We recently had the opportunity to collaborate with Shawn "swyx" Zhang to share some of our learnings so far (from building Codeium) that would generalize to anyone interested in building their own "Copilot for X" product.

The TL;DR is that to build a “Copilot for X”, you must:

• Estimate inference scale: By reasoning about the tradeoff between latency and output quality
• Build first party: Third-party APIs will wreck your unit economics
• Figure out Realtime Infra: ChatGPT, Dall-E, etc are too slow to keep flow
• Optimize prompts: Context window is limited; what info most improves output?
• Merge model outputs and UX: Raw model output is unintuitive; long tail of UX.

Read the whole post on swyx's L-Space Diaries newsletter!