Key takeaways
- Choose by task and constraints, then test on your own data
- Where the data may go often decides between hosted and open weight models
- Build so the model can be swapped later without a rewrite
Start with your constraints, not a leaderboard
Leaderboards reshuffle every few weeks. Your constraints change slowly, so they are the better place to start. Four questions cover most of it.
- What is the task? Reasoning over long contracts, writing code, calling tools, classifying short messages, voice, or reading images.
- How sensitive is the data? Public, internal, personal, or regulated, such as health records or bank KYC.
- Where must it run? A public API, a private tenant in your cloud account, or servers you control.
- What volume and speed do you need? A hundred calls a day from an internal team is a different problem from a million calls from a consumer app.
The main families
- Anthropic Claude. Strong at careful reasoning, long documents, coding and agentic tool use. Available directly and through AWS Bedrock and Google Vertex AI.
- OpenAI models. Broad capability and a large ecosystem of tools, available directly and through Azure AI Foundry.
- Google Gemini. Strong with images, audio and very long context, and a natural fit if your data already sits in Google Cloud.
- Open weight models such as Llama, Qwen and Mistral. You run them on your own infrastructure. Worth it when data must not leave your environment, or when volume is high enough that hosting costs less than paying per call. The trade-off is that you now operate the model yourself.
You will probably use more than one
Most production systems route requests. A small, quick model sorts incoming WhatsApp messages into "order", "complaint" and "other". A stronger model handles the complaint that needs a careful reply. Cost and latency stay low, and quality stays high where it matters.
Test on your own data
Collect 50 to 200 real examples with the answers you would accept. Run every candidate model through the same set and compare accuracy, cost per task and response time. Expect the order to change. The model that wins the public benchmark is often not the one that handles your purchase orders best.
Leave yourself a way out
Put a thin adapter between your application and the provider. Keep prompts and evaluation sets under version control. When a new model comes out, re-run the evaluations and let the numbers decide. Changing models should take a measured afternoon of work.
Have a question about this guide? Write to us or ask Colony, our AI guide.