A reference architecture for AI powered web applications

This is the shape most of our AI applications end up with. Your project may need only some layers on day one. It still helps to know where each piece will go before the first feature ships.

The layers

1. Interface. A web or mobile front end, for example Next.js and React, that streams responses and shows sources and pending actions clearly.
2. Application API. Your own service layer handles authentication, permissions, rate limits and business rules. The browser never calls a model provider directly. If it does, your API key is one view-source away from being public.
3. AI orchestration. Prompts, tools, agents and retrieval, built with something like LangGraph, the Claude Agent SDK or the OpenAI Agents SDK, with MCP for tool access.
4. Data. Your operational database, a vector index such as pgvector or Qdrant, and object storage for documents.
5. Model providers. Claude, OpenAI, Gemini or open weight models, reached through a thin adapter so one can be swapped for another.
6. Observability. A trace for every request, evaluation results, cost per feature and alerts.

Principles

• Permissions first. Decide what the user is allowed to see before retrieval runs.
• Prompts are code. Version them, review them in pull requests and test them.
• Evaluations in CI. A change that lowers answer quality should fail the build, the same way a broken unit test does.
• Stream and show the work. People trust a system more when they can see its sources and steps.
• Plan for failure. Providers have slow days. Set timeouts, have a fallback model and show a clear message when a call fails.

Deployment

Containers on AWS, Google Cloud or Azure, or a platform such as Vercel for the front end. Secrets in a managed vault. Separate development, staging and production environments, each with its own keys and its own spending limit, so a load test in staging cannot burn through the production budget.