Short answer: building a single, narrow AI feature into an existing product typically takes 3 to 6 person-weeks; a retrieval-based assistant over your own documents 8 to 18; an agent that acts in other systems 15 to 25 or more. Running cost is set by tokens per request multiplied by request volume, and for most features it lands between a fraction of a cent and a few cents per request. The rest of this guide shows how to work out both figures for your own case.
About these numbers. Effort ranges are planning estimates for an experienced team, not survey data and not a BBR price list. The per-token prices used in the worked examples are illustrative placeholders chosen to make the arithmetic easy to follow. Real prices differ by provider and model and change often, so take current figures from the provider’s pricing page and put them into the same formula.
The build cost
The build follows the same rule as all software: effort multiplied by rate, plus contingency. Rates by region and team type are covered in our MVP cost guide. What is different about AI work is where the effort goes. The call to the model is a small part; the data, the evaluation and the handling of wrong or slow answers are the bulk.
| Component | Typical effort | What drives it |
|---|---|---|
| Prototype and feasibility check | 2–4 person-weeks | Usually a separate first stage; answers “is the quality good enough?” on your data |
| Single-call feature (summarise, draft, classify, extract) | 1–3 | Prompt design, structured output, validation, interface, error states |
| Document ingestion and indexing for retrieval | 2–5 | File formats, scanned documents, tables, how often content changes, number of sources |
| Retrieval quality (chunking, hybrid search, reranking, citations) | 1–3 | How precise answers must be; how similar documents are to one another |
| Permission-aware retrieval | 1–3 | Whether different users may see different documents, and how those rules are stored |
| Each tool or system an agent can use | 0.5–2 | API quality, authentication, write actions versus read-only |
| Agent loop, state, limits and approval steps | 2–4 | Number of steps, need for human approval, recovery from failed steps |
| Evaluation set and test runner | 1–3 | Collecting real examples, defining correct answers, automated scoring |
| Guardrails (input checks, output validation, abuse limits) | 1–2 | Public-facing versus internal; sensitivity of the domain |
| Conversational interface with streaming, history and feedback | 2–4 | Less if the feature sits inside an existing screen |
| Usage metering, per-customer limits, cost dashboard | 1–2 | Essential in multi-tenant products |
| Provider fallback and model routing | 0.5–1.5 | How critical availability is |
| Testing, deployment, project management | Add 20–30% | |
Three build examples
A. Summaries inside an existing product. A “summarise this thread” button in a support tool. Single-call feature (2), small evaluation set (1), guardrails and limits (1), metering (1): 5 person-weeks, about 6 with overhead. At $50/hour roughly $12,000; at $150/hour roughly $36,000.
B. Knowledge assistant over company documents. Staff ask questions and get answers with source links, respecting department-level access. Ingestion (3), retrieval quality (2), permissions (2), evaluation (2), guardrails (1), chat interface (3), metering (1): 14, about 18 with overhead. Roughly $36,000 at $50/hour; $108,000 at $150/hour.
C. Agent that prepares and files records. Reads incoming requests, looks up the customer in a CRM, drafts a response and a record update, and waits for human approval. Four tools (5), agent loop and approvals (4), evaluation (3), guardrails (2), review interface (3), metering and fallback (2): 19, about 24 with overhead. Roughly $48,000 at $50/hour; $144,000 at $150/hour.
If the AI feature is part of a new product, these figures sit on top of the product itself: accounts, billing, admin and the rest. See SaaS development cost for those blocks.
How token pricing works
Hosted language models are billed by the token. A token is a fragment of text; in English, one token averages roughly three-quarters of a word, and other languages and source code generally use more tokens for the same content. Five things about the pricing model matter for budgeting:
- Input and output are priced separately, and output tokens cost several times more than input tokens.
- Everything you send counts as input on every call: the system prompt, tool definitions, retrieved documents, and the whole conversation so far. A chat gets more expensive with each turn because the history is sent again.
- Model tiers differ widely in price. Within one provider’s range, the largest model can cost ten or more times as much per token as the smallest.
- Reasoning or “thinking” tokens, where a model works through a problem before answering, are billed as output even though the user never sees them.
- Discounts exist for repeated and non-urgent work: cached input is billed at a reduced rate, and batch processing that can wait hours is commonly offered at a substantial discount.
A worked formula for running cost
Cost per request = (input tokens × input price) + (output tokens × output price)
Monthly cost = cost per request × model calls per task × tasks per month
For the examples we use two made-up models. These are not any provider’s actual prices.
| Illustrative model | Input price per million tokens | Output price per million tokens |
|---|---|---|
| “Small” | $0.50 | $2.00 |
| “Large” | $5.00 | $20.00 |
Example 1: support answer with retrieval
Each request sends a 500-token system prompt, 3,000 tokens of retrieved passages and a 100-token question (3,600 input), and receives a 400-token answer.
| “Small” model | “Large” model | |
|---|---|---|
| Input: 3,600 tokens | $0.0018 | $0.018 |
| Output: 400 tokens | $0.0008 | $0.008 |
| Per request | $0.0026 | $0.026 |
| 20,000 requests per month | $52 | $520 |
| 500,000 requests per month | $1,300 | $13,000 |
The same feature varies tenfold with model choice alone. Whether the small model is good enough is a question only your evaluation set can answer, which is why evaluation is a cost-control tool as much as a quality one.
Example 2: a multi-step agent
An agent completes a task in 8 model calls. Context grows as tool results accumulate, averaging 6,000 input tokens and 300 output tokens per call on the “large” model.
- Per call: (6,000 × $5 ÷ 1,000,000) + (300 × $20 ÷ 1,000,000) = $0.030 + $0.006 = $0.036
- Per task: 8 × $0.036 = about $0.29
- 5,000 tasks per month: about $1,440
Two lessons. Agents multiply cost by the number of steps, so a cap on steps is also a cap on spend. And whether $0.29 is expensive depends entirely on the task: for something that saves a person ten minutes it is trivial; for a free-tier convenience feature it is not.
Check the unit economics
Put the running cost next to what the customer pays. If a plan costs $30 a month and a typical user triggers 300 retrieval requests on the “large” model, the AI cost is about $7.80, a quarter of the revenue, before hosting and support. The same usage on the “small” model costs $0.78. That single comparison often decides the model, the plan limits and whether the feature is included or sold as an add-on.
Levers that reduce the running cost
| Lever | How it helps | Cost of using it |
|---|---|---|
| Use the smallest model that passes evaluation | The largest single saving, often an order of magnitude | Requires an evaluation set to prove it |
| Route by difficulty | Small model for routine requests, large model only when needed | Routing logic and its own evaluation |
| Prompt caching | Providers bill a repeated prompt prefix (system prompt, tool definitions, a fixed document) at a reduced rate | Prompts must be structured with stable content first; cache lifetimes are short |
| Response caching | Identical or near-identical questions answered from your own cache at no model cost | Invalidation when source content changes; care with per-user data |
| Send less context | Better retrieval means fewer, more relevant passages; summarise or trim old conversation turns | Retrieval tuning effort |
| Limit output | Output is the expensive side; ask for concise, structured answers and set a maximum length | Minimal |
| Batch processing | Non-urgent jobs (nightly classification, backfills) at a discounted rate | Results arrive hours later |
| Cap agent steps and retries | Bounds the worst case per task | Some tasks end unfinished and need a fallback |
| Do not call the model at all | Rules, search or a database query for the cases that do not need language understanding | Ordinary engineering |
| Per-user and per-tenant quotas | Protects against abuse, bugs and one heavy customer | Metering infrastructure, which you need anyway |
The cost of evaluation
Evaluation is the line most often missing from AI budgets, and the one that pays back fastest.
- Building the set. Collecting 50 to 300 real examples and agreeing the correct result for each takes days of time from someone who knows the domain. That person is usually on your side, not the developer’s, so plan for their time.
- Running it. Every run sends the whole set through the model, and automated grading with a second model doubles the calls. At a few hundred examples this costs a few dollars per run, which is negligible next to the engineering time it saves.
- Human review. Automated scores drift from human judgement. A periodic manual check of a sample keeps them honest.
- Re-running on change. New prompt, new model version, new document source: run the set again. When a provider retires the model you launched on, the evaluation set is what turns the migration into a day’s work instead of a month of uncertainty.
How to build and use an evaluation set is covered in our guide to integrating an LLM into a product.
Other running costs
| Item | Planning note |
|---|---|
| Embeddings | Priced per token and far cheaper than generation. Indexing a document collection is normally a small one-off cost, repeated only for changed content |
| Vector storage | Free if you use the vector support in a PostgreSQL database you already run; a dedicated vector database adds a monthly fee |
| Document parsing and OCR | Metered per page by most services; matters for scanned or image-heavy archives |
| Speech-to-text and text-to-speech | Metered per minute or per character; voice features can cost more than the language model |
| LLM logging and monitoring tools | Free tiers to low hundreds of dollars a month; or plain logs in your own database |
| Provider rate limits | Not a fee, but higher throughput tiers may require prepayment or spending history. Check before a launch |
| Maintenance engineering | Model version migrations, prompt adjustments, evaluation upkeep, new content sources |
A one-page budget
- Define the task and the volume. How many times per user per month, and how many users?
- Prototype first. Two to four weeks to find out whether quality is sufficient, and on which model tier.
- Measure tokens per request in the prototype, not by guessing. Include system prompt, retrieved context and history.
- Apply the formula with current provider prices, for an expected case and a heavy-use case.
- Compare with revenue per user. Decide on limits, tiers or add-on pricing.
- Estimate the production build from the component table, plus 20–30% for testing and management.
- Add upkeep: a few days per quarter as a floor.
- Set alerts and hard caps with the provider and in your own code before launch.
If step 5 does not work on paper, it will not work in production, and it is far better to learn that in week three than in month six. Our guide to building an AI product covers the wider process, including when the right answer is not to use AI at all. If you would like help with the estimate, BBR’s AI development service begins with exactly this exercise.