The Real Cost of an AI Feature in Production

The real cost of an AI feature in production is not the API bill

Direct answer: the real cost of an AI feature in production is the sum of four layers rarely budgeted together: data preparation, model calls, guardrails and monitoring, and drift maintenance. The model call, the one you see first on a pricing page, is often less than half of the total three-year bill. The rest of this article breaks down these four layers, gives an order of magnitude by feature type, and explains when it is better not to build at all.

A Swiss SME looking to add a support assistant, a document extraction module, or a scoring engine usually gets a single number: the per-token price of the chosen model. That number is reassuring because it is precise. It is also almost always incomplete. The running cost of an AI feature in production follows a different pattern than a regular software module, because it keeps costing money after launch, at a pace driven by real user behavior rather than the original spec.

The four layers that make up the bill

Each layer shows up at a different point in the feature's life cycle. Conflating them is the single most common reason budgets end up 30 to 50 percent short, based on experience reports from comparable projects.

1. Data preparation

Before the first model call, documents or records need cleaning, structuring, and chunking. For a RAG system (retrieval-augmented generation), this layer includes indexing, chunk-size decisions, and continuous updates to the underlying knowledge base. It is the most underestimated layer, because it looks like ordinary integration work and no model provider bills it separately on a pricing page.

2. Model calls

This is the visible layer: a price per million input tokens, another per million output tokens. The math gets harder once an agent chains several calls for a single task (intent classification, planning, tool selection, execution, verification), or once conversation history accumulates inside the context sent with every request. An agent that runs five internal steps to answer one user question can multiply the apparent cost of a simple direct answer by five.

3. Guardrails and monitoring

Off-topic response detection, sensitive-data filtering, per-feature and per-user logging, budget-overrun alerts: this layer produces no visible value for the end user, but its absence turns a minor incident into a five-figure bill. A retry loop that keeps hammering the same call, or a context leak that inflates every request, can turn a projected monthly bill of a few hundred francs into several thousand within days, with no warning if nothing was built to detect it.

4. Maintenance and drift

A model changes version, a provider adjusts pricing, user behavior evolves, and edge cases multiply. A feature that shipped at 95 percent reliability drifts if nobody regularly re-evaluates its real-world answers. This layer also covers prompt updates, regression tests against already-solved cases, and switching models when a cheaper competitor reaches sufficient quality for the task.

An order of magnitude, not a quote

The figures below come from published observations on North American projects, at a larger scale than most Swiss SMEs operate at. They are useful as a proportion between layers, not as a pricing grid to apply directly: a single feature added to an existing product costs far less than a platform built from scratch.

The practical consequence: negotiating a provider's per-token price only moves one of four layers, often the one that weighs least over three years once the feature is live.

A worked example, for illustration

Take a support assistant answering first-line questions for a mid-sized SME: 200 conversations a day, three exchanges on average per conversation, an average context of 2,000 tokens per call once history and retrieved documents are included. That is roughly 1.2 million input tokens a day, a volume that comfortably fits standard pricing tiers from current model providers and, on this layer alone, usually costs a few hundred francs a month, not more.

The calculation changes shape once the other three layers are added. Building the initial knowledge base (indexing product documentation, terms of service, resolved ticket history) requires one-off integration work, followed by continuous updates as new documents appear. Guardrails require detecting out-of-scope questions (legal, sensitive billing) and routing them to a human instead of an invented answer. Maintenance requires a regular review of a sample of real conversations, to confirm answers stay correct as the product catalog changes. Added up, these three layers almost always exceed the cost of model calls over time, even for a volume as modest as this example.

This is not a universal rule: a very high-volume, short-context use case (classifying a form field, for instance) shifts the balance toward inference. The method stays the same: cost the four layers separately before comparing offers.

How to estimate the cost before you start

A useful estimate is built before writing a single prompt, from measurable data rather than assumptions.

1. Count the real expected requests per day, not a theoretical volume: active users multiplied by interactions per user.
2. Estimate the average context length sent with each call, conversation history and retrieved documents included, not just the user's message.
3. Count calls per task if an agent orchestrates several steps, not one call per visible interaction.
4. Add a realistic retry rate (10 to 20 percent is a common starting point for integrations with unstable third-party systems).
5. Cost the integration with existing systems separately: CRM, ERP, ticketing. This is often the longest layer to deliver, not the most visible on an API bill.
6. Set aside a 15 to 25 percent reserve on the total, for the guardrails and drift that only appear after a few weeks of real usage.

When a custom AI feature is not the right choice

It is not always the right answer, and saying so clearly avoids months of unnecessary development.

If expected volume stays low (a few dozen requests a day) and the task is well defined, an existing tool (a packaged support agent, a CRM extension already in place, a no-code automation) often covers the need for a fraction of a custom build's budget, without layers 3 and 4 described above. Building a custom feature makes sense when the task touches proprietary data, a business flow specific to the company, or a volume that justifies the monitoring investment. If none of these conditions hold, the return on investment of a custom build rarely arrives before twelve to eighteen months, and the maintenance layer keeps running even when real usage stays marginal.

Another common case: a company that has no process yet to measure the quality of the answers produced. Adding an AI feature before knowing how you will evaluate whether it works is building without a control plan. It is better to first define a representative, even modest, set of test cases than to launch a feature nobody can tell, six months later, whether it has drifted. A third case worth naming plainly: when the underlying process is still changing month to month. An AI feature built on top of a workflow that has not stabilized yet mostly automates the churn, and the maintenance layer ends up rebuilt as often as the process itself, which defeats the point of automating it in the first place.

What a Forward Deployed Engineer approach changes in this calculation

A large share of budget overruns comes from a choice made too early: building the full architecture before observing real user behavior over the first weeks. The approach we follow on engagements, close to what forward deployed engineering covers, is to ship a narrow scope on real data from the first weeks, then adjust layers 3 and 4 based on what actually breaks, rather than provisioning them on assumption. We followed this logic on a wealth-management platform where integration with existing systems accounted for most of the work, far ahead of the model call itself: the codebase of that platform, built on a base of roughly 106,000 lines, grew through measured iterations rather than a plan fixed at kickoff.

It is also why the breaking points of an agent in production rarely hit the model first: we cover these breaking points in a dedicated article on what breaks first when an AI agent goes into production. For an SME choosing between a packaged module and a custom build, the same questions settle it: real volume, data sensitivity, the ability to measure quality over time. These are also the questions we ask systematically before pricing a fixed-scope or staff-augmentation engagement, and what the studio's day-to-day work looks like on this type of project.