# How to add AI to an existing web app

> Start with one job a user already does by hand, ground the model in data you already hold, and set accuracy, latency and cost budgets before writing code. Ship behind a feature flag to a small group, measure against those budgets, then widen. The integration work, not the model, is what takes the time.

Most teams asking this question already have the hard parts. There is a database with real data in it, an authentication system, a deployment pipeline, and users who will notice if something breaks. That is a much better starting position than a greenfield AI project, and it changes the order the work should happen in.

## Which feature should you build first?

Pick a job somebody already does by hand, repeatedly, inside your product. Support agents pasting the same explanation. Ops staff reading PDFs and copying five fields into a form. Users scrolling a list because search only matches exact words.

Those make good first features for three reasons. The value is measurable, because you know roughly how many minutes the manual version takes. The training data already exists, in the form of what those people did last month. And the definition of "correct" is available from someone in the building, which you will need shortly.

Avoid, for a first feature, anything whose output goes straight to a customer unreviewed, anything that spans more than two systems, and anything where nobody can say what a good answer looks like.

## Why grounding matters more than the model

A general model knows a great deal about the world and nothing about your business. Ask it about your refund policy and it will produce something that reads like a refund policy, which is worse than useless.

Grounding is the fix: retrieve the relevant text from your own systems, put it in the prompt, and instruct the model to answer only from it. This is retrieval-augmented generation, and it is where the engineering effort actually goes.

In practice that means deciding what a retrievable unit is (a document is usually too big, a sentence too small, a section about right), keeping a searchable index of those units next to the source records, and passing the model the handful most relevant to the question along with where each came from, so the answer can cite them.

Your existing database is often enough to start. Postgres with `pgvector`, or even full-text search, will carry a first feature further than teams expect. A dedicated vector database is a scaling decision, not a starting requirement.

## What budgets should you set before writing code?

Three numbers, agreed before implementation, because each one changes the design:

**Accuracy.** Assemble thirty to fifty real examples with known-good answers, and decide what percentage must be right for the feature to be worth shipping. Thirty examples in a spreadsheet is a perfectly respectable eval set, and it is thirty more than most teams have.

**Latency.** A search box needs to feel instant; a nightly document pipeline does not. That single decision determines whether you can afford a large model, whether you need streaming, and whether the work belongs in a queue.

**Cost per call.** Multiply the expected token count by the provider's price, then by your expected monthly volume. Do this on a napkin before you build, because it is the number most likely to kill the feature later, and the design that fixes it (a smaller model, cached retrieval, a cheaper first-pass filter) is much easier to adopt at the start.

## How do you ship it without risking the product?

Put the feature behind a flag and turn it on for a handful of users who know they are early. Log every request and response, including the retrieved context, so that when someone reports a bad answer you can see exactly what the model was looking at.

Keep a human in the loop anywhere the feature writes, sends, or spends. An agent that drafts and asks for approval is roughly as useful as one that acts autonomously, and it fails far more gracefully.

Then compare against the budgets you set. If accuracy is short, the fix is almost always retrieval rather than prompting: the model usually cannot see the right text. If cost is short, look at how much context you are sending before you look at anything else.

## Where the time actually goes

Teams consistently expect the model work to dominate and find that it does not. On a typical first feature, prompt design is a few days. Retrieval (deciding what to index, keeping it in sync as records change, making the results good) is a couple of weeks. Evaluation, the interface, permissions, error states, logging and the rollout are the rest.

That is a normal software project with an unusual component in the middle, which is good news: it means your existing engineering practice mostly applies. The parts that are genuinely new are the eval set and the habit of treating the model as unreliable by default.

## The sequence, condensed

1. Choose one repetitive job inside the product.
2. Write down thirty real examples and their correct answers.
3. Set accuracy, latency and cost budgets.
4. Build retrieval over data you already hold.
5. Ship behind a flag to a small group, logging everything.
6. Measure against the budgets, fix retrieval first, widen.

None of this requires a platform migration, a new team, or a strategy deck. It requires picking something small enough to finish and being honest about whether it works.

---

Published by Vuewer. AI features and web applications, shipped to production.
Canonical version: https://vuewer.com/blog/how-to-add-ai-to-an-existing-web-app