What an AI integration project actually looks like, week by week

Why the timeline is the post#

Most "AI projects" that go wrong go wrong the same way. Someone watched a demo, imagined a six-week build, then realised six months in that the model was answering on vibes and the team could not tell whether the thing was helping or hurting. The fix is not a cleverer model — it is a more honest process.

This is the structure I run, end to end, for any non-trivial LLM integration. It is six weeks of calendar time, give or take, and most of the steps are non-negotiable.

Week 1 — diagnose the workflow, not the model#

The first week is spent with the team, on the ground, watching how the work actually gets done. Which inbox do the hard questions land in? What does the human do next — and how do they decide? Where do they search, and what do they paste into the answer? Which part of the response is template, which part is judgment, which part is a copy-paste from another system?

I leave week 1 with a one-page brief: the workflow, the human steps that can be automated, the human steps that must stay human, and the three or four failure modes I am most worried about. No model has been chosen. No code has been written. If we cannot write that brief, we do not start.

Weeks 2–3 — the unglamorous 60%: data and retrieval#

Most of the project budget goes here. The work is: identify every document, ticket, transcript, or database row the agent needs to read; clean it; chunk it; embed it; wire it to a vector store; and write the retrieval logic that picks the right context for the right question.

This is also where I make the boring infrastructure decisions. Postgres plus pgvector if the data is already in Postgres. A dedicated Qdrant or Weaviate if the volume is high. A local Ollama-served embedding model if the data is private. None of it is glamorous; all of it determines whether the agent gives good answers or confidently makes things up.

I also instrument everything from day one. Every query, every retrieved chunk, every prompt, every response, every latency. If we cannot replay a conversation in three months, we have built a black box, not a product.

Week 4 — guardrails and evals (how you actually know it works)#

By week 4 we have a working retrieval-augmented agent answering real questions. Now we have to prove it answers them well.

I write an evaluation set of fifty to two hundred real examples drawn from the team's actual inbox — the things they wish they did not have to answer twice. Each one has a known-good answer or a known-good behaviour. I run the agent against that set every time the prompt, the model, or the retrieval changes. If the score drops, we do not ship.

Guardrails sit next to evals. Output schema validation. PII detection on the way out. A refusal path when retrieval returns nothing. A confidence threshold below which the agent hands off to a human instead of guessing. The expensive production failures are rarely bad answers — they are confident bad answers that the system had no reason to suppress.

Week 5 — pilot with humans in the loop#

Week 5 is the pilot. The agent runs in production, but every response is reviewed by a human before it leaves the building. We measure agreement: how often the human would have sent the same answer, how often they had to rewrite it, and how often they had to throw it out entirely.

This is also where I tune the prompts, the retrieval thresholds, and the handoff rules based on what the human reviewers actually flag. I do not tune on vibes. I tune on the disagreements. If the agent is wrong about a specific class of question, I fix that class — I do not just patch one example.

I am explicit with the team during pilot: this is not a test of the agent. This is a test of whether the workflow as we have redesigned it is something humans actually want.

Week 6 — handoff and runbook#

If the pilot survives, week 6 is handoff. I write the runbook: how to deploy a model update, how to roll back, how to read the eval dashboard, who to call when latency spikes, and what to do when the agent makes a mistake in production. I hand the keys to the team that will own it, and I stay on retainer for the first month.

A runbook is not a manual. It is a short, opinionated document that tells the on-call engineer exactly what to do when they get paged at 02:00. If I cannot write one in an afternoon, the system is too complex to hand off.

What makes these projects fail#

The same failure modes come up again and again:

• Scope creep. "While we are at it, let's also do invoicing." No. Finish one workflow end to end, then expand.
• Skipping evals. "We'll add evals after the pilot." No. The pilot is the eval.
• Skipping the pilot. "The demo looked great, let's just ship it." No. Real users break real systems.
• Letting the model pick the workflow. The model is a tool. The workflow is a business decision. Keep them separate.
• Treating retrieval as a feature. Retrieval is the product. If the agent cannot find the right context, nothing else matters.

If you are weighing an integration and want a second opinion on whether the workflow is ready, the services page lists what I actually take on. If you already know you want to start, tell me the workflow and I will tell you honestly whether it is a six-week project or a six-month one.

---

Frequently asked questions#

Do all AI projects take exactly six weeks?#

No — six weeks is for a single, well-scoped workflow with clean data. A multi-workflow rollout, a regulated industry, or a greenfield data architecture can easily run three to four months. Anything advertised as faster is either skipping a step you will regret, or the workflow was trivial to begin with.

Can you skip the pilot and go straight to production?#

You can, and I will not stop you, but I will also not put my name on it. Skipping the pilot means the first users to break the system are your customers, and the classes of failure that week 5 is designed to surface show up in production instead. The pilot is cheap insurance.

What if our data is a mess?#

That is the most common starting point, and it is exactly why weeks 2 and 3 exist. I do not pretend the data is clean. The first deliverable in those weeks is an honest audit: what is structured, what is semi-structured, what is in someone's head, and what needs to be cleaned before any model touches it. Sometimes the right answer is "do not start yet" — and that is a useful answer too.

---

Sources: Anthropic, Building Effective Agents; OpenAI, A Practical Guide to Building Agents.