Putting It All Together

The first internal tool I shipped was a support dashboard: a place where I could see the incoming questions my teammates were drowning in. It told me what was happening, but it still left a human to answer every message. The interesting work started when I asked a harder question — could the system draft the answer itself, and only pull a person in when it wasn't sure? That question turned a dashboard into the de-identified support-automation assistant I kept refining afterward.

In modules 06 through 08 you built that assistant's parts one at a time: a first real Anthropic call, a knowledge surface it could read from, and an agent that decides what to do. On their own, each part is a demo. This module is where they stop being demos and start being a system — a small, scoped, runnable support-triage assistant that reads a customer message, answers from what it knows, and escalates when it's out of its depth.

Think of it like a triage nurse. A nurse doesn't treat everyone the same way: they take in the complaint, match it against what they already know, and decide in seconds whether to handle it or call the doctor. Our assistant does the same three things — read, match, decide — and the whole point of this capstone is wiring those three moves into one honest flow.

By the end you'll have a single file you can run that takes a message and returns either a grounded answer or a clean hand-off to a human — no café, no cartoon business, just the real shape of the assistant this path has been building toward.

Before any code, let's name the three responsibilities and how the earlier modules feed them. Keeping the build this small is deliberate: every line has to earn its place, because a support assistant that's hard to reason about is a support assistant you can't trust.

• Retrieve — pull the most relevant fact from a small knowledge base. This is the read-only "what do we already know" surface you met as MCP resources in module 07.
• Triage — make one real Anthropic call (module 06's first call, module 08's agent) that classifies the message, drafts an answer from the retrieved context, and rates its own confidence.
• Escalate — a plain Python decision: if the model isn't confident, hand the draft to a human instead of sending it.

The data flows in one direction: a message comes in, retrieve grounds it, triage reasons over it, and escalate decides who replies. There's no parallel orchestration and no hidden state — just a short pipeline you can hold in your head.

Mid-Module Checkpoint

Before you write a line: of the three moves — retrieve, triage, escalate — which one is the only one that should ever talk to the language model, and why does keeping the other two as ordinary Python make the system easier to trust?

A support assistant is only as honest as the facts you give it. Rather than hoping the model remembers your product, we hand it a tiny knowledge base and a function that finds the closest entry. This is the same instinct behind the retrieval system in my support-automation work: ground the model in your truth so it can't confidently invent policy.

Notice the last line. When nothing matches, retrieve returns an empty string on purpose — it does not guess. That empty result is a signal we'll use in a moment to push the model toward low confidence instead of a confident hallucination.

Here is the only place the system talks to the model. We give it the retrieved context, ask it to classify the message, draft a short answer using only that context, and — crucially — report a confidence on the first line so the rest of our plain-Python flow can act on it. We use claude-haiku-4-5: it's fast and cheap, which is exactly what you want for high-volume triage where most messages are routine.

Two design choices are doing the heavy lifting. First, "use ONLY the provided context" turns the empty-string signal from retrieve into low confidence instead of a guess. Second, forcing the confidence onto the first line means the next step doesn't have to re-parse free text — it reads one number and decides.

Escalation is where most "AI support" demos quietly cheat — they send every answer and hope. Ours makes the boundary explicit. We read the confidence the model reported and, if it's below our threshold (or if we can't read it at all), we route the draft to a person.

The except branch is the quiet hero. If the model ever ignores our format, we don't crash and we don't send a mystery message — we fail safe, toward a human. In production support, the cost of a wrong confident reply is far higher than the cost of one extra hand-off, so the default when in doubt is always "ask a person."

Mid-Module Checkpoint

Predict the behavior: a customer asks "why was I double-charged this month?" — a billing question your KB only partially covers. Walk through retrieve, triage, and needs_human. Where in that path does the system get its chance to escalate instead of bluffing?

Now the three moves become one function. This is the whole assistant: ground the message, reason over it, decide who answers. It's short enough to read in a breath — and that's the feature, not a limitation.

Run it on the export question and you'll get a grounded, confident answer. Run it on something the KB never heard of — "can I get a refund in EUR?" — and watch it escalate instead of inventing a policy. That contrast is the entire lesson of this path made concrete: the system is trustworthy precisely because it knows its own edge.

A Reflective Prompt Before You Move On

Your threshold is 0.4. If support leadership told you "we'd rather over-escalate than ever send a wrong answer," would you raise or lower it — and what does the opposite choice cost in human workload? There's no single right number here; the point is that the threshold is a business decision you can now reason about, because you built the lever yourself.