Field Kit

For a while this page displayed what each run cost, and those numbers were fabricated: a hardcoded constant plus your character count divided by four. Fixing it exposed a second problem. Prompt caching had never fired at all, because the cache marker sat on a prompt too short to qualify on either model.

Both are fixed, and everything below is what I rebuilt. The lesson is narrower than test your code: measure the thing you are claiming. I had written confidently about model routing on a page whose own instrumentation was guessing.

And I shipped this page as three separate tools, each with its own name, icon, and accent color. Three was branding, not architecture — the corpus, the guardrails, and the judgment underneath never changed. It took an outside eye pointing out how machine-made the symmetry felt before I collapsed them into the single question box above.

Every answer draws on a written record of nine projects I have worked on — the case file rendered on this page. The first version scored that record against your question and passed the model only the best four entries. It kept dropping the one that actually fit. Nine entries is small enough that a vector database would be pure overhead, so now I send all of it and let the model choose, with a lexical ranking attached as a hint rather than a filter.

That also made caching possible. A cached prompt only works on an exact prefix match, so the corpus sits in front of the cache marker and your question sits behind it. My first design cached the retrieved subset, which changed with every question, so nothing ever hit. Both models now carry the same cached prefix — system prompt plus corpus — and each keeps its own cache entry. The thresholds I measured still stand: 1,024 tokens on Sonnet, 4,096 on Haiku, and the shared prefix clears both. The trade-off: that prefix runs about 6,000 tokens on Haiku and 8,500 on Sonnet, a one-hour cache write costs twice a normal request, and a read costs a tenth. You break even on the third question.

Every question goes to a Haiku 4.5 classifier first, which decides one thing: is this a genuine either/or, or an open question? An either/or gets answered by Haiku too, where the first words appear in a few seconds — the router itself accounts for a third or so of that — a clean binary wants speed more than depth. Everything else goes to Sonnet 5, where the answer needs judgment and a few more seconds is a fair price. The router itself costs a few hundred milliseconds and a fraction of a cent — the footnote shows that time on its own, and folds the cost into the one total under every answer.

This page used to route by tool instead — three fixed tools, each pinned to a model. Collapsing the tools forced the routing decision down to where it belonged: the question. If the classifier errors, the run falls back to the judgment path and the footnote says so.

The model returns JSON matching one schema I define, constrained to that shape rather than asked for it. The page validates the result again before rendering, so an answer it cannot draw becomes a handled error instead of a broken page.

Having a real contract made an old problem obvious. Back when this page was three separate tools, one of them used to rate each option out of five and compute a weighted total to two decimals. It could not measure these options, so those numbers were invented. The single schema now names which option wins each point and why, which was the only part carrying information.

A model will answer a question it has no business answering in the same tone it uses when it knows. Asking for hedging in the prompt works some of the time, which is not good enough. So confidence is a required field — HIGH, MED, or LOW, no hiding in prose — and the unsure section, what it cannot account for and which facts you never gave it, is a required object the prompt has to fill in. A response missing that structure fails validation before it reaches the page.

The most useful thing it produces is not the position, it is the list of what you have not told it. That sits directly under every answer.

Token usage only exists once the model has finished, long after the response headers have gone out, so it cannot travel in a header. The server appends it as a framed trailer at the end of the same stream and the client strips it off before the JSON parser sees it. When a number is not known yet, the panel says so rather than estimating.

Answers render as they arrive, so the first words appear in a second or two instead of ten. That means reading half-built JSON — unfinished arrays, fields that do not exist yet — which every renderer treats as an expected state before validating the finished object in full.

Rate limits, timeouts, overload and malformed output each map to a specific message that says what happened and whether trying again will help. Requests are validated on the way in and responses on the way out.

You cannot eyeball a system that answers differently every time, so there is a suite that runs real questions against the real thing and grades what comes back. Every question is asked 3 times, because one good answer proves nothing. Last full run on Sep 4, 2026: 328 of 337 checks passed across 9 questions.

Most checks are plain code: exactly two options, no invented numbers, and every cited company actually exists in my corpus. One re-derives the cached prefix for two different questions and fails if a single character differs, which is precisely the bug that sat here unnoticed.

A recent catch: I added my current role at Lyft, where the work is still in progress. The prompt asks each example to lead with a failed first attempt, so the model produced one — a failure at my current job that never happened. Entries can now be marked in progress, and a check fails the run if any past-tense outcome is attributed to them.