On the evening of 11 September 2026 the project opened its Post-LM gate with a new instrument, the Latency Lab (tools/latency/lab.py). It drives a fixed set of scripted journeys through the normal launcher on the live system and breaks every turn into named stages. The journeys ask for status, the time, a recall, a project, a HUD action, the last turn’s latency and a short chat, each three times: 21 turns per run. The same evening, two small changes cut the median turn time by 59%. This entry describes the measurement, the two changes and the limits of the result.
The baseline
The first run, at commit 8daf99045, labelled “pre-optimisation”, recorded:
| Measure | p50 | p95 |
|---|---|---|
| Whole turn, end of input to end of turn | 3,083 ms | 5,194 ms |
| End of speech to first answer audio | 2,749 ms | 3,362 ms |
| End of speech to first model token | 2,649 ms | 3,180 ms |
All 21 journeys answered correctly. The lab’s bottleneck ranking split the time between inference (59.1% of recorded stage time) and voice (34.2%). Two stages dominated. The provider request had a p50 of 1,704 ms. The acknowledgement stage, the short spoken cue JARVIS plays so the owner knows it heard, had a p50 of 1,138 ms.
That second figure was the surprise. The acknowledgement was meant to make waiting feel shorter, but the code waited for it to finish before sending the request to the model. Every cognition turn paid for the cue and then paid for the model.
Change one: the acknowledgement stops blocking
Commit 689483e18, “the acknowledgement no longer blocks the provider call”, overlaps the two. The provider request now starts while the cue is still playing. Two constraints were kept. The answer’s first clause still waits for the acknowledgement to finish, under the barge-in watcher, so JARVIS never talks over itself. An interruption during the acknowledgement now stops it.
On the same 21 journeys:
| Measure | Before | After | Change |
|---|---|---|---|
| Whole turn p50 | 3,083 ms | 1,723 ms | -44% |
| Whole turn p95 | 5,194 ms | 4,147 ms | -20% |
| First answer audio p50 | 2,749 ms | 1,610 ms | -41% |
| First token p50 | 2,649 ms | 1,357 ms | -49% |
The acknowledgement stage on the critical path fell from 1,138 ms to under 1 ms. Every journey still answered correctly.
The first comparison also exposed a flaw in the instrument. The lab’s automated comparison labelled this run “regressed” overall. A few sub-100 ms intervals had moved by tens of milliseconds and pushed the verdict over its 10% threshold, even though every felt-latency number had improved by more than 40%. The same commit changed the comparison so that the overall verdict follows the primary rows (whole turn and first audio, p50 and p95). Small intervals are now reported as secondary movements. The fix was recorded in the commit message rather than applied silently.
Change two: send the tools the turn needs
The next bottleneck was the prompt itself. Every hosted turn carried all 101 tool schemas, about 15,500 prompt tokens, whether the owner asked for the time or asked to reorganise a folder. The local-model path already had a selector that projects only the tools relevant to a request. Commit 66c1cbf3a applied it to the hosted backend as well.
The selector falls back to the complete catalogue when its evidence is weak, so an uncertain request keeps full recall. The projection costs about 7 ms p50. After the change, a sampled warm turn carried 5,242 prompt tokens and 19 tools instead of 15,512 tokens and 101 tools.
With both changes, measured against the same baseline:
| Measure | Baseline | Both changes | Change |
|---|---|---|---|
| Whole turn p50 | 3,083 ms | 1,270 ms | -59% |
| Whole turn p95 | 5,194 ms | 3,923 ms | -24% |
| First answer audio p50 | 2,749 ms | 1,107 ms | -60% |
| First token p50 | 2,649 ms | 947 ms | -64% |
| Provider request p50 | 1,704 ms | 1,295 ms | -24% |
This time the lab’s overall verdict was “improved”, and all 21 journeys answered correctly.
What got slower
The comparison is honest about costs too. Prompt building went from 0.27 ms to 7.4 ms p50 because the projection now runs there. Policy validation went from 3.9 ms to 11.1 ms, and tool execution from 5.1 ms to 10.2 ms. All three are small next to the seconds removed, but the lab reports each as a regression rather than folding it into the total.
Why the p95 moved less than the p50
The median fell by more than half, and the 95th percentile by about a quarter. The slowest turns were still dominated by the provider. After both changes, inference accounted for 88% of recorded stage time. The lab’s own explanation of one slow warm turn read: “The slowest stage was provider request at 3.40 s (94% of the turn).” Those two changes removed the delays the project controlled. What remains is mostly the hosted model’s response time, which the project classifies as an upstream constraint. It keeps that segment in its reports as out of budget and does not quietly raise the budget.
What this does not show
These are typed turns, so speech capture and recognition are excluded, and the voice path has changed since. The local C013 voice was integrated two days later. Twenty-one turns per run cannot support a p99. The constitution’s cognition first-audio budget is a p50 of 600 ms or less and a p95 of 900 ms or less. At 1,107 ms p50 and 1,929 ms p95 for first audio, the result was a large improvement that still sat above both budgets. The project’s latency work kept going after this point, and none of these figures is a release qualification.