How a request moves through JARVIS

JARVIS is one owner’s locally hosted personal AI system. It runs on a single desktop PC. This page follows one request from start to finish so that you can see how the separate systems fit together. Each step links to the page that covers it in depth.

Here is the short version. A model helps decide what to do, but it never decides whether to do it. Every effect in the world passes through one governance kernel that can refuse it. Every result is checked by reading the world back afterwards. The answer you hear can only claim what that check found.

Step 1: hearing the owner

A spoken request starts at the microphone. Audio goes into a 30-second pre-roll ring buffer, so the first syllable is not lost while the system decides whether speech has started. Echo cancellation removes JARVIS’s own playback from the input, and a self-speech gate stops JARVIS from hearing itself as the owner. Silero voice-activity detection segments the speech. It refuses to fall back silently to a simpler energy detector; that fallback has to be chosen deliberately.

Transcription runs locally on the GPU with faster-whisper. A measurement recorded in the code’s own documentation (July 2026) gives 165 ms for local transcription against 2,479 ms for the hosted transcription service. That figure is a single recorded comparison, not a current benchmark. A semantic end-of-turn model then decides whether the owner has finished or is only pausing.

Two checks follow before anything is interpreted. Speaker verification admits only the owner. Addressee detection asks whether the sentence was said to JARVIS at all. When either check is uncertain, the system stays silent, because a missed request is cheaper than a wrong action. Details are on the voice page.

A typed request skips all of this and enters at the next step.

Step 2: choosing a lane

A lane classifier puts every utterance into exactly one of three lanes within 30 ms:

The project sets separate first-audio budgets for the lanes: reflex p95 ≤ 300 ms, and cognition p50 ≤ 600 ms and p95 ≤ 900 ms. These are targets. The reflex lane has a recorded reading over its budget, and that reading is still owed a fresh owner-present measurement.

Step 3: working out what was meant

Most spoken requests are vague: “move it to the other screen”, “open the page from earlier”. A director component turns vague phrasing into a deterministic plan of known commands, or into nothing. Words like “it” and “that” resolve through a context stack. The stack has slots for things like the current subject, project, artifact and device, and each slot is RESOLVED, STALE or UNBOUND. When a referent has gone stale, JARVIS asks instead of guessing.

A context compiler then builds the prompt for the model within a token budget. It never silently truncates a required section. The tools offered to the model come from the canonical command table, and the model cannot add or edit a tool. External tool servers must be listed by the owner and pinned by digest, and their results are labelled untrusted.

Step 4: the model proposes

Production cognition today uses a hosted commercial language model, reached through a single ProviderGateway. A local model is being trained to replace it, but no local candidate has qualified. The training page covers that work.

Whatever the model suggests is collected into a ToolProposal. A proposal is data. It is not an instruction and it grants no authority. Plans are capped at a depth of four, and they stop at the first step that cannot be verified.

Step 5: the runtime disposes

The proposal now enters the governance chain. Its order in code differs from the usual simplified description in three ways:

• The E-stop is checked first, and checked again at several later points.
• Approvals are consumed inside the policy decision, not in a later stage.
• “Validation” is several layers, not one.

A command outside the grant allowlist is refused. A separate guard for irreversible action classes can refuse the step outright. PolicyEngine.decide is fail-closed: any unexpected error becomes DENY. Twelve action classes always need owner approval, including sending messages, deleting data and printing. That approval is bound to the exact object and expires. Every decision goes into a hash-chained audit trail. The full chain is on the governance page.

Step 6: acting, then checking

Only now does anything happen. The executor normalises the command, computes the state it expects afterwards, reads the world before acting, performs the effect and discards the effect’s own return value. Then it observes the world again. The result is decided by comparing that observation with the expectation:

• VERIFIED: the world now matches the expectation.
• REJECTED, UNVERIFIED, FAILED or REFUSED: anything else. The plan halts.

The effect’s own return value is thrown away because an application saying “done” is not evidence that the job is done. On the desktop, a window action that does not match is rolled back rather than reported as done. The computer operation page explains how. A ToolReceipt records the outcome as true, false or unknown, and the receipt goes into an append-only evidence store.

Step 7: answering

The spoken answer and the on-screen answer are rendered separately. Speech points at the result, while the HUD carries the detail. A persona governor enforces answer-first form and epistemic marking: an answer says whether something is confirmed, inferred, incomplete or unknown. A fabrication guard checks that any claim about a state or event is backed by a measurement.

Speech comes from C013, the owner-selected local voice, or from an announced fallback voice. The voice is never swapped silently. The owner can interrupt at any point, with a barge-in budget of 120 ms (14.8–27.6 ms measured in tests). Only the words the owner actually heard stay in the conversation context. The HUD page describes how every value on screen carries its source and the time it was observed.

The same kernel for every other route

Missions, automations, gestures and protocols all feed the same governance kernel. None of them has a separate path to the world. Memory has its own single gate: every write goes through one MemoryRouter, which records where the information came from and refuses writes from untrusted origins. See memory.

What is not proven yet

• Live readings are dated. The typed conversation and planning readings come from 16 September 2026, and the governance registry probes from 17 September 2026. The project treats live evidence as expiring after 24 hours, so these are records of what happened then, not claims about today.
• The full spoken loop has not been qualified. Full-duplex voice is waiting on an owner-present sitting. The interrupted-speech journey is fixture-tested only. Audible first-audio latency with the C013 voice has not been measured.
• The local model is not the brain. Production cognition is hosted. No local candidate has passed its gates.
• Some context features lag in reachability. Context-stack registry rows were demoted during an internal reachability audit. Later rows record them between TESTED and PRODUCTION-REACHABLE.
• No release. The project’s own certification matrix reads “JARVIS-READY-V1: NO”, and the final gate has not passed. See capabilities for rung-by-rung status and bench for measurements.