A voice assistant is judged by how it sounds, and a sound is the easiest thing to show on a website. This site shows none. This entry explains how JARVIS’s voice pipeline works, how its speaking voice was chosen, what its qualification still lacks, and why the project has decided not to publish audio.
Fourteen stages
The repository generates a map of the voice path from the source code itself (docs/training/BRAIN_AND_VOICE_RUNTIME_MAP.md). It covers 14 stages and 24 code anchors, and reports “0 absent”, meaning every anchor resolved to real code. In order:
- Capture. Microphone audio is held in a ring buffer, so the first syllable survives while the system decides whether speech has started.
- Speaker identity. Input is classified OWNER, NOT_OWNER, UNCERTAIN or UNAVAILABLE. Only OWNER is admitted, so the stage fails closed.
- Endpointing. The system decides whether the owner has finished or is pausing mid-sentence. On the owner’s own end-of-utterance corpus the text model reaches 94.6% balanced accuracy. It held 4 of 37 finished utterances too long and cut off 0 of 11 mid-thoughts. That residual is why an MIT-licensed audio turn-taking model was approved in the licence register.
- Transcription. faster-whisper runs locally on the GPU. A measurement recorded in the code puts it at 165 ms against 2,479 ms for the hosted transcription API, and local latency stays flat as utterances get longer. A public-synthetic screen on 21 September showed that
int8_float16precision keeps accuracy on the frozen cases while using about 1.9 GiB less GPU memory. - Inbound prosody. Urgency, terseness and energy are measured against the owner’s own baseline.
- Lane selection. Reflex, cognition or mission. This decides whether anything is said to cover the wait.
- Identity into the prompt, then 8. prompt assembly and 9. generation. Cognition runs on a hosted model today. The first streamed token starts the clause chunker.
- Fabrication guard. If an answer asserts a state or an event, the guard checks that something was actually measured that says so.
- Persona enforcement. The answer is accepted, rewritten or rejected. A rejection triggers regeneration that names the rule the answer broke.
- Spoken and display split. Speech points at the payload, and the HUD carries it.
- Synthesis. The pinned voice speaks, or an announced fallback does. The pipeline never substitutes a voice silently.
- Playback and barge-in. The owner can take the floor at any time.
Two stages repay a closer look. Echo handling stops JARVIS hearing itself: an echo canceller and a self-speech gate sit in front of voice detection. In a probe with synthesised over-talking speech, 96.5% of the owner-side speech passed during double-talk. Interruption keeps only what the owner actually heard. The turn manager has a 120 ms stop budget, measured at 14.8 to 27.6 ms in tests. Clauses that were never played are cancelled and dropped from context, so after a correction the conversation continues from the correction.
How the speaking voice works
JARVIS speaks through a locally run, open-source text-to-speech model with no fine-tuning, conditioned on a short reference clip. The configuration is identified as C013, a candidate from the project’s voice lab. It was integrated on 13 September (e20bae2ec) and runs on the owner’s GPU from an immutable package. At load, the package is checked against its manifest digest and fails closed on any mismatch. It “never falls back to a different checkpoint”. No pitch, formant, rate, equaliser or effect processing is applied, and nothing in the loop writes audio to disk.
If the voice cannot speak, a local fallback voice does, and the change is announced. The design phrase is “honest about who spoke”. A hosted voice is retained as a rollback path. On 18 September the full conversation loop assembled with this voice in the normal launcher’s check mode.
How it was chosen, and what that does not mean
The owner chose the voice through scored and listened comparisons. The project’s rule is that owner preference alone never grants qualification. Listening approval is one gate among several, and the others are technical.
The voice’s technical qualification is pending. Its long-form non-regression check failed. First audio, streaming, acronym pronunciation, audible cancellation, restart recovery and fallback have not been measured in qualification. The voice is configured on the integration branch and has passed its assembly check. It has not been proven live for audible latency, and it is not release-qualified.
A dedicated instrument now exists for the audible first-audio measurement (b1367642b, 20 September). It rejects any fallback voice and binds the exact package. The project describes it as an implemented and tested instrument, not an audible qualification. The measurement itself is reserved for an owner-present sitting.
Why there are no samples here
This site does not publish voice audio. That covers the speaking voice, the candidates and any video narrated in it.
The reason is the qualification state described above, not modesty. The voice has not passed the project’s own gates, and the qualification plan (docs/training/VOICE_QUALIFICATION_AND_AUDIO_REGRESSION_PLAN.md) sets conditions a voice must meet before it is presented as finished. Publishing a sample would present an unqualified voice as a finished one. A recording would also outlive the configuration it came from, and nothing on a static page would show that the audio was recorded before qualification and may not match the voice JARVIS uses later.
The site also does not publish similarity scores from the voice campaign. They are internal comparison figures that only mean something inside the setup that produced them, and published on their own they would suggest a precision they do not have.
What the site does publish is the pipeline, the failed and unmeasured gates, and the date the voice was integrated. When the voice passes qualification, that will be recorded here too. Until then, the voice exists in the owner’s room and nowhere else.