Training the local language model

JARVIS’s everyday reasoning runs on a hosted commercial language model, reached through the project’s provider gateway. The project calls this Path-1: “hosted cognition is production, the local model is experimental”. In parallel, the owner is trying to build a local model that fits on one 12 GB consumer GPU and behaves the way JARVIS must. It has to stay grounded in real system state, be honest about what it did and did not do, and never give itself authority.

Between 9 and 22 September 2026, that effort went through about a dozen trained candidates and bounded probes on three base-model families. None has qualified. This page explains how the work is done and reports every outcome. The interesting part is not a leaderboard number. It is the discipline: frozen gates, checkpoints chosen before results were seen, independent review, and a refusal to promote anything that fails.

What the model is allowed to be

Whatever model runs, it only proposes. The policy engine, approval registry, E-stop and memory router stay outside the model as the single authorities. See governance. Memory and changing personal facts stay outside the weights and are retrieved at runtime. The HUD is driven by a runtime state contract, never by HTML a model generated. A local server must bind to loopback, verify the pinned base, revision and adapter digest on every streamed chunk, and never fall back silently.

The deliverable was always “a pinned open base model plus a small adapter”, not a model pretrained from scratch. The adapter is meant to shape behaviour, while facts that change stay outside the weights.

Choosing a base model

Each base was pinned to an exact revision and fetched anonymously, with pinned artifact hashes and no remote code. The licence values above are what each model card’s metadata says. They are not a legal review. The project’s licence register states the rule behind this caution: “A LICENSE file is not the licence.”

Fitting on one 12 GB GPU

Everything has to fit on one RTX 5070 (12,227 MiB of VRAM) with 32 GB of system RAM, alongside speech recognition, the voice and the HUD.

• The recipe. For the 14B base, the model is quantised to 4-bit NF4 with double quantization and BF16 compute. LoRA adapters of rank 16 (alpha 32) go on the attention query and value projections of all 40 language layers. That is 80 modules and 9,830,400 trainable parameters, with the vision weights frozen.
• Hardware proven first. A pilot ran two real optimizer updates, saved a checkpoint, and resumed in a fresh process from step 2 to step 3. The project recorded that this “proved hardware feasibility”, not quality.
• Measured, not assumed. Every GPU job runs under a watcher. It samples dedicated and shared GPU memory, enforces an allocator cap (10,468,982,784 bytes for training), and treats any fallback to host memory as a failure. The first full campaign’s final update peaked at 9,502,195,712 reserved bytes, at about 282 processed tokens per second.
• Failures are kept. Qwen3-14B’s first long prefill failed on a 2.5 GiB attention allocation, and a memory-efficient attention path fixed it (verified against reference outputs to about 1e-6). Qwen3.5-9B’s first training smoke test ran out of memory, and nested activation checkpointing fixed it with identical loss and gradients. Gemma 4’s first full prefill also ran out of memory, and 256-token chunked prefill passed.
• Coexistence (Gate A). On 22 September 2026, one run loaded the Gemma inference recipe with the speech recogniser, the voice worker, the HUD, the body services and a browser, then shut everything down cleanly. The whole-GPU peak was 11,025,776,640 bytes, under the 11.5 GiB ceiling. That run is explicitly not latency-certified, not uncontended and not proof of production routing.

The model-visible contract

Before the final campaigns, the project froze exactly what a model is allowed to see: tool-schema projections, command names, persona instructions, and the pinned tokenizer and chat template, all bound by hash. This CP-7 contract covers 134 commands and 373 bound source files (frozen 19 September 2026). A training corpus without a matching CP-7 receipt refuses to open.

Context size is a real constraint, and the ledgers measure it:

• Offering the full 101-tool payload of the time cost 20,975 prompt tokens, over the local 16,384-token limit. A schema-preserving compaction brought a representative prompt down to 14,561 tokens.
• With Gemma’s proven 4,096-token window, a synthetic “list files” request needs 5,992 input tokens against a 3,072-token allowance. The fallback of offering every tool needs 35,937. This defect is recorded as unresolved.
• The 22 September plan answers this with bounded capability discovery: at most two discovery calls, then one action completion, with complete schemas only for the selected tools. It is in progress, not integrated or qualified.

Campaign outcomes

The early foundation runs were small, and they mostly taught what not to trust. Validation loss fell while behaviour did not improve, and the project wrote that down: “Reduced validation loss is not claimed as demonstrated general behavioral improvement”.

The frozen-contract campaigns on Ministral-3-14B each trained from a fresh base, reloaded in a fresh process, and were scored on unchanged gates. Validation and adversarial sets are release-blocking. DEV scores are diagnostic only and are not pass rates toward release.

Some automatic failures are exact-wording checks rather than substantive errors. The project refused to weaken them.

Diagnosing the failures

After Recovery004, the team stopped launching full runs and used bounded probes on training-derived cases to find the mechanism:

• Exact fit versus transfer. Replaying 12 exact training prompts showed Recovery004 had learned 6, partly learned 2 and failed 4: “partial train fit plus poor transfer”. A separate 40-case battery tests whether learning carries over to new variations.
• Interference. Scoring checkpoints along the run gave 29, then 36, then 32, then 32 transfer passes out of 40, at updates 24, 48, 72 and 93. Behaviour learned mid-run was lost by the end. The project read this as interference or forgetting, not as a sign that more epochs were needed.
• Preservation probes. Three matched probes from the same parent tried preservation interleaving, extra output-projection capacity and a token-reweighted objective. They reached 31, 32 and 31 transfer passes, and none earned inclusion.
• A data defect. A generalization map found that 704 targeted replacement rows had displaced broad-behaviour rows, leaving 82 of 786. Architecture001 restored those anchors and added 512 new original rows in counterfactual pairs, sampled so each positive and negative pair lands in the same optimizer step.

Architecture001 still failed. Among other errors, it proposed owner approval while authority was withheld, added an unrequested HUD mode change, and accepted an expired receipt.

Checkpoints, custody and sealing

• Checkpoints chosen in advance. Architecture001 saved every 4 updates but ranked only updates 32, 64, 96 and 125, on a frozen diagnostic with a declared rule. Update 96 was frozen before any canonical evaluation, and “no checkpoint was switched after inspecting canonical results”.
• Fresh-process reload. Every candidate is reloaded in a new process. Later probes also require the reloaded likelihoods to reproduce exactly.
• Custody. Checkpoints are identified by tree hashes, and Architecture001 is preserved under an immutable custody tag. An early defect, in which step counters could disagree yet report “complete”, was found and fixed.
• A sealed test set. A SEALED final set has never been opened. It opens only after a single frozen checkpoint passes development selection.
• Independent review. Judgments about outputs come from separate read-only reviewers. A judgment by the integrator alone is labelled “not independent”.

The negative decision (21 September 2026)

Four more bounded probes followed: more exposure, added MLP down-projection capacity, a precision comparison, and untuned Qwen3-14B. None met the frozen viability requirements. The decision: “Do not start another full training run or select an architecture from these probes.” It is framed as “a negative feasibility decision, not an assertion that every possible local model or adapter is incapable”. The same record says plainly: “JARVIS remains incomplete”.

The stronger-local study (now)

The same day, the owner authorised a bounded study of stronger local models. It allows at most five new full campaigns, and zero have been used. A probe must meet all of these viability gates:

• Exact: at least 10 passes with no failures.
• Transfer: no critical failures and at most 4 failures in total.
• Anchors: at least 7 passes with no failures.
• Memory, latency and coexistence proof.

Qwen3.5-9B failed the transfer gate. After a controlled continuation to 48 updates, it reached 32 of 40 transfer passes, still with too many failures. Its greedy “thinking” mode often produced no final answer within the deployment budget (8 of 12 cases). The adapted model measured a first-final-token p95 of 939 ms and about 18 tokens per second of sustained decode, without uncontended certification.

Gemma 4 12B is the current subject. Chunked prefill of 4,096 tokens took about 3.45 s. Gate C passed independent review on 22 September 2026: one real optimizer update with a 4,096-token backward pass, exact loss parity, and same-base checkpoint reload. Four bounded specialization contrasts evaluated that day all failed the gates; the recorded contrasts had 9 or 10 transfer failures out of 40 against a gate of at most 4 (see JARVIS Bench). A reasoning-mode profile was stopped at its futility bound. One more matched, exposure-only continuation (16 to 48 updates) finished at 16:25 EDT on 22 September 2026. Its mean exact-target loss fell from 2.35 to 0.86, and its behavioural evaluation is pending. The decision rule was registered beforehand: if fit improves but transfer still fails, “do not automatically extend again”.

What is not proven yet

• No local model is development-qualified, integrated, live-proven or release-qualified. Production cognition is hosted.
• The SEALED test set has never been opened.
• The context-delivery defect is unresolved, and bounded tool discovery is not integrated.
• Coexistence is a single observation, without latency certification or production routing.
• The latest Gemma continuation has no behavioural result yet. A lower training loss is not a behavioural improvement.
• Every step after training on the production ladder is unreached: SEALED evaluation, integration, release-candidate certification, owner-present checks, promotion, and running without the build assistant.