What broke, and what it taught

Failure and recovery lab

Real problems from the JARVIS build: the symptom, the diagnosis, the repair, the test that now guards against it, and how the repair was verified. Problems without a repair yet are listed as open.

Repaired and verified: 9
Repaired, verifying: 2
Open: 3

Systems covered: Browser safety, Cognition / language model, Cognition / planning, Computer operator, Engineering, Engineering evidence, Evidence and verification, Fabrication / scene, Governance / HUD scene, HUD / Engineering, Memory, Memory / skills, Self system.

Repaired and verified Computer operator 23 Sept 2026

The Save As dialog that pretended to listen
1. Symptom
  Save As showed the typed file name but kept its own. Pressing Enter raised “Confirm Save As” for the source file, and something other than the test answered it: the source was rewritten in place and no copy was made.
2. Diagnosis
  Setting the name by a window message changes what the dialog displays, not what it uses. An observe-only reproduction, sampling every ~265 ms and sending nothing, still saw the confirmation answered.
3. Repair
  The name is typed only while the focused control is verified to be the file-name field, then read back. JARVIS never answers a confirmation: any extra dialog makes the result refused and uncertain.
4. Guard
  Disabling confirmation detection turns the test red.
5. Verified
  Live Save and Save As passed afterwards, checked by re-reading file bytes and identity (on the operator lane).
Lesson. What a dialog shows is not what it will do. Verify the effect, and never click through a question you did not expect.

Capability: Verified Save and Save As
Repaired and verified Self system 23 Sept 2026

A helpful duplicate sensor that overrode a real reading
1. Symptom
  A canonical merge test failed after new RAM, CPU and disk producers were added.
2. Diagnosis
  They were added on the claim that none existed. That claim was never checked: the second producer marked disk UNAVAILABLE over a real reading.
3. Repair
  The duplicate producer was reverted.
4. Guard
  The merge test that caught it.
5. Verified
  37 tests passed after the revert.
Lesson. Check the claim that something is missing before building it. Qualify the existing producer instead of adding a second.
Repaired, verification pending Cognition / language model 23 Sept 2026

The persona example that taught a fabricated check
1. Symptom
  Local-model candidates kept claiming to have checked things they had not, even after targeted training.
2. Diagnosis
  The system prompt’s own example of a “no record” answer described an inspection that never happened, and it appeared in every training row and every test case. The model was copying its instructions.
3. Repair
  Persona policy 2.1.0 allows “no record” only after a search shown in the same turn.
4. Guard
  521 tests passed; the negative control fails as intended.
5. Verified
  Not verified yet.
Lesson. Before blaming the model, read what you taught it. The policy is landed but not yet active; it ships with contract v2.

Read the journal entry Capability: Grounded “no record” answers (persona policy 2.1.0)
Repaired and verified Computer operator 23 Sept 2026

“Visible” windows that were minimised
1. Symptom
  Keep-visible reported success for windows that were minimised, invisible or off screen.
2. Diagnosis
  It only checked whether the window was covered, not whether it was rendered at all.
3. Repair
  A window only counts as visible if it is rendered: visible, not minimised, non-empty and on a monitor.
4. Guard
  8 new tests that fail on the old code.
5. Verified
  Live re-proof on the operator lane, with 0 foreign window changes.
Lesson. Never report verified without observing the thing you claim.

Capability: Keep a window visible, move others aside
Open HUD / Engineering 22 Sept 2026

The engineering panel showed the previous run’s ID
1. Symptom
  After a new thermal bench solve, the values on the engineering panel were new, but its provenance line still named the previous run. The new run ID appeared only after a restart.
2. Diagnosis
  Seen on camera during the 22 September 2026 capture session. The published values were correct and labelled calculated; only the provenance line was stale, and a full restart cleared it. The cause has not been established yet.
3. Repair
  Not repaired yet.
4. Guard
  No regression test yet.
5. Verified
  Not verified yet.
Lesson. Provenance has to be at least as fresh as the value it describes, or it misleads more than it helps.

Read the journal entry See it on camera Capability: Local solver execution
Open Cognition / planning 22 Sept 2026

A two-step request ran its first step and silently dropped the second
1. Symptom
  Asked to “run the tensile bar case, then open the 3D model”, JARVIS ran the solve, reported “Done”, and never opened the model. The plan it recorded had only one step.
2. Diagnosis
  The recorded plan contained a single step (engineering.solve), and that step verified, so the turn reported success. Nothing in the turn compared the plan with the full request. Why the planner dropped the second step has not been established yet.
3. Repair
  Not repaired yet.
4. Guard
  No regression test yet.
5. Verified
  Not verified yet.
Lesson. Verification only checks the steps that exist. A request that asks for two things needs a check that the plan asks for two things.

Read the journal entry
Open Memory 22 Sept 2026

A stated fact was stored as a standing instruction
1. Symptom
  Told “remember that the website launch footage was recorded tonight on monitor two”, JARVIS answered “Noted, sir. Standing instruction on …”. Recall and forgetting worked, but the memory was filed under the wrong kind.
2. Diagnosis
  The remember path classified a statement of fact as a standing instruction. Recall and a governed forget both behaved correctly afterwards; the classification itself is the fault. Its cause has not been established yet.
3. Repair
  Not repaired yet.
4. Guard
  No regression test yet.
5. Verified
  Not verified yet.
Lesson. Keeping kinds of memory apart only helps if what enters is filed under the right kind.

Read the journal entry See it on camera Capability: Memory consolidation, supersession and forgetting
Repaired and verified Governance / HUD scene 22 Sept 2026

An emergency-stop fence that was created but never entered
1. Symptom
  After an E-stop trip, the 3D scene route could still deliver model bytes: a reproduction got 200 bytes back after the stop.
2. Diagnosis
  The route created its stop fence but built the response outside it, so the fence’s final check never ran.
3. Repair
  The response is now built inside the fence.
4. Guard
  32 regression tests, including a trip in the middle of loading.
5. Verified
  A live proof after the fix returned 403 and withheld the bytes.
Lesson. A safety check that exists is not the same as a safety check that runs. The live proof, not the code review, found it.
Repaired, verification pending Fabrication / scene 22 Sept 2026

0.9999999999999999: parts that could never be sliced
1. Symptom
  Some objects asked for a geometry decision forever and could never reach the slicer.
2. Diagnosis
  Scale is a running product. Applying a factor and then its inverse left 0.9999999999999999, and the scene compared scale exactly with 1: this happened for 36 of 299 factors between 0.01 and 2.99.
3. Repair
  One tolerant “is this the identity?” check is now shared by the scene, readiness and targeting.
4. Guard
  A regression test with a mutation control.
5. Verified
  Not verified yet.
Lesson. Exact equality on floating-point results is a bug waiting for the right number.
Repaired and verified Engineering 22 Sept 2026

CAD-kernel noise read as a design change
1. Symptom
  The first what-if live run treated a clone of the design as changed.
2. Diagnosis
  The bounding box differed by 2 × 10⁻⁷ mm, and exact equality counted that as a change.
3. Repair
  The tolerance is now explicit (10⁻⁶ mm).
4. Guard
  Covered by a regression test.
5. Verified
  The what-if proof then passed 213 of 213 checks across two boots.
Lesson. Every comparison of measured geometry needs a stated tolerance.

Capability: Conversational what-if engine
Repaired and verified Engineering evidence 22 Sept 2026

An observation in the closing second read as “after”
1. Symptom
  The evidence-class live proof rejected a valid observation.
2. Diagnosis
  An observation made within the same whole second as the run’s close was read as coming after it.
3. Repair
  The boundary was fixed.
4. Guard
  A regression test for the same-second case.
5. Verified
  The evidence-class proof passed across two boots.
Lesson. Time boundaries need an exact rule, written down, and a test at the edge.

Capability: Calculated vs measured evidence classes
Repaired and verified Browser safety 22 Sept 2026

The loopback filter let fixture.localhost through
1. Symptom
  While designing a navigation proof, the team found that names like fixture.localhost and .local hosts passed the browser’s loopback filter.
2. Diagnosis
  The filter only matched hosts that started with “localhost”, so the claim that the browser refuses loopback was overstated.
3. Repair
  Any *.localhost and *.local name is now refused.
4. Guard
  Negative-control cases in the navigation proof.
5. Verified
  The navigation proof’s three negative controls then sent 0 requests (on the operator lane).
Lesson. A security claim is only as good as the cases it was tested against.

Capability: Governed browser navigation with negative controls
Repaired and verified Memory / skills 22 Sept 2026

10 million row visits to rank a few skills
1. Symptom
  Ranking skills by past attempts was slow.
2. Diagnosis
  The action history was scanned once per skill.
3. Repair
  One shared snapshot is read once.
4. Guard
  Output must stay record-for-record identical.
5. Verified
  Synthetic benchmark: 189.47 ms → 0.042 ms, 10,000,000 → 10,000 row visits, identical records.
Lesson. Measure before optimising, and prove the output did not change.
Repaired and verified Evidence and verification 6 Aug 2026

256 “live” capabilities that did not prove what the word said
1. Symptom
  On 28 July 2026 the build ledger recorded 256 of 341 capabilities as LIVE. Many of those readings could not show that a real owner action reached the code, or that the booted system itself had answered.
2. Diagnosis
  Two gaps. An audit at the end of July (AUDIT 03) found that for dozens of capabilities no production entry point constructed the code at all. And a probe could pass by exercising code near the registered path, in the same process as the test, rather than by asking the booted system.
3. Repair
  Reachability from a real entry point became a precondition for LIVE, and on 6 August the evidence rule was tightened: LIVE is reached “by asking the process, not by proximity”, with no test doubles, a real booted process on a real port, evidence written to disk, and expiry after 24 hours. 108 registry entries were demoted from LIVE in one step.
4. Guard
  Every adapter carries an is_test_double flag, and a probe whose call graph touches a double can produce IMPLEMENTED at most. LIVE decays after 24 hours unless probes run again.
5. Verified
  The census of 17 September 2026 recorded 9 of 342 capabilities as LIVE under the strict rule. The website shows that figure with its date.
Lesson. A smaller number you can defend beats a large one you cannot. The drop was a correction, not a loss of features.

Read the journal entry