JARVISLabs

Tier 1 mutation kill rate

Tier 1 mutation kill rate

Then 51.8% 30 Aug 2026 Now 73% 18 Sept 2026 improved

  • measured or observed

project target 90%

Show the numbers
Date Value Evidence Revision Samples Note
30 Aug 2026 51.8% MEASURED caec91cb 527 Share of deliberately injected bugs that the test suite catches.
11 Sept 2026 72% MEASURED fb174c77 583 Share of deliberately injected bugs that the test suite catches.
17 Sept 2026 73.5% MEASURED f02fd5e7 611 73.5% of 611 injected defects were caught across all 44 tier-1 modules.
18 Sept 2026 73% MEASURED 22ee5da8 611 Share of deliberately injected bugs that the test suite catches.

Test: tools/mutation/mutate.py over 38 tier-1 modules (up to 14 mutants each): killed 273 of 527.

Protocol

Test definition

tools/mutation/mutate.py over 38 tier-1 modules (up to 14 mutants each): killed 273 of 527.

Limits

Limitations

  • Module set grew over time (38 -> 44), so ratios are not strictly like-for-like; 90% floor not met; the 2026-09-18 reading is 3 kills below the 449/611 reading of 2026-09-17.
  • Different mutant population from the 2026-09-09 figure; not directly comparable.

Log

Recorded values

Date Value Evidence Samples Source
30 August 2026 51.8% MEASURED 527 docs/analysis/mutation-tier1.json (git history)
11 September 2026 72% MEASURED 583 docs/analysis/mutation-tier1.json (git history)
17 September 2026 73.5% MEASURED 611 docs/orders/post-lm/HANDOFF.md (2026-09-17); docs/analysis/mutation-tier1.json
18 September 2026 73% MEASURED 611 docs/analysis/mutation-tier1.json (git history)

Source paths refer to the private JARVIS repository and runtime records. They are listed for audit and are not published.