t27.aiРусский

Surviving mutants

You will learn

What a surviving mutant tells you about the test that let it live.

A surviving mutant is the most specific bug report you will ever get: here is a fault, here is the test that should have caught it, here is why it did not. The widget counted them where nobody was looking: 367 PRs landed on t27 in 14 days, and 11 of them deleted tests or asserts; one PR, titled to sound like an implementation, deleted all 26 tests in its file and nothing failed. Every one of those 11 was a mutant that survived -- and it survived because a human let it, which is exactly the failure mode mutation testing exists to catch.

Try it

Open the test-touch widget and find the PR that deleted all 26 tests; then say what check would have caught it at review time.

Open the interactive lesson →

Did the agent touch its own tests? Every PR merged on t27, checked
Did the agent touch its own tests? Every PR merged on t27, checked ↗

Every pull request merged on gHashTag/t27 in 14 days, bee lane and human lane: which added tests, changed them or removed asserts, with the exact test lines from git.

specs/tools/trios/tri/x7-mutate.t27

// SPDX-License-Identifier: Apache-2.0
; specs/tools/trios/tri/x7-mutate.t27 -- tool gHashTag/BrowserOS:tri/x7-mutate, the `tri x7-mutate` command of the trios loop CLI
; Generated by apps/website/scripts/tools-from-trios-tri.mjs from gHashTag/BrowserOS:trios/bin/tri at 7366096248df; do not edit.
; The CLI the loop timers run (`tri drift` holds ~/.local/bin/tri to the tracked copy). It is another program than
; the Rust tri of gHashTag/t27 and the Zig tri of gHashTag/trinity, so the ID is repository-qualified.
; A card is data and carries no test block. ASCII only (L3). phi^2 + 1/phi^2 = 3 | TRINITY

module tool_trios_tri_x7_mutate;

pub const KIND : str = "tool";
pub const FAMILY : str = "tri-cli";
pub const ID : str = "gHashTag/BrowserOS:tri/x7-mutate";
pub const REPO : str = "gHashTag/BrowserOS";
pub const QUALIFIED_ID : str = "gHashTag/BrowserOS:tri/x7-mutate";
pub const SCHEMA : u32 = 2;
pub const COMMAND : str = "tri x7-mutate";
; The case arm of the dispatcher and the first line of it that does the work.
pub const VARIANT : str = "case arm `x7-mutate)`, line 1900";
pub const SOURCE : str = "trios/bin/tri";
pub const ENTRY : str = "trios/bin/tri";
pub const SOURCE_COMMIT : str = "7366096248dfb09893e9a7d1a0f1c56ced13d16e";
pub const ROUTED : bool = true;
pub const DISPATCH : str = "exec python3 \"$HOME/skills/xilinx7-bitstream-loop/scripts/x7.py\" mutate \"$@\"";
pub const DOCUMENTED : bool = true;
pub const HELP_LINE : str = "tri x7-mutate [SPEC] -- mutation catalogue: every defect must fail the spec tests AND the Vivado sweep";
pub const CATEGORY : str = "xilinx7 MVP loop (bitstream back half as .t27, skill xilinx7-bitstream-loop)";
pub const ABOUT : str = "mutation catalogue: every defect must fail the spec tests AND the Vivado sweep";
pub const ABOUT_SOURCE : str = "`tri help` (the heredoc under the help arm of trios/bin/tri)";
pub const ACTIONS : [0]str = [];
pub const ACTIONS_ABOUT : [0]str = [];
pub const ARGS : [1]str = ["[SPEC]"];
pub const AGENTS : [0]str = [];
pub const AGENTS_NOTE : str = "No source binds an agent letter to this command: the trios CLI is not named by docs/agents/AGENTS_ALPHABET.md or .claude/agents/*.md of gHashTag/t27.";
pub const WHEN_TO_USE : str = "mutation catalogue: every defect must fail the spec tests AND the Vivado sweep";
; public/term/tri-x7-mutate/meta.json lists 1 run(s) of `tri x7-mutate`; recorded with the live ~/.local/bin/tri,
; not a build of SOURCE_COMMIT. The site plays it at the end of this card.
pub const CAST : str = "term/tri-x7-mutate/session.cast";
pub const WITNESS : str = "source-parse";
pub const WITNESS_SOURCE : str = "gHashTag/BrowserOS:trios/bin/tri at 7366096248dfb09893e9a7d1a0f1c56ced13d16e, read as text (the help heredoc and the top-level case arms); the CLI was not run";
pub const ENABLED : bool = true;

Open the lesson's spec in the player ↗

All lessons