A test says yes or no
You will learn
How a test compares what the code gave with the answer you wrote down first, and why a yes means only what the test asked.
You ask an agent for a function that gives the larger of two numbers. Do not read its code yet: first write down the answer you expect for 3 and 9, which is 9. A test runs the code and the judge compares: yes if it gave 9, no if it gave anything else. The judge sees only the answer, so a yes means the code agrees with you for those inputs and nothing more. A second version of the function with a slip in it, always giving the leading number, gives 3, and the judge says no.
Try it
Write down your answer for 3 and 9, run both versions, and read what the judge says to each. Then try 9 and 3 and see which version still gets a yes.
Write down the answer you expect, run two versions of a function, and watch the judge say yes to one and no to the other. Lesson 16 of the vibecoding course.
specs/vibecoding/test_judge.t27
// SPDX-License-Identifier: Apache-2.0
; specs/vibecoding/test_judge.t27 -- vibecoding lesson: a test says yes or no
; You ask an agent for a function that returns the larger of two numbers. You do not read
; every line it wrote. You write down the answer you expect and let a test judge: the
; judge says yes when what the code gave equals what you wanted, and no when it does not.
; larger is what the agent should write. larger_wrong is a plausible answer with a slip in
; it: it always returns the first number. The constants below are what each one gives for
; the inputs 3 and 9; the tests are the judge. The last test is the negative control: the
; judge must say no to the slip.
; phi^2 + 1/phi^2 = 3 | TRINITY
module vibecoding_test_judge;
pub fn larger(a: u8, b: u8) -> u8 {
if (a > b) {
return a;
}
return b;
}
pub fn larger_wrong(a: u8, b: u8) -> u8 {
return a;
}
; What I expect for larger(3, 9), written down before I run anything.
pub const WANT : u8 = 9;
; What larger(3, 9) gives: 3 is not above 9, so it returns b.
pub const GOT : u8 = 9;
; What larger_wrong(3, 9) gives: its first argument.
pub const GOT_WRONG : u8 = 3;
test the_judge_says_yes_to_working_code {
assert GOT == WANT;
}
test the_judge_does_not_see_inside_only_the_answer {
assert GOT >= 3;
assert GOT >= 9;
}
test negative_control_the_judge_says_no_to_the_slip {
assert GOT_WRONG != WANT;
assert GOT_WRONG < WANT;
}
All lessons
Module 1 · What vibecoding is
You describe, an agent writes, you run it and check. What a program is, and five myths put to a test.
Module 2 · Your desk
An editor with an agent beside it, ten terminal commands, and reading an error message.
Module 3 · Save points with git
A commit is a save point, a branch is a safe try, and a key is a pair of which one stays private.
Module 4 · Prompting
The parts of a prompt, an example in place of an adjective, and one small step at a time.
Module 5 · The working loop
Four phases from idea to ship, the loop of ask, run, read and fix, and what to do when it is stuck.
Module 6 · A test as the judge
A test says yes or no, a planted bug shows the test works, and a diff is read before it is accepted.
Module 7 · The parts of an agent
What an agent answers with, what it can do with tools, and what it keeps between messages.
Module 8 · A first bot
A bot is a message loop, a bot comes from a spec, and a secret stays out of the repository and the prompt.
Module 9 · Your own project
A spec for your own idea, one thing measured after it runs, and an honest plan for the next month.