About Satyrn Evals

Why

Small models need help, and help is only help if you can measure it. Before Satyrn, we built remedies for failures we had not diagnosed — and the failures turned out to be our own harness's. Evals exists so a claim about a model or an engine is a measurement, not a story. Why that needed its own harness, rather than an existing agent benchmark, is in why Satyrn has its own evals.

How

Evals captures a task, runs an attempt command in an attempt worktree under the eval's confinement extension, preserves the patch and transcript, and grades the retained evidence offline. The engine seam is an executable command, so the suite runs against a fake command and never imports engine internals.

What

Tasks, confined cells, a launcher, grading from retained evidence, and the census that classifies where an arm actually fails. The vocabulary is the glossary; the physical run is the architecture.