Satyrn Evals and Satyrn Engine

You have a great idea to speed up AI. Your agent encourages your brilliance and starts building. And building. Four quadrillion tokens later, nobody understands anything, you're painted into a corner, and you find it isn't actually solving a problem.

Because you didn't start with evidence.

Satyrn Evals is a deep, deliberately over-deep way to find pathologies and experiment with remedies. Small models need help. Evals gives you the confidence that you're on the right track, and that you can deliver something into a solution -- Satyrn Engine.

The evidence is in the numbers: a pre-registered comparison of /implement against bare Pi on medium-build tasks, read once when complete, with where it did not help and what it costs. That reading ran under a harness that has since been retired, and it is marked unconfirmed until it is re-derived; the numbers page says what has been re-derived since.

New here? Your first run is the guided journey: what an engine is, what evals are for, and then a real task run end to end on ollama, both arms, and how to send what you saw back to the people building it.

The shape. Evals is the star; Engine is the petri dish where remedies get tested. Later, Engine becomes a routine-Python tool for Laptop AI. For now, we are a group of people learning how to measure and investigate for Laptop AI. Read how it works for the whole story in diagrams.

The honest scope. Satyrn Engine is not ready to be an everyday addition to your agent. In 0.1 it is a good implementer for medium-sized tasks in routine Laptop AI Python work.

Join. Follow us on Mastodon — TBD.

FYI. Two repositories today (satyrn-engine, satyrn-evals), both under pauleveritt; this site lives at pauleveritt.github.io/satyrn-evals/. After 0.1 they fold into one repository under satyrn-ai.

Part of the SatyrnAI project

SatyrnAI builds tools for Laptop AI — link TBD.