# Eval gate

Every diff runs your tests and our evals, in order, before a pull request opens. A failing diff goes back to the agent with the failure attached.

The eval gate is the reason Kestrel's pull requests get merged. An agent can write a diff that looks right and still breaks the build, touches files it shouldn't, or fixes the test by deleting it. The gate catches those before anyone is asked to review.

![The diff runs through six checks in order. It fails the evals and goes back to the agent with the failure; the revised diff passes every check, and the pull request opens.](/illos/eval-gate.svg)

## The checks

Checks run in order and stop at the first failure, so a diff that doesn't build never waits for the test suite.

| # | Check | Fails when |
|---|---|---|
| 1 | Build | The repo doesn't build |
| 2 | Lint | Your linter reports an error |
| 3 | Types | The type checker reports an error |
| 4 | Tests | Any test fails, or a test was deleted or skipped |
| 5 | Evals | The diff goes beyond the task, or breaks a past fix |
| 6 | Review | A second model finds a bug the tests didn't cover |

## Set your commands

Kestrel finds your build, lint and test commands on the first run and keeps them in [memory](/agents/memory). Set them yourself to be sure:

```yaml kestrel.yaml
evals:
  build: npm run build
  lint: npm run lint
  types: npx tsc --noEmit
  tests: npm test
  suites: [kestrel/scope, kestrel/regressions]
```

## When a diff fails

The agent gets the failure: the check, the command, and its output. It revises the diff and the gate runs again from the top. After three tries the run stops with the status `needs_review`, and the trace shows the failure it couldn't fix.

```ts TypeScript
const result = await run.wait();
if (result.status === "needs_review") {
  console.log(result.gate.failed.check, result.gate.failed.output);
}
```

```python Python
result = run.wait()
if result.status == "needs_review":
    print(result.gate.failed.check, result.gate.failed.output)
```

> [!WARNING]
> Turning a check off with `evals: { lint: false }` lets diffs through that fail it. Kestrel notes every check that was off in the pull request, so reviewers know.