# Fix flaky tests

Point Kestrel at a test that fails one run in ten. It finds why, fixes it, and proves the fix by running the test until a failure would have shown.

A flaky test is the perfect task for an agent: easy to state, slow to chase, and simple to check. This guide takes one from a failing CI job to a merged fix.


### Find the test

Pick the test and how often it fails. Your CI history has both. Here, `checkout.test.ts` fails about one run in ten.

### Ask for proof, not a guess

A fix for a flaky test is only proven by running the test many times. Tell Kestrel how many with `--repeat-tests`. Twenty passes in a row makes a 1-in-10 flake very unlikely to have survived.

```bash Terminal
kestrel run "fix the flaky checkout test" --repeat-tests 20
```

```ts TypeScript
await kestrel.runs.create({
  repo: "acme/web-app",
  task: "Fix the flaky checkout test",
  evals: { repeatTests: 20 },
});
```

### Let the agent reproduce it first

The agent runs the test in a loop until it fails, then reads the failure. In the trace you'll see a `reproduce` step before any edit. A fix that comes without a reproduction is a guess, and the evals reject it.

### Read the fix

When all twenty runs pass, Kestrel opens the pull request. The description names the cause, for example a timer that wasn't awaited, and the trace shows every repeat.

```bash Terminal
✓ sandbox ready              0.8 s
✓ reproduced on try 7
✓ 20 of 20 repeats passed
✓ pull request #418 opened
```

### Teach it the pattern

If the cause is common in your codebase, add it to memory so the next run checks for it first.

```bash Terminal
kestrel memory add --repo acme/web-app --topic tests "Await clock.runAllAsync() after advancing fake timers"
```


> [!TIP]
> Start a run for each flaky test at once. They queue and run side by side, and each opens its own pull request.