KestrelDocs

Runs

Network errors and busy models are retried with growing waits between tries. A run is never started twice, because each request carries an idempotency key.

Fig. 1Retry with backoff
Retry with backoff float trip(float t, float t0, float hold) { // out to the wall (0.4 s) and back (0.4 s), from t0; 0 outside float u = t - t0; if (u < 0.0) return -1.0; if (u < 0.4) return ease(u / 0.4); if (u < 0.8 && hold < 0.5) return 1.0 - ease((u - 0.4) / 0.4); if (hold > 0.5 && u < 1.2) return 1.0 + ease((u - 0.4) / 0.8) * 0.85; return -1.0; } float field(vec2 p, float t) { if (p.x < 18.0 || p.x > 702.0) return 0.0; vec2 A = vec2(110.0, 230.0), W = vec2(400.0, 230.0); float open = sm(2.5, 2.8, t) * (1.0 - sm(3.6, 4.2, t)); float v = blob(p, A, 26.0, 0.8); // the service's wall: a dense band that thins when it opens v += exp(-pow(p.x - W.x, 2.0) / 60.0) * step(abs(p.y - 230.0), 90.0) * (0.85 - 0.6 * open); v += blob(p, vec2(610.0, 230.0), 34.0, 0.6); // three tries, the gaps doubling: 0.2, 0.9 (+0.7 after the bounce), 2.3 (+1.4) float k = trip(t, 0.2, 0.0); if (k >= 0.0) v += blob(p, mix(A, W - vec2(18.0, 0.0), k), 10.0, 0.95); k = trip(t, 0.95, 0.0); if (k >= 0.0) v += blob(p, mix(A, W - vec2(18.0, 0.0), k), 10.0, 0.95); k = trip(t, 2.35, 1.0); if (k >= 0.0) v += blob(p, k <= 1.0 ? mix(A, W, k) : mix(W, vec2(600.0, 230.0), k - 1.0), 10.0, 0.95); return v; }
A call fails and is tried again after a short wait. Each failure doubles the wait, with a little jitter, until a try succeeds.

What is retried

The SDKs retry a request when it fails for a reason that may pass: a dropped connection, a timeout, a 429, or a 5xx. They don't retry a 4xx that will fail the same way again, like a bad key.

Waits double each time, with up to 25% jitter, so a thousand clients that fail together don't all come back together.

TryWait before it
1none
2about 1 s
3about 2 s
4about 4 s
5about 8 s

A Retry-After header always wins over the table.

Change it

const kestrel = new Kestrel({ maxRetries: 2 });
// or for one call
await kestrel.runs.create({ repo: "acme/web-app", task: "Bump lodash" }, { maxRetries: 0 });

Idempotency keys

A retried POST /runs could start the same task twice. The SDKs send an Idempotency-Key header with each create, and reuse it on every retry, so Kestrel starts the run once and returns the same run to every try.

Terminal
curl https://api.kestrel.dev/v1/runs \
  -H "Authorization: Bearer $KESTREL_API_KEY" \
  -H "Idempotency-Key: 4f1c2a8e-ci-build-8812" \
  -d '{ "repo": "acme/web-app", "task": "Bump lodash" }'
Note

Keys are kept for 24 hours. Use something tied to the job, like a CI build number, so a re-run of the same build doesn't start a second Kestrel run.

Retries inside a run

Inside a sandbox, Kestrel also retries the steps that fail for passing reasons: a model that is busy, or a package registry that times out. Those retries show in the trace as extra tries on one step, not as new steps.

Was this page helpful?

Open the dashboard