A durable /goal agent loop, built on Inngest, can survive worker crashes without losing progress or re-running finished work. Killing the worker with kill -9 mid-attempt let a new worker resume about 15 seconds later, with every finished turn preserved exactly once. The key boundary found: state survives a dead worker indefinitely, but the run itself only survives as long as the function's retry budget, fixed by raising retries from the default (a few minutes) to 20 (hours). Risky agent-generated code runs only inside a disposable sandbox, never on the worker, after a plain child-process timeout failed to stop a runaway while(true) loop.
Table of contents
Key takeawaysWhat is a /goal loop, and what breaks when it runs for hours?How do you build a /goal loop out of durable steps?What happens when the worker dies mid-attempt?What happens when the model provider times out or drops the connection?How long can a single attempt run?How does the loop keep its progress across hours and human reviews?Where does the agent's untrusted code run?How do you tell whether a change to the loop actually helped?How do you watch a loop that runs for hours?Run it on your own goalQuestions this post answers
What's the difference between state durability and run durability in Inngest's durable execution model?
Step state survives a dead worker indefinitely, but the overall run only survives as long as the next function that must execute keeps retrying. If a worker stays down past the function's retry window, the run fails even though every finished step's results are still saved; raising the function's retries (e.g. to 20) widens that window from minutes to hours. Anyone designing long-running agent workflows can study durable execution trade-offs like this on daily.dev.
How do you safely run untrusted AI-generated code that might hang or crash?
Run only the code-execution step inside a disposable sandbox, never on the worker that drives the loop, because a plain child-process timeout isn't enough; killing a wrapper process can still leave a child spinning on an infinite loop and pinning a core. Reading, editing and typechecking source stays on the worker since those never execute the code, and only the grader step touches the sandbox, which has no network access. Developers isolating AI-generated code execution can track patterns like this via daily.dev.
How do you detect when an AI agent is overfitting to its own evaluation checks?
Track the generalization gap between train and holdout scores: a loop that learns to satisfy its own check report shows a train score that the holdout score doesn't match. Holdout cases are scored only once, at the end, on the final best attempt, and are never shown to the agent during attempts, keeping the gap measurement clean. Teams tuning agent evaluation setups can follow techniques like this on daily.dev.