Reliable Workflow Design
Use this guide to turn an objective into an acyclic, evidence-producing workflow with explicit contracts, context boundaries, verification, and stop conditions. Read Custom Workflow Authoring first if you have not built a workflow definition yet.Choosing an Execution Shape
“Use a workflow” is not one decision — it covers several execution shapes with different costs and guarantees. This section is written as agent-facing guidance: it is the self-prompt an orchestrating agent should run before the first tool call on a new request, and it doubles as documentation for humans who want to steer that choice explicitly.Multi-item routing rule: Enumerate requested implementation items and prove their dependencies before launch. Run independent items as separate concurrent top-level workflow runs with bounded concurrency, one explicit worktree and root failure boundary per item. Preserve ordered composition only for real code, artifact, contract, decision, approval, or merged-result dependencies.The shapes, cheapest first:
The self-prompt: pre-launch workflow architecture
For every non-trivial workflow task, perform a short workflow-architecture pass before the first launch. Choose the execution shape before starting substantive work; reconnaissance already counts as inline execution. Derive the task’s implementation lifecycle needs, whole-codebase research needs, independent work slices, competing strategies, exact API/type/build contracts, schema or generated-artifact contracts, state-transition/lifecycle behavior, deterministic stop conditions, and required evidence. For coding tasks, that pass also infers repository intent from repo-level behavior before objectives and acceptance criteria freeze: mine git history (includinggit log --show-signature), merged PRs, issues, commits, and review comments for unwritten conventions — commit signing, message style and issue linking, changelog discipline, PR size and review norms — weighing the requesting user’s own activity highest so the authored contract captures norms no doc states. Non-coding tasks mine their analogous available context sources (issue trackers, long-form docs, chat or comment threads, prior artifacts) the same way. Inferred conventions fill contract gaps; they never override the stated objective or explicit repository docs.
Use this compact coverage matrix internally (it may stay concise for a straightforward task), and let every unresolved material row change the graph choice:
- Which stages may repeat?
- Does each iteration create distinct tracked work?
- What is the current frontier before each repeated stage?
- Could any proposed parent edge target an ancestor or the node itself?
- Are nested child workflows composed through boundaries rather than recursive
runinvocation? - Does resume/replay rely on stable per-iteration identity and call order?
- Is the outcome provable? If success can be stated as evidence (tests green, artifact exists, behavior demonstrated, reviewer approves), the task fits a workflow. If no proof is possible or needed, inline is probably fine.
- Is there structure? Multiple subtasks, dependencies, handoffs, or parallel slices rule out inline execution. A single focused evidence-gathering pass does not.
- Is there a loop or gate? Any “until Y”, “fix until passing”, review/approval gate, or unknown-length repair cycle requires a workflow that enforces the stop condition, never an improvised inline retry loop or an overextended subagent call.
- Is it one task or a queue of tasks? “Address all open issues” or “fix every ticket assigned to me” is a factory request, not one workflow. Enumerate and dependency-classify the items first, then follow Task queues and software factories: independent items become bounded concurrent top-level per-item runs; dependent items share one ordered composed graph; independent dependency clusters become separate top-level runs.
- Does an installed graph supply complete coverage? Run a named workflow only if its objective, inputs, lifecycle, and produced evidence cover every material row. Do not force-fit a broad-but-partial match (When to Use Workflows).
- What routing signals shape the graph? Broad repository uncertainty points to repository-focused Fan-out-and-synthesize; independent slices to Fan-out-and-synthesize; plausible-but-wrong contract risk to Adversarial verification or a task-specific verification stage; competing architectures or implementations to Generate-and-filter or Tournament; an explicit repeat-until condition to Loop until done; implementation work to a task-specific worker/reviewer loop; and exact API/build/schema requirements to dedicated deterministic gates.
- Does a tested graph solve only part of the task? Author one custom parent and nest that definition with
ctx.workflow(...), placing the missing research, verification, or deterministic gates around it instead of copying its prompts and gates. - Is it only specialist evidence-gathering? If the parent keeps control, no completion gate is needed, and the work is bounded (a debug pass, a parallel research fanout, one noisy investigation), inline subagents are enough—and cheaper than a workflow.
- Is it truly tiny? Deterministic, low-risk, single-file/no-test/no-review—answer or edit inline and stop.
@bastani/atomic/workflows/builtin, and call ctx.workflow(...). Nested children preserve their stages and guarantees within the expanded graph up to maxDepth, but they remain under the parent’s root lifecycle and failure boundary.
Choose the cheapest complete graph. Routing cues are not a reason to add decorative stages: avoid duplicated research and review loops. Before launch, state the selected graph, why one broad builtin is sufficient or insufficient, the evidence each major stage produces, and the stop/repair conditions. A simple direct match can be one sentence; a composed graph should briefly name its children and task-specific gates.
Stage model and thinking-level assignment
Before launching an authored workflow, assign every model stage a role, failure cost, primary model, thinking level, and fallback policy. Read Model Selection for the role defaults, but treat thinking levels in benchmark rows as measurement configurations, not production defaults. Reservemax for high-cost-of-error roles or an explicit user request; use high for demanding mapping, lifecycle analysis, compatibility, planning, synthesis, triage, and repair; use medium for user-impact review and final reporting; and keep deterministic checks as tool nodes with no model call.
Print this compact assignment before launch, with a short cost/quality rationale for each model stage:
max mechanically. Call workflow({ action: "models" }), use only each returned entry’s fullId and availableThinkingLevels, and if the role level is unsupported choose another catalog model or leave the stage unpinned rather than inventing a suffix. An empty or unavailable catalog is not a reason to fabricate a model or level. Deterministic typechecks, tests, schema checks, runtime probes, and artifact inspection remain durable tool gates rather than model self-report.
When an arbitrary task-specific workflow has plausible-but-wrong contract risk, design a bounded evidence-backed adversarial loop:
- Give a fresh-context, grumpy/skeptical-but-fair reviewer the literal objective. It should aggressively seek realistic counterexamples without inventing requirements or accepting hand-waving and circular worker-authored evidence, then emit a structured verifier plan: exact probe, inputs, command/assertion, expected success condition, and requirement/risk covered.
- For known contracts, author direct task-specific
ctx.tool(...)gates up front. For adversarially discovered risks, let the model select high-value probes in structured output, but execute the selected compile, test, schema generation/validation, runtime, and artifact-inspection checks authoritatively through durable workflow-ownedctx.tool(...)calls. The model must not self-report outcomes. - Feed the actual tool results to a skeptical evaluation stage. It classifies failures and emits one consolidated, evidence-backed, bounded repair payload for the implementation child.
- After repair, rerun the deterministic verifier tools until the declared pass condition succeeds or the iteration budget is exhausted. Define pass, repair, failure, and iteration-limit conditions before launch.
ctx.tool for workflow-owned external checks and side effects that benefit from durable checkpointing. Leave pure transformations as ordinary TypeScript; do not wrap every model-stage action in a tool call. A custom-loop pre-launch declaration must name the skeptical reviewer, deterministic verifier gates, how model-selected plans become tool executions, how evidence reaches evaluation/repair, and the bounded success/failure condition.
Judging task complexity
Complexity is a property of risk, not effort. Score a task on five axes and let the worst axis dominate — complexity is not the sum:
A one-line change to a serialization format is complex (high failure cost, exact contract). A 500-line mechanical rename is simple (zero uncertainty, type-checker-verified). The common trap is judging by effort instead of risk: long-but-mechanical is simple; short-but-contractual is not.
Fast tells, usable in the first 30 seconds:
- Done-condition test: if the success condition does not fit in one sentence, the task is complex or underspecified — clarify before guessing.
- The “and” test: “fix X and update docs and add a test” is three tasks in one sentence; enumerate and classify each.
- Loop words: “until it passes”, “keep trying” make the task at least moderate — iteration is expected.
- Working-memory test: more than about three interacting constraints at once means complex.
- Two or more distinct phases with a real handoff (research → implement, implement → verify), not just steps.
- The done-condition needs proof — tests, builds, review, or a contract check. If “how do you know it works?” is a fair question, a verification stage is waiting to exist.
- Iteration is expected — an anticipated repair loop, not a straight line.
- Failure cost is high — even a one-line change gets adversarial verification.
- The work outlives one attention span — losing mid-task state is a real risk.
Scoring rubric
When the ladder is ambiguous, score the task on six dimensions (0–2 each):
Interpretation:
- 0–3 total: inline. Adding stages creates more work than value.
- 4–6 total, Iteration ≤ 1, no gate: inline subagents when the parent should retain control, or a small named/custom workflow when tracking and artifacts matter.
- 7+ total, or Iteration = 2, or Verifiability = 2 with a review/approval gate: a real workflow. Prefer a named workflow when one fits the whole task; otherwise author a custom graph, nesting proven children where sub-problems overlap.
- Any single hard signal overrides the arithmetic: an explicit loop/stop condition, an approval or evidence gate, or a request for durable/background execution puts the task in workflow territory regardless of total score.
reads.
Task queues and software factories
Some requests are not one task but a queue of them: “address all open issues”, “fix every Linear ticket assigned to me”, “burn down the TODO backlog”, or “implement issue A and create a PR after; also implement issue B and create a PR after”. One monolithic worker loop would process the queue serially in a growing context and make unrelated work share one root failure boundary. Do not confuse splitting a queue across runs with splitting one objective across slices. Queue triage separates unrelated implementation items into top-level lifecycles; Stacked implementation slices keeps one dependent objective in one parent and verifies each ordered child slice before the next. Interpret ordering words locally unless a cross-item dependency is explicit. “Implement A and create PR A after; implement B and create PR B after” normally meansimplement A → validate A → PR A and implement B → validate B → PR B; those two item lifecycles may run concurrently. It does not mean PR A → start B. Serialize only when the user or repository evidence says, for example, “implement B after A is merged”, “B builds on A’s branch”, “use A’s generated schema in B”, or “do these in order”. Do not infer a cross-item sequence from list order or from “create a PR after” when “after” naturally refers to that item’s own implementation. Prove the dependency before serializing independent workflow items. If wording remains materially ambiguous after dependency research, ask one grouped clarification instead of silently serializing.
Triage before dispatch:
- Enumerate every requested item.
- Inspect stated issue, PR, branch, and approval dependencies.
- Check whether each prerequisite is already merged into the base each run will use. A merged prerequisite does not serialize current items when every base contains it. An unmerged prerequisite delays only the item or dependency cluster that consumes it; unrelated items remain eligible for separate concurrent workflow runs under the queue’s bound.
- Check likely shared files, API contracts, migrations, generated artifacts, and release or deployment effects. A shared unmerged contract can create a dependency even when items edit different files.
- Classify items as independent, dependent, or clustered.
- Dispatch independent items or clusters concurrently with an explicit concurrency bound; preserve dependency order inside each cluster.
- Report an item → run ID → worktree → branch → result/PR map. After each terminal lifecycle notice, inspect that run’s status detail before updating its result/PR fields.
Workflow run isolation and Git worktree isolation are separate guarantees. A top-level run provides its own context, progress, lifecycle controls, retry state, and root failure boundary. A worktree provides a separate checkout and Git state; it is not an operating-system sandbox. Several worktrees inside one sequential root do not create concurrent top-level runs or independent root failure boundaries, while concurrent writer runs without separate worktrees can still conflict. Use both for independent implementation items.
A natural-language request for a worktree does not configure runner isolation. Inspect the named workflow’s inputs first. Each per-item definition must declare and implement its reusable-worktree and branch inputs, and the dispatcher must pass distinct values explicitly. With
worktreeFromInputs, a missing target is created as a detached checkout from baseBranch, while an existing same-repository worktree is reused as-is. Neither case checks out the feature branch named by a separate branch input, so the item workflow must enforce that branch step itself.
Supported example: two independent top-level issue runs with a bound of 2. First save this complete project workflow as .atomic/workflows/issue-to-pr.ts, then run /workflow reload. It is a user-defined workflow built only from supported authoring APIs, not a bundled workflow name that Atomic installs by default.
run starts. The first durable tool then creates or checks out the requested feature branch, so worktree setup’s detached checkout never becomes the implementation branch. The item run owns branch setup → implementation → bounded review/repair → deterministic checks → push → PR creation. A failed review or check fails that item before push/PR.
Inspect the new target with workflow({ action: "inputs", workflow: "issue-to-pr" }). Then issue these two ordinary named-run tool calls in the same dispatch turn and end the turn. Interactive named launches return after startup admission instead of waiting for terminal completion, so the two run bodies overlap. Starting exactly two item runs and admitting no third until one ends enforces the bound of 2; the top-level tool has no batch-only worker loop or hidden concurrency field.
action: "statusDetail" and a detail object. Read detail.status and detail.error. For a completed run, read its declared outputs from detail.result and require a string detail.result.pr_url before filling that item’s result/PR fields; do not infer the PR URL from the lifecycle notice or stage prose. A completed detail without the required result or pr_url is a reporting-contract failure.
For a failed run, record detail.error and leave the PR field as no PR when the failure occurred before create-pr. If failure may have occurred during or after that durable tool, inspect its status/tool detail or the GitHub PR list before retrying so the dispatcher does not create a duplicate PR. In either case, free the dispatcher slot, keep unrelated top-level runs active, and do not treat a failed run’s partial result as successful output. Only after these per-run inspections should the dispatcher fill the final map:
The second failure does not cancel, pause, or roll back the first run, and it does not block unrelated later items from using an open dispatcher slot. A first item’s review, repair, or check failure must not block unrelated items; if it would, reconsider whether the queue was placed in one root workflow by mistake.
This example uses top-level named runs, not nested
ctx.workflow(...) children. Each launch appears in top-level status, gets its own lifecycle notices and controls, and owns an independent root failure boundary. Nested children are hidden from top-level run lists and expand inside one parent graph; a failed child call normally fails its parent, and parent exit cancels in-flight children. Use nested children to preserve ordered composition inside a truly dependent item or cluster, not to claim separate root lifecycles for independent queue items.
The factory self-prompt is: enumerate → inspect and classify dependencies → fan out top-level runs where independent → compose where dependent → dispatch in bounded waves → report the map.
Prompting the choice
Humans can steer the shape directly:- Name the shape or installed workflow. “Do this inline”, “use subagents to investigate”, or “write a custom workflow for this” overrides automatic scoring.
- State acceptance criteria. Verbatim criteria make the objective provable and define reviewer and reducer contracts.
- State the loop. “Iterate until tests pass” or “review and fix until approved” defines a hard workflow stop condition.
- State the evidence. A QA video, test output, generated artifact, or reviewer sign-off tells the graph which gates it needs.
- State the boundary. “Work in a separate worktree”, “do not create a PR”, or “stop after implementation” separates implementation from final actions.
- State the queue policy. Say how to split, order, isolate, and bound queued items; otherwise Atomic runs the dependency-triage and bounded-dispatch playbook before implementation. Ordinary list order and per-item “create a PR after” wording do not create a cross-item dependency.
The Run Contract
A run’s contract is its objective plus its acceptance criteria. Only the user may change it. Every stage that receives a change must hand it to the next stage. This is the single most important rule for getting predictable results out of a multi-stage run, and it is the rule most often broken by accident.Only the user may change the contract
A workflow launches with a contract: the objective and, when supplied, explicit acceptance criteria. Two parties relate to it very differently:- You may amend it at any time. A mid-run message — steering, a follow-up, resume text — is authoritative. If you say “also handle the detached path,” that is a new requirement, and the run adopts it from that moment.
- Agents may not amend it at all. An implementer that notices a nearby bug, a cleaner abstraction, or a missing feature has found deferred work, not a new criterion. It records the observation and keeps building to the contract.
Amendments must reach the next stage
An amendment that stays inside the session that received it is invisible to everything downstream. That produces the failure this rule exists to prevent:You steer the implementation stage to add a requirement. The implementer adopts it and builds it. The reviewers were launched with the original criteria, so they score the added work as unrequested scope and the original criteria as contradicted. The run then burns review loops arguing about a contract mismatch nobody can see.So every builtin stage prompt carries a steering propagation contract:
- Restate every objective-relevant steering message in your report or handoff artifact, under an explicit
Contract amendments receivedheading, verbatim when short. - Keep user-authored amendments visibly separate from your own observations, so the next stage can tell a required clause from an agent proposal.
- Treat amendments inherited from an upstream stage as contract clauses. Cover them in acceptance and traceability work; never classify them as out-of-scope.
- Resolve ambiguity before implementing. Use
intercomto ask the supervisor or originating stage when one is reachable; otherwise state the conflict and implement the narrowest reading consistent with the launch contract. - Propagate nothing else this way. Tool preferences, working style, and your own ideas are not amendments.
ctx.task, ctx.chain, and ctx.parallel prompt carries the contract automatically. Do the same in a custom workflow:
Scope discipline
The mirror of “only the user may amend” is that the agent holds the line. Every builtin implementation stage carries this contract:Before writing code, state the goal in one sentence and list the acceptance criteria. That list is the contract. Freeze it.While implementing:
- Done means the contract, not “good.” When all criteria pass, stop. Polish, refactors, and “while I’m here” fixes are new work, not this work.
- Every addition must trace to a criterion. If you cannot point at the criterion a change serves, do not make it. Log it instead.
- Keep a deferred list, not a growing diff. When you notice a bug, smell, or missing feature outside the contract, write one line in a deferred note and move on. Surface it at the end.
- Distinguish blockers from improvements. Change scope only if a criterion is impossible or wrong as written — and say so explicitly before proceeding, rather than silently absorbing the work.
- Watch for the tells. “It would be cleaner if…”, “we should also…”, “this really ought to…” mean you are about to move the goalpost. Stop and check the contract.
- Prefer the smallest diff that satisfies the contract. Fewer files touched, fewer abstractions introduced, no speculative generality for futures nobody asked for.
Protect the contract from compaction
A long-running stage gets compacted, and compaction ranks lines individually rather than preserving whole instructions. That ranking has a bias worth knowing: an objective is verbose and restated, while the constraint that bounds it is usually one line. Rank them independently and the constraint is the cheaper deletion — so what survives is coherent, actionable, and missing its boundary conditions. A prohibition removed from context reads as permission. Wrap contract text inkeepContext so it survives verbatim regardless of the compression ratio:
keepContext is a pure string helper, not a ctx.* primitive: it creates no graph node and has no side effect, so call it anywhere a prompt is assembled. It is idempotent, so composing already-wrapped text will not nest.
Tag:
- role constraints that bound a stage to part of the work — “research only”, “review and report, do not repair”;
- acceptance criteria and immutable contracts a later stage is judged against;
- explicit prohibitions;
- identifiers a stage must not lose, such as a target branch, worktree path, or run ID.
reads.
Every builtin does this for its own invariants: the steering propagation contract, the literal objective contract, scope discipline, worktree discipline, per-run acceptance criteria, and the research/review role constraints are all protected. See Compaction for the retention mechanism.
Tagging is not only for workflow authors
The tags are plain text, so they work anywhere text becomes a stage prompt — you do not need to be writing a workflow definition to use them. Two cases matter in everyday use, and both apply to an agent driving theworkflow tool on your behalf.
Run inputs. Workflows inject their inputs into stage prompts, so anything you tag in an input is inherited by the stages that receive it:
send amendment is authoritative and stages must carry it forward, but it is one short message arriving late into an already-long session, competing against the entire transcript for retention. Tagging it keeps it alive until the stage acts on it:
Practical consequences
- Steer freely — it is the supported amendment channel. You do not need to restart a run to add a requirement.
- Say what you mean as a requirement. “It would be nice if…” reads as guidance; “also handle X” reads as a clause. Stages are told to distinguish them.
- Expect amendments in the reports. If a stage received one and its report has no
Contract amendments receivedsection, the amendment did not propagate and downstream stages will not honor it. - A growing diff with no new criteria is a defect. That is the tell that scope discipline slipped, and it is a legitimate reason to stop a run.
Scope-Guard Starter Pattern
Use a scope guard when a worker may find valid adjacent work and a later reviewer or repair stage could treat that finding as part of the current task. The guard is an independent reviewer built from existing workflow composition. It controls scope only: code reviewers and deterministic checks still decide whether the candidate is correct. Do not add awatchdog field, stage option, or custom runtime primitive for this pattern. Choose the lightest existing shape that fits the boundary:
Canonical scope contract
Create one inspectable contract artifact before guarded work starts. Treat it as immutable for that run and include:- the literal objective;
- required scope and allowed files or systems;
- explicit non-goals;
- stage boundaries and expected lifecycle order; and
- acceptance criteria and required evidence.
reads where the primitive supports it, tell fresh stages to read the needed sections, and keep Intercom messages short. A fresh guard must not rely on a sibling transcript or hidden graph state.
Decision contract and actions
For each proposed material expansion, the guard records one evidence-backed classification and action:
Use a stable key for each proposal, such as
public-error-shape or transport-timeout. Keep one row per key, merge repeated evidence into that row, and cap the log (the examples use 20 entries). Do not let the guard and worker echo the same finding back and forth. The persisted decision artifact is the source for later review and repair stages; chat messages only steer the open turn.
A useful decision record contains key, classification, concrete evidence, and action. A guard failure or missing coordination channel never means approval.
Fallback policy
Pick and document one policy before the run:
Use
block for risky public contracts, data changes, security behavior, releases, or publication. warn is a practical default when a boundary review can replace live steering. Never degrade silently from block to warn or from guarded execution to off.
Ordinary intercom is mandatory in every workflow model stage. noTools: "all", restrictive tools allowlists, and excludedTools continue to restrict every other tool but cannot remove Intercom, so live steering remains available.
Lifecycle, topology, and context rules
- Keep the graph acyclic. A boundary guard is an ordinary downstream reviewer node. Live Intercom steering is activity inside already-running parallel stages, not a new graph edge.
- Never make a guard watch itself, recursively start another guard, reopen a terminal task, or add a dependency from the current frontier to an ancestor. Complete all turns on a retained guard before starting downstream dependency work.
- Messages admitted before a worker generation closes drain through that stage boundary. Late messages do not reopen or mutate its terminal workflow state. Give each live branch a bounded stop rule;
ctx.parallel(...)releases downstream work only after all started branches settle, even when one finishes first. - Persist decisions under stable keys. Pause/resume, model fallback, durable replay, and nested workflows then reread the artifact instead of sending duplicate interventions.
- Omit
groupfor ordinary use. The worker, guard, nested workflows, and delegated subagents inherit the top-level workflow invocation’s stable Intercom group. Set an explicit group only for intentional isolation; an override isolates that stage from ordinary same-group peers while leaving it steerable from the invocation context, which retains directional list/send/live-ask control over the subgroups it owns. - Use
context: "fresh"for guards, reviewers, and judges. They should see only the contract, candidate, decision artifacts, and current files. - Use
context: "fork"plusforkFromSessionFilefor implementation, debugging, and repair roles that need continuity with an owned earlier session.context: "fork"alone does not name a fork source; an initial worker with no prior lineage may start fresh. A later continuation should use the earlier worker’ssessionFilewhen available. Do not fork an independent guard from the worker it judges. - Send a forked continuation only the delta after the fork point: new evidence, the decision artifact, any human answer, and the next action. Keep the full shared contract in its canonical file.
candidate → validation → approval → push/publish, a guard at the candidate or validation boundary must not reject the patch merely because it is unpushed or unpublished. Only the later publication stage owns that action.
Runnable boundary-task example
Use a fresh task when one check at a material boundary is enough. This complete project workflow keeps the worker lineage coherent, saves a structured decision log, and sends ambiguity toctx.ui before the continuation:
prepare candidate → scope boundary → optional human prompt → continue worker. Each step is new downstream work; no edge points back to the original worker.
Runnable retained-stage example
Usectx.stage(...) when one independent checker needs a retained conversation. Run its tracked prompt() once, then use sendUserMessage(...) for a bounded post-prompt turn on that same session; a second tracked prompt() on the finalized stage is invalid.
sendUserMessage(...) starts one retained follow-on turn after that node finalizes; it does not create or reopen graph work. The follow-on updates the artifact directly only when evidence changes, and it finishes before the human prompt or worker continuation starts.
Runnable live-parallel example
Use a live peer only when steering during generation adds clear value. Both branches omitgroup, so Atomic places them in the workflow invocation’s same Intercom group. The guard first performs a bounded Intercom status handshake and returns; later blocking intercom.ask calls can reopen its retained conversation for classification. After both parallel branches settle, a fresh task reads that transcript and persists the final deduplicated decision artifact. Normal late sends are not part of this handshake.
warn runs that task as a boundary check, block requires ctx.ui, and off records that no guard approval exists.
Fast inference for workflow stages
Stages select faster inference the same way they select any other model: by naming it. Where a provider publishes a fast variant, its canonical selectable ID is the base model ID plus-fast, so a stage pins openai-codex/gpt-5.6-sol-fast — with an optional thinking suffix, openai-codex/gpt-5.6-sol-fast:medium — in its model field, and lists fast and normal IDs in its fallback model fields in whatever order it wants:
fast marker. Normal and fast IDs are distinct fallback candidates and distinct modelAttempts entries, and stage model metadata carries the exact selected ID, so a run’s records show precisely which one served each attempt. Graph node cards keep their dependency metadata focused on topology.
workflow({ action: "models" }) lists fast variants alongside their normal siblings, so a workflow can discover them at runtime. See Providers for which providers publish fast variants and what each one sends upstream.
Pin fast variants deliberately in broad workflows. Parallel fan-out and fallback attempts multiply provider requests, and priority-tier requests are billed at a higher rate.
Context Engineering
A workflow is an information-flow system, not just a list of prompts. Most workflow failures come from missing, stale, oversized, or poorly-routed context. Design every stage boundary deliberately.Locally Scoped Stage Prompts
Stage prompts should define local contracts, not describe the full workflow runtime. Write prompts as if the stage could be executed independently from a fresh session with only the listed inputs. A useful compact shape isRole · Goal · Success criteria · Constraints · Tools · Output · Stop rules; omit sections that do not change behavior. Include:
- the stage’s current objective and what is out of scope for this stage
- the exact files, artifacts, child outputs, or user inputs it may use; put long inputs before the final instruction
- context-dependent tool routes and permission boundaries, without describing tools the stage cannot call
- the expected output format and length, or the schema it must return when the workflow item is schema-enabled
- the checks, tools, or deterministic commands it should run when relevant, plus evidence required for progress or completion claims
- the success criteria and blocker conditions that let this stage stop
context: "fork" or forkFromSessionFile for coherent long-running implementation stages that need continuity from their own earlier work. Use context: "fresh" for unbiased reviewer, evaluator, and gate stages so they inspect the current files and explicit artifacts rather than inheriting the implementer’s assumptions. When continuity is needed across fresh stages, pass it explicitly through files, declared outputs, and reads.
Context-Mode-Aware Prompt Text
Context mode is an execution property configured withcontext/forkFromSessionFile; the model cannot act on context mode, so keep it out of prompt text:
- Never describe the stage’s own context mode. Sentences like “you are running in a fresh context window”, “your context is clean/non-forked”, or “this is a forked session” add tokens without changing behavior. State the concrete action, inputs, and success criteria instead.
- Fresh stages must not reference invisible context. A fresh stage has no “previous conversation”, cannot see sibling stages, and does not know the surrounding graph, so instructions like “compare against previous workflow reasoning” or “this runs in parallel with the locator pass” do not help and may confuse the model. Phrase the same intent stage-locally (“compare the working tree against the baseline branch”; “do your own scan; do not assume any other stage’s output is available”) and pass any state the stage needs through files, declared outputs, and
reads. - Forked continuation prompts send only the delta. A forked stage already carries the role, contracts, guidance, and output format from its own earlier prompts, so repeating them uses more tokens and can make the two copies diverge. Send what changed since the fork point — new artifacts, updated state, the next action — plus a one-line pointer back (“the contracts and report format established earlier in this thread still apply unchanged”) instead of re-injecting the full text.
- Keep one canonical copy of shared contracts. When fresh and forked variants of a stage share guidance, render the full contract only in the prompt that first establishes it and reference it from continuations. If a continuation needs a contract restated (for example, after a schema change), that is a new contract version, not a repeat.
Context Fundamentals
Treat context as a finite attention budget. Include only information needed for the current decision, place critical constraints near the beginning or end of prompts, and use progressive disclosure instead of loading every possible reference up front. Common context sources:- System instructions: persistent behavior and guardrails.
- User inputs: workflow inputs and human-in-the-loop decisions.
- Retrieved documents: files, search results, logs, API responses, and artifacts.
- Message history: useful for continuity, but grows quickly in long-running stages.
- Tool outputs: often the largest source of context bloat.
Context Degradation Patterns
Watch for these failure modes in long or multi-stage workflows:
Use compaction, file references, and bounded loops before context fills with transcript noise. In attached workflow stage chat, manual compaction shows
Compacting context..., threshold compaction shows Auto-compacting..., and overflow recovery shows Context overflow detected. Auto-compacting... in the same animated status row used for normal model work. That label is a fact about the stage session rather than about the pane, so detaching to the graph and reattaching while compaction is still running restores the same reason-specific label instead of falling back to the generic Working... row; it clears as soon as the compaction ends. A successful compaction leaves the normal expandable ✻ Context compacted boundary in the transcript; the boundary is reconstructed from the durable session and has a typed live fallback if the refreshed session snapshot is temporarily unavailable.
Compression and Artifact Handoffs
Optimize for tokens per completed task, not the smallest prompt. Aggressive compression can force later stages to rediscover information. A compressed handoff includes:- objective and current status
- decisions already made
- files, symbols, commands, and artifact paths with evidence
- open questions and known risks
- rejected alternatives when they matter
- next action expected from the downstream stage
output with outputMode: "file-only" and reads for research bundles, logs, plans, diffs, reviewer reports, and any other stage product that can grow. In the downstream stage prompt, say Read the file at ${artifactPath} before continuing. Do not inject full session tails, all previous stage outputs, or every prior review round into later prompts by default; pass the latest relevant artifact paths and make older history discoverable from a ledger or index file.
Three rules make that work in practice:
- One owner per artifact. The runner writes the stage’s final assistant message to
outputafter the stage ends, automatically writes the companion transcript outside the repository tree, and appends one instruction telling the model that its final message becomes the artifact. Your prompt does not need to restate any of that — describe the deliverable, not the plumbing. If a late admitted turn displaces the intended content, search the transcript withrgrather than assuming the curated artifact holds every later turn. A prompt may write other files freely; only the declaredoutputpath is runner-owned and overwritten at stage end. - Do not read an artifact back just to return it.
outputMode: "file-only"exists so the parent receives a compact reference. CallingreadFileon that artifact and returning its text as a workflow output cancels the saving and drops the whole report into the caller’s context window. Return the reference and a*_pathoutput instead. - Return paths from the workflow. Declared outputs are consumed by the calling session, so a workflow’s
resultshould be a reference plus explicit*_pathoutputs. Callers that need the body read the path; callers that only need the outcome pay nothing for it. When a detail is missing from the curated artifact, search its companion transcript withrgand inspect a narrow range.
reads passes paths rather than content: a stage reads the file when it runs, so the artifact must hold the real report at that moment.
Multi-Agent and Parallel Patterns
Use parallel stages to isolate context and separate independent work, not merely to assign role labels. Good parallel branches have distinct evidence-gathering or review angles:- locator / mapper: where relevant files and systems live
- analyzer: how the current implementation works
- pattern finder: how similar code is written elsewhere
- external researcher: what upstream docs or APIs require
- reviewer/evaluator: whether outputs satisfy the validation contract
Filesystem Context
Use files when workflow context grows too large:- write large tool outputs to files and return concise references
- store plans, state, and reviewer findings in structured markdown or JSON
- pass artifact paths via
reads; prompt agents withRead the file at <path>...rather than pasting artifacts into{previous} - for review loops, pass the latest review-round artifact first and let a ledger/index point to older rounds only when needed
- give parallel branches separate output paths to avoid write conflicts
- use
grep, globbing, and line-range reads instead of loading entire logs - clean scratch files or keep them under run-specific directories
Evaluation and Quality Gates
Build validation into the workflow instead of waiting for a final manual check. Useful gates include:- deterministic checks: tests, typechecks, linters, schema validation, command exit codes
- rubric checks: completeness, correctness, evidence quality, risk coverage, user fit
- reviewer stages: fresh-context reviewers that inspect artifacts and current files
- LLM-as-judge stages: direct scoring, pairwise comparison, or rubric-based grading for subjective outputs
schema on that workflow item. Keep that stage’s prompt narrow: tell it the specific check to perform, the files/tools it may use, the evidence to report, and the structured decision it must return. Require progress and completion claims to map to current tool results; when evidence is unavailable, the stage should identify the unverified claim or blocker rather than infer success.
When using LLM judges, reduce bias by defining score anchors, requesting observable evidence and criteria-based justification, calibrating against examples, and keeping length/order effects in mind. Do not ask for chain-of-thought or reconstructed internal reasoning. Track pass rates and failures over time for reusable workflows.
Tools, MCP, Memory, and Hosted Execution
Constrain each stage to the tools it needs. Too many tools increase ambiguity and token cost; too few tools force brittle workarounds. Tool descriptions should make inputs, side effects, and error handling clear. Use per-stagemcp allow/deny lists when a workflow needs external systems but some stages should remain read-only or isolated. Use memory or durable project knowledge only when cross-run continuity is required; otherwise prefer explicit inputs and artifacts.
Hosted or remote agent workflows need additional design work: sandbox setup, dependency caching, auth boundaries, artifact transfer, concurrency limits, and multiplayer/session handoff behavior. Optimize startup before the user begins the run; do not make each stage rebuild its environment.
Task Fit and Project Design
Before turning a process into a workflow, confirm that it suits automation:
For complex workflows, structure the implementation as a pipeline: acquire context, prepare prompts/artifacts, process with LLM stages, parse or validate outputs, and render the final result.
Design Checklist
Before implementing or shipping a non-trivial workflow, answer these questions:- Purpose and fit: What concrete outcome should the workflow produce? Is the task naturally multi-stage, parallel, resumable, or reusable? What is out of scope?
- Inputs: Which values should be declared as inputs? What is the narrowest schema type? Which defaults are safe?
- Common pattern: Which common workflow pattern best matches the task, and where does the actual design intentionally diverge?
- Stage decomposition: For each stage, what question does it answer, what context does it need, what output should it return, and what model/tool/MCP requirements does it have?
- Local stage contract: Can this stage prompt stand alone with its current objective, inputs/artifacts, expected outputs, tools/checks, and success criteria, without unexplained workflow internals or future-stage assumptions?
- Prompt vocabulary: Do stage, reviewer, and reducer prompts describe the concrete action, available evidence, and success criteria that the stage can see locally, instead of assuming the model knows the workflow graph’s name or surrounding context? Avoid phrasing like “the create-PR workflow stage” or “this Foo workflow” unless that name is explicitly supplied as user-visible context or materially affects behavior.
- Information flow: For every edge between stages, is
previousenough, or should the handoff use structured returns, files,reads,output, oroutputMode? - Output contract: Which outputs should be declared in
outputs, which stage/task/child results shouldrunreturn for those keys, and what runtime type must each value have? If another workflow may call this workflow as a child, which non-default outputs should the parent rely on? - Context size: Can downstream stages succeed from the handoff alone? Should large transcripts, logs, or research bundles be summarized or saved as artifacts?
- Control flow: Should the workflow use
ctx.chain,ctx.parallel,ctx.ui, bounded loops,failFast, orfallbackModels? - Acyclic topology: What node and dependency shape can each branch, bounded loop, and nested workflow boundary materialize? Which stages repeat, does each iteration create distinct tracked work with stable identity and call order, and what is the current frontier before each repeat? Could any proposed parent edge target the node itself or an ancestor? Are nested children composed through
ctx.workflow(...)boundaries rather than recursiveruninvocation? Redesign or stop before launch if any self-edge or back-edge remains. - Scope control: Could valid adjacent findings expand the patch? If so, where will a fresh scope guard read the immutable contract, how will it classify and persist bounded decisions, which
warn/block/offfallback applies, and which worker session owns any forked continuation? - User experience: Are stage names readable in status and graph views? Is the final output compact? Are important artifacts saved with stable paths?
- Validation: What success criteria, review gates, deterministic checks, or evaluator stages prove the workflow did the right thing? Are model gates schema-backed instead of regex/prose-matched, and do adaptive gates run as focused model stages with explicit tool/check instructions?
- Final actions: Does the workflow distinguish implementation/review convergence from post-approval final actions such as PR/MR/review creation, release tagging, deployment, or publication? Are reviewers and reducers prompted to approve and hand off when implementation and validation criteria are proven and only an explicitly authorized final action remains?
Common Mistakes
- Do not invent workflow names; list first.
- Do not guess input keys; inspect with
inputsorgetfirst. - Do not call
create,update, ordeleteon the workflow tool; definitions are code-authored. - Do not use legacy workflow tool fields like
agent,stage, or run-controlname. - Do not pass strings or path objects to
ctx.workflow(...); import the workflow definition from@bastani/atomic/workflows/builtinor another TypeScript module first. - Do not create a self-edge or a dependency edge from the current frontier to an existing ancestor. Cyclic workflow graphs are unsupported; redesign or stop before launch when a cycle cannot be removed.
- Do not model a bounded loop by reopening an earlier node beneath its downstream work. Create distinct tracked work per iteration and keep retained-session follow-up as non-topological activity when it adds no dependency work.
- Do not claim TypeScript or workflow discovery proves a dynamic workflow acyclic. Discovery diagnoses imports and definition shape; execution, replay, and DBOS hydration are the runtime topology boundary.
- Do not rely on undeclared child outputs; returning a key that is not declared in
outputsfails the run. Declare every child-workflow field you expose inoutputs— includingresult— and return values matching those schemas fromrun(see Outputs). - Do not expect to select or rename child outputs at the call site; parent workflows receive the child’s declared output contract as
child.outputsafter checkingchild.exited === false, and a partial declared-output map whenchild.exited === true. - Do not expect named workflow runs to block the chat turn; they are background tasks.
- Use
interruptorpausewhen the user asks to pause specific live work resumably; usequitfor a graceful run-level process boundary. - Keep stage names readable because they appear in workflow status and UI.
- Do not ask a stage to reason from workflow or stage names that are only orchestration labels. Model stages see their local prompt, artifacts, tools, and reads; describe the concrete action and evidence instead of referring to an implementation-specific nickname.
- Do not write stage prompts that depend on hidden workflow-wide awareness; make each model stage locally scoped and self-described (Locally Scoped Stage Prompts).
- Do not parse model gate decisions from ad-hoc prose with regular expressions; configure
schemaon a focused workflow item and consumeresult.structured. - Do not make reviewers fail an implementation gate solely because an authorized final action has not run yet. Represent that remainder as a post-approval next action (for example
finalActionRemaining/nextAction) and let the final stage perform it. - Do not let scope guards approve correctness or turn follow-up findings into blockers. Keep scope decisions separate from code review and deterministic validation, and do not reject expected pre-publication state assigned to a later lifecycle stage.
- Return compact structured decisions and save large artifacts to files; artifact handoffs should still use files when the next stage does not need the whole payload in context.
Workflow Best Practices
This playbook helps coding agents and workflow systems produce better results. Treat an agent as a capable engineering partner that needs a clear objective, tight scope, explicit validation, and occasional steering. Most weak agent runs fail for predictable reasons: the goal is vague, the scope is too broad, validation is missing, or the agent keeps following the wrong signal. This playbook addresses these failure modes. The examples below are synthetic and intentionally generic. Replace placeholders like[component], [test command], and [workflow] with your own project details.
The core loop
The core workflow pattern is:- Define the end state.
- Constrain the blast radius.
- State what counts as done.
- Run the agent or workflow.
- Inspect status before reading details.
- Steer only when the run is off track, blocked, or missing criteria.
- Require evidence before accepting the result.
- Ask for a summary, handoff, or next-step plan.
Prompt anatomy
A strong workflow prompt usually includes:Objective
What should be true when the work is complete?Context
What does the agent need to know before acting?Scope
What is the agent allowed to change?Non-goals
What should the agent avoid?Done criteria
How will we know the work is complete?Stop conditions
When should the agent stop and ask instead of guessing?Core principles
1. Start with the end state
Describe what should be true at the end, not just what the agent should investigate. Bad:2. Keep scope tight
Agents often expand into nearby cleanup, which can help, but most workflow runs should stay bounded. Use phrases like:Only touch files required for this behavior.Do not refactor unrelated code.Preserve existing behavior for [case].Make the smallest correct change.
3. Separate implementation from validation
Relevant evidence, not the agent’s claim, determines whether a change is done. Evidence can include:- a targeted test,
- a broader regression test,
- a smoke command,
- a typecheck or lint command,
- a structured output contract check,
- or a clear manual verification step.
4. Prefer evidence over speculation
When something fails, steer the agent back to the observable signal: the error, failing test, log line, user behavior, or broken contract.5. Use staged thinking
For ambiguous work, separate the flow into stages:6. Steer, do not micromanage
The best steering messages are short and corrective. They add constraints, redirect attention, or provide a decision. Usually, state only what changed instead of rewriting the whole prompt.7. Treat failed validation as the next task
A failed test becomes the next objective.8. Interrupt stale or wrong work
If a run is solving the wrong problem, based on outdated assumptions, or duplicating another run, stop it. Continuing usually creates more cleanup.9. Inspect at the right level
For long-running workflows, do not start by reading every log. Check:- overall status,
- current stage,
- blocker or failure reason,
- relevant stage details only if needed.
10. Ask for synthesis before handoff
Before switching from investigation to implementation, or from implementation to review, ask for a concise synthesis:Common Workflow Patterns
For workflows larger than one tracked task, choose a small control-flow pattern before writing prompts. Workflow authors should favor these common patterns by default: naming the pattern up front keeps the stage graph understandable, makes validation gates explicit, and helps reviewers see why work is split across model sessions. Reach for a bespoke structure only when none of these patterns fit. The first six patterns below have runnable builtins. For example, a migration workflow can nest fan-out-and-synthesize for call-site fixes, adversarial-verification per patch, and loop-until-done while tests still fail. Import and compose the builtin definitions instead of copying their prompts/graphs. Scope guard and Stacked implementation slices are authoring starter patterns rather than builtins; compose scope guard’s boundary-task, retained-stage, or live-parallel form from current primitives, and use stacked slices to unroll dependent implementation children through existingctx.workflow(...) boundaries. Constructive quorum is an accepted reviewer-coordination pattern used by goal and ralph; it is prompt guidance rather than a standalone builtin.
These patterns organize work inside one root lifecycle. They do not replace the task-queue rule: independent whole implementation items normally get separate top-level runs and failure boundaries, while real dependency clusters may use these patterns inside each cluster run. Constructive quorum shapes bounded deliberation inside parallel reviewer stages; stacked implementation slices split one objective inside that lifecycle; queue triage splits separate whole items.
Constructive quorum relies on existing Intercom mechanics: every workflow invocation gets its own stable Intercom group, and parallel stages and delegated subagents inherit it when they can use Intercom. Reviewers can therefore reach siblings without authoring group plumbing; keep the evidence exchange bounded and leave quorum counting to the deterministic reducer.
Pattern diagrams
1. Classify-and-act
Builtin definition and contracts: Six composable pattern builtins.- Make the classifier return a structured category and confidence, not free-form prose.
- Keep each action branch isolated with the minimum tools and context it needs.
- Add a fallback or human-input branch for low-confidence classifications.
2. Fan-out-and-synthesize
Builtin definition and contracts: Six composable pattern builtins.- Partition by files, sources, claims, candidates, or work items that can be evaluated independently.
- Save each branch to a separate artifact and pass paths with
readsinstead of inlining all branch output. - Treat synthesis as a barrier: it waits for every branch, deduplicates, resolves conflicts, and cites evidence.
3. Adversarial verification
Builtin definition and contracts: Six composable pattern builtins.- Give verifiers fresh context and a concrete rubric with pass/fail evidence requirements. For task-specific contract risk, use a grumpy/skeptical-but-fair persona that seeks realistic counterexamples, stays within the literal objective, rejects hand-waving and circular worker-authored evidence, and reports only actionable evidence-backed defects.
- Separate adversarial probe design from authoritative execution. Require a structured verifier plan with each exact probe, inputs, command/assertion, expected success condition, and covered requirement/risk; then run selected compile, test, schema generation/validation, runtime, or artifact checks through durable workflow-owned
ctx.tool(...)calls. Actual tool results—not model self-report—feed judgment and consolidated repair. - Known contracts may use direct task-specific
ctx.tool(...)gates designed before launch; uncertain risks may use model-selected probes executed by those deterministic tools. Rerun the tools after repair until the declared pass condition or iteration limit. - Ask verifiers to find blockers and not rewrite the candidate unless you explicitly assign them to repair it. Keep pure transformations as ordinary TypeScript rather than wrapping every model-stage action in
ctx.tool. - Decompose the rubric into named criteria and score each in its own call. Compound rubrics can latch onto one salient factor; the reference scan reports 76.4% for the best single criterion versus 78.3% for a three-criterion ensemble (§4.3).
- Aggregate by mean plus an explicit veto for genuinely disqualifying findings, never a unanimity AND across verifiers: unanimity makes false-reject grow as 1−(1−p)^K while the false-accept it buys only decays as (1−p)^K. See Verification scaling.
- The shipped
adversarial-verificationbuiltin acceptscriteriaas a record of criterion names to descriptions or as acriteria.mdMarkdown string; the sharedverification-criteriamodule also canonicalizes string lists andCriterionInputlists. Its public doors areparse_rubric,normalize_criteria,select_criteria, anddecide_verification, using theCriterion,CriterionInput,CriterionScore, andFindingshapes;NoCriteriaandEmptyCriterionare explicit rubric errors. - A
criteria.mdrubric may have a#title, an optional##section whose heading containsground truth(normally## Ground Truth Note; the first such section wins), and must include a##section whose heading containscriteri(normally## Criteria) whose### Name {#id}headings own non-empty criterion bodies. HTML comments are ignored; an omitted{#id}is slugged to lowercase alphanumeric/underscore text (up to 40 characters), with a fallbackcriterionid and encounter-order_2/_3deduplication.parse_rubricrejects a rubric with no criterion headings or an empty criterion body. VERIFICATION_SCALEanchors integer scores from 1 (certainly fails) through 20 (verified correct).select_criteriapreserves the requested id order and rejects unknown ids;decide_verificationaccepts only with quorum, a mean at or above the policy threshold, and novetofinding, while an invalid report remains metadata rather than a score.- Keep a scoring family in the
SHARED HEAD ‖ VARYING TAILlayout fromverification-prompts: the byte-identical head contains the task, ground-truth note, candidate bodies (or caller-provided read paths), and scale anchors in that order; the tail contains only the criterion name and description plus the output-format instruction. Candidate-specific bodies stay in the shared head, not the varying tail, so sibling criteria can reuse the cached prefix. - Inline the whole candidate family only while every body is at most
32 * 1024UTF-8 bytes (MAX_INLINE_CANDIDATE_BYTES). If any body is larger, switch the whole family to caller-bound paths, preserving path order and duplicates; an oversized pathless family is rejected rather than guessed. warm_first_fan_outschedules the first-seen step for each prefix before releasing the remaining steps, establishing the provider’s warm prefix before sibling criteria or pair slots vary. Warm failures are observed without fail-fast, the remaining phase is still attempted before the error is rethrown, and successful results return in input order.- The builtin input defaults are
verifier_count=3,max_repairs=2,accept_mean=14on the 1–20 scale, andreask_limit=1; omittedcriteriauses thetask_fit,evidence, andcompletenessrecord. A round expects one schema-valid score for every criterion/verifier cell, and the normal call shape is criteria length multiplied by verifier count. - Invalid criterion reports are written as invalid artifacts and re-asked in bounded waves up to
reask_limit; an invalid or missing report is counted ininvalidCountonly and is never converted into a fail vote or included in the mean. If the required quorum is still missing after the re-asks, the round isindeterminaterather than silently narrowing the decision. score_table_pathnames the durableverification-summary-<round>.jsonfor the final round. Its object containsscores(criterion_id, integerscore,evidence, andfindingswithfindingplusseverity),mean,invalidCount, thedecision(accept,repair, orindeterminatewith its corresponding mean/findings or missing count), and foldedusage;review_report_pathcarries repair guidance or quorum evidence.
4. Generate-and-filter
Builtin definition and contracts: Six composable pattern builtins.- Generate more candidates than you need, then filter hard by an explicit rubric.
- Dedupe before judging so near-identical candidates do not dominate the shortlist.
- Use this for exploration, naming, design options, hypotheses, and lightweight eval ideas.
- When the filter ranks candidates rather than applying a threshold, use the same judge guidance as Tournament: graded per-criterion integer scores rather than binary keep/drop, a Bradley–Terry preference from the score gap so near-ties stay near-ties, and K repeats with candidates swapped between the A and B slots. See Verification scaling.
- For a custom ranking filter, reuse the shared
verification-criteriamodule and itscriteria.mdparser rather than inventing a binary keep/drop rubric; stable criterion ids let the judge select the same criteria in each comparison. See Adversarial verification for the accepted shapes and score decision.
5. Tournament
Builtin definition and contracts: Six composable pattern builtins.- Use pairwise comparison when absolute scores are noisy or subjective.
- Randomize or balance presentation order where possible to reduce order bias.
- Keep the judge rubric short and require rationale tied to observable criteria.
- Have judges emit graded per-criterion integer scores rather than a binary winner, then derive a Bradley–Terry preference from the score gap so near-ties stay near-ties.
- Repeat each pair K times with the candidates swapped between the A and B slots; the swap cancels positional bias within the pair and variance falls as O(1/K). In the reference scan’s discrete-judge study, 26.7% of pairs tied at K=1; with slot swaps, the reported K=1→16 result moved from 74.7% to 77.5%.
- See Verification scaling for score granularity and call-budget trade-offs.
- The shipped tournament inputs use
num_attempts=4andmax_concurrency=4;n_evaluations=2repeats each criterion/directed pair,pivots=1selects the second comparison phase’s pivot candidates, andseed=0drives the deterministic schedule.criteriais optional and accepts a markdown rubric, a string-to-description record, a string list, or aCriterionInputlist; omission uses the shipped three-criterion Correctness, Completeness, and Evidence and task fit rubric. Optional orderedmodelsids are assigned round-robin to attempt slots. comparisons_pathpoints tocomparisons.json, whose ledger records the task and seed,params(n,pivots,n_evaluations, and normalizedcriteria), per-jobcomparisonsrows (a,b, phase, criterion id, repeat, slot-swap flag, scores or aninvalidmarker, preference, and judge artifact path), aggregatepairs, weights/counts, the completeranking, and optional model assignment. Itsbudgetrecords planned versus executed judge stages, including re-asks; invalid reports remain auditable rows and an all-invalid pair remains marked invalid rather than becoming a score.
6. Loop until done
Builtin definition and contracts: Six composable pattern builtins.- Define both success and escape conditions before the loop starts.
- Keep a durable ledger of attempted work, findings, failures, and validation evidence.
- Bound loops by iterations, budget, or convergence criteria so exhausting a bound produces an inspectable failure instead of letting the loop continue indefinitely.
- Materialize every iteration as distinct tracked work with stable iteration identity and call order. Never represent repetition by a self-edge, a back-edge to an ancestor, or reopening an ancestor below its downstream work.
- Record a progress magnitude in the ledger beside the boolean stop bit; a flat or decreasing series is the stall signal that the loop is burning iterations without moving.
- Treat the trend as a monitoring and escalate-to-human signal, never a kill switch: the explicit stop condition remains authoritative. See Verification scaling.
- The builtin defaults
max_iterations=5,progress_scoring=true, andprogress_repeats=1; setprogress_scoringfalse to omit advisory scoring, whileprogress_repeatsis the repeat count passed to the scoring primitive. Each scored iteration adds aprogressentry toprogress-ledger.jsonwithscore,perRepeat(null for an invalid repeat),trend, and the classifierwindow; the ledger also emitsprogress_curve,final_trend, andprogress_disclaimer. - Progress scores use the anchored 1–20 scale and average valid repeat scores per checkpoint.
classify_trenduseswindow=3,riseDelta=1.5, andfallDelta=-1.5; it compares equal leading/trailing halves of the trailing two windows, drops an odd middle sample, and classifies inclusive threshold crossings asrising,flat, orregressing. A short series isflatevidence. - The trend is monitoring and escalation evidence only: it never kills, terminates, or approves a loop, and the explicit evaluator stop condition remains authoritative.
progress_curve,final_trend, andprogress_disclaimerare advisory outputs, not alternate closure signals.
7. Constructive quorum
This prompt-level reviewer pattern is used by thegoal and ralph builtins; it does not add a reducer or quorum mechanism.
- Give every reviewer an independent preliminary assessment before it reads sibling findings or verdicts.
- Run exactly one bounded evidence-exchange round. Share concrete findings and evidence, challenge blocking claims, and stop rather than opening a second round.
- Change a verdict only through evidence, never deference. Each reviewer emits its own final structured verdict and records whether deliberation changed it and which evidence caused the change.
- Let the existing deterministic reducer count the final votes; deliberation shapes votes but does not replace quorum counts or the
stop_review_loopcontract.
Stacked implementation slices starter pattern
Use this authoring pattern when one implementation objective should land as a stack of small, independently verified changes. It is not a queue dispatcher: the slices belong to one dependency chain, so slice N+1 starts only after slice N is verified. During the pre-launch architecture pass, enumerate the slices in the coverage matrix. Give every slice its own objective, acceptance criteria, changed-file scope, and verification gates. Target roughly 100–500 changed lines between verification points by default, but treat that as a reviewability default rather than a law: keep a genuinely atomic mechanical change or generated-artifact refresh in one slice, and do not split a small objective just to reach a count.goal or ralph from @bastani/atomic/workflows/builtin, or use a task-specific child when neither builtin matches. Before each child, use a durable ctx.tool(...) step to create or check out the slice’s explicit branch in its worktree. worktreeFromInputs creates a missing target with a detached checkout and reuses an existing target as-is; base_branch and git_worktree_dir do not create or check out a feature branch by themselves. Create slice N+1’s branch from slice N’s verified branch, then pass that previous branch as base_branch and give the child a distinct git_worktree_dir.
The parent should verify each child before creating the next boundary. If a gate fails, stop at the first failed gate, report that slice as unverified, and retain the earlier verified slices and their branch/worktree records. Do not roll earlier slices back and do not continue past the failure.
The calls below are deliberately unrolled. Repeat the downstream shape for the planned slices, giving every call a fresh child boundary and distinct tracked nodes; do not reopen an ancestor or add a back-edge.
prepareSliceWorktree tools run before their child boundaries and use git worktree add -b, so each child starts in a named feature branch. Once the path exists, the child’s worktree binding reuses it as-is; base_branch remains the comparison base for its reviewers. The child owns implementation, review, repair, and acceptance, while the parent owns branch/worktree setup and the stop boundary.
Use ralph or a task-specific child in the same positions when its input contract fits better. For a longer stack, keep the same explicit downstream shape: create each next named branch from the previous verified branch, pass that previous branch as the next child’s base_branch, and use a distinct worktree. Do not replace the chain with a loop that points back to an ancestor. A final handoff can report slice → branch → worktree → verified/failed from the explicit inputs and preparation records without reopening completed child work.
Verification scaling
This is authoring guidance for custom workflows, not a description of shipped builtin inputs:- Use an anchored 1–20 integer scale as the default score granularity.
- Providers expose no token logprobs, so a K-sample average is the substitute; K=16 parity costs roughly 16× the call cost, making K a budget decision.
- Treat pool diversity as a bet on the selector’s oracle ceiling. In the reference scan’s pivot tournament, best-of-3 selection reached 86.5% ±1.1 against 79.4% pass@1 with a 92.1% oracle ceiling, while best-of-5 reached 88.0% ±0.6 against 78.7% pass@1 with a 96.6% oracle ceiling. A chance-level selector can make a more diverse pool worse, so widen the pool only once the judge beats chance.
- Self-verification—having the same model judge its own rollouts—still gained +7.1 over pass@1 in the best-of-3 comparison (86.5% versus 79.4%) and +9.3 in the best-of-5 comparison (88.0% versus 78.7%).
- For a cheap operating point, an author can use one pivot and K=2 repeats for a best-of-3-shaped comparison budget; this is an authoring recipe, not a shipped default.
- For the shipped primitive references, see Adversarial verification for criteria parsing, warm-first scoring, re-asks, and score summaries; Tournament for inputs and
comparisons.json; Loop until done for progress ledger/trend outputs; and Goal/Ralph for re-verification and convergence evidence.
Choosing a common workflow pattern
- Pick classify-and-act when routing correctness matters more than breadth.
- Pick fan-out-and-synthesize when the work divides cleanly into independent slices.
- Pick adversarial verification when the main risk is a plausible but wrong answer.
- Pick generate-and-filter when output quality depends on exploring a large option space.
- Pick tournament when multiple whole-solution strategies should compete under one rubric.
- Pick loop until done when the workflow should continue until evidence says it is finished, not until a preselected number of stages completes.
- Pick constructive quorum when several fresh-context verifiers judge one artifact and a simple tally could hide a defect or preserve one verifier’s misreading; use one bounded evidence exchange before each verifier emits its own final vote.
- Pick scope guard when valid adjacent findings could expand a worker or repair stage beyond its immutable contract; choose a boundary task by default and live parallel steering only when timing requires it.
- Pick stacked implementation slices when one dependent implementation objective needs ordered, independently verified layers. Keep the 100–500 line range as a default with atomic-change escapes; create or check out each named branch before its child, create each next branch from the previous verified branch, pass that previous branch as
base_branch, use a distinctgit_worktree_dir, and stop at the first failed gate.
Steering patterns
Tighten scope
Signal: The agent starts expanding into adjacent cleanup, unrelated files, or broad refactors. Steer:Add missing done criteria
Signal: The agent has a plan, but no clear completion criteria. Steer:Redirect an off-track stage
Signal: The workflow is investigating the wrong area or solving the wrong problem. Steer:Respond to a blocked prompt
Signal: The workflow asks for approval, a choice, or clarification. Steer:Turn failed validation into the next task
Signal: Tests, typecheck, lint, build, or smoke checks fail. Steer:Ask for synthesis
Signal: The workflow has gathered information, but the next action is unclear. Steer:Pause, stop, or rerun
Signal: A run is stale, duplicated, superseded, or based on outdated assumptions. Steer:Copy-paste templates
Start a workflow
Tighten scope
Add acceptance criteria
Redirect a stage
Handle failed validation
Ask for synthesis
Turn findings into implementation steps
Prepare a release gate
Concrete examples
Example 1: Fixing a failing test
Scenario: A package has one failing unit test after a recent change. Initial objective:[targeted test command], then [nearby test command].
Outcome: Small fix applied, regression test passes, and the workflow reports exact commands and results.
Example 2: Repairing a workflow definition
Scenario: A custom workflow no longer returns the expected structured output. Initial objective:Example 3: Investigating before implementing
Scenario: A user-reported bug is ambiguous. Initial objective:Anti-patterns
These anti-patterns target run prompts; Common Mistakes covers workflow tool and authoring mistakes.Quick reference
Before starting a workflow, include:- Objective
- Context
- Scope
- Non-goals
- Done criteria
- Validation command
- Reporting requirements
- Stop conditions
- Queue dependency classification, concurrency bound, and item → run/worktree/branch map (when several implementation items are requested)
- What changed?
- Why was this the right fix?
- What evidence supports it?
- Which commands were run?
- What still might be risky?
- Is anything blocked or unresolved?