home · docs · sub-agent supervision
Sub-agent supervision.
When carlos delegates, the supervisor view is the marquee surface. Per-agent intent, live tool-call stream, progress, state, token spend and burn-rate, diff preview, and explicit steer / interrupt / stop verbs. Single-agent by default; delegation is the exception.
The architectural commitment
The event log is the source of truth. Every agent writes an append-only event log to SQLite (WAL mode). Live state, the roster, the focus pane, lineage, resume-after-crash, the skill-proposal review queue: all projections over those logs. The TUI never owns state; it reads from the log and the in-memory projection.
Crash recovery, time-travel debugging, and the "what was that sub-agent doing at 3:42?" question all reduce to "replay the log." If a projection disagrees with the log, the projection is wrong by construction.
Ten-state machine
The set is deliberately fine-grained because conflating any two of these is the documented legibility-failure mode in real supervisors (systemd, OTP, Claude Code's "thinking vs hung" issue).
| state | meaning |
|---|---|
spawning |
Process/context created; system prompt + tool allowlist + budget being injected; first provider call not returned. |
queued |
Admitted but waiting on a concurrency slot. Distinct from spawning because attribution of spawn failures matters. |
running |
Actively in its provider-call / tool-call loop. |
awaiting-input |
Blocked specifically on the user or parent. Resolver: a human. |
blocked |
Waiting on an external dependency (sibling artifact, rate-limited API, file lock). Resolver: the system. |
paused-by-user |
The user explicitly froze this agent. Different cause, UI affordance, and resume semantics from blocked. |
compacting |
Agent is summarizing or handing off context near the window limit. |
cancelling |
Stop requested but the agent is mid-tool-call. Must reach a safe boundary before terminating. |
done |
Terminal, success. |
failed |
Terminal, unrecoverable for this attempt. Retries are new attempts with new IDs, never resurrections. |
orphaned |
Pathological terminal-ish: supervisor lost the child (crash, OOM, heartbeat lost). May still burn tokens; this state is loud. |
Eleven rows. carlos's SPEC says ten, but in practice orphaned is a separate state, so it's included here. Legal transitions are encoded as (state, event) → state | error in internal/agent/state.go. Illegal events are rejected at runtime; the TUI never infers state from side-effects.
queued → running(cannot skipspawning)done → anythingfailed → runningwithout going throughspawning(no resurrection)paused-by-user → done(a paused agent did not finish)compacting → donedirectly (must return torunningfirst)
Parent-child contract
When an agent delegates, the parent sends a structured task spec, not a free-form prompt. Four parts, every time.
What the child must accomplish, one paragraph. Concrete, scoped, falsifiable. No "explore the codebase and report back."
Typed result schema. The child returns a struct, not free text. The parent parses, validates, and routes the result without re-reading prose.
Restricted allowlist. A research child cannot run shell commands. The supervisor enforces; the child cannot escape.
Explicit token cap, turn cap, wall-clock cap, success criteria, definition-of-done. A child without a budget is a leak.
The child returns a compressed deliverable plus references to artifacts written to disk (content-addressable at ~/.carlos/artifacts/<sha256>). Full transcripts stay in the child's event log; they don't pollute the parent's context window.
Three verbs: steer, interrupt, stop
Each labeled in the status bar with a distinct keybinding so they're discoverable, not memorized.
| verb | key | semantics |
|---|---|---|
| steer | s | Inject a [steering] message into the agent's event log; delivered at the next tool-call boundary, never mid-inference. Agent keeps its context and in-progress work. |
| interrupt | i | Abort the current turn (soft). Keep the session and context alive; return control to the user. Tool calls already completed stay in place; they are not rolled back. |
| stop | x | Terminate the agent. Default is graceful drain: signal stop, wait for the agent to reach a safe boundary, then transition to done / failed. Drain timeout escalates to hard-kill if the agent doesn't reach a boundary within the budget (OTP-style shutdown timeout). |
Never hard-kill mid-side-effect. A file-write or HTTP request in flight completes (or fails atomically) before transition to terminal.
Claude Code's documented "Enter queues but doesn't interrupt" failure mode is exactly what these three verbs are designed against. s steers without aborting, i aborts without killing, x kills with a drain budget. Each is reachable in one keypress from the supervisor.
Roster & focus pane
Roster. One row per agent (including the top-level agent), lineage by indentation, terminal-state rows dim or collapse. Columns: short ID, state badge (text + color, never color alone), one-line intent, model, tokens-in/out, cost, current tool, 60-bucket burn-rate sparkline, elapsed time.
Sorting and filtering. k9s-style. / opens a fuzzy filter; sortable by any column. Default sort surfaces the four priority-ordered monitored conditions (next section) above normal-running.
Focus pane. Transcript tail of the selected agent. Most recent reasoning + last tool call by default, scrollback on demand. Streaming tokens render from a coalesced in-memory projection (250-500ms flush cadence).
Status bar. Current verb labels (s steer, i interrupt, x stop, Enter focus), discoverable without memorization. Mode chip (mode=X (cap N)) shown when the supervisor satisfies ModeReporter.
What the user monitors for
Priority order. This list drives the default roster sort.
- Agents awaiting input. Block downstream progress; only a human unblocks them. Loudest signal.
- Runaway cost. Burn-rate spike. Irreversible spend.
- Confidently-wrong / drifting outputs. Verification-failed flags, citation-missing markers.
- Stuck agents. No heartbeat, no progress, not compacting.
- Everything else (normal running, done).
Safety rails
Non-negotiable. Each is enforced by the supervisor, not by the model.
- Restart intensity (OTP pattern). If more than MaxR (default 3) retries occur within MaxT (default 60s), the parent supervisor terminates the subtree and surfaces a circuit-breaker alert.
- Per-run and per-subtree token/cost caps. Hard ceilings, enforced before each provider call.
Budget+Tracker(Phase 5a). - Spawn-depth cap (default 1; leaf agents cannot spawn) and concurrency cap (default 3-5 parallel children, mode-gated to 0/1/5).
- Heartbeat-based orphan detection. 5s heartbeat emit + 10s sweep; agents past 2x interval transition to
orphanedat next startup viaRecover. - Worktree isolation for side-effecting sub-agents. Each agent that may write to shared state gets an isolated git worktree under
~/.carlos/frames/<frame>/worktrees/<agent-id>/. "Apply" is a separate, user-gated merge step.
Worktree isolation also means you can let a sub-agent edit files without polluting your working tree. If the deliverable is no good, the merge step is where you say no.
When to delegate vs stay single-agent
Two layers, easily conflated. The frame's mode (solo / tight / orchestrator) is the supervisor's enforcement: how many in-flight children it allows at all (cap 0 / 1 / 5; SpawnCapFor in internal/frame/frame.go). The system prompt's Frame block then tells the model how to behave under that cap. New personal frames ship in orchestrator.
What each mode tells the model:
solo: do the work yourself. Sub-agent delegation is opt-in, only spawn when the user explicitly asks or the task is plainly beyond a single session.tight: single-task focus, no tangents. The supervisor caps in-flight children at 1, so when delegation does happen it runs sequentially.orchestrator: delegate by default. Anything more involved than a single-line edit, a one-shot lookup, or a trivial fix goes to a sub-agent; split the work across parallel sub-agents when the parts are independent. The user opted into this mode, so no per-turn confirmation.
Research evidence still informs the design: DeepMind arXiv:2512.08296 finds that multi-agent coordination nets negative once single-agent baselines exceed ~45% on a task class, and sequential / decision-dense work degrades 39 to 70% under MAS. The user picks which trade-off matches the task class through /mode; the carlos design choice is to surface that as a mode selection rather than a per-turn overlay.
The Agent tool's Description() stays policy-neutral: it documents the tool's mechanics (objective, output_format, tool_allowlist, atomic typed return) and points the model at the Frame block for when-to-delegate guidance. This lets the same tool surface change behavior per mode without re-registering. Implementation: internal/agent/agent_tool.go + internal/agent/sysprompt.go.
Skill proposals share this surface
An induced skill is just another on-disk artifact referenced from the event log, reviewable as a markdown diff in the same focus-pane/approval UX as any other deliverable. There is no separate skill-review subsystem; skill PROPOSALs, file edits, plans, and research outputs all flow through the same plan/preview/apply gate.
If you can review a diff, you can review a skill proposal. If you can apply a plan, you can apply a skill. carlos refuses to invent a second approval surface for the second kind of artifact.
Related reading
- Permissions: the layered allow/deny gate every sub-agent tool call passes through.
- Memory: event log, FTS5 summaries, and the user model.
- Skills: induced skills as reviewable artifacts.
- Agent loop comparison: how carlos's loop differs from Claude Code's.