robot-voice-chat — Agent Orchestration Spec¶
robot-voice-chat is the voice/chat app that hosts the mission brain — one instance per
colleague, on the central brain host, attached to a robot: an agentic
MissionExecutor layered on top of the single-turn voice/chat pipeline. It handles multi-step
planning, parallel long-running tasks, triggers and replanning, the HTTP event ingress for
capability servers, a Claude-backed planner LLM, teaching mode with process-memory integration,
mid-mission barge-in (safety stop → pause → confirm), and a live mission pipeline view in the UI
(every planned tool call colored by state).
Repo: robot-voice-chat
This spec covers the brain's behaviour, not its operation — running an instance is Running a brain.
This document is self-contained for an engineer who knows this codebase (ports-and-adapters,
Hydra DI via composition.py, FastAPI + Socket.IO, McpToolBackend). External contracts it
depends on are restated inline.
0. Capability model — the brain as a pure MCP client¶
Model updated (2026-07): direct MCP tools, not skill instantiation
The R1 stack converged onto galaxea_agent, and the old skill_control server was
removed with no successor. The brain no longer discovers and instantiates ROS "skills"; it
calls MCP tools directly on the robot's servers. Sections below that still show
skill_control / instantiate_skill / navigation describe the earlier control/data-plane
split and are being migrated — the current deployment model is brainctl plus a robot manifest
(Running a brain).
The brain is a pure MCP client (no rclpy). It drives the robot by calling tools directly on
the robot-side MCP servers; it never talks to ROS. The deployed surface it attaches to (via a
robot manifest resolved by gen_mcp_config.py):
| Server | Port | Layer | What the planner calls |
|---|---|---|---|
r1-abstraction |
9220 | L4/L5 | The robot-abstraction contract over galaxea: perceive, pick, place, approach, plan_execute, set_gripper, verify_held, navigate_to_named, stop, … |
grasp-service (grasp) |
9210 | L3 | grasp(object) — the reliable grasp skill (side/top/two_hand FSMs). |
mcp-perception-buffer (perception) |
9202 | L5 | Ambient observation: what_do_you_see, what_did_you_see, add_watch/remove_watch. |
robot-mcp-memory (memory) |
9103 | — | Process/fleet memory: find_processes, save_process, record_outcome, Fleet admin. |
ros2-memory (episodic) |
9203 | — | Temporal queries ("what did you see / where was I N s ago"). |
All of them are binabik servers and so speak SSE at /sse (galaxea's own servers speak
streamable-HTTP at /mcp, but the brain never reaches them directly — r1-abstraction fronts
them). In local dev a single g1_robot stdio server stands in for the robot; in production
the manifest attaches r1-abstraction + grasp-service (see §10).
A "capability use" is a direct tool call. grasp("the box") blocks and returns
{ok, held, side, …}; perceive("a bottle") returns {ok, pose, dims, …}; what_do_you_see()
returns the current scene. There is no discover→instantiate→observe→stop protocol.
Observation is mostly pull; watchers push. The Async Task Contract (robot-mcp-kit) carries
watch_events from the perception buffer (a watcher firing) and long-running task completions to
POST /api/agent/events; most observation is the pull buffer, not pushed events.
Safety stop (§9) calls the robot's stop tool (r1-abstraction) and cancels in-flight async
tasks/watchers.
Illustrative examples
The eta_below / zone_entered / navigate_to examples in §1–§14 illustrate the generic
async-task/watcher machinery; the concrete tool surface is the table above.
1. The Async Task Contract¶
This backend takes one dependency on robot-mcp-kit (for the shared pydantic TaskEvent /
TaskFailure models and signing.verify). Everything else is in-repo.
1.1 Consumer view¶
Capability servers POST events to this app:
POST /api/agent/events
X-RVC-Server: navigation
X-RVC-Timestamp: <unix-seconds> # ±300 s tolerance
X-RVC-Signature: sha256=<hex> # HMAC-SHA256(secret, f"{ts}.{raw_body}")
Body: TaskEvent (robot_mcp_kit.models) —
{schema_version, event_id, task_id, server, tool, type, seq, ts, payload}
type ∈ progress | trigger_fired | task_done | task_failed | watch_event
task_failed payload = TaskFailure {error_code, message, recoverable, suggestions}
Delivery is at-least-once and per-task ordered; dedupe on event_id. Reconciliation: every
task-capable server exposes get_task(task_id) / cancel_task(task_id) /
list_tasks(states=["running"]) MCP tools — poll after restart or event silence. Async-task
tools return a TaskInfo JSON ({task_id, state: "running", est_duration_s, ...}) immediately;
the executor recognizes this shape (§5.4).
2. Architecture overview (what goes where)¶
backend/src/robot_voice_chat/
├── agent/ # NEW package — the mission brain
│ ├── executor.py # MissionExecutor: the agentic loop (§5)
│ ├── mission.py # Mission, PlanStep, MissionEvent dataclasses + states (§4)
│ ├── plan_tools.py # set_plan / update_plan / finish_mission local tools (§5.3)
│ ├── reactions.py # ReactionTable: event-pattern → tool call, zero LLM hop (§6)
│ ├── task_registry.py # live TaskHandle map + reconciliation (§5.4)
│ ├── trace.py # TraceRecorder — full mission trace (always on) (§8)
│ └── teaching.py # teaching flag + distillation prompt + save to memory (§8)
├── services/llm/anthropic.py # NEW AnthropicLLM adapter (§7)
├── services/usage_ledger.py # NEW priced-record ledger + robot-side ingest (§3a, BIN-240)
├── services/budget.py # NEW spend limits, evaluated in-process (§3b, BIN-240 phase 4)
├── services/llm/swappable.py # MODIFIED: the budget gate lives in `_take()` (§3b)
├── api/routes/agent.py # NEW /api/agent/* routes incl. events + usage ingress (§3, §3a)
├── pipeline/… # MODIFIED: start_mission routing (§5.1), barge-in (§9)
├── services/tools/mcp.py # MODIFIED: per-server event_secret + motion flag exposure,
│ # harvests a `usage` key off every tool result (§3a)
└── config/… # NEW groups + agent section (§10)
frontend/src/… # Mission pipeline view + Teach button (§11)
Design rules: the planner lives here, MCP servers are capabilities not coordinators; the fast path (the base single-turn pipeline) stays untouched for small talk; one mission at a time.
3. Event ingress — POST /api/agent/events¶
Route module api/routes/agent.py:
- Read the raw body (signature is over raw bytes), look up the server's
event_secretfrom the MCP config by theX-RVC-Serverheader,robot_mcp_kit.signing.verify(secret, ts, body, sig). Failures → 401 (logagent_event_rejected). Unknown server name → 401. Server entries without anevent_secretconfigured cannot post (401). - Parse into
TaskEvent; unknownschema_version> supported → 400. - Dedupe on
event_id(in-memory LRU set, size 4096). Duplicates → 200{"status": "duplicate"}. - Hand to
MissionExecutor.on_task_event(event)(and to the ReactionTable first, §6). Events for unknowntask_ids are loggedagent_event_orphanand dropped with 200 (the task may belong to a previous process life). - Rate-limit: 120/min per server via the existing slowapi limiter.
Other routes (admin-token not required; these power the main UI):
| Route | Behavior |
|---|---|
GET /api/agent/missions |
last N missions (id, instruction, state, started/ended, step summary) from the in-memory store + trace dir |
GET /api/agent/missions/{id} |
full mission snapshot (plan, steps, events) — same shape as the mission_plan socket event |
POST /api/agent/teaching |
{"enabled": true} → arm teaching for the next mission (§8); GET returns current state |
3a. Usage ingress — POST /api/agent/usage (BIN-240 phase 2)¶
A robot-side spender (the grasp recovery tier, a vision/segmenter server) reports what it spent
here, so the brain's cost ledger covers the robot, not only itself. Route module
api/routes/agent.py; pricing and the dedupe gate are services/usage_ledger.py.
- Authenticated by one fleet-wide
RVC_USAGE_SECRET, resolved throughservices.credentials, never the per-serverevent_secret§3 uses — attaching that secret to a manifest entry forcesgen_mcp_config.pyto emit the server'snameas a tool-name prefix, which would rename a robot server's tools the moment it started reporting spend. NoX-RVC-Server/secret match required the way §3'sevent.server != servercheck requires —X-RVC-Serveris carried into the rejection log only; the record's ownservicefield is what the ledger reads. - Body is a
robot_mcp_kit.usage.UsageRecord, sent unpriced (confidence: "unpriced",cost_micro_usd: null) — the brain prices it on arrival via the samePriceTablea brain-side call is priced against.schema_versiongate mirrors §3's, againstrobot_mcp_kit.usage.USAGE_SCHEMA_VERSION. services.usage_ledger.record_reported_usagerejects (400, logged, never priced) a record that already carries a cost, or whoserobot_iddisagrees with this brain's own. A blankrobot_idis stamped with this brain's.- A record carrying
gpu_seconds(a time-billed call — Modal, say) is priced asunpricedrather than aRates-derived zero: the table has four token classes and no per-second one, so pricing it anyway would produce a false$0.00. - Dedupe is durable, unlike §3's in-memory-only LRU: an in-memory LRU catches a retry
within one process life, and
usage_totals(where=("record_id", …))catches the rest, for as long as the raw row survivesretention_days— because at-least-once delivery here means real money, not an orphan event a reconciliation poll can shrug off. - No
executordependency, unlike §3 — a brain with agent orchestration disabled still has robot-side spend worth recording. - Rate-limit: 600/min (higher than §3's 120/min — a dropped usage report is lost money, not a retryable orphan event).
The other channel needs no route at all. services/tools/mcp.py's _harvest_usage reads a
usage key off every tool result through McpToolBackend._dispatch — the chokepoint call()
and call_on() both funnel through — prices each entry the same way, and strips the key before
the planner ever sees it. No secret, no URL: it rides the answer to a call the brain already
made. This is the channel to reach for by default; the HTTP route exists for spend a tool
result can never carry (a bounded recovery loop that hits its deadline and returns nothing).
See Integrating a robot → reporting spend for the robot-developer side of this contract.
3b. Spend limits — POST/GET/DELETE /api/admin/cost/limits (BIN-240 phase 4)¶
An operator caps this brain's robot from Admin → Cost, and the cap is enforced in-process,
never from a poll — services/budget.py reads the same mounted ledger services/usage_ledger.py
already writes to, at the one place every billable brain-side call already passes through
(SwappableLLM._take(), services/llm/swappable.py).
robot_idis the only scope. BIN-364 decision 1 (one brain per robot) makes a brain limit and a robot limit the same limit, so there is no scope selector — a limit set here always governsusage_ledger.ROBOT_ID.POST/DELETE400 when it is unset (a workstation brain has nothing to cap).- Three windows:
session(filtered bysession_id, resets when Reset rolls the session — it is not a time bound, the same reasoncost_view._session_reportfilters the same way),daily(UTC midnight),mtd(the 1st, UTC). POST /api/admin/cost/limits—{window, limit_micro_usd, hard_stop}. 400 on an unknown window or a non-positive limit. Admin-gated, like every other cost route that can name a scope's configuration. Persisted to aspend_limitstable on the metrics DB's mounted volume — the same disk the spend being measured against it already lives on — so a limit survives a container recreate the way the ledger does.DELETE /api/admin/cost/limits/{window}— removes one; a no-op, not an error, when none was set.GET /api/admin/cost/limits— the configured limits, for the form to load.GET /api/admin/costand the ungatedGET /api/cost/summaryboth carry abudgetsarray (one entry per configured window: spend so far, the limit,hard_stop,pct, atierofok/warn_50/warn_80/over, andincompletewhen part of the window aged pastretention_daysinto the daily rollup, which carries norobot_iddimension to recover the rest from). It is on the ungated summary route too, deliberately: the escalating banner has to work for a user who has never logged into admin, the same reasoning that keeps that route ungated at all — aggregates only, no key identity.
Enforcement. SwappableLLM._take() calls budget.check() after promoting any queued model
switch (a switch is not a spend, so it is never blocked) and before returning the serving
provider. A blocked verdict raises BudgetExceeded, caught at the turn boundary — the pipeline
orchestrator's turn handler and the mission executor's step loop — and surfaced as a clean
refusal (emit_error for chat, a failed mission with the gate's own sentence as the summary,
never "internal error: …"). Because chat, the mission planner, the surveillance observer
(judge=planner) and the recovery distiller all resolve through the same SwappableLLM, one
gate covers all four; robot-side spend cannot be gated this way, since a report arrives after the
call already happened, on another host.
When a hard-stopped window is first found over its cap, budget.check() also calls
ActivityGate.stop_all_for_budget() — the same set_on(False) the operator's own toggle uses —
for every registered billable poller (today: the observer, §9's set_surveillance_enabled).
This is deliberately not routed through the idle gate's own suspend/resume bookkeeping: a
budget stop must stay off until an operator raises the limit or disables the stop, and the idle
gate's "resume exactly what I suspended" rule would otherwise switch a budget-stopped poller
back on at the next UI interaction.
Warnings and enforcement both dedupe per (robot_id, window, tier) for the log line and per
(robot_id, window) for the poller stop, so a poller ticking every few seconds does not spam a
log line or re-flip a switch the operator has since overridden — cleared the moment that
window's spend drops back under the cap (a session Reset, a new UTC day), so a later re-crossing
enforces again.
The general "any service polling an external API must surface a warning in the UI while it
runs" guardrail (coding-guidelines.md) is answered here only for the two cost-incurring
pollers left after BIN-306 retired mcp-perception-buffer — the observer and the Modal
segmenter GPU, both already in BillableServiceBanner. A generic who-is-polling registry is
explicitly out of scope for this phase and belongs to its own issue.
4. Domain model (agent/mission.py)¶
MissionState = Literal["planning", "running", "paused", "succeeded", "failed", "cancelled"]
StepStatus = Literal["pending", "running", "succeeded", "failed", "skipped"]
@dataclass
class PlanStep:
id: str # "s1", "s2", … assigned by set_plan
title: str # human-readable, shown in the UI ("Drive to the mailroom")
tool: str | None # dotted tool name if known ("navigation.navigate_to")
status: StepStatus = "pending"
detail: str = "" # live detail line: progress %, ETA, result snippet, error
task_id: str | None = None # linked async task, if any
started_at: datetime | None = None
ended_at: datetime | None = None
@dataclass
class Mission:
id: str # "m-" + uuid4.hex[:6]
instruction: str
state: MissionState
steps: list[PlanStep]
created_at: datetime
summary: str = "" # set by finish_mission
teaching: bool = False
used_process_ids: list[str] = field(default_factory=list)
notes: list[str] = field(default_factory=list) # what the run observed going wrong
notes is copied off the per-mission blackboard at finalization (§5.6) rather than read live,
because the blackboard is cleared by the next mission — so a question asked two runs later
would be answered with the wrong run's diagnosis. It is what lets a debrief say
reason='pre_grasp_failed', approach='side', attempts=2 instead of only "it failed" (BIN-350).
A MissionStore (in-memory dict + the trace files of §8 for history) holds the current mission
and recent finished ones. Single-mission invariant: starting a mission while one is
planning/running/paused is rejected — the fast path tells the user the robot is busy and what
it's doing (it has the mission summary available).
5. MissionExecutor (agent/executor.py)¶
One asyncio task per mission, owned by a singleton MissionExecutor built in composition.py
with: planner LLMProvider (§7), ToolBackend (existing MCP backend), EventEmitter,
ScenarioRegistry (for the system prompt base), AppState, MemoryClient (thin wrapper that
calls memory.find_processes / record_outcome through the ToolBackend), config (§10).
5.1 Entry: routing from the fast path¶
The fast-path LLM has one additional local tool (not an MCP tool — intercepted by the
dispatcher exactly like use_predefined_answer):
start_mission(instruction: str, ack: str)
description: "Call this when the user asks for a multi-step physical task (fetch, deliver,
patrol, find-and-bring …) rather than a question or a single gesture. `instruction` is the
task restated with all context resolved; `ack` is one short sentence to say immediately."
On call: speak/emit ack through the normal output path, then executor.start(instruction).
The fast turn ends; the mission runs in the background. While a mission is active, every
subsequent user turn is routed into the mission (executor.on_user_turn(turn)) instead of the
fast path — status questions, corrections, and "stop" all land in the mission conversation. Small
talk unrelated to the mission stays in the mission conversation too; the planner decides and can
answer briefly via say.
What a fast-path turn carries (BIN-350). It used to be exactly two messages — the scenario system prompt plus the operator's sentence — which meant a turn arriving after a mission had no idea one had run. Reported at 09:33, four seconds after the brain had itself summarised a failed grasp: "Grasp was failing because you had the robot at maximum height…" → "I'm a robot assistant… What would you like me to do?" Not a bad answer to the question; an answer from a blank context. The composition is now:
[ system ] scenario prompt + injection guard # deployment config
[ system ] debrief of the mission that just finished # live, while it is recent
[ user/assistant … ] the recent exchange # bounded window
[ user ] the operator's sentence, whole # unchanged
Four constraints shape it, and each is a defect avoided rather than a preference:
- The transcript is recorded in
EventEmitter, not in the orchestrator. The operator is usually replying to something a mission said, and mission narration andask_backnever reach the fast path — so a log kept by the chat path would omit the very sentence being answered. The emitter is the one place every turn on screen passes through, which also means the window and the screen cannot disagree. - Remembered operator text stays wrapped in
<user_input>. The injection guard asserts that everything inside that block is untrusted; history smuggled in unwrapped would make the assertion false for most of the prompt. - The debrief is a leading system message.
_to_anthropic_messagesjoins leading system messages into the top-levelsystemand downgrades any later one into a<system-note>user turn, so its position is load-bearing. - The window never opens on an assistant turn. A slice can begin mid-exchange, and the Messages API rejects a conversation whose first message is the assistant's — a long mission narration would otherwise have turned every following turn into a 400. Consecutive turns by one speaker merge first, so one mission's narration cannot fill the window either.
MissionExecutor.DEBRIEF_WINDOW_S (30 min) bounds the claim "the mission that just
finished"; Reset (abort_mission) clears the transcript alongside the cost session, because
the client wipes its own message list there.
And the operator's diagnosis is offered to the recovery path. classify_recovery (§8)
refuses an unrecovered failure — "no recovery lesson to store yet" — correctly, since the
trace holds a failure and no fix. The human's sentence is the missing fix, so the fast path
emits feedback_offer (§11.1) after a substantive post-mission turn and one click files it
verbatim through POST /api/agent/feedback (recovery.operator_lesson): no LLM call, no
classifier verdict, nothing billed, retrieved by the same lexical instruction match as a
distilled lesson. A click rather than an inference, because a stored lesson enters the
planner's grounding on every similar instruction afterwards — the heuristic only decides
whether to ask.
5.2 The loop¶
async def _run(mission):
processes = memory.find_processes(mission.instruction, k=3) # executor-driven, not LLM
messages = [system prompt (§5.5) + processes] + [user: instruction]
while mission.state in ("planning", "running", "paused"):
step_llm: response = await planner.chat(messages, tools=self._tools())
execute tool calls (possibly several in parallel via asyncio.gather)
append assistant msg + tool results to messages
if no pending work and no tool calls → nudge once, then fail with NO_PROGRESS
if awaiting events/user (LLM said so via wait_for_events) →
suspend on self._wakeup (asyncio.Event), set by:
· on_task_event(...) — task progress/trigger/done/failed
· on_user_turn(...) — user spoke/typed mid-mission
· the mission's own max_wall_clock_s deadline (checked at the top of the loop)
· ONLY while blocked on ask_user: agent.wakeup_timeout_s (default 60) →
inject a nudge naming the unanswered question
each wakeup: drain queued events → append as user-role messages (formatted §5.4) → next LLM step
finalize (§5.6)
Budgets (config, hard caps): max_llm_steps (default 40), max_wall_clock_s (default 1800),
max_cost_usd optional (computed from usage if the provider reports it). Exceeding one →
mission failed, summary explains the budget, all live tasks cancelled.
The loop is woken by events, not by a clock (BIN-456). It previously also woke every
wakeup_timeout_s (60 s) regardless, injecting a synthetic "still waiting; tasks: …" message so
the sleeping loop had something to react to. That was a cadence, and the message existed only to
serve the sleep — one planner turn per minute per waiting mission, saying nothing new. It also
collapsed two different states into one: "the mission made no progress" and "the mission is
asleep" looked identical.
The single exception is a mission blocked on ask_user, where the timeout is the only exit rather
than a cadence — nothing will arrive on its own, so without a nudge an unanswered question would
hang to the full max_wall_clock_s instead of the planner closing the mission out incomplete.
The implementation keys on whether the mission is awaiting an answer, not on which tool requested
the wait, so a safety stop's continue-or-abort is covered by the same rule.
Budget consequence: max_llm_steps is consumed less than before, since the idle turns that
burned it are gone. A mission that only ever waits now terminates on max_wall_clock_s.
5.3 The planner's tool surface (agent/plan_tools.py + MCP)¶
The planner sees, in one flat list:
- All MCP tools from the existing
ToolBackend(dotted names, unchanged schemas). - Local mission tools (executed in-process, never reach MCP):
| Tool | Effect |
|---|---|
set_plan(steps: [{id, title, tool?}]) |
Replace the mission plan. Must be the planner's first call. Emits mission_plan (§11.1). |
update_plan(steps) |
Same shape; used on replanning. Steps already terminal keep their status (match by id); new ids start pending; removed pending steps become skipped. |
mark_step(step_id, status: running|succeeded|failed|skipped, detail?) |
Manual status control for steps without a 1:1 tool call. |
say(text) |
Speak/emit to the user mid-mission via the existing SpeechOutput + emitter (chunked TTS path reused). |
register_reaction(event: {server, task_id?, type, match?}, call: {tool, arguments}) |
Add a prepared reaction (§6). |
wait_for_events(reason: str) |
Explicitly suspend until a real wake — a task event, an operator turn, or the mission deadline. The loop also auto-suspends when a step ends with running tasks outstanding and no tool calls. No periodic poke on this path (BIN-456). |
ask_user(question, options?, allow_free_text?) |
say + emit ask_back (§11.1) + suspend until on_user_turn. The answer is an ordinary operator turn, so both UI surfaces reply with send_message. |
finish_mission(outcome: succeeded|failed, summary: str) |
Terminal. |
Step linkage: every MCP tool call may carry step_id (the executor injects an optional
step_id property into each MCP tool schema, mirroring the existing response_text injection,
and strips it before the MCP call). When present: step → running on dispatch; for short tools →
succeeded/failed on return; for async-task tools the step stays running, linked via
task_id, and is resolved by task_done/task_failed events. Progress events update the
step's detail ("ETA 9 s · 82 %"). Steps and events both emit UI updates (§11.1).
5.4 Async tasks: recognition, registry, reconciliation (agent/task_registry.py)¶
- A tool result parsing as JSON with
task_id+state: "running"registers aTaskHandle{task_id, server, tool, step_id, mission_id, server_key}.serveris a display string andserver_keyis the address — the two are not interchangeable, and conflating them is what made reconciliation a no-op for two months (BIN-358, below).serveris the tool name's dotted prefix, so it is empty for the bare-named tools every robot server exposes;server_keycomes fromToolBackend.tool_server(name)and is what the backend routes on. Usehandle.routeto address the server andhandle.labelin text. - Incoming
TaskEvents resolve handles; formatted for the LLM as compact user-role messages:[task nav-7f3a navigation.navigate_to] trigger_fired: eta_below 10 (eta 9.4s)/[task …] failed DOOR_CLOSED (recoverable): The mailroom door is closed. suggestions: ask_reception, retry_after_s:300.progressevents update UI/step detail but are not appended to the LLM conversation (token hygiene). Thewait_for_eventstimeout summary that used to carry the latest one no longer exists on this path (BIN-456). - Reconciliation: on app startup and whenever a server reconnects (hook into the existing
reconnect loop in
services/tools/mcp.py), call itslist_tasks(states=["running"]); any registered handle missing there is synthesized into atask_failedwitherror_code: "SERVER_RESTARTED". If no mission is active, orphan running tasks on servers are cancelled (cancel_task) and logged. A server advertising only part of theget_task/cancel_task/list_taskstrio is logged at warning — the kit installs all three together, so a subset means something is broken — while a server with none of it is simply not a task server and is skipped silently.
A server key is not a tool prefix, and a tool name is not an address (BIN-358)
Reconciliation existed, was unit-tested, and had never once run against either
server that owns tasks. Everything downstream of it — SERVER_RESTARTED failures,
orphan cancellation — was dead for as long as it existed. Three distinct assumptions had
to be wrong at the same time, and each one alone was enough:
- Addressing. The tool was built as
f"{server}.list_tasks", but the argument is a server key (cfg["name"] or cfg["url"]), and only a named server's key doubles as its prefix. Robot servers deliberately have noname— bare tool names are a hard requirement of the planner's routing (integrating a robot §0) — so the key is a URL and the call became"http://rap-1:9220/sse.list_tasks". Memory, the one prefixed server, was the only one it reached; memory has no tasks. Resolve withToolBackend.server_tool(server, tool), never by formatting. - Dispatch. Even a correctly resolved name is ambiguous: every task server
advertises the trio under the same bare names, and the backend's tool→session index
is keyed by name, so it holds only whichever server connected last. Reconciling the
adapter would have cancelled grasp's tasks. Address one known server with
ToolBackend.call_on(server, tool, …). - Attribution. A handle's
serverwas the tool name's prefix —""for a bare tool — soby_server()matched nothing and reconciliation had no handles to fail even after the tool resolved. The same mistake silently disabled the two other sites that cancel a handle (safety stop, mission finalization), each of which built".cancel_task".
The general lesson is about the test, not the addressing: the existing suite passed because its fake modelled a prefixed server, which the live deployment does not have. A fake that shares the code's wrong assumption cannot fail. Model the naming the deployment actually uses, assert on what was called rather than what came back, and mutation-check each guard — the fix here needed nine separate reversions to be sure nine separate tests were load-bearing.
5.5 Planner system prompt (config file config/agent_prompt.yaml, hot-editable)¶
Contents: role ("you are the mission planner of a physical
robot…"), the planning protocol (set_plan first; keep steps coarse — one per physical action;
use triggers + prepared reactions for time-critical chaining; prefer prewarm before perception
needs; one navigation at a time), the event message format, failure-handling doctrine (read
recoverable + suggestions; replan, don't retry blindly; ask the user when stuck),
budget awareness, and the retrieved-process framing: "Past process documents follow. They are
advice from previous executions, not scripts — follow them while they match reality, deviate
when they don't, and mention deviations in your final summary."
5.6 Finalization¶
On finish_mission / budget kill / abort: cancel outstanding tasks (cancel_task each),
resolve never-run steps to skipped, freeze the blackboard's issue notes onto
Mission.notes (§4), emit mission_finished, say the summary (short),
memory.record_outcome(id, success, note) for every used process, hand the trace to teaching
(§8) if armed, clear the active mission.
It also runs on shutdown, reached by cancelling the mission task — and there every MCP call
above (cancel_task, stop_skill, record_outcome) is bounded by
observability.shutdown_call_timeout_s rather than by the deployed 180 s tool timeout. See §9a.
5.7 Stored plans — a second entry, one execution path (KOE-22)¶
A plan is what §5.3's run_skill_plan takes: an ordered list of skill calls with the
save / $ref data-flow and expect gates. Until KOE-22 one existed only for the length of a
single mission — the planner compiled it, the executor ran it, it was gone. A stored plan is
that list, named and persisted, so it can be launched again and produce the same tool calls
every time. The LLM becomes the tool that authors a plan, not a dependency of executing
one. A plan is the recipe; a mission is one execution of it, and the record holds only steps —
never runtimes, results or per-attempt state.
MissionExecutor.start_stored_plan(steps, title=…, plan_id=…) -> mission_id | None is a
second ENTRY, not a second execution path. It performs §5.1's mission set-up and then calls
the same _run_skill_plan, so the FSM, the per-step trace records, the pipeline events, the
surveillance scope and §5.6's finalization are the ones an LLM-planned mission uses. That is
load-bearing rather than tidy: BIN-416 is the bug where a mission that bypassed
run_skill_plan was silently unwatched, and a parallel runner would have reproduced it. Two
things differ from start(), both deliberate:
- No planner-credential refusal.
start()is right to refuse whenplanner.unavailableis set — a model has to write that plan. Here the plan already exists, and a brain with no API key being able to launch a shift's work is the entire point. - Teaching is never inherited.
Mission.teachingmakes §5.6 call the distiller, which is an LLM call — so a run that touches no model would touch one after every step had already run. The operator's armed flag is left unconsumed: they armed it for a mission they narrate.
The mission's step list is seeded from flatten_plan(steps), which makes display slot i the
same as mission.steps[i]; that mapping is why the step statuses can be driven from the run
here and not on the LLM path, where set_plan's coarse steps and the compiled skill steps
are unrelated lists.
REST surface (api/routes/plans.py, registered in api/router.py):
| Method | Path | What |
|---|---|---|
GET · POST |
/api/plans |
list (id, name, description, step count, timestamps) · create |
GET · PUT · DELETE |
/api/plans/{id} |
the full plan · update any subset of {name, steps, description, layout} · delete |
POST |
/api/plans/{id}/duplicate |
copy under a new name |
POST |
/api/plans/{id}/run |
launch; {overrides?} merges per-launch arguments by step index. Returns {mission_id, plan_id} |
The id is opaque (p-<hex>) and never the name, because rename and duplicate both have to
work. Since BIN-537 two plans may no longer share a label, though: the name is refused
case- and padding-insensitively on create, rename and from_mission, and duplicate takes the
first free (copy N). Two plans called "Bobbin vertical" are distinguishable only by that
opaque id, in a customer-facing view on a robot. Enforced in the route rather than as a UNIQUE
column deliberately — a constraint needs a migration, and a database already holding a duplicate
pair would then reject every write to either row, turning two confusingly-named plans into two
unopenable ones. PUT also honours an optional if_unmodified_since against the plan's
updated_at, so two tabs stop silently clobbering each other. Every record carries a
schema_version.
The plan language has three readers, and one file measures all three¶
agent/plans.py validates, agent/plan_executor.py executes,
frontend/src/lib/programming/planModel.ts edits — and
frontend/src/lib/programming/planShapes.fixture.json is the shared list of shapes each is
measured against. This is the design rule, not an implementation note: the language exists in
three places, so anything not pinned to that file is free to drift, and every defect BIN-537
found sat in exactly that gap.
The executor was the reader outside the contract until BIN-537, and the consequence was a
validator that checked key names and never values. max_iterations: "3x" passed validation,
was stored, passed validation again at launch, and then raised ValueError out of the
executor's unguarded int() — an operator reading a Python exception for a plan two validators
had blessed. Three rules follow, each of which was a defect before it was a rule:
- Values are typed and ranged, not just named.
max_iterations1–100,search.max_attempts0–20,optionala real boolean,expect/until/title/saynon-empty strings,timeout_sabove zero. The ceilings matter as much as the types: nothing bounded them, somax_iterations: 100000was a valid plan that ran its body a hundred thousand times against a robot — the step-list cap bounds the list, not the work. - The runtime clamps as well as the validator refusing. A validator-only bound is one that a
from_missionsave, a hand-edited row, or a plan stored before the check existed walks straight past. So the executor's coercions degrade to a plan failure and log, rather than raising — and the fixture is asserted in both directions, refused shapes included. - One copy of each bound. They live in the executor, which defines the language, and the
validator imports them. A second copy is exactly how a plan gets blessed with a value the
runtime then clamps, which is this issue one layer up.
_refsalready followed the same rule by mirroring_resolve.
A step may also declare its own wall-clock budget, timeout_s. Without one a single MCP call is
bounded only by the transport's 180 s, so a hung tool held the whole plan with the step showing
running and §5.5's watchdog unable to tell it from slow-but-fine. Exceeding it is a hard
failure rather than a failed gate, so a loop does not retry — retrying a hung tool is how one bad
call becomes max_iterations of them.
Schema 2 added the canvas layout, and a step id to key it by (KOE-38). Both are optional and the reasoning behind each is load-bearing:
- The coordinates are a plan-level
layout({"<step id>": {x, y}}), besidestepsand never inside one._STEP_KEYSis a closed vocabulary of what the executor reads, and it is the check that catchesexpectsinstead ofexpect— a typo that silently removes a step's success gate. Lettingx/ythrough would have bought a layout hint with that guarantee. - A step may carry an
id, and that is what the layout is keyed by. It is the only key correct under reordering:apply_overridesaddresses steps by 0-based index, which moves the moment anybody inserts or deletes one, andtitleis free text two steps may share. Refused on save unless it is a non-empty string that no other entry in the tree uses — an id two steps share cannot say which of them a position belongs to. - The id is optional everywhere, and every reader works without it. No planner-authored plan
has one (including one stored through §5's
from_mission), and no plan written before schema 2 has one. So/runandapply_overrideskeep their index key (an id-keyed override would work on some plans and silently miss on the rest),flatten_plan's display allow-list drops it so no canvas handle reaches the mission trace, and the editor writes one only onto a step that has a position, which is what keeps re-saving an untouched plan byte-identical. - The read is tolerant, the write is strict. The steps are the plan and the coordinates are a
convenience, so an absent, partial or nonsense layout — including a coordinate for a step that
no longer exists — opens and runs, with the block falling back to the derived layout;
plans.coerce_layoutis the one definition of what a stored coordinate is, and refusesNaN, infinities and booleans.POST/PUTmodel each pair asfloat, so nonsense is a 422 rather than a row in the database. Omittinglayouton aPUTleaves the stored arrangement alone — a rename from the Config panel must not discard it — and{}clears it. layoutis on the record, not on the list row. It is only ever wanted by the editor that is about to draw the plan, and the list view draws no canvas.- Adding the column needed a forward migration, which the store owns rather than this module
— §8a → Migration. Note that the row's own
schema_version(this record's) andstore.schema.SCHEMA_VERSION(the file's) are two different numbers doing two different jobs; KOE-38 moved both.
Validation runs on save and on launch, against the live tool inventory — unknown tools,
unknown parameter names and $save.path references no earlier step produces, all reported at
once with 400. The two are not redundant: save sees what was written, launch sees the stored
steps with this launch's overrides merged in, against the tools attached now. A plan saved
while the grasp service was up and launched after it detached fails there rather than
mid-motion. The inventory is ToolBackend.tool_definitions() — the robot's MCP surface and
only that, deliberately not the planner's _tool_defs(), which both adds mission-local tools
the FSM cannot dispatch and injects step_id into every MCP schema. It is the same list the
Programming view's skill library reads (KOE-24), so a plan cannot be authored against tools
the validator would then reject.
The validator recurses into loop bodies, if/then/else branches and per-step
search blocks, because all three execute real tool calls; checking only the top level would
bless exactly the plans that fail on the robot. It also enforces the executor's own nesting
rules — loops do not nest, branch steps are plain skill steps — since the executor's failure
for a nested loop is step is missing 'tool', reported against a step nobody wrote.
Where they live is the design decision, not the schema. A stored plan is a definition
(§8a): content a human authored and named. It is the plans table of the one brain store, and no
retention sweep may touch it. KOE-22 originally put it in a sibling data/metrics/plans.db
because that directory was the only path brainctl bind-mounted — the right local call, since it
provably survived a deploy, and the wrong global one, since the directory name then described a
third of its contents. BIN-531 consolidated it; §8a is the state model.
Managing plans from the Config panel is KOE-23 (Using the brain → Plans); authoring them by dragging skill blocks is the Programming view, KOE-24.
6. Prepared reactions (agent/reactions.py)¶
ReactionTable: list of {event_pattern, tool, arguments, once: true} entries scoped to the
mission. on_task_event consults it before the executor queue; on match it immediately
dispatches the tool call (fire-and-forget through ToolBackend) and appends a synthetic
note to the mission conversation: [reaction] eta_below fired → called perception.prewarm(...).
Pattern fields: server, type, optional task_id, optional match (subset-match on
payload.trigger). This is the zero-LLM-hop path for "prewarm perception 10 s before arrival".
7. Planner LLM — AnthropicLLM adapter (services/llm/anthropic.py)¶
An adapter implementing the LLMProvider port (chat + chat_stream with tools), selected by the
Hydra group llm_planner (the fast chat path uses group llm).
- SDK:
anthropic, clientAsyncAnthropic(api_key=...), key fromANTHROPIC_API_KEYvia the usual${oc.env:...}resolver,SecretStrin the schema. - Default model:
claude-opus-4-8(exact string). Request shape:thinking={"type": "adaptive"},output_config={"effort": cfg.effort}(default"high"),max_tokens=16000. Notemperature/top_p/top_kand nobudget_tokens— these 400 on this model family. - Translation: the app's
Messagelist → Anthropicmessages(system text → top-levelsystemwithcache_control: {"type": "ephemeral"}on the stable prefix — the system prompt + tool defs repeat every step, caching matters); appToolDefinition→{name, description, input_schema}; responsetool_useblocks → the app'sToolCallmodel (parseinputas dict, keepidfor thetool_resultround-trip);stop_reason "tool_use"→ has_tool_calls. Tool results go back as{"type": "tool_result", "tool_use_id": …, "content": …}user-role blocks — extend the app'sMessagemodel with an optionaltool_resultsfield if it can't express that today. - Streaming (
chat_stream) is required by the port: useclient.messages.stream(...)and yield text deltas; the executor itself uses non-streamingchat. - Retries/timeout: reuse
services/resilience.pyconventions;request_timeout_sconfig default 120 (planner steps can think).
# config/llm_planner/anthropic.yaml
_target_: robot_voice_chat.services.llm.anthropic.AnthropicLLM
api_key: ${oc.env:ANTHROPIC_API_KEY}
model: claude-opus-4-8
max_tokens: 16000
effort: high
request_timeout_s: 120.0
# config/llm_planner/groq.yaml — reuses the existing GroqLLM target (no new code), so a
# deployment can run single-provider.
The root defaults list carries - llm_planner: anthropic. The composition root builds the adapter
only when agent.enabled, and construction is lazy and failure-tolerant: if the key is missing, it
logs a warning, disables missions, and keeps the app up (chat-only dev deployments boot without an
Anthropic key).
8. Trace + teaching (agent/trace.py, agent/teaching.py)¶
TraceRecorder (always on): every mission appends to the mission_events table of the brain
store — one row per: instruction, retrieved processes (ids+scores), each LLM step (request size,
tool calls), each tool dispatch/result, each task event, each user turn, plan snapshots, final
outcome + summary + timings. This is observability and the teaching input.
A trace is a record (§8a). Retention deletes whole missions — the missions row and its
events together — past agent.trace_retention_days (default 30), at startup.
read() returns the same dicts the JSONL lines held (ts and kind beside the payload, not
nested under it), because mission analysis and recovery classification consume them: BIN-531 moved
where a trace lives, not what it is. Writes go through the store's queue, never SQLite on the
caller's thread — record is driven from the event loop by the emitter, and the file version did
synchronous open/write there (BIN-203).
Teaching: POST /api/agent/teaching {"enabled": true} (UI button) arms a one-shot flag; the
next mission records teaching: true. On finalization, teaching.py:
- Renders the trace into a compact text transcript (tool calls + args + result snippets + failures + fixes + user corrections; drop progress noise).
- One distillation call on the planner LLM with a fixed prompt (in
config/agent_prompt.yaml): produce a process document in exactly this markdown format — therobot-mcp-memorydoc format (frontmatter:idslug,title,task_pattern,tags, body sections## Steps,## Pitfalls & fixes,## Operator notes). Include the format spec verbatim in the prompt. memory.save_process(markdown)via ToolBackend; emitteaching_saved {process_id}; disarm. Distillation failure: log + emitteaching_failed, never crash finalization.
Recovery lessons: retrieval has exactly one door. A stored lesson enters the planner's
grounding on every similar instruction afterwards, so the conditions on reading one are part of
the contract rather than of a call site. MissionExecutor.recall_lessons is the only reader —
the mission prompt and the admin transparency demo both go through it, and a source guard in the
suite fails the build if a third one appears. It owns the recoveries feature lock, recovery_k
and the relevance floor; the store owns the robot scope. Two rules follow, and each was a defect
before it was a rule (BIN-545):
- Locked means neither written nor read. The feature gate belongs on retrieval as much as on the two write methods: lessons stored while a developer profile was active must not reach a customer profile's prompt after a switch.
- A lesson is scoped to the robot that learned it. Every episode is stamped with
ROBOT_PROFILEand retrieval filters on the same value through the same resolver — a tagged lesson is read only by a reader carrying that tag, an untagged one by any reader, and an untagged reader takes only untagged lessons. One resolver because a writer and a reader that disagreed would reject everything the instance had written, and "no lessons injected" is indistinguishable from the empty store this loop starts from.
The second rule generalises: a filter over a store that is usually empty needs a positive
signal when it withholds something (recovery_profile_filtered, with the counts) — otherwise
"working" and "rejecting everything" produce identical output.
8a. State model — records, definitions, beliefs (services/store/)¶
The vocabulary is robot-mcp-kit's; this section is the brain's declarations
Kind, Schema and the two guards moved to the kit in BIN-549 so four services share one
vocabulary, and BIN-574 widened it to four kinds (adding derived) and to per-column
granularity. The brain is the last consumer still running its own
services/store/kinds.py copy — adopting the kit's is
BIN-547's remaining work, not a second design.
The kit's spec
(robot-mcp-kit §4.9) owns the vocabulary and the guards; what follows
owns which kind each of the brain's tables holds, and the reasoning that produced the
taxonomy in the first place.
Everything the brain remembers is one SQLite file, data/brain.db, on the one directory
brainctl bind-mounts ($SHARED/brain/<name> → /app/data). This section owns what may live
there and what may not; the mount, the retention knobs and the boot line are
running-the-brain.md's.
Before BIN-531 there were three databases in a directory called metrics (the cost ledger plus
KOE-22's plans and KOE-26's palette, all there because it was the only mounted path), mission
traces as loose JSONL, recovery lessons as loose JSON with no query and no retention, and warnings
nowhere at all.
The rule: everything we write goes in a database¶
Pascal's decision, following the persistence audit (BIN-541 → BIN-546):
Read from three places only: the config tree, the env file, and the key store. Write everything to a database — never
json.dump/yaml.safe_dump/write_texta data file from application code. Every written thing has a retention policy, and something actually runs it.
"Database" is about the file type and the interface — a driver, a schema, transactions — not
about the data model. The engine is settled once, for every service: SQLite, one file per
service, behind a thin store class. Zero ops on a 4090 already running seven containers and on
a robot that is a container with no systemd, reset once or twice a week; backup stays one file
copy on a bind mount; and no network dependency for data whose whole job is to survive when the
network is down. A document shape, where one is genuinely wanted, is a TEXT column plus
json_extract() and an indexed generated column.
The line between config and data is authorship, and it is the useful test: a file a human is
meant to edit is config; anything the application produces is a row. So system_prompts.yaml
stays a file (config that wants a mount, BIN-539) while the operator's Context facts became a
table — the application writes them on the operator's behalf, through an API. The price table is
the case that needed splitting rather than deciding: see below.
A CI test enforces it per repo, because the rule decays in a month otherwise —
tests/unit/test_writes_go_to_the_database.py walks src/ and fails on a write-shaped call
(json.dump, yaml.safe_dump, write_text, open(…, "w"), os.replace of a temp file,
NamedTemporaryFile, fp.write(json.dumps(...))) outside an allowlist that is a dict keyed by
module path with a reason, not a set, so an exemption has to finish the sentence. It also fails
a stale exemption. Reads are not flagged, so the config loader and load_dotenv need no entry.
The brain's copy is the reference the other three repos lift (BIN-547 → BIN-548/549/550).
The kinds, and why the split does the work¶
Asking "should all of this be in a database?" is unanswerable while the state is one heap.
Separated, each kind answers itself. services/store/kinds.py declares which kind every table
holds, and a table that declares none fails a test.
| Kind | What it is | Lifecycle | Tables |
|---|---|---|---|
| Record | what happened — append-only, historical, never invalidated by anything the robot does afterwards | persist indefinitely; pruned by retention | missions, mission_events, recoveries, events, events_daily, logs, model_prices |
| Definition | what a human authored — intent, not observation | persist indefinitely; removed only by an explicit human act | plans, skill_visibility, spend_limits, context, store_meta, notice_mutes |
| Belief | an observation with a validity window | must never survive a restart as fact — emptied at every boot | notices |
| Derived | a projection of another table, with no content of its own — a search index | never pruned; its owner rebuilds it from its source | none in the brain |
A mission from last month is exactly as true as it was. A stored plan does not become false because the robot rebooted. But after a reboot the held object is not old, it is unknown — and that difference is what the third row is for.
The third row had to stay empty until BIN-592, and what changed is the observer. A belief needs the identity of the epoch it was observed in. For anything the robot observes — a held object, a pose — that epoch has to originate at the robot, and the world service still exposes none (BIN-213's scope; BIN-533 waits on it). Persisting one of those would hand it a longer lifetime with no way to refute it, which is BIN-396's phantom held object.
notices is different because this process is the observer: "the robot exposes no stop
tool" is the brain's own reading of the tool surface it is holding, so the epoch it needs is the
brain's own boot, which the brain trivially knows. That is what lets it be stamped, and what
makes the boot flush safe rather than lossy — nothing has re-checked the condition after a
restart, so the row goes and is re-raised on the next observation.
Invalidation is not retention, and the two are deliberately different code. Retention deletes
by age, and a belief is not made false by time passing; _invalidate_beliefs deletes by
epoch and does not route through the prune chokepoint. Conflating them would let a knob called
"retention" acquire the power to invalidate. Note also that a belief table is absent from
PRUNED_TABLES and a guard asserts the overlap is empty.
And the mechanism is chosen by whether the row has a definition half. A notice is belief all
the way through, so deleting it is honest. A waypoint is part definition — a human named the
place — so BIN-533 will resolve its pose to unknown while keeping the row, precisely so the
name survives to say "re-teach me". Same taxonomy, two mechanisms.
The fourth row is empty here for a different and less interesting reason: the brain has no search
index. It was added to the shared vocabulary for robot-mcp-memory's processes_fts
(BIN-574) — the first three all answer "what does
elapsed time do to this content?" and a projection has none to answer for, so pruning one
independently of its source is wrong in both directions.
events_daily is a RECORD, not derived, and the distinction is the useful part of the
definition. It is a roll-up of events — so it looks like a projection — but the events rows
it summarises are deleted by the same sweep that writes it. Once that happens the aggregate cannot
be reconstructed, which makes it content in its own right. Derived means rebuildable from a live
source, not merely "computed from something else".
The decision procedure for a belief: can it be re-observed?¶
- If it can be re-observed, do not persist it. Reading is cheaper than remembering and always
correct. BIN-499 settled this for the held object: observation supersedes memory on every read,
so a belief that outlived a reboot is refuted the first time anyone looks — and until then it
blocks missions, which is BIN-396. The same holds for poses, joints, the scene inventory and
camera frames:
binabik-world-stateanswers on demand. Persisting_armswould be a regression with a longer lifetime, not an improvement. - If it cannot be re-observed, persist it with the identity of the epoch it was observed in. BIN-213 is the case that does not dissolve: a taught waypoint pose is map-frame and boot-relative, there is nothing to compare it against, and only a human can re-teach it.
So the honest answer to "should it all be in a database?" is no, and the exceptions are
principled rather than accidental. Most of the brain's in-memory state — the blackboard, the
conversation log (BIN-350's bounded window), _arms/_carrying — is in memory correctly, and
a consolidation that swept it in would make the brain worse.
Invalidate, do not delete¶
The rule for a belief, when there is one: prune records past retention, never prune definitions,
and for beliefs stamp the epoch and resolve a foreign epoch to unknown rather than removing
the row. The waypoint store already does exactly this, and it is the worked example of a
definition and a belief sharing one row — a row in r1-abstraction's own SQLite store since
BIN-550, and one whose kinds are declared per column since BIN-574:
kinds={"waypoints": Kind.DEFINITION}, # what retention reads: a DELETE takes the row
columns={"waypoints": {
"name": Kind.DEFINITION, "name_lower": Kind.DEFINITION,
"x": Kind.BELIEF, "y": Kind.BELIEF, "yaw_deg": Kind.BELIEF,
"session": Kind.BELIEF, # the epoch stamp IS part of the observation
"saved_at": Kind.RECORD,
}},
The name is a definition; the pose is a belief. Deleting on boot would throw away the fact that a place called "Table right" exists and needs re-teaching, which is strictly more useful than silence: stale can say "you taught this in an earlier session; re-teach it", absent can only say nothing.
The table-level kind is not a second judgement. It must be exactly conservative_kind() of the
columns — strongest wins, RECORD < BELIEF < DEFINITION — and the kit refuses a Schema where it
is not. Both directions of getting it wrong are real: declaring this row a RECORD would let a
retention rule delete it to expire the pose, taking the name; declaring it a BELIEF is
unprunable and therefore passes a "may retention delete this?" check, yet would resolve the
name to unknown along with the pose — the one thing the design exists to prevent. That is why
the rule is an order and not a boolean. Schema.columns_of_kind("waypoints", Kind.BELIEF) is what
the unknown resolution reads, and the definition columns are exactly what it must leave alone.
Two epochs, and they are independent. The brain restarting loses in-memory state, so a naively
persisted belief would survive and lie. The robot restarting rebuilds the map frame, so poses
mean something else — and the brain may not have restarted at all. One "boot id" cannot cover both:
the robot's epoch has to originate at the robot and travel through the snapshot, and
binabik-world-state's snapshot carries no boot/session/uptime identifier today (verified during
BIN-499). Exposing one is BIN-213's scope, which is why no belief is persisted yet and three tests
fail if one arrives: nothing may be declared a belief, every table must declare a kind, and the
live belief holders must provably write nothing.
The one document that had two authors: the price table¶
The rate table is both config and written data, which is why it took a decision rather than a
classification (BIN-465 → BIN-546/BIN-547). BIN-465 deliberately made it operator-editable on a
mount so a rate correction reaches a running brain with no rebuild — and then had a background
loop rewrite that same file every day. One document, two authors, and only a convention
(providers: is the machine's, overrides: is yours) keeping them apart.
Split along authorship:
| Kind | Home | Written by | |
|---|---|---|---|
endpoints:, overrides:, aliases: |
config | pricing.yaml, mounted, read-only to the brain |
a human, by hand |
providers: |
config (the seed) | the same file | a one-shot operator script, committed to the repo |
| the fetched catalogue | record | model_prices in brain.db |
the daily refresh loop |
Resolution is endpoints: → overrides: → model_prices → providers:, merged per token
class, so one hand-written line leaves the other three tracking the daily refresh. The seed
stays at the bottom rather than being deleted, because it is what prices a brain that has never
fetched: no egress, refreshing switched off, or the first minute after a start. unpriced is
unchanged — a missing rate is loud and never zero.
model_prices is a record, not a definition (nobody authored the numbers) and not a belief (a
quoted rate is not invalidated by a restart, and resolving one to unknown after a reboot would
un-price the ledger). It is deliberately not prunable, for events_daily's reason: outliving
the fetch that produced it is the point, and "a failed fetch leaves last-known-good rates in
place" stops being true the moment age can delete a rate. Its retention policy is the upsert —
one row per (provider, model), replaced in place.
Two constraints the store inherits¶
- BIN-203 — never touch SQLite on the caller's thread.
BrainStore.enqueueputs the write on a queue drained by a dedicated writer thread, and drops (bounded, counted, warned about on close) when the queue is full. That wasSqliteMetricsRepository's, and it moved down because it is the pattern every hot write path needs rather than a detail of metrics: a synchronous insert from a structlog processor once stalled the loop badly enough that a zero-IO REST handler took 10 s. Cold paths — operator CRUD, retention — stay synchronous and undroppable, because the queue's overflow behaviour is right for a diagnostic and wrong for a plan somebody just named. - BIN-347 — a free read stays free. Nothing in the store may reach a paid path, and it cannot: the package imports no world-state client, no HTTP client and no provider SDK, which is asserted structurally rather than left to its current callers. If a later phase needs the robot's epoch, it arrives through a caller.
Migration¶
Startup folds the five old homes in, once, and parks each source in a migrated/ subdirectory
rather than deleting it. It reads every source through SQLite, never by copying files:
plans.db was measured at 4 KB with an 82 KB write-ahead log, so a copy of the .db alone would
have moved a database with no rows and — on that file — no schema either. Idempotence is matched to
what each table can key on: a natural primary key (INSERT OR IGNORE), a per-mission skip, and for
events, which has nothing to collide on, a store_meta marker written inside the same
transaction as its rows. A store written by a newer build is refused rather than written
through.
Changing the schema afterwards needs a forward step, and SCHEMA cannot be it. Every
statement in the DDL is IF NOT EXISTS, which is what makes replaying it on an already-migrated
file a no-op — and exactly what makes it useless for a table that already exists. A column added
to CREATE TABLE IF NOT EXISTS plans reaches a fresh brain.db and never reaches the deployed
one, where the rows are. So each SCHEMA_VERSION bump carries a step in _MIGRATIONS, keyed by
the version it upgrades to, run in ascending order for any file below it and before the DDL
is replayed, because an ALTER TABLE has to see the table as it currently is. A fresh file
(version 0) skips them — SCHEMA already creates everything at the current shape, and altering a
table that was just created with the column would fail. Every added column carries a DEFAULT
that reads as "this row predates it", so the reader stays tolerant rather than clever.
KOE-38's plans.layout is the first such step, and the version gate is what makes it safe to
re-run: a second ALTER TABLE for a column that is already there is an error, and it would take
the whole brain's startup with it rather than one endpoint. Both halves are tested against a file
built by hand with the version-1 DDL — a fresh store passes with the migration deleted, and a
fixture derived from the current DDL would grow the column by itself and stop testing anything.
A NEW TABLE is not a bump, and getting that backwards is expensive in both directions. The
whole job of the version is to decide whether a build may open a file, and the only answer that
costs anything is the refusal — a container that will not start. CREATE TABLE IF NOT EXISTS
already reaches a deployed file, because the DDL is replayed on every open; and a build that
predates a table reads every table it does know and never writes a column it does not, so
additive DDL is readable by an older build. Bumping for one would make every rollback a boot
failure in exchange for nothing. BIN-547 added context and model_prices with no bump for
exactly that reason. The rule, stated once: CREATE TABLE IF NOT EXISTS needs no version; an
ALTER TABLE needs one.
The mechanism is four things — SCHEMA_VERSION, the store_meta.schema_version row,
_MIGRATIONS keyed by the version it upgrades to, and the SchemaTooNew refusal — and it is
what BIN-548/549/550 lift alongside the write guard. Its own tests are the ones worth copying:
emptying _MIGRATIONS must leave the column absent (otherwise the DDL replay is quietly doing
the work and the forward-migration test proves nothing), the steps must run in ascending order,
and the refusal must happen before anything is written.
The steps are DDL strings rather than callables, deliberately: a step that has to read rows to reshape them would need a function, and BIN-546 struck data migration from the whole programme — every store that moves creates its table and drops its file reader, so nothing has old rows to reshape. Widen the type the day a step needs the connection.
Do not confuse this number with a record's own version. store.schema.SCHEMA_VERSION says
what the file holds; plans.PLAN_SCHEMA_VERSION, on every plan row, says what shape a plan record
is in (§5). They move for different reasons and a change can move one, the other, or both.
9. Barge-in: safety stop → pause → confirm¶
The rule is halt motion immediately, then ask.
- The
stop_pipelinesocket event and a spoken "stop" (fast-path classification gets astop_missionlocal tool; the mission-routed turns get the same via the planner — but do NOT wait for the planner: the executor pre-scans mission-routed user turns for a config word listagent.stop_words(["stop", "halt", "stopp", "arrête", "fermati"]) and acts before the LLM). - On stop: (1) for every server flagged
motion: trueinmcp.yaml,cancel_taskall its live tasks and call itsstop_navigation-class tool if configured (safety_stop_toolper server entry); (2)output.interrupt()(drop queued speech); (3) mission →paused, steps of cancelled tasks →failedwith detail "safety stop"; (4)say"Stopped. Should I continue or abort?"; (5) suspend. - Next user turn while
pausedgoes to the planner with a system note: mission is paused after a safety stop; user reply follows; eitherupdate_plan+ continue, orfinish_mission(failed/ cancelled). Continue = replan from current state (the pose is whatever it is now).
9a. Shutdown — what SIGTERM does to a running brain (services/shutdown.py)¶
The same family as §9: something has to stop, and the robot must not be left executing a goal with nobody behind it. The difference is that here the brain is not staying to ask.
brainctl stops a container with SIGTERM and a 30 s grace period before the daemon SIGKILLs
it (BIN-535; the operator's half is Running a brain
§Stopping). Everything below is what the app does with
those seconds. Before BIN-552 the answer was "possibly minutes, unbounded" — executor.aclose()
awaited the cancelled mission task with no timeout, and _finalize's robot calls ran at the
deployed call_timeout_s of 180 s.
One deadline, per-step caps, and a reservation. Every awaitable step runs under one
ShutdownBudget (observability.shutdown_budget_s, 20 s). Per-step caps are maxima; the budget
is the hard total. The mission teardown's cap is reserved from the start, so a slow drain
cannot starve the step that records finished.
The order is a durability argument:
| # | Step | Cap | Why there |
|---|---|---|---|
| 1 | halt the robot | shutdown_call_timeout_s |
before any draining — nothing else will stop a motion this brain started |
| 2 | drain the in-flight pipeline turn | shutdown_drain_timeout_s |
§19.4, pre-existing |
| 3 | cancel background loops, stop the pollers | 2 s each | local; may not touch the reservation |
| 4 | executor.aclose() → §5.6 finalization |
shutdown_teardown_timeout_s |
records the finished trace event |
| 5 | close the store | unbounded, always runs | lands the queued writes, reports brain_store_writes_dropped, truncates the WAL (§8a) |
| 6 | close the tool backend and recorder | what is left | after durability, deliberately |
Step 6 is last because McpToolBackend.aclose blocks the event loop synchronously, so no
wait_for can interrupt it — ahead of step 5 it was up to ~10 s of best-effort socket teardown
standing between a mission's trace and the disk.
SIGTERM halts, and only the robot this brain is driving. halt_for_shutdown fires the
capability stop tool when — and only when — a mission is active here. Six instances share a
host and stop raises the robot's cooperative flag, so an unconditional halt would let one
instance's deploy stop a colleague's mission. clear_stop is not called afterwards: the next
plan to run clears it, under a brain that means to move the robot.
A mission ended by a deploy stays cancelled — MissionState is shared with the frontend —
but its trace summary says so, which is what distinguishes it from an operator pressing Reset.
And a teardown that overruns anyway is not the end of the story. Its terminal event is lost,
and the startup sweep writes one on the next boot: services/store/lifecycle closes out every
mission holding a mission_started and no finished with state: "interrupted" — a word no
_finalize can produce, stamped at the mission's last known moment rather than at boot, with
missions.updated_at restored so the Missions view is not reordered. The count goes on the boot
line (interrupted=N), so interrupted=0 reads as "the last stop was graceful". This is also
the backlog cleanup: every mission alive during any deploy before BIN-535 is in that state,
because docker rm -f sent no SIGTERM at all.
10. Config¶
# config/config.yaml — defaults list
- llm_planner: anthropic # NEW group
agent: # NEW section (Pydantic AgentConfig, validated)
enabled: true
max_llm_steps: 40
max_wall_clock_s: 1800
wakeup_timeout_s: 60 # nudge interval while blocked on ask_user ONLY (BIN-456)
trace_retention_days: 30 # delete whole missions older than this at startup
recovery_retention_days: 0 # 0 = keep every learned lesson forever
stop_words: ["stop", "halt", "stopp"]
prompt_file: config/agent_prompt.yaml
memory_server: memory # name of the process-store MCP server ("" disables retrieval)
observability:
# ONE store for everything the brain remembers (§8a). MUST be under `data/`, the only directory
# brainctl bind-mounts ($SHARED/brain/<name> → /app/data); anywhere else is erased by every
# `brainctl update` (BIN-240, BIN-461, BIN-531).
brain_db_path: data/brain.db
log_persist_level: WARNING # records at/above this reach the `logs` table — AND the
# admin Logs page, which reads that table since BIN-592
log_retention_rows: 5000 # newest N persisted log records kept at startup
# The SIGTERM budget (§9a). Must fit inside brainctl's BRAIN_STOP_TIMEOUT (30 s) together with
# the ~3 s synchronous store close; the brain checks that at boot and warns if it stops fitting.
shutdown_budget_s: 20 # hard total for every awaitable teardown step
shutdown_drain_timeout_s: 10 # the in-flight pipeline turn
shutdown_teardown_timeout_s: 10 # executor.aclose() — reserved from the start of the budget
shutdown_call_timeout_s: 5 # each robot/memory call while shutting down (NOT call_timeout_s)
config/tools/mcp.yaml server entries gain optional keys (extend McpServerConfig):
In deployment the server list is generated from a robot manifest (robots/<name>.yaml) by
gen_mcp_config.py; the entries below show the McpServerConfig optional keys. A current R1
attach looks like:
servers:
- name: r1-abstraction # the L4/L5 contract over galaxea
transport: sse
url: "http://rap-1:9220/sse"
motion: true # included in the safety stop
safety_stop_tool: stop # called unconditionally on stop
- name: grasp # the L3 grasp skill
transport: sse
url: "http://rap-1:9210/sse"
- name: perception # ambient "what do you see" buffer (opt-in on the robot)
transport: sse
url: "http://rap-1:9202/sse"
event_secret: ${oc.env:PERCEPTION_EVENT_SECRET} # enables watch_event ingress for this server
- name: memory # central fleet/process memory (default)
transport: sse
url: "http://phyai4090:9103/sse"
- name: episodic
transport: sse
url: "http://rap-1:9203/sse"
blocking_tools: ["grasp"]
Every ${…} above is substituted by the launcher, and an entry whose URL variable stays unset is
dropped by gen_mcp_config.py — that is how the opt-in servers disappear from an instance that
did not ask for them. Which variable comes from where (shared vs per-instance) is a brainctl
concern: Running a brain.
.env.example gains: ANTHROPIC_API_KEY=, PERCEPTION_EVENT_SECRET= (generate once per
fleet — python -c "import secrets; print(secrets.token_urlsafe(32))"; it is shared in
brain.env and the same value goes into every robot's perception-buffer launch env). In local dev the robot is a single g1_robot stdio server
instead of r1-abstraction + grasp.
11. Frontend: mission pipeline view + Teach button¶
11.1 Socket.IO events (server → client; declared in types/events.ts)¶
| Event | Payload |
|---|---|
mission_started |
{mission_id, instruction, teaching} |
mission_plan |
full snapshot: {mission_id, state, steps: [{id, title, tool, status, detail, task_id}]} — emitted on set_plan/update_plan AND on every step change (snapshot > diffs; trivially correct UI) |
mission_event |
{mission_id, kind: tool_call|task_event|reaction|say|user_turn, text, ts} — the activity log line |
mission_finished |
{mission_id, outcome, summary} |
ask_back |
{mission_id, question, options, allow_free_text, ts} — a structured clarification; the mission is blocked until an operator turn lands. mission_id is load-bearing in the UI: a live question is scoped to the mission that asked it, so an unanswered one expires with its mission instead of blocking the next (BIN-160). Rendered as a chat card and as the Visual view's interrupt |
feedback_offer |
{mission_id, instruction, text, ts} — the operator has just explained why the last run went wrong, and the brain offers to keep it as a lesson (BIN-350; see §5.1). One click → POST /api/agent/feedback, which files the sentence verbatim. Nothing is stored until they click, and the card is absent under a profile with recoveries locked |
teaching_state |
{armed: bool} / teaching_saved {process_id} / teaching_failed {reason} |
visual_stream_available |
{enabled, reason?, ts} — whether this viewer should try negotiating a WebRTC head-camera stream (BIN-262). Emitted on entering the Visual view, before any offer. To that sid alone, like the two below: signalling is 1:1 by construction, and an SDP answer names one viewer's session. reason is present exactly when enabled is false and names which side declined — "off in this brain" (video_stream.enabled, which defaults to true since 2026-09-11 — so this reason means somebody set it) and "this robot advertises no stream_start" (STREAM_ENABLE) send an operator to different hosts |
visual_stream_answer |
{sdp, ts} — the robot's complete SDP answer, relayed verbatim. The brain never parses it; it is a broker, not a peer. No trickle ICE, so nothing follows this |
visual_stream_failed |
{reason, ts} — no stream for this viewer, in words an operator can act on. Deliberately not an error: the view has not broken, it stayed on the ~0.7 s visual_frame poll, and a red banner over a working-if-slow camera trains people to ignore banners |
Client → server (BIN-262; before it there were none, and the teaching toggle is still REST
while stop still reuses stop_pipeline):
| Event | Payload | Ack |
|---|---|---|
visual_stream_offer |
{sdp, type} — one complete SDP offer, post-gathering |
{accepted, reason} |
visual_stream_stop |
{} — release this viewer's encode |
— |
Both are ungated: not _require_control, not _visual_exec, and not
Feature.PLANNER. Watching has always been ungated (see visual_mode), the sender's
shared-encode tee removed the cost argument for restricting the stream to the controller, and a
camera view is not a grounded goal for the planner — showing the picture faster does not change
what anyone can command. visual_stream_offer returns the established {accepted, reason} ack
envelope because the answer arrives on a separate event: without it a lost offer is
indistinguishable from a slow one, which is BIN-197's shape.
These are the mission events only. The base pipeline events (status, transcription,
response/response_chunk, tool_start/tool_result, error) are defined by the backend's
own socket layer — read them there. One behaviour worth repeating here because the header UI
depends on it: status is also emitted to a client
on connect (BIN-121), so a fresh page load knows the real resting state — listening for an
always-on VAD mic — instead of assuming idle until the first turn completes.
11.2 UI¶
MissionPanel (rendered above/inline with the conversation feed whenever a mission is active or
recently finished; Zustand missionStore fed by the events above):
- Pipeline view — a vertical list (timeline) of plan steps, each a card with title, tool name
(mono, dimmed), live
detailline, and a status color:
| status | color treatment |
|---|---|
pending |
neutral gray, hollow dot |
running |
blue, pulsing dot + subtle progress shimmer; show detail (ETA/%) |
succeeded |
green, check icon |
failed |
red, ✕ icon; detail shows error_code + message |
skipped |
amber/strikethrough |
Use semantic Tailwind tokens consistent with the existing theme system (data-theme aware),
not hard-coded hex. On update_plan the list re-renders from the snapshot — steps may appear/
reorder (replanning is visible to the user, which is the point).
- Activity log — collapsible feed of mission_event lines under the pipeline (tool calls,
trigger fires, reactions, says).
- Header row — mission instruction, state chip (same color language), elapsed time, and a
Stop button (emits stop_pipeline). When paused: highlighted "Paused — continue or
abort?" with two buttons that just send_message("continue") / send_message("abort").
- Teach button in the main Header: toggles POST /api/agent/teaching; armed state shows a
pill ("Teaching next mission"); on teaching_saved show a toast with the process id.
- mission_finished collapses the panel into a summary card in the conversation feed.
GET /api/agent/missions/{id} lets the panel rehydrate after a page reload (fetch on mount if
the store is empty but /api/agent/missions reports an active mission).
11.3 The Programming view — authoring a plan with no planner (KOE-24)¶
A third top-level view beside Chat and Visual Command (uiStore.appMode: 'chat' | 'visual' |
'program', switched from the header's three-segment control). Chat and Visual both ask the
planner LLM to produce a plan; this one has no model in it at all — the operator composes the
step list directly, saves it through the plan store (KOE-22) and launches it. That is what makes
a run deterministic, inspectable, repeatable, and independent of provider availability and cost.
| Piece | Module | Responsibility |
|---|---|---|
| Skill library | components/programming/SkillLibrary.tsx |
The live tool inventory, grouped by server, searchable; HTML5 drag and an Add button (the keyboard path, and the one a jsdom test can drive) |
| Canvas | components/programming/PlanCanvas.tsx |
Blocks, connections, pan/zoom — React Flow (@xyflow/react, MIT) |
| Parameter form | components/programming/SchemaForm.tsx + lib/programming/schemaForm.ts |
Controls derived from a tool's MCP input schema; a per-field literal↔$reference toggle |
| Plan model | lib/programming/planModel.ts |
Blocks → running order, validation, draft ⇄ PlanStep[]. Pure: no React, no canvas, no network |
| Network | lib/programming/api.ts + hooks/usePlans.ts |
GET /api/tools; the plan writes and the launch. The list read is KOE-23's shared ['plans'] query, so a save here refreshes the Config panel's card. A refusal is relayed whole: KOE-22 answers detail: {message, problems: […]} and reports every problem at once, so showing only the first — or the message alone — would throw away the part that says what to fix |
| Editor state | stores/programStore.ts |
The draft, the selection, dirty, the launched mission id |
Three invariants, each with a test that was watched failing.
- No model call, asserted on the wire. The view's whole network surface is
PROGRAMMING_ENDPOINTS—/api/toolsand/api/plans.ProgrammingView.no-llm.test.tsxdrives mount → author → save → launch against afetchthat throws for any other path (importing the allow-list from the module under test, so it cannot drift) and asocket.emitspy that must never fire, sincesend_messageis the other route into the planner. Per BIN-347: assert what was called, not what came back. - The library is the live inventory. Attaching a server makes its tools appear and detaching one stops them being offered, with no frontend change; tested by rendering the same component against two inventories.
- Forms are generated, not written. No per-skill code exists. A skill that gains a required parameter grows a required field and a validation error the day its server advertises it; tested by rendering one schema, then its successor.
Linear sequences only. plan_executor runs an ordered list, so a branch, a merge or a loop
on the canvas is an error, not something to flatten — a plan that ran in a different shape from
the one on screen is worse than one that refuses to save. Offering branching is a follow-up
against the executor first and the editor second.
Launch reuses §11.2. POST /api/plans/{id}/run returns a mission id; the view renders that
mission through the same MissionCard, and maps its steps back onto blocks positionally,
refusing to paint anything when the step list does not line up with the canvas.
Block positions are persisted, sparsely (KOE-38 — the record side is in §5). Only the blocks
somebody moved are stored; canvasLayout.deriveLayout answers for the rest, which is every block
in a plan written before schema 2. The positions live on the draft, not in PlanCanvas, so a
move marks the plan dirty and the unsaved-changes guard fires — a stored position discarded on
the way out of the view is worse than one that was never stored. canvasLayout.positionCommits
drops a change that puts a block where it already is, so the guard cannot fire on a plan nobody
touched.
planModel.planWriteFrom produces the steps and the layout in one walk, deliberately: the
layout is keyed by the id written onto the step, and two functions deciding that key separately
is a save whose coordinates name ids the steps do not carry — which looks exactly like "positions
are not saved" and is invisible until somebody reopens the plan.
Position still means nothing to the running order. orderBlocks decides it and reads no
coordinate. Note that a mouse drag cannot be synthesised in this jsdom (React Flow drags through
d3-drag on window, which does not deliver to a listener added with window.addEventListener) —
but the library moves a selected node with the arrow keys through the same onNodesChange, and
that is what the wiring tests drive. See components/programming/reactFlowTestEnv.ts.
The view is lazy-loaded so the chat-only bundle does not carry the canvas.
11.4 Tool inventory — GET /api/tools¶
The library's source (api/routes/tools.py). Composes ToolBackend.servers_detail(), which
reads the same _tools_by_server table ToolBackend.tool_definitions() flattens — and therefore
the same surface MissionExecutor.skill_inventory() hands the KOE-22 plan validator, and the same
names plan_executor resolves a step's tool against. One inventory, so the library cannot offer
a block the validator would then reject; a unit test asserts the two name sets are equal rather
than leaving it to coincidence.
{"data": {"servers": [{"key", "name", "url", "transport", "connected", "prompt_fragment",
"tools": [{"name", "description", "blocking", "input_schema"}]}],
"tool_count": 1,
"injected_args": ["response_text", "step_id"]},
"error": null}
input_schema is the tool's own schema: the arguments the brain injects for the planner
(response_text, §5.3's step_id) are stripped, because they are stripped again before the call
reaches MCP and a generated form must not ask an operator to fill in a field the skill has never
heard of. Those names live once, in services/tools/base.py
(INJECTED_ARG_KEYS / strip_injected_args), rather than as literals at each injection and
stripping site.
The local mission tools of §5.3 are deliberately absent: they are the planner's controls, executed in-process, and not steps a stored plan can contain.
Always 200, ungated. A brain with tools=none, or with every server down, answers with an empty
or disconnected inventory — "nothing is attached" is the library's subject matter, not an API
failure. The route touches no LLM and dispatches no tool call; tests/integration/
test_tools_routes.py installs a provider that raises on every entry point and asserts the
request still succeeds.
12. Deployment — where it is documented¶
The per-server hand-cloning + systemd model this spec was written against is gone. Deployment is one command per host and is not documented here:
- Brain host — Running a brain (
brainctl: every command and flag,brain.env, central memory, HTTPS). - Robot host — R1 robot stack (
r1ctl: the galaxea servers,r1-abstraction,grasp-service, and the opt-in perception/episodic/vision windows). - From zero on your own machine — Set up your own stack.
Two properties of this app are worth stating because the runbooks rely on them: order between
hosts doesn't matter — the brain's 30 s reconnect loop adopts late servers and reconciliation
(§5.4) handles restarts — and secrets are never in a manifest, which references ${VAR} and
resolves it from the launcher's env file.
13. Tests¶
Unit (tests/unit/agent/), all with fake LLM/ToolBackend/clock:
- Executor happy path: scripted planner (set_plan → navigate_to (async) → wait → trigger → reaction fires → task_done → finish) — assert step statuses, emitted socket events, conversation contents, summary.
- Replanning:
task_failed DOOR_CLOSEDwakes the loop; planner'supdate_planpreserves terminal steps, marks removed ones skipped. - Budgets: step/wall-clock caps fail the mission and cancel live tasks.
- Routing:
start_missiontool call hands off; mission-active turns route toon_user_turn; second mission rejected with busy message. - Barge-in: stop word pre-scan cancels motion tasks + calls
safety_stop_tool, pauses, resumes via planner. - Task registry: TaskInfo-shaped results registered; reconciliation synthesizes
SERVER_RESTARTED; orphan events dropped. - Events route (integration, TestClient): valid HMAC accepted; bad signature/skewed timestamp/unknown server → 401; duplicate event_id → deduped; reaches a stub executor.
- AnthropicLLM (respx/mocked SDK): message+tool translation both directions; no sampling params in requests; tool_result round-trip; streaming yields deltas.
- Teaching: trace → distillation prompt contains failures/fixes; save_process called; distillation error doesn't break finalization.
- Frontend:
npm run typecheck+ a MissionPanel render test over a snapshot fixture covering all five step states.
14. Acceptance criteria¶
- All backend gates (
pytest,ruff check,ruff format --check,mypy,import-linter) and frontend gates (typecheck,build) pass. import-linterenforces thatagent/is application-layer — it may importservices/**/base.pyports and models, never concrete adapters.- The bring-up runbook (§12) runs end-to-end on a dev machine against the real capability servers
(perception with
frame_source=folder). - The mailroom scenario works: plan visible in UI with correct colors, prewarm fires via a prepared
reaction at
eta_below:10before arrival, failure injection (mapfailures:entry) triggers visible replanning, and the teaching round-trip stores and later retrieves the process. - Chat-only
RVC_DEPLOYMENT=dev(no Anthropic key, no servers) boots clean. - The Programming view (§11.3) works with every model provider unreachable. A plan can be authored, saved, reopened, duplicated and launched with no LLM call and no planner socket message anywhere in the journey — asserted on the wire, not read off the code.
Appendix A — Repository & tooling conventions¶
Scope. The brain is the reference implementation of those conventions, and the MCP client, not a server — the MCP-server rules describe the companion servers it attaches. Config shape: the reference Hydra repo; the guidelines' Configuration and DI sections are written from it.
The full list — repo structure, uv/ruff/mypy/pytest tooling, the pre-commit hook set,
ports-and-adapters, the two config shapes (Hydra where implementations are selectable, a frozen
env dataclass where they are not), secrets, structlog, the CI gate, and what an MCP
server owes its consumer (transport choice, blocking tools, the ~30 s call timeout, reconnect
tolerance, the hardened systemd unit) — is
Coding guidelines.
It lives there and only there: this appendix was one of five near-identical copies, and they
drifted from each other and from the code.
One addition that is this repo's alone: the CI job has a frontend half — npm test
(vitest), npm run typecheck, npm run build, plus a blocking npm run audit:prod over
the shipped dependency tree with dev advisories left advisory (BIN-169, and
SCA in the coding guidelines).