binabik-world-state — one snapshot every reasoning layer reads¶

Repo: binabik-world-state · Port: :9240 · Status: deployed on rap-1 and read by the brain — r1ctl status and the brain's /api/world_state are the live answer.

This line said the opposite until 2026-09-10 (BIN-543)

It read "built and merged; never run against a real robot" for weeks after the service went live, while the same page's §"Measured, deployed (rap-1)" quoted latencies taken on that robot — so the document contradicted itself and gave a reader no way to tell which half was current. The repo README carried the identical claim. A deployment state decays and prose does not: ask r1ctl status (→ world 9240 up) or curl localhost:8001/api/world_state on the brain host, and believe those over any sentence here.

The single "what is the world like" surface: the robot's body, its cameras, and the scene. The brain's planner, the L3 skill services and the surveillance watchdog all read it — and nothing else — for state and pixels.

brain / L1  ─┐
grasp / L3  ─┼──▶  binabik-world-state  ──▶  r1-abstraction  ──▶  galaxea  ──▶  ROS
supervisor  ─┘            (:9240)                 (:9220)

Why it exists¶

None of the three reasoning layers could see the robot's own body, and none ever saw an image. The planner knew only what was held in which gripper; the L3 recovery tier got goal text and tool results with no pose at all; the watchdog's robot_state() read gripper loads and returned ok: true unconditionally.

So the robot parked itself in absurd postures and nobody noticed — the only component whose job was to notice was structurally unable to. And an open-ended instruction ("assemble the thing in front of you") had no entry point, because perceive(prompt) requires you to already know what to look for.

Why in front of the abstraction¶

Nothing in it is R1-specific. Caching, staleness, refresh budgets, the posture summary and the per-consumer projections would be rewritten for every robot if they sat behind the seam. In front, they are written once and a new robot inherits all of it by implementing the contract.

That gives the service one hard rule: it is ROS-free. It composes MCP tool calls and never subscribes to a topic. The day it needs rclpy, it has fallen behind the abstraction and become per-robot code. It still deploys on the robot host — deployment and architecture are separate choices. See architecture §2a.

Tools¶

Tool	Cost	Returns
`world_state(refresh, trigger, blocks, caller, cameras)`	a bare call is free — it never captures; `refresh=true` buys one, `mission_start` / `step_failed` refill only a stale scene, `settled_after_motion` only marks it outdated, and a `blocks` selection without `scene` cannot spend at all	`{ok, body, scene, cameras, blocks, stamps}`, plus a free `frames` section when `cameras` names any (BIN-392)
`get_frame(camera, annotated, cameras, caller)`	a plain frame is free and uncapped; `annotated=true` is a paid capture sharing the scene fill budget — ceiling, floor and `by_caller` (BIN-536)	`{ok, camera, annotated, image_jpeg_b64, width, height}`, plus `budget` on the annotated path
`note_motion()`	free	marks the scene outdated — what the brain sends after every motion burst (BIN-391)
`note_scene(scene, caller, age_s)`	free — it installs a capture the caller already paid for and issues no contract call at all; that grounder run is counted once in the same rolling window (BIN-561)	`{ok, stored, objects, source, stamp, budget}`

`body` — free, and kept current by a background poll¶

Eight contract reads issued in parallel: joint_state, ee_pose ×2, get_pose, gripper_state ×2, robot_state, fault_state. Yields posture, motors (joints with at_limit), ee, grippers, base and flags, each as one capability block, plus the freshness envelope.

flags comes from fault_state(), and it is deliberately not a field of robot_state's (BIN-372). The block was hard-coded unavailable until the contract had a real fault read, because the only thing to hand was robot_state() — gripper loads, ok: true unconditionally — and no faults versus cannot see faults is the one distinction this service exists to keep. The contract shape and the rules behind it are integrating-a-robot §2a's, not restated here; what this service adds is that the two reads stay separate, each with its own per-tool deadline (WORLD_FAULT_DEADLINE_S). robot_state was measured exceeding the body deadline 5 times out of 5 on rap-1 (BIN-362) — which is why flags was dark in that report — so a fault verdict routed through it would have inherited the latency and reproduced the bug on the read meant to fix it. robot_state's narrative still travels beside the verdict as notes.

Neither read counts toward degraded. For fault_state that is because an absent verdict is the expected answer on the Gazebo sim — there is no hdas driver, so no /hdas/* topics — and marking every sim snapshot stale over a signal the sim cannot have would turn the staleness mechanism into noise on the platform most of the suite runs on.

A background poll keeps these warm, so a request serves from memory (BIN-353). Until then "always current" was aspirational: body was read on demand, per request, at a measured 2.4–3.6 s on rap-1. Every consumer paid multi-second latency for data that costs nothing to keep warm — the planner per decision point, the watchdog per tick, each open World panel per poll, times however many brains are attached. It is on by default because it is free; see the note under rule 1 for why a poll of ROS reads is not the thing the standing cost rule forbids.

The cache is a latency device, not a freshness claim, and three details keep it honest:

it is served only while younger than WORLD_BODY_SERVE_MAX_AGE_S — past that the request pays the round-trip rather than being handed something the service itself would call stale;
stamp/age_s/stale are recomputed per request, so a cache crossing the staleness threshold starts saying so instead of carrying a stale: false frozen at composition time — the latched-value failure in miniature;
served_from says poll or live, so a consumer can tell.

The poll's lifetime is the read path's, at both ends (BIN-381/BIN-554). It is armed by the first body read — not by a startup hook, and emphatically not by a session, since FastMCP hands lifespan to the low-level MCP server where it runs per session: that meant nothing polled at boot and one client disconnecting stopped the poll for every other client, so with two brains on one robot (BIN-348) brain A leaving cost brain B its warm cache. It now stops itself when no reader has wanted a body for the idle window, and the next read re-arms it.

That symmetry is the point. ensure_body_poll's own docstring claimed that "a service nobody talks to generates no ROS traffic", which was true only until the first read of the process's life — nothing ever stopped the loop, so eight tool calls a second continued with every brain detached and every mission finished. An idle stop driven by sessions would have rebuilt BIN-381's bug with a timer in it, so it is driven by reads, and two things follow from that:

a read still in flight is a reader, however old its stamp. Body reads queue behind manipulation on the same adapter and have been measured at 100 s (BIN-378), so a busy robot must not read as an unused one — the poll would stop in the middle of the mission that most wants it warm;
the self-stop is a plain return between rounds, never a cancel. Stopping a round mid-flight is the exact window that dropped seven of eight tool calls (BIN-514), and a loop that simply declines to start another round cannot reach it.

The cost of stopping is one live read for whoever arrives next — the same price BIN-381 already accepted for the first read after a restart — which is why the window has to sit well above any live consumer's own cadence, and above the serve window: a poll that stops before its cache goes unservable is re-armed by every request and serves nobody.

A failing round backs off exponentially, capped, resetting the moment a round learns anything. What that bounds is log volume, not load: a round against a down adapter fails all eight reads and each failure logs a warning, so a fixed interval meant eight warnings a second, indefinitely — which is how an operator learns to skim the lines that matter. One warning per outage, the rest at debug, and an INFO naming the count when it recovers.

The trigger is a round that learned nothing at all, deliberately not a round that is merely degraded. get_pose fails on every call whenever SLAM is unlocalised and fault_state is unreadable across the whole Gazebo sim, so backing off on either would stop polling a robot that is answering perfectly well — this service's recurring defect (BIN-353/BIN-362/BIN-369) wearing a new hat, a threshold sitting underneath the normal cost of the thing it measures.

body used to claim age_s: 0.0 no matter what the reads cost

Which meant WORLD_BODY_STALE_AFTER_S had never been applied to anything — a threshold that existed, was documented, and was inert, while a snapshot whose reads took 3.6 s reported itself instantaneous. The stamp is now taken before the round, since the oldest read in it is the honest bound.

Making the threshold real is also why its value moved: it has to sit above the worst-case poll cycle (interval + per-read deadline), or every healthy snapshot reports stale. That is the third instance in this service of a threshold sitting underneath the real cost of the thing it measures — after BIN-362 and BIN-369 — so load_settings() now warns at startup when any of them does, naming both numbers. It caught this very field during BIN-353.

Each read is individually bounded, and that bound is a work cutter rather than a wedge detector like WORLD_CALL_TIMEOUT_S. Parallel only makes a round cost its slowest member, so one pathological tool still costs every reasoner: galaxea's get_robot_pose spends ~5.1 s inside a TF lookup for map → base_link before failing, on every call, whenever SLAM is not localised. With a deadline, base reports "did not answer within Ns" — a reason deliberately distinct from a transport failure, because slow means go look at the robot and down usually means restart a service — and the other six answer at their normal speed (BIN-345).

There are two caps, not one, and the reason is a measurement (BIN-362). Two numbers on rap-1 bracket the problem: the healthy body reads cost ~3 s together under real load, and the unlocalised TF pathology above costs ~5.1 s. A single threshold cannot serve both — anything above 3 s stops cutting the pathology, anything below it cuts data that was going to arrive — and those two demands are only 2 s apart. So get_pose carries its own tighter cap (WORLD_POSE_DEADLINE_S) and the other six share a generous one (WORLD_BODY_DEADLINE_S). Cutting the pathological read then costs the base block a reason instead of costing every reasoner five seconds, which is the trade BIN-345 wanted.

Both caps are work cutters on free reads, so getting them wrong is silent in a specific way worth knowing about: a cut read is indistinguishable from a robot that cannot answer, so too tight a cap shows up as blocks blinking in and out of the World panel and the planner's prompt rather than as an error anywhere. That is how a 2 s cap survived — it was discarding robot_state on about half of all reads (BIN-362), and later a load-dependent get_pose (BIN-369), with nothing in the logs. The service now logs each read's duration at debug and a warning naming the tool and the limit on every cut, which is what makes these numbers measurable instead of guessed.

The values are in the repo's env table, and they are measurements — not preferences

Current defaults live in one place, binabik-world-state's README env table; this page owns the reasoning, so it deliberately does not restate them — it went stale for exactly that reason. What belongs here are the bounds any value has to respect: above the ~3 s healthy read, below the ~5.1 s pathology. Both were re-measured on a loaded rap-1, and the first pass at the tighter cap was taken on an unloaded host, which is why it was wrong (BIN-369). The repo's tests pin that bracket rather than the literals, so a re-measurement has to argue with the bounds.

The posture sentence is the highest-value field:

"torso upright, left arm tucked, right arm extended forward 0.61 m, both grippers empty
 — left_arm_joint1 is at its limit"

Handing a language model 18 raw joint angles and expecting "that arm is in a silly position" is asking it to do trigonometry on every turn. It will not, and the failure is silent — the planner simply never mentions posture. So the numbers are reduced once, here, and the raw values stay underneath for consumers that do geometry (the L3 skills get numbers; the planner gets the sentence).

The limit warning is appended rather than merged, so a reader hits the problem last, where it reads as a warning instead of another descriptor.

`scene` — paid, cached, event-refreshed¶

Objects with ids, labels and poses, plus stamp/age_s/stale, the refresh reason, and the budget status.

Two staleness thresholds, and the band between them is the design (BIN-353). The first marks the inventory stale but still served; the second withdraws it altogether:

Threshold	What happens past it
`WORLD_SCENE_STALE_AFTER_S`	`stale: true`, and the objects are still returned. A recently-stale inventory answers "was there a cup here a moment ago", which is why motion invalidates the value rather than clearing it
`WORLD_SCENE_UNUSABLE_AFTER_S`	`available: false`, with a reason naming the age in human units and the way out. The objects are dropped

The second exists because honesty about age turned out not to be enough. Measured on rap-1: a scene block reading available: true, carrying objects, with age_s: 172223 — 47.8 hours — going into the operator's World panel and the planner's prompt, long after those objects had been picked, moved and cleared. stamp, age_s and stale were all right there in the payload; available: true is what a consumer keys on. Same failure class as BIN-343 one level up: a block whose availability flag and its actual epistemic state disagree.

Every available scene block says source, and an empty one no longer claims nobody has looked (BIN-561). source is world-state when this service measured the inventory itself, otherwise the name of whoever published it, with a matching qualification in reason — a caveat rather than a failure reason, so it appears only when there is something to qualify (BIN-343). A service that serves readings it did not take owes the reader that distinction.

"scene never captured" is gone with it. It is the house failure mode in four words — an absence reported as a fact — and it went into the operator's World panel and the planner's prompt while an operator was looking at a freshly grounded inventory a few hundred milliseconds away. The claim the service can defend is narrower: it holds none, none has been captured through it and none published to it, and here is each way in. A capture that errored still reports its own failure, because a broken robot and an idle service call for different actions.

Keeping them as two thresholds rather than lowering one is deliberate. Collapsing them would trade a 47-hour lie for throwing away good data at two minutes, and the stale-but-usable band is genuinely useful. A withdrawn scene also stays distinguishable from never-captured: one means wait for a capture, the other means go and fix the camera.

The two rules¶

1. What is free is updated all the time; what costs is updated only when needed¶

Both halves are one sentence, and reading only the second is how the free half stayed unimplemented for months. Body reads and plain frames cost nothing, so they happen uncapped — and are kept warm by a background poll rather than merely being permitted on demand (BIN-353, below). The planner and the L3 recovery loop should be looking at what the robot is looking at. A scene fill runs the grounder and bills (BIN-139: the whole capture is the billable unit, not the segmenter), so it goes through a policy:


Fills on	mission start · a step failed · an explicit ask — each the moment before something reads the labels
Never on	a timer, never on a bare read, never on a reasoning event, and never because the robot moved (BIN-391)
Ceiling per window	`WORLD_SCENE_MAX_FILLS` (40) captures per `WORLD_SCENE_WINDOW_S` (3600), rolling — the volume guard. `0` switches captures off entirely
Floor between captures	`WORLD_SCENE_MIN_INTERVAL_S` (5) — the rate guard: a burst of triggers buys one capture
What counts against it	every capture: a scene fill and an annotated `get_frame`, which are the same grounder and the same bill (BIN-536)
Every refusal	returns a reason naming the spenders and how long until the ceiling frees a slot, and is counted into `budget.refused`

trigger is what decides whether a call may spend, and its default cannot.

`trigger`	Kind	May it capture?
`read` — the default	none	Never, at any staleness. A plain read is not an event, and a stale scene is not a reason to spend
`mission_start` · `step_failed`	physical	Refills a stale scene; leaves a fresh one alone
`settled_after_motion`	invalidating	Never. Marks the held inventory outdated and buys nothing — same free effect as `note_motion()`, reachable from a read (BIN-391)
`explicit` (or `refresh=true`)	operator	Overrides freshness and buys one — the operator's refresh button
`decision_point` · `replan`	reasoning	Never, at any staleness — see the rule below
anything unrecognised	—	Falls back to `read`. A cost guard fails closed: a typo costs a cache-served scene with a reason, not a bill

A physical-world event may capture; a reasoning event never may (BIN-353)¶

This is the axis that decides spend, and it is arithmetic rather than taste. A capture is ~15 s and a bill. A planner turn is already multimodal — it sees frames via BIN-289 — so capturing on that turn is a second vision call, in series, on the critical path, interpreting the same image the planner can already see for free. The governing constraint, stated as a rule: never two VLM calls in series for one decision.

So the split is by what kind of thing happened:

Physical — a mission is starting from an unknown state, or a step failed. The world changed and something is about to read the labels, so re-perceiving buys new information that is about to be used.
Invalidating — the robot moved and came to rest. The held inventory is known wrong, so say so; buying a replacement is a different question, and nothing has asked it yet.
Reasoning — a model thought about something. Nothing in the world changed, so there is nothing to re-perceive. The frame is what a reasoner needs; the inventory exists for the L3 skills that act on stable object ids.

Movement invalidates; a need for the labels buys (BIN-391)¶

The refinement of the rule above, and the one that came from watching the bill. settled_after_motion was classified physical, so motion + a stale inventory filled — and since motion is exactly what makes the inventory stale, that pair was not an edge case, it was every stop.

Stated as a rule:

A physical event invalidates. What buys a capture is a need for the labels — and the triggers that spend are exactly the ones that name such a need.

In practice, for this service:

Telling it the robot moved is free. The brain sends note_motion after every motion burst (or reads with trigger="settled_after_motion", which does the same thing on the way through), the inventory is kept and labelled outdated, and nothing captures. "We have not looked since the robot moved" and "there is nothing here" are different answers, and only the first is true after a drive.
The labels are bought at the point of use. A mission starting, a step failing, or an operator pressing refresh. A capture taken then is also better data than one taken on a timer, because it is taken when something is about to read it.
The operator always sees it. An outdated inventory presents as outdated with a reason rather than as a confident object list.

The brain's Visual view is a separate cache, and it does re-segment on standstill

Don't read the rule above as "nothing captures after a drive anywhere". The Visual view holds its own segmentation overlay — the click-to-pick target — and it re-segments once the robot stands still, on the bounded budget described in robot-voice-chat's README. BIN-391 removed that for a day and Pascal restored it: standing still after a drive is when an operator reaches for an object, so the overlay is bought just before it is used, and the world-state update it costs is accepted.

What the rule governs is this service's inventory, which has different readers (the planner, the watchdog, an L3 recovery controller) and no operator staring at it — so it is labelled outdated and refilled when one of them needs the labels.

A refused reasoning trigger says so and names get_frame, because the caller is not wrong to want a current view — only about which tool provides it, and a refusal with no alternative is a guard people route around.

Two properties make this hold rather than merely being written down. The classification lives on the trigger, not at the call sites, since a call site can be added without reading any of this and an enum member cannot be classified without choosing; a test fails if a member is added unclassified. And an unclassified trigger fails closed to the kind that cannot spend — wrongly refusing costs a stale scene carrying a reason, wrongly allowing costs a bill nobody chose.

Free is not a licence to ask the same question twice (BIN-554)¶

The rule's free half says updated all the time, not fetched again for every caller. world_state composes its sections concurrently, and two of them needed the camera list — the cameras block, and the name validation behind frames — so a single request made two list_cameras round trips for a value that cannot change while a robot is up. It is now one: a lock coalesces the concurrent pair, and the held outcome carries across requests.

What makes that correct rather than merely fast is the invalidation, and it is shaped by the transport. ToolClient opens a fresh session per call, so there is no connection to watch and nothing announces that the adapter behind a cached capability list has been replaced by a restart. The closest observable is a round trip that could not be completed — after which every call runs over a new session by construction — so any contract call that cannot reach the adapter drops the list. Deliberately over-inclusive in the cheap direction: a false positive costs one free round trip, a false negative advertises the cameras of a robot that is gone, and refuses a camera name on the strength of it.

Two failures are deliberately not that signal, and both exclusions are load-bearing rather than tidy. A deadline cut is this service's own work cutter firing, on a call whose true cost is by definition unknown — counting it would make the cache inert on rap-1, where get_pose is cut on every call whenever SLAM is not localised. A not ok reply is the adapter answering; the session was fine and the tool said no, which is fault_state across the entire sim.

And what is held is the round trip's outcome, not its value. A failure is remembered as a failure, with its reason, so "we could not ask" can never be replayed as "this robot has no cameras" — the distinction that decides whether an unknown camera name may be refused (BIN-302/BIN-392), and the one place a cache here could do real harm by inventing a permanent capability limit out of one bad round trip. The adapter draws the same line one layer down, treating an ok: true with no list as a missing feature rather than as a robot with no cameras.

Naming blocks is a stronger guarantee than naming a trigger (BIN-353)¶

world_state(blocks=[…]) selects among body · scene · cameras. Because scene is the only section that can ever spend, a selection that omits it cannot capture whatever refresh and trigger say — a structural guarantee rather than one that depends on getting a string right. Asking for ["scene"] alone also skips the seven body reads, so a caller wanting only the inventory stops waiting for state it did not ask for. An unrecognised name degrades to the full snapshot rather than to an empty one — a typo must not become missing data, and the caller most likely to make one is a model writing a tool call — and is reported back so the mistake is visible.

The blocks that comes back lists what the snapshot contains, which is not always what was asked for (BIN-554). Naming cameras adds a free frames section whatever the blocks selection says, and the list used to be built from the request — so blocks=["body"], cameras=["head"] returned a section the list did not mention. That defeats the field's one job: BIN-353 added it so a stored snapshot stays interpretable without the request beside it, and a section silently omitted from the list is precisely the case that cannot be read back later. It is now derived from the sections actually composed, so the two cannot drift again.

The asymmetry is deliberate and worth stating, because it otherwise reads as an oversight: frames comes back in blocks and cannot be passed in it. A frame needs a camera name, so blocks=["frames"] could not say which one — and making it selectable would make the default (all blocks) mean a frame from every camera on every read, the expensive default the cameras argument exists to avoid.

BIN-139 shipped an automatic re-segment every 6 s on an empty scene; BIN-299 found a 4 s perception poll silently re-targeting a grasp in flight. Both were timers nobody chose, so the policy class contains no timer and a test fails if one appears.

BIN-347 is the third, and it was inside this service. trigger defaulted to explicit — in the MCP tool and again in the method behind it — and explicit is the verdict that overrides freshness and spends. So refresh=false did not mean what it says: every planner read, watchdog tick and panel poll ran the grounder, at ~14.7 s and a Gemini call each, until the 40-fill budget ran out and the scene silently went cold. The guardrail the perception buffer was retired for breaking, reappearing in its replacement. The lesson for the next cost guard is not "add a default" — it had one — but that a free-read default is a claim about the wire, and has to be tested there: the guarantee is now pinned at the policy, at the snapshot, and end-to-end through a real session asserting capture_scene was never reached.

A failed capture does not spend budget — a grounder that errored produced no scene, and charging for it would end the session early with nothing to show.

Which calls bill, and the two doors they go through (BIN-536)¶

There are two paid calls, and for a while only one of them was guarded. capture_scene is the obvious one. get_frame(annotated=true) is the other: the labelled overlay carries the scene inventory's object ids, which means the grounder ran to produce them — the same bill, one boolean apart, in the same function. It had no budget, no counter, no interval floor and no attribution, was reachable by any MCP consumer at any rate, and appeared in none of budget.fills / budget.refused / budget.by_caller. The brain capped its own use at 3 per mission, which is why no bill ever appeared — a cap in one consumer, not a control in the service, and it never applied to a second brain, an L3 skill, or anything added later.

The lesson is the shape rather than the omission: a guard that lives inside one function guards one function. So the cost question is asked on two axes, and reaching the robot at all goes through one of two doors:

Axis	Question	Mechanism
tool	does this call bill?	`cost_of(tool, arguments)` — a table of every tool the service calls, plus the one argument that turns a free tool into a paid one. Unclassified counts as paid
event	may this trigger spend?	the trigger table above, unchanged

Door	Used by	Rule
`_read`	body reads, `list_cameras`, plain frames	Refuses to issue a paid call. Logs an ERROR naming the tool, and a test asserts a full free exercise produces no such refusal — so a third paid path on a free code path fails the suite instead of quietly spending
`_spend`	`capture_scene`, the annotated frame	The only way a paid call is issued: verdict first (nothing is called when the answer is no), charge on delivery. Refuses to meter a free call, which would burn the ceiling on a ROS read

The annotated frame is charged on delivery, and that is not a formality. An adapter that runs the grounder for get_frame(annotated=true) must echo annotated: true in the payload; without the echo the paid and free variants are byte-for-byte identical and the world service cannot know whether it was billed. Measured against the R1 on 2026-09-09: the adapter's get_frame(camera) had no annotated parameter at all, and FastMCP silently drops arguments its schema does not name, so every annotated request on this robot had been arriving as a plain frame. Budgeting the request would therefore have spent the entire ceiling on plain frames and cooled the scene for work nobody did. Instead nothing is charged, a warning names the missing echo, and the answer says annotated: false with a reason — which also closes a correctness gap, because a consumer told it has object ids on an image that has none will ground its language to nothing.

The R1 now declares the parameter and refuses it (BIN-555), at the adapter and at binabik-r1-vision, and BIN-558 decided the overlay is not built at all — so on today's fleet this path never charges and the design above is exercised only by its refusal branch. That is not a reason to relax it: charging on the echo is what keeps the ceiling honest for any robot that does implement one, and it is the branch that produced the annotated: false a consumer reads. The reasoning behind the decision lives in integrating-a-robot, which owns the contract.

So: read annotated in the answer; never infer it from the request. It is present on every frame. false on an annotated request means no ids and no charge, with the reason saying which — the budget refused (and you get the free plain frame plus budget), or this robot has no overlay. The requirement on the robot side is integrating-a-robot's.

A scene can also arrive from outside, and that is not a third door (BIN-561)¶

The cache used to be fillable only by this service's own capture_scene, and the capture an operator actually presses does not go through this service. The brain's Re-segment now resolves capture_scene on the planner's MCP surface, which reaches the adapter directly — because binabik-world-state is deliberately absent from that surface (BIN-357: world_state(refresh=true) bills, so listing :9240 there would hand the LLM a paid button). Both paths end at the same tool on the same robot. Neither told the other it had run.

Measured on the deployed stack, 2026-09-09: one Re-segment, two capture_scene calls, the Gemini grounder and the SAM 3 segmenter each running twice for one camera view that had not moved — while world_state reported scene: {available: false, reason: "scene never captured"} to the planner, the L3 skills and the watchdog, and would have gone on saying so however many times the operator re-segmented. Two consequences, and the second is the expensive one: the planner is lied to in the direction that reads as innocent, and the next reader that wants a scene here pays for the same picture again.

note_scene(scene, caller, age_s) brings the picture to the cache instead: one capture, one bill, and everybody sees it.


Not a third door	It issues no contract call. The two doors govern the calls this service makes; this is an inbound write with nothing to fetch, so no paid artefact is obtainable through it. A test asserts the robot is never touched — after a loop tick, so "no call" cannot mean "no call yet"
Counted, once	One grounder run, one `record_fill`, attributed — the same rolling-window ledger a local fill lands in, so `budget.by_caller` names the brain that paid. A published fill hidden from the accounting would read `fills: 0/40` on a robot whose grounder had run forty times, and the ceiling would be one in name only
Not charged twice	The publisher's capture never passed through `_spend`, so this is its only entry. Double-charging is as wrong as double-capturing
Never refused on cost	`consider()` is not consulted. Permission is asked before spending; accounting happens after the fact. Turning away a picture already paid for would be the double-capture this fixes, in a cost guard's costume. It can still bring the shared ceiling closer for everyone — correct, because the grounder really did run — and since BIN-536 that is a wait, not a dead end

The direction was chosen over the alternative (point the operator's capture at :9240 so it goes through _spend), which remains the better end state for the planner surface and is not foreclosed by this. Publishing wins today for three reasons: the world service keeps only the normalised object list and the Visual overlay reads the raw capture payload, so routing through it would make the service the schema owner for a UI overlay it has never seen; publishing works for any payer — an L3 recovery, a robot-side script — not only the callers that can be re-pointed at a new URL; and it needs nothing on the planner's paid surface, which is what BIN-357 was protecting.

Three details keep the two writers honest. age_s back-dates the stamp by however long ago the capture was taken, because scene() stamps its own fills at the moment the call starts (a capture costs ~7 s and is honest about it) — so a publication stamped on arrival would present as strictly fresher than the identical local capture. The interval floor is anchored monotonically, or a back-dated fill would walk it backwards and let a burst through on the strength of an older event. And the cache refuses to move backwards in time at all: a local capture completing after a fresher publication keeps the publication, and the source moves with the objects because one function writes both.

What this deliberately does not do is second-guess the publisher. A capture grounded at a pose the robot has since left puts labels over whatever is at those pixels now; the check for it belongs to whoever holds the epoch, which on the brain side is capture_scene_now's (BIN-207). A later note_motion invalidates a published scene exactly as it does one of ours.

So a publisher owes four things (BIN-564; the brain's capture_scene_now() is the reference implementation, and the only caller today):


Publish only a capture you kept	This service cannot tell that your robot moved mid-capture — the epoch is yours. Dropping the capture but publishing it anyway installs exactly the drifted masks your own check exists to refuse
Send the `caller` you already read with	Same string as your `world_state` calls (`BRAIN_NAME` for a brain, BIN-415), or `budget.by_caller` grows two rows for one payer and answers "who burned the 40 captures" wrongly rather than not at all
Send a truthful `age_s`	Zero is generous, not wrong — but a capture costs ~7 s and claiming it as instantaneous makes it beat the identical local fill in the ordering race
Keep it off your own critical path	Free at this end does not mean instant on the wire. A publication is a side benefit; a service that is slow, absent or refusing must not delay whatever you captured for. Best-effort, bounded, and never able to fail the capture

Publish every capture you kept, including automatic ones: same bill, same picture, and withholding the automatic ones would leave most of the spend invisible to the ceiling. An objects: [] inventory is a real answer and is published like any other — a grounder that looked and found nothing is a fact somebody paid for.

The ceiling has an end, and it is not a restart (BIN-536)¶

WORLD_SCENE_MAX_FILLS was a session ceiling on a long-running daemon, so "session" meant "since the last r1ctl restart". Two consequences, both bad in opposite directions:

exhaust it and the scene goes permanently cold — every consumer reads a stale-or-unavailable inventory forever — with the refusal's own advice being "restart the service", which the brain-side operator being refused cannot do and which discards the warm body cache and the scene already paid for;
and it was never persisted, so a restart did clear it. Neither a durable budget nor a resettable one; a ceiling whose only reset is a process bounce gets cleared by a process bounce.

It is now a ceiling per rolling window (WORLD_SCENE_WINDOW_S, one hour), which fixes both ends at once: the scene un-colds by itself, and bouncing the process buys nothing that waiting would not. A refusal says how long — "capture budget spent (40/40 in the last 60min, spent by brain-giovanni (38), brain-pascal (2)) — the next one frees up in 9min, or raise WORLD_SCENE_MAX_FILLS" — and budget carries window_s, frees_in_s and a lifetime fills_total beside the windowed fills, with by_caller scoped to the same window as the ceiling it explains.

This reverses the original argument that "a rate limit still permits unbounded spend over a long session, which is exactly what an idle robot does overnight". That stopped holding when BIN-347 and BIN-353 landed: READ never fills, a reasoning trigger never fills, and an idle robot generates no physical events — so an idle robot spends nothing whatever the window is. The overnight spend is prevented by the trigger model now; what the process-lifetime counter was still doing was making exhaustion permanent.

\"Never on a timer\" is about the bill, not about the word poll

The body poll added in BIN-353 is a timer, and it does not contradict the rule above — reading it as a contradiction is the mistake worth heading off, because it is the reason the free half of the rule went unimplemented while the paid half was enforced twice.

The standing guardrail is that no service may regularly poll a costly external API. What BIN-306 retired the perception buffer for, and what BIN-347 reintroduced by accident, was a Gemini grounder running on a cadence nobody chose. The body poll calls five ROS reads behind the adapter: the only cost is ROS traffic, and Pascal's rule says such things should be current all the time.

Two things keep the distinction from eroding. The poller is structurally unable to reach the paid path — it calls the body composer, which knows only the seven body tools — and because that exact regression has now shipped twice, a test drives real cycles and asserts on what was called, not on what came back. And the paid tier keeps every guard it had: a ceiling per window, a floor between captures, the event-kind check, and no timer anywhere in the policy class.

2. Absent is never reported as healthy¶

A block whose read failed is still returned, marked stale, with a degraded reason. Three states must stay distinguishable:

there is nothing there — we looked, the table is empty
we have not looked recently — there may be six objects, from before the robot drove off
we cannot look — the vision service is down

Collapse those and a planner confidently places an object where something used to be. So objects: [] with stale: true and a reason is not the same payload as objects: [] with stale: false. Motion invalidates the scene rather than clearing it — a stale inventory still answers "what was there a moment ago", and dropping it would say the table is empty.

The same rule governs joint limits: when the URDF has not arrived, the range keys are absent rather than defaulted, because at_limit: false on every joint is exactly what a healthy robot looks like. See the warning in integrating-a-robot §2a.

The per-layer input contract — who gets what, and who has to ask (BIN-392)¶

One service, three consumers, and three different default diets. Decided 2026-08-18; it revises what BIN-288, BIN-289, BIN-290 and BIN-292 shipped, so this is a correction to the design rather than a bug report against a merge.

Layer	Gets by default	Asks for
L1 — the planner	the free body half only: posture, joints + `at_limit`, EE and base poses, grippers, faults	the object inventory and any camera frames, by name, via `read_world_state`
The supervisor	the free body half, every tick	a picture every `vlm_every_n_ticks`, or on suspicion
L3 — skill services	nothing. The interface is prepared, never wired	whatever the skill's owner chooses to switch on

The reason is attention, not API cost. A plain get_frame runs no grounder, so pixels are free at this service — which is exactly why the first design pushed a frame to the planner on every turn. They are not free at the consumer's own model: every turn carried ~1.5k image tokens and a cached object list the planner had not asked for and usually could not use, competing with lessons, waypoints and operator facts for one context window. And a planner that must ask has to know why it is asking, which is a property worth having on its own.

It also fixed a mismatch nobody had noticed: the question most likely to need a picture is a gripper or grasp check, and that needs a wrist camera. The every-turn head frame could never answer it.

What each layer's rule protects¶

L1 pulls. read_world_state(cameras=[…], objects=…, why=…) is a brain-local tool, deliberately not the world service listed on the planner's MCP surface — because a manifest entry hands the model the real tool with refresh included, and the whole condition on giving the planner this capability was that the paid path stays unreachable. The local tool has no refresh in its schema at all: absent, not defaulted false, which is the difference between a parameter a model cannot pass and one it might be talked into. BIN-347 is the cost of getting that wrong — a default argument turned every "free" read into a ~15 s Gemini capture until the session budget ran out. Every pull is logged with the planner's own why, so "how often does it actually want pixels, and what for" is a number rather than an impression.

The supervisor rations. Free deterministic checks on every tick; a look every N ticks or on suspicion, where suspicion is a body check that reported a problem. That ordering is what makes rationing safe rather than merely cheaper — the tick that most needs a picture must not be the one made to wait its turn — and suspicion is spent once rather than latched, since otherwise one bad body reading turns the eye back into a per-tick poller, which is the cost bug arriving by the door marked safety. Off by default stays off by default.

L3 is offered, never enforced. The kit's RecoveryAgent(observe=…) is an optional hook, so every skill built on the template inherits the possibility and none inherits the behaviour. That held until grasp-service gave WORLD_URL a loopback default, which made the hook live on any robot running the world service — i.e. every R1 — with nobody opting in. The default is empty again. The skills belong to whoever owns them; preparing an interface and enabling it are different acts.

cameras is the contract addition that makes "give me more" cheap

Without a plural, a consumer asking for more has to take everything. world_state(cameras=[…]) and get_frame(cameras=[…]) are the plural of BIN-302's single camera, and they keep its promise: each camera gets its own {ok} envelope so one dead camera costs its own entry rather than the request; a name is refused with the real list when the robot can enumerate its cameras, and attempted anyway when it cannot — because refusing a camera on the strength of a list we could not obtain would invent a capability limit.

Capability blocks — where robot-dependence is handled (BIN-317)¶

Most of this snapshot is robot-dependent. The R1 reports joint angles; the next robot may not. That difference is resolved here, once — the alternative is the planner, each L3 skill, the supervisor and the frontend each learning which robot can do what, which is what the abstraction exists to prevent.

So every part of the snapshot is the same shape:

{"label": "Robot motor states", "available": true,  "joints": [...], "at_limit": [...]}
{"label": "Robot motor states", "available": false, "reason": "this robot does not implement joint_state()"}

Three properties are load-bearing:

An unavailable block carries no data. Not joints: [], not at_limit: false, not {}. It is enforced in the block constructor rather than asked of callers, because the caller who breaks it is the one adding an empty list so a .get() stops raising.
The label travels in the payload. The operator's World State panel renders Robot motor states: Not available by iterating blocks, with no per-capability and no per-robot code — so a capability added later appears in the UI the day the service emits it.
"Not implemented" and "failed" read differently. An unknown-tool error from any transport is normalised to "this robot does not implement joint_state()" — an integration gap — while a timeout stays a failure. They call for opposite actions, and a transport string in front of an operator answers neither.

Two narrow exceptions, both deliberate. notes carries something the robot did say while still not answering the question — robot_state()'s narrative, or the scene's refresh budget — under its own key so it cannot be read as the capability's value. And available is not complete: cameras on a robot that cannot enumerate is available (the head camera works) and incomplete (the list is an assumption), which is one usable camera plus a caveat rather than a claim.

flags is now the second block of that second kind, and the clearest illustration of why the two axes are not one (BIN-372). On the physical R1 it is available and qualified: fault: false covers the one signal the robot can read, while e_stop and self_collision cannot be read at all — a hardware latch with no state topic, and a MoveIt planning result rather than a published state. So the block reports its verdict, names both unreadable signals in its caveat, and carries the adapter's reason for each under unreadable. What it never does is emit e_stop: false: a signal we cannot read has no value, and the reasons are a statement about what we cannot see rather than about the robot. A bare green "Faults: none" is exactly the misreading the whole chain exists to prevent.

A caveat must never become an unconditional sentence

BIN-343's guard was "no available block carries a reason" — true only for as long as no body block had a legitimate caveat. flags is the first, so the guard had to be narrowed rather than dropped: an allow-list of blocks permitted a caveat, each defended in review, paired with a test that drives every listed block with nothing to qualify and requires it to fall silent. Without that second half, moving an unconditional failure sentence from reason to caveat would reintroduce BIN-343 in the one place the guard no longer looks — which is the shape of almost every regression in this service's history.

Running it¶

r1ctl starts it in window 26 on :9240, on by default — unlike vision, perception and episodic, it polls nothing and calls no external API, so there is no "did the operator ask for it" question and up asserts it like the adapter and grasp. WORLD_ENABLE=0 skips install and launch, and removes :9240 from the required set. Ports, windows and flags → r1-robot-stack.md.

It needs no sourced ROS env and no --system-site-packages venv, which follows from being ROS-free: a plain uv sync is the whole install.

It binds 0.0.0.0, and that is load-bearing. The brain runs centrally, on another host, so a loopback bind makes the World panel, the planner's world block and the watchdog's snapshot all unreachable — while every check on the robot still passes: the port is open, r1ctl status says world 9240 up, and the service's own suite connects over loopback by construction. It shipped that way and nobody noticed until a curl from the brain host (BIN-344). WORLD_STATE_HOST overrides it; r1ctl narrows it to loopback only where a tailscale serve proxy already owns the port. FastMCP's DNS-rebinding protection is disabled for the same reason — a client arriving as http://rap-1:9240 is otherwise refused by Host header.

Measured, deployed (rap-1, 2026-08-15): a plain world_state() is ~2.2 s — almost entirely the get_pose deadline above; it drops to ~0.6 s once nav2 localises. refresh=true is ~15 s, which is what a grounder costs and why it only ever happens when someone asks.

Configuration is a single frozen env dataclass; the full table is the repo's README.

Testing¶

The logic lives in modules with no transport and no clock (posture, freshness, refresh, snapshot), so CI tests it for real rather than through mocks — including "an hour passed and nothing spent", which an injected clock makes testable without waiting an hour. tests/test_server_surface.py then stands up the real FastMCP server on an ephemeral loopback port and drives it with a real client, because the seam every unit test fakes is the seam nothing tests (BIN-179).

No ROS, no robot, no GPU, no credentials — which is the architectural claim restated as a test property, not a convenience.

Not done yet¶

Every item that stood here has shipped, and the deployment is what found the bugs — recorded below rather than deleted, because what a design missed is worth more than what it planned.

Done since: note_scene has a caller (BIN-564). This entry stood open for exactly as long as the mechanism had none — recorded rather than glossed, because a mechanism with no caller looks exactly like a fix. The brain's capture_scene_now() now hands its payload over on the separate WorldStateClient connection it already reads on, so an operator Re-segment now is one grounder run and both caches hold it. Nothing about the planner's MCP surface changed: :9240 stays off robots/r1.yaml (BIN-357).

Done since: it runs on rap-1 (the "never run against a real robot" caveat is what BIN-343 through BIN-347 cost); all four consumers read it — the planner prompt (BIN-288), the multimodal planner (BIN-289), the supervisor (BIN-290), the L3 skills (BIN-292), the operator panel (BIN-313); and cameras is real (BIN-302) — list_cameras returns head, left_wrist, right_wrist and each returns a distinct frame.

What the first deployment found, none of which any test could see:


BIN-343	An available block carried a `reason` explaining a failure that never happened — `motors: available` with 18 joints, captioned "joint_state() unavailable". `capability()` now separates `reason` (why unavailable, dropped when available) from `caveat` (why an available block is qualified)
BIN-344	The service bound loopback, so the brain could never read it — while every check on the robot passed. Guarded on the module, since a session test connects over loopback too
BIN-345	The three top-level reads were a tuple of `await`s — sequential, not concurrent — and one slow tool bounded nothing
BIN-346	`gripper_state` reached this service without the contract's `ok`, so a working gripper read as `Grippers: Not available` (fixed in `r1-abstraction`)
BIN-347	Every plain read billed — see rule 1 above

The pattern across all five: each one passed every check that existed, from the robot's side. Four of the five are invisible to any test that fakes the transport or runs on the same host, and the fifth was pinned by a test asserting the opposite of its own name. The standing lesson is in coding-guidelines.md — verify from the consumer's position, not the producer's.

Settled (BIN-348): the scene cache and the session fill budget are per world-service process, and the service is per robot, not per brain — so two brains attached to the same robot share both.

Sharing the cache is right and stays. There is one physical world; a scene one brain paid for is just as true for the other, and re-capturing it would be paying twice for the same frame.

Sharing the budget is the awkward half, because cost belongs to whoever pressed the button. That became more than a curiosity when BIN-353 landed: before it, only a human pressing Refresh scene ($) could spend, so a cold scene always had a person behind it — now physical events spend on their own, so one brain's mission can drain another's budget with nobody having touched anything.


Done	Spend is attributed. `world_state(caller=…)` names the asker, fills land in `budget.by_caller`, and the exhaustion reason names the spenders ranked so the brain to go and talk to is first. A fill nobody named counts as `unattributed` rather than being dropped, so the arithmetic closes. Attribution is bookkeeping and changes no decision — if `caller` could influence whether a fill happens it would become a way to talk the cost guard into spending
Decided	The ceiling stays shared, and the limitation is documented rather than engineered away. Per-caller budgets were the alternative and are not being built
Consequence	The brain does pass `caller` as of BIN-415 — `BRAIN_NAME`, which `brainctl` injects as `brain-<name>`, so it is the same string as `brainctl ls`. Production attribution reads `{"brain-pascal": N}`. It used to read `{"unattributed": N}`, left that way on the reasoning that with one brain per robot there is nobody to attribute to; that stopped being true (six live instances on the 4090, two of them pointed at the same host), and `unattributed` is the one value the field cannot use

Two brains on one robot share the 40-fill ceiling — known, and accepted

What happens: one brain's missions can exhaust the other's fills. The second operator sees the scene go cold with no way to know why, because the spend happened in someone else's session. Nothing warns them; budget.exhausted is true and the reason names the spenders, but only if someone reads the payload.

Why it is accepted: per-caller ceilings would need a decision about what an exhausted brain sees when another brain's cache is fresh, which is real design work. The attribution is the part that makes the limitation explainable, and the prerequisite if this ever needs building.

The "nobody runs this configuration" half of that argument has expired. It was written when one brain per robot was the deployment. On 2026-08-24 the 4090 was running six instances, two of them attached to the same host — so a shared ceiling with two spenders is now a configuration that exists, not a hypothetical. What changed with it is that the spenders are nameable (BIN-415); what has not changed is the ceiling.

What to do if you hit it: read budget.by_caller in the snapshot. It names who spent the fills, ranked, and budget.frees_in_s says how long until the ceiling admits one more — since BIN-536 the answer is usually just wait, because the ceiling is per rolling hour rather than per process lifetime. If waiting is not acceptable, raise WORLD_SCENE_MAX_FILLS or run a world service per brain — and reopen BIN-348, because at that point the configuration exists.