Skip to content
AuthorPascal DateJuly 14, 2026 Rev1.0

robot-mcp-memory — Implementation Spec

robot-mcp-memory is the per-robot process memory MCP server for the robot fleet. It stores process documents — markdown files distilled from successfully executed (or instructively failed) missions in "teaching mode" — and retrieves the most relevant ones when a new instruction arrives, so the planner LLM can reuse what the robot has already learned.

Repo: robot-mcp-memory

The store is per-robot and local-first: a SQLite FTS5 index plus a directory of human-readable markdown files, with no central service.

Note

Learned-good skills (vision configs) may in future be persisted here alongside processes and merged into mcp-skill-control's catalog. That is an additive use and does not change the interface described here.

1. System context

Each robot runs a chat/voice app (robot-voice-chat, FastAPI, port 8000) whose agentic MissionExecutor plans missions with an LLM and calls tools on capability MCP servers (HTTP/SSE, on-robot). This server is one of them (port 9103). Siblings: robot-mcp-navigation (9101), robot-mcp-perception (9102).

The flow this server enables:

  1. Operator presses Teach in the UI → the chat app records the next mission's full trace.
  2. Mission ends → the chat app distills the trace into a markdown process document (the chat app does the LLM distillation, not this server) and calls memory.save_process(...).
  3. On every later mission start, the chat app's executor calls memory.find_processes(instruction, k=3) and injects the matches into the planner's context as advice.
  4. After a mission that used a process, the executor calls memory.record_outcome(...) so stale or flaky processes lose weight over time.

Unlike navigation/perception, everything here is fast — no Async Task Contract and no robot-mcp-kit task manager. All tools are short request/response MCP tools. This server does not depend on robot-mcp-kit and posts no events.

2. Repo layout

robot-mcp-memory/
├── pyproject.toml                  # package: robot_mcp_memory; deps: mcp, pydantic, pyyaml
│                                   # (sqlite3 is stdlib; FTS5 ships with CPython's sqlite)
├── README.md
├── src/robot_mcp_memory/
│   ├── __init__.py
│   ├── server.py                   # FastMCP("memory") + tools (§5) + entrypoint
│   ├── store.py                    # ProcessStore: files + SQLite FTS5 (§4)
│   └── docmodel.py                 # ProcessDoc parsing/serialization (§3)
├── deploy/robot-mcp-memory.service
└── tests/
    ├── test_docmodel.py
    ├── test_store.py
    └── test_tools.py

Tooling: Python ≥ 3.11, uv, ruff, mypy --strict, pytest, GitHub Actions CI. Entrypoint/transport convention as siblings: console script robot-mcp-memory, MCP_TRANSPORT (stdio|sse), MCP_PORT default 9103. FastMCP name must be "memory".

3. The process document format

One markdown file per process under MEMORY_DATA_DIR/processes/ (default data/processes/), filename {id}.md. YAML frontmatter + free markdown body:

---
id: fetch-package-mailroom            # kebab-case slug, unique; server enforces
title: Fetch a package from the mailroom
task_pattern: "fetch|get|bring ... package|parcel ... mailroom"   # free-text hint, also indexed
tags: [delivery, mailroom]
robot: g1-lab-01                      # provenance (env ROBOT_ID at save time)
created: 2026-06-12T14:00:00Z
updated: 2026-06-12T14:00:00Z
success_count: 3
failure_count: 1
last_used: 2026-06-12T16:20:00Z
source_mission_id: m-9af31c           # the teaching mission that produced it
---

## Steps
1. navigation.navigate_to(mailroom) with trigger eta_below:10
2. on trigger → perception.prewarm("mailroom, parcel shelf")
3. perception.find_object("package labeled <addressee>")
...

## Pitfalls & fixes
- After 18:00 the door is often closed → DOOR_CLOSED is recoverable: ask reception via ...

## Operator notes
- Visitor packages are on the lower shelf.

docmodel.py provides ProcessDoc (pydantic) with parse(text) -> ProcessDoc / render() -> str (round-trip stable), validating frontmatter and slug format. The body is opaque advice text — the server never interprets steps; only the planner LLM reads them.

The markdown directory is the source of truth; SQLite is a rebuildable index. Files are human-editable: on startup and on a reindex tool call, the store rescans the directory and rebuilds the index (with an mtime-based incremental scan on startup).

4. ProcessStore (SQLite FTS5)

MEMORY_DATA_DIR/index.db:

CREATE TABLE processes (
  id TEXT PRIMARY KEY, title TEXT, task_pattern TEXT, tags TEXT,
  robot TEXT, created TEXT, updated TEXT,
  success_count INTEGER, failure_count INTEGER, last_used TEXT,
  path TEXT, mtime REAL
);
CREATE VIRTUAL TABLE processes_fts USING fts5(
  id UNINDEXED, title, task_pattern, tags, body, tokenize='porter unicode61'
);

Search (find(query, k)):

  1. FTS5 MATCH over title+task_pattern+tags+body with bm25 ranking. Sanitize the query (strip FTS operators, OR the remaining terms) so arbitrary natural-language instructions never produce syntax errors.
  2. Re-rank: score = bm25 * reliability, where reliability = (success_count + 1) / (success_count + failure_count + 2) (Laplace smoothing).
  3. Return top k with score and a reliability field; the chat app decides how to present them.

Future

Embedding search is a planned upgrade inside this repo (swap or augment find). The tool surface does not change for it.

Store API: save(doc) -> ProcessDoc (insert or update by id; bumps updated), get(id), find(query, k), record_outcome(id, success: bool) (increments counters, sets last_used, rewrites the file's frontmatter), delete(id), list(limit, offset), reindex(). All writes go file-first, then index (a crash leaves at worst a stale index, fixed by reindex).

5. MCP tools

All tools return JSON strings. Their docstrings tell the LLM that processes are advice from past executions, that it should deviate when reality disagrees, and that find_processes is normally called automatically by the system (though the LLM may still call it mid-mission for sub-tasks).

Tool Behavior
find_processes(query: str, k: int = 3) §4 search → {"processes": [{id, title, score, reliability, success_count, failure_count, updated, markdown}]}. markdown is the full document — the chat app injects it verbatim.
get_process(id: str) Full document or {"error": "not_found"}.
save_process(markdown: str) Parse + validate (ProcessDoc.parse); slug collision ⇒ update that process (preserve counters, bump updated) unless frontmatter differs in source_mission_id, in which case suffix the id (-2). Returns the stored doc's frontmatter.
record_outcome(id: str, success: bool, note: str = "") Update counters/last_used; if note non-empty, append a line under an ## Outcome log section (create if missing) with date + success flag + note.
delete_process(id: str) Remove file + index row.
list_processes(limit: int = 50, offset: int = 0) Frontmatter summaries (no bodies).
reindex() Rescan directory, rebuild index. Returns counts.

6. Configuration, deployment, wiring

Env var Meaning Default
MCP_TRANSPORT / MCP_PORT transport / SSE port stdio / 9103
MEMORY_DATA_DIR processes dir + index.db location data/
ROBOT_ID provenance written into saved docs unknown

Systemd unit deploy/robot-mcp-memory.service: same shape as siblings (User=robot, Restart=always, MCP_TRANSPORT=sse, MCP_PORT=9103, ReadWritePaths=/opt/mcp-servers/robot-mcp-memory/data). No RVC_CALLBACK_URL — this server posts no events.

Chat app wiring (backend/config/tools/mcp.yaml):

  - name: memory
    transport: sse
    url: "http://127.0.0.1:9103/sse"
# memory tools the executor calls directly (find_processes / record_outcome) are invoked
# through the same MCP client; no blocking_tools entry needed.

Fleet sync (documented in the README): the processes/ directory is plain markdown — git init it or rsync it between robots, then run reindex() after a sync.

7. Tests

  1. docmodel: parse/render round-trip; invalid frontmatter rejected; slug validation.
  2. store: save→find by phrasing variants ("get me a parcel from the mail room" finds fetch-package-mailroom); reliability re-ranking (a 0/5-failure process ranks below a 5/0 one at equal bm25); FTS query sanitization (queries containing ", *, (, NEAR don't crash); record_outcome rewrites frontmatter and survives reindex; a file edited on disk is picked up by reindex.
  3. tools (FastMCP in-memory client): full surface; save collision behavior; not-found paths.

8. Acceptance checklist

  • All gates pass (pytest, ruff check, ruff format --check, mypy --strict).
  • MCP_TRANSPORT=sse robot-mcp-memory serves on 9103; chat app logs mcp_server_connected server=memory.
  • Demo: save_process a hand-written mailroom doc, then find_processes("bring me the parcel from the mail room") returns it ranked first with the full markdown.
  • Kill -9 between file write and index write, restart → reindex heals.

Appendix A — Repository & tooling conventions

Standard "how we build repos here". This is an MCP server — all sections apply, including MCP servers specifically.

Structure — src/<package>/ layout (never flat); config/ Hydra tree; tests/{unit,integration}/; committed README.md (quickstart + how-to table + troubleshooting table), CHANGELOG.md (Keep-a-Changelog + SemVer), .pre-commit-config.yaml, .github/ (CI + dependabot.yml), optional multi-stage non-root Dockerfile, .env.example committed / .env gitignored.

Tooling — uv for env + deps (uv sync, lockfile committed), Python ≥ 3.11, optional extras for heavy/optional deps (the --extra observability pattern). ruff (lint + format), mypy --strict, pytest + pytest-asyncio, coverage gate ≥ 80%. Pre-commit: trailing-whitespace, end-of-file-fixer, check-yaml/toml/merge-conflict/large-files, ruff (--fix), ruff-format, and a local mypy hook run via uv run (so it picks up the pydantic.mypy plugin). CI on every PR: pytest, ruff check, ruff format --check, mypy, pip-audit. Dependabot weekly for the uv ecosystem.

Architecture — ports-and-adapters: every capability with >1 implementation or that touches hardware/network gets an abc.ABC port in base.py; concrete adapters live beside it; application code imports only ports. Hydra + Pydantic config: one config-group dir per port, one YAML per implementation carrying _target_; swap implementations by changing one line in the root config.yaml defaults list — no if provider == … ladders. Validate the composed config through Pydantic dataclasses at startup (fail fast). Deployment presets under config/deployment/<name>.yaml selected via an env var.

Secrets — never in YAML or git. .env + OmegaConf ${oc.env:KEY} resolvers, typed SecretStr; startup refuses to boot if a required key is empty.

Logging — structlog JSON by default, pretty renderer toggle for dev; bind correlation IDs (request/turn/mission) via contextvars; log duration_ms per stage. Subprocess — always argv lists, never shell=True. Testing — fakes at the port boundary (don't patch internals); respx for HTTP-level mocking; integration via httpx.AsyncClient + ASGITransport.

MCP servers specifically — one repo per capability from a copy-me template; transport stdio if the consumer supervises it (1–2 co-located), else sse/streamable_http with its own systemd unit/container/logs/restart policy and the consumer just holds a URL. Tools namespaced server.tool; mark slow perception tools "blocking", else fire-and-forget; expect a ~30 s call timeout; tolerate clients health-checking via list_tools(). Resilience: the consumer reconnects on an interval — design for clean restart, never assume connection order at boot. Deploy: systemd unit with hardening (NoNewPrivileges, PrivateTmp, ProtectSystem=strict), bind localhost, dedicated user, journald.