Phase 3.5 extraction & assembly

XML-style delimited context formatter with locked ordering (directive → history → memories by category → tools) and per-assembly session nonce (ADR-0070).

Milestone 3.5.C.5 — Context Formatter

Status: Completed Goal: Track C — Context Assembly Engine
Phase: 3.5 — Extraction & Context Assembly
Estimated effort: 2 days
Track: Track C — Context Assembly Engine
Depends on: 3.5.C.4 (packer v2)
ADR: ADR-0070


Why This Milestone Exists

Memories are untrusted input and must be wrapped as data with a per-assembly nonce (SECURITY.md §7.3). Conversation history is trusted and must stay as native role messages. Planning docs disagreed on history position; this milestone locks order in code + ADR and corrects ARCHITECTURE.md’s stale “history last” line.


Non-Goals

  • Full AssembleContext orchestrator (budget → retrieve → score → pack → format) — 3.5.C.6
  • gRPC / proto changes
  • Amending ADR-0038 (gRPC contract) or ADR-0042 (self-hosted LLM adapter)

File layout (as shipped)

services/context/app/formatter.py   # ContextFormatter, FormattedContext, CATEGORY_ORDER
services/context/app/config.py      # IBEX_CONTEXT_FORMATTER_NONCE_BYTES
services/context/tests/test_formatter.py

Settings / env:

  • IBEX_CONTEXT_FORMATTER_NONCE_BYTES (default 16) — secrets.token_urlsafe nbytes

Final Locked Ordering

1. Directive (always first — sets behavior before any content)
2. Conversation history (native role: content lines — not XML-wrapped)
3. Memories by category (procedural → factual → preference → behavioral → episodic)
4. Tool schemas (if applicable)

Rationale for History in position 2: “lost in the middle” — history stays near the start where attention is reliable.

Memory wrap:

<ibex_memory nonce="{nonce}" id="{memory_id}" category="{category}">
{content}
</ibex_memory>

Success signals

  • CATEGORY_ORDER = procedural → factual → preference → behavioral → episodic
  • History at position 2 as native role: content (not XML-wrapped)
  • ADR-0070 accepted (new ADR; not ADR-0042)
  • Golden-output exact-match fixture (directive + 2 history turns + 3 memories)
  • Nonce uniqueness + spoofed-tag neutralization tests
  • ARCHITECTURE.md format/budget order aligned with locked ordering
Edit on GitHub

Last updated on