ADR-0057: Memory write-path persistence and orchestration

• Status: Accepted
• Date: 2026-08-27
• Authors: IBEX Harness team
• Milestone: 3.C.5 Write-path persistence & orchestration
• Tracking issue: #620

Context

Track C ships a documented nine-step write pipeline (tracks):

validate → PII → dedup → embed → near-dup → conflict → DB write → cache update → index

Milestones 3.C.1–3.C.3 implement stages 1–6 (ADR-0054, ADR-0055, ADR-0056). No milestone owned steps 7–9. Track D is read/ranking; 3.C.4 is multi-label only. Phase 3 exit gate 3.E.2 requires POST /v1/memories end-to-end.

Also: ESCALATE_PENDING and pending_supersede_targets exist only on ephemeral WriteContext today — a conflict worker has nothing durable to poll.

Options considered

1) Ownership of steps 7–9

1. Expand 3.C.4 to include insert/cache/index — overloads multi-label scope.
2. Pull write orchestration into Track D — Track D is read/ranking.
3. New milestone 3.C.5 before 3.C.4 (chosen). Multi-label attaches labels at insert time and needs a real insert transaction.

2) Insert + supersession transaction shape

1. Staged writes with compensating actions on partial failure — complex; easy to leave orphan edges or active duplicates.
2. Single Postgres transaction wrapping insert + apply_supersession / relationship edge writes (chosen). Cache and vector upsert run after commit (best-effort / retryable); do not hold the DB txn open for Redis or ANN.

3) Exact-hash unique violation (race after ADR-0055)

Concurrent inserts against the partial unique index can fail after the exact-dedup stage passed. Options: surface a 500, or map to the existing bump-retrieval path and return the documented 409 DUPLICATE_CONTENT / bump semantics. Chosen: map to bump — never a bare 500.

4) Durable escalations for ESCALATE_PENDING

1. Invent pending_review on memories.status — forbidden without migration; ADR-0056 already rejected inventing it casually.
2. New relationship_type like contradicts_pending — overloads the committed graph with workflow state (memory_relationships has no status column).
3. Dedicated org-scoped table memory_conflict_escalations (chosen design). Worker polls status = pending. Implementation deferred to milestone 3.C.5 (#620); this ADR locks the design so the gap is not silent.

Decision

1. Milestone 3.C.5 owns steps 7–9 and the HTTP write handler that backs the existing POST /v1/memories contract (do not invent a new REST shape).

2. Sequencing: 3.C.1 → 3.C.2 → 3.C.3 → **3.C.5** → 3.C.4.

3. After conflict stage returns, the orchestrator:

   • Opens one DB transaction
   • Inserts the new memory (gets DB-generated id)
   • Applies each pending_supersede_targets id via apply_supersession (and inserts other edges as decided by conflict outcomes)
   • Commits
   • Then best-effort cache update + vector upsert/index refresh

4. On unique violation for active (org_id, agent_id, content_hash): call the existing bump path and return 409 with existing_memory_id per API docs.

5. Escalation durability schema sketch (implement in 3.C.5):

   memory_conflict_escalations (
     id UUID PK,
     org_id UUID NOT NULL,
     new_memory_id UUID NOT NULL,      -- after insert
     candidate_memory_id UUID NOT NULL,
     status TEXT NOT NULL,             -- pending | resolved | dismissed
     reason TEXT,
     subject_key TEXT,
     created_at TIMESTAMPTZ,
     resolved_at TIMESTAMPTZ NULL
   )

   Rows are written after the new memory insert succeeds for ESCALATE_PENDING decisions. Prefer RLS + explicit org_id filters. Do not add pending_review to memories.status in this design.

Consequences

• 3.C.4 and 3.E.2 are blocked on 3.C.5 for a correct write product surface.
• Until 3.C.5 ships, stages 1–6 remain service-internal (composed integration tests exercise them without HTTP).
• Conflict workers gain a pollable escalation table only when 3.C.5 lands.