Phase 3 core memory substrate

Interval-overlap-first conflict detection after near-dup: auto-supersede sequential facts without LLM; escalate only on overlap or missing validity. ADR-0056.

Milestone 3.C.3 — Temporal-interval-aware conflict detection

Status: In progress (feature/m3-c-3-temporal-conflict)
Goal: Track C — Write Pipeline
Phase: 3 — Core Memory Substrate
Estimated effort: 4 days
Track: Track C — Write Pipeline
ADR: ADR-0056 (planning text said ADR-0042; that id is taken by the self-hosted adapter)


Why This Milestone Exists

This is the single biggest correctness fix in Phase 3. The original design treats any pair of near-duplicate memories above a similarity threshold as a candidate contradiction and fires an LLM call to classify it. The flaw: "User prefers Python" (March) and "User is switching to Go" (June) are not a logical contradiction — they are a sequential fact change, correctly modeled as supersedes, not contradicts.

The original conflict resolution table already distinguishes these types, but the detection step does not use time at all before calling the LLM — it relies entirely on the LLM prompt to figure out "these are actually sequential, not contradictory" from text alone, which is fragile and wastes LLM calls.

ADR-0056 captures the conflict model, interval semantics, and reviewable tradeoffs.


Non-Goals

  • Changes to the conflict resolution taxonomy (supersedes/contradicts/merge — kept as-is)
  • Inventing pending_review status without a migration (use existing CHECK + contradicts edge)
  • Subject extraction beyond what is needed for the interval check
  • GDPR deletion or archive flows
  • Production LLM worker (ships pluggable ConflictClassifier seam; default noop escalate)

Orientation (indicative)

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

  • open the live tree and follow existing patterns before inventing new ones
  • research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
  • advance the design beyond the sketch where measurement or code reality says so
  • land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

  • Memory service / repositories
  • Write pipeline / safety
  • Database schema / migrations

Suggested naming (provisional)

Rename freely to match the change that actually lands.

  • Branch: feature/m3-c-3-temporal-conflict
  • PR title: feat(memory): temporal-interval-aware conflict detection (m3.c.3)

ADR-0056 — Temporal-interval-aware conflict detection

See ADR-0056. It documents:

  1. Why interval-overlap-first beats similarity-only gating
  2. spaCy subject extraction (reuse en_core_web_md from ADR-0054)
  3. Missing valid_from → escalate (never silent skip)
  4. Status taxonomy without inventing pending_review
  5. Pending supersede targets until the new memory id exists post-insert

Redesigned Algorithm

1. Near-duplicate candidate supplied by Milestone 3.C.2 (cosine > threshold).

2. Temporal interval check (fast path):
 - Extract subject from new memory and candidate memory using spaCy dependency parse.
 - If new memory valid_from > candidate memory valid_from
 AND subjects refer to same attribute/entity
 AND intervals do not overlap:
 → classify as SUPERSEDES, no LLM call.
 → (persist later) memory_relationships supersedes + status=superseded.
 → DONE.

3. Interval overlap check (slow path, only if fast path did not fire):
 - If validity intervals overlap (or valid_from missing):
 → escalate to ConflictClassifier (LLM seam; noop default → escalate_pending).
 - If no interval overlap and not same attribute/entity:
 → no conflict, proceed to write.

Success signals

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

  • ADR-0056 written (planning referred to ADR-0042; id already taken)
  • Sequential-fact fixture classified as supersedes with zero LLM calls
  • Overlapping-interval fixture escalates via classifier seam
  • memory_relationships row helpers for supersedes / contradicts (org-scoped)
  • Integration tests drive persist helpers against Postgres (full write HTTP API later)
  • ibex_memory_conflict_llm_calls_total + ibex_memory_conflicts_total{outcome} metrics

Prerequisites

  • Milestone 3.1.1 merged (valid_from/valid_until columns and idx_memories_validity index in schema)
  • Milestone 3.C.2 merged (near-duplicate candidates supplied as input)
  • ADR-0056 accepted with implementation
Edit on GitHub

Last updated on

On this page

0%