Phase 3 core memory substrate

Wire existing composite_score() into the semantic search read path so find_similar ranks by category-conditional composite score after retrieval — not raw cosine or ts_rank_cd order.

Milestone 3.D.2 — Composite Scoring in Search Ranking

Status: Complete
Goal: Track D — Read/Ranking Pipeline
Phase: 3 — Core Memory Substrate
Estimated effort: 2 days

ADR: ADR-0053 (composite scoring v2 formula and half-life table); FTS sentinel in ADR-0058


Why This Milestone Exists

Milestone 3.D.1 returns search hits in raw retrieval order (cosine similarity for vector, ts_rank_cd for GIN supplemental rows). Category-conditional composite scoring and the per-category half-life table already ship in Track B (app/scoring/, write hot-cache ZADD). 3.D.2 wires that scorer into the read path so find_similar and POST /v1/memories/search order results by composite_score() after merge/hydrate. HTTP similarity remains the retrieval metric; rank reflects post-composite order.

FTS supplemental hits use a fixed relevance sentinel (0.5) for composite scoring only — they have no cosine score. See ADR-0058.


Non-Goals

  • Re-implementing decay math or changing CATEGORY_HALF_LIFE_DAYS (shipped in 3.2.1)
  • Changing composite formula weights (validated separately in Phase 3.5's benchmark)
  • Introducing a relevance gate (that's Phase 3.5, milestone 3.5.C.3)
  • Read-side hot cache consumer (3.D.3)
  • Changing write hot-cache scoring (write/cache.py)

Orientation (indicative)

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

  • open the live tree and follow existing patterns before inventing new ones
  • research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
  • advance the design beyond the sketch where measurement or code reality says so
  • land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

  • Memory service / repositories
  • Vector store / search
  • Caching / Redis

Suggested naming (provisional)

Rename freely to match the change that actually lands.

  • Branch: feature/m3-d-2-composite-scoring-v2
  • PR title: feat(memory): wire composite scoring into search ranking (m3.D.2)

Working notes

Per-category half-life table (reference — implemented in 3.2.1)

Python
CATEGORY_HALF_LIFE_DAYS: Final[dict[str, float]] = {
 "factual": 180.0, # facts decay slowly
 "procedural": 120.0, # how-to knowledge is durable
 "preference": 45.0, # preferences drift moderately
 "behavioral": 30.0, # patterns can shift faster
 "episodic": 14.0, # events are most time-sensitive (original constant, now scoped correctly)
}

The half-life table lives in app/scoring/half_life.py today. Optional follow-up: expose values via environment variables for ops tuning without redeploy — not a 3.D.2 blocker.


Success signals

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

  • Formula weights unchanged (0.40/0.25/0.20/0.10/0.05) — scorer reused from 3.2.1
  • Regression test proves old factual outranks fresh episodic at equal vector relevance — unit (test_read_ranking_unit.py) + HTTP integration (test_composite_ranking_http.py)
  • Half-life table is data, not hardcoded per-branch logic — CATEGORY_HALF_LIFE_DAYS in half_life.py (3.2.1)
  • Composite scoring applied to search ranking results (app/read/ranking.py, find_similar)
  • Repo guards / CI checks still pass

Prerequisites

  • Milestone 3.D.1 (semantic search read path)
Edit on GitHub

Last updated on