Phase 3.5 extraction & assembly

Scoring-time relevance floor on the memory read path before composite_score (ADR-0068), distinct from ADR-0053 retrieval min_similarity.

Milestone 3.5.C.3 — Composite Scorer v2 (Relevance-Gated)

Status: Completed Goal: Track C — Context Assembly Engine
Phase: 3.5 — Extraction & Context Assembly
Estimated effort: 2 days
Track: Track C — Context Assembly Engine
Depends on: 3.5.C.2 + composite ranking from 3.D.2
ADR: ADR-0068


Why This Milestone Exists

Without a scoring-time gate, a near-zero relevance candidate with high confidence / usefulness / access frequency can outrank a highly relevant memory when retrieval is opened below the production ANN cutoff. Reweighting alone leaves that failure mode open. A hard relevance floor excludes irrelevant candidates before they enter the weighted sum.

Category-conditional half-lives and formula weights already shipped in 3.D.2 and are out of scope here (see 3.D.2 Non-Goals, which deferred this gate to 3.5.C.3).


Non-Goals

  • Packing (3.5.C.4)
  • Changing the proto wire contract
  • Changing CATEGORY_HALF_LIFE_DAYS or composite formula weights
  • Adding scoring code under services/context/

File layout (as shipped)

services/memory/app/scoring/relevance_gate.py   # passes_relevance_floor()
services/memory/app/scoring/composite.py        # score() / composite_score() unchanged (pure sum)
services/memory/app/read/ranking.py             # gate inside rank_hydrated_hits before composite_score
services/memory/app/read/repository.py          # passes Settings.composite_relevance_floor
services/memory/app/config.py                   # composite_relevance_floor (default 0.15)

Settings / env:

  • IBEX_COMPOSITE_RELEVANCE_FLOOR / IBEX_MEMORY_COMPOSITE_RELEVANCE_FLOOR (default 0.15)

This is distinct from retrieval-time vector_search_min_similarity / min_similarity (ADR-0053, default 0.70).

Planning sketches that placed a MemoryScorer under services/context or reused ADR-0040 were stale: ADR-0040 is the Anthropic provider adapter; context assembly consumes already-ranked results and does not own scoring.


Design

Python
# Gate FIRST in rank_hydrated_hits — before composite_score
relevance = relevance_for_composite(ranked)  # vector cosine or FTS sentinel 0.5
if not passes_relevance_floor(relevance, relevance_floor):
    continue  # never scored
composite = composite_score(hit.composite_inputs(relevance, now=reference))

Floor must be < 0.5 (settings enforce this). Gate semantics: FTS sentinel 0.5 fails only when floor > 0.5.


Empirical floor choice

Swept {0.10, 0.15, 0.20, 0.25, 0.30} against gold-set default retrieval semantics (min_similarity=0.5 in bench_ranking_quality.py). Neutrality of precision@5 / recall@10 / MRR versus baseline 1.0 under that cutoff is by construction (hits already have relevance ≥ 0.5). The gold set also includes gold.noise.relevance_gate_bait; the bench runs a separate opened-retrieval relevance_gate_probe (min_similarity=0.0) that fails unless the bait is excluded — see ADR-0068. Default floor 0.15.


Success signals

  • Relevance floor gate unit-tested: below excluded; at/above scored; empty / all-below → []
  • Classic noise case (similarity≈0.05, high non-relevance components) excluded, not merely ranked low
  • Gold-set metrics preserved on default queries (precision@5 / recall@10 / MRR = 1.0); opened-retrieval gate probe excludes synthetic bait
  • ADR-0068 accepted and linked; half-lives / weights untouched
  • No scoring or proto changes under services/context/
  • AssemblyOptions weight overrides documented as unwired until 3.5.C.6 (apply after the gate when connected)
Edit on GitHub

Last updated on