Milestone 3.5.B.1 — Extraction Prompt v2 and Structured Output ContractCopy Anchor Link

Status: Completed Goal: Track B — Extraction Pipeline
Phase: 3.5 — Extraction & Context Assembly
Estimated effort: 3 days
Track: Track B — Extraction Pipeline
ADR required: ADR-0063 — Extraction output schema v2 Depends on: 3.5.A.1

Why This Milestone ExistsCopy Anchor Link

Define what gets extracted and in what shape, before wiring up execution. The original prompt design was reasonable as a v1 — the redesign keeps its five-category taxonomy as the default label set but changes the output schema to be multi-label and temporally aware, consistent with the Track C (Phase 3) conflict-detection redesign that requires valid_from/valid_until on memories.

categories maps onto the existing write-path labels[] contract from ADR-0048 / milestone 3.C.4 ({label, confidence} pairs, enum-checked against factual / preference / behavioral / episodic / procedural). Extraction caps at 1–3 labels per memory (write path allows up to 5).

Non-GoalsCopy Anchor Link

• Actual execution/provider wiring (3.5.B.2)
• Cost tiering and batching (3.5.B.2)
• Quality evaluation harness (3.5.B.4)

Orientation (indicative)Copy Anchor Link

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

• open the live tree and follow existing patterns before inventing new ones
• research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
• advance the design beyond the sketch where measurement or code reality says so
• land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

• Extraction pipeline
• Workers / task runtime

Suggested naming (provisional)Copy Anchor Link

Rename freely to match the change that actually lands.

• Branch: feature/m3-5-b1-extraction-prompt-v2
• PR title: feat(worker): extraction prompt v2 and structured output schema (m3.5.B.1)

ADR-0063 — Extraction Output Schema v2Copy Anchor Link

Why multi-label, not single-category: A statement like "I always deploy on Fridays, but this week I'm switching to Mondays because of the incident" is simultaneously behavioral (a pattern) and episodic (a one-off change) and procedural (an instruction going forward). Forcing a single category loses information the scorer needs later. Each category is a {label, confidence} object matching POST /v1/memories labels[] (1–3 items for extraction).

Why temporal fields on extraction, not just storage: The extraction LLM is the only place with access to the conversational signal of "this replaces X" or "this is only true until Y." Pushing temporal reasoning to the write pipeline's dedup step throws that signal away. valid_from defaults to the turn's timestamp; valid_until is nullable, explicit, and validated client-side to match memories_valid_interval_chk (valid_until IS NULL OR valid_until > valid_from).

Why keep overall confidence scoring: Unchanged from v1 — confidence is a float in [0.0, 1.0] (same semantic as write-path memory confidence). A future review gate at < 0.5 may be wired in 3.5.B.2; the live write orchestrator does not invent a pending_review status on memories.status (ADR-0056 / ADR-0057).

Extraction SchemaCopy Anchor Link

# Illustrative — exact path may differ (live: services/worker/app/extraction/schema.py)
from __future__ import annotations
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, Field, field_validator, model_validator

VALID_LABELS = frozenset({"factual", "preference", "behavioral", "episodic", "procedural"})
MemoryLabel = Literal["factual", "preference", "behavioral", "episodic", "procedural"]

class ExtractedMemoryLabel(BaseModel):
    label: MemoryLabel
    confidence: float = Field(ge=0.0, le=1.0)

class ExtractedMemory(BaseModel):
    content: str = Field(min_length=5, max_length=10_000)  # plus UTF-8 octet_length <= 10000
    categories: list[ExtractedMemoryLabel] = Field(min_length=1, max_length=3)
    confidence: float = Field(ge=0.0, le=1.0)
    valid_from: datetime | None = None
    valid_until: datetime | None = None

    @field_validator("categories")
    @classmethod
    def categories_unique_known(cls, v: list[ExtractedMemoryLabel]) -> list[ExtractedMemoryLabel]:
        # reject unknown labels + duplicates; map 1:1 onto write-path labels[]
        ...

    @model_validator(mode="after")
    def valid_interval(self) -> ExtractedMemory:
        # mirror DB: valid_until IS NULL OR valid_until > valid_from
        ...

class ExtractionResult(BaseModel):
    memories: list[ExtractedMemory] = Field(default_factory=list)

    @field_validator("memories")
    @classmethod
    def cap_per_turn(cls, v: list[ExtractedMemory]) -> list[ExtractedMemory]:
        if len(v) > 10:
            raise ValueError("at most 10 memories per turn")
        return v

Extraction Prompt v2 DeltaCopy Anchor Link

Keeps original rules 1–4 unchanged, adds:

5. A memory may belong to 1-3 categories if applicable (e.g. a stated
 one-time preference change is both "preference" and "episodic"). Avoid forcing
 a single category if the content spans multiple. Emit each as
 { "label": "<enum>", "confidence": <0..1> }.
6. If the memory describes something true only for a limited time or scope
 (e.g. "for this migration", "until Friday"), set valid_until to the best
 estimate of the ISO-8601 end date. Otherwise omit it (memory is indefinite).

Success signalsCopy Anchor Link

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

• ExtractionResult.model_validate() round-trips 50 hand-labeled example conversation turns without validation errors
• Multi-category output verified against ≥10 crafted examples that span 2 categories
• valid_until correctly parsed for ≥5 crafted "temporary fact" examples, correctly None for indefinite facts
• ADR-0063 published with example extractions for both the single- and multi-category cases

PrerequisitesCopy Anchor Link

• 3.5.A.1 Celery worker skeleton