Phase 3.5 extraction & assembly

Terminate sticky sessions, buffer turns in Redis, and enqueue extract_session_memories via an internal worker HTTP endpoint (Celery apply_async).

Milestone 3.5.D.4 — Extraction Task Enqueue

Status: Completed Goal: Track D — Proxy Integration
Phase: 3.5 — Extraction & Context Assembly
Estimated effort: 1 day
Track: Track D — Proxy Integration
Depends on: Milestone 3.5.D.2 (handler wiring)


Why This Milestone Exists

The proxy needs to mark sessions completed and trigger memory extraction once per completed session. Checkpoints store hashes only (ADR-0032 / ADR-0064), so turn text is buffered transiently in Redis on the chat path and read at terminate time.

Transport decision: the proxy does not speak Celery’s Redis wire protocol. It calls an internal worker HTTP endpoint that runs extract_session_memories.apply_async in Python. See ADR-0072 (supersedes the earlier LPUSH sketch on this page).


Orientation (indicative)

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

  • open the live tree and follow existing patterns before inventing new ones
  • research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
  • advance the design beyond the sketch where measurement or code reality says so
  • land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

  • Proxy service (HTTP terminate, turn buffer, enqueue client)
  • Session store (CompleteByExternalID)
  • Worker Starlette enqueue surface + Celery task

Success signals

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

  • POST /v1/sessions/{external_id}/terminate sets status=completed (sticky id + agent scope)
  • Extraction enqueue fires on first completion and on retry-after-failure recovery; never on active; never twice for the same successfully Ack’d buffer (ADR-0072)
  • Proxy enqueues via worker HTTP (apply_async in Python) — no hand-rolled Celery Redis envelope in Go
  • Turn text comes from the Redis turn buffer (or enqueue is skipped if empty) — never fabricated
  • Local Go/Python unit + worker coverage for D.4 surfaces
  • Repo guards / CI checks still pass

Prerequisites

  • Milestone 3.5.D.2 (handler wiring)
  • Phase 3.5 Track A (worker skeleton)
  • ADR-0072
Edit on GitHub

Last updated on