Phase 3 core memory substrate

Mirror Phase 1's TestSecurity_SEC* pattern for the memory substrate: 10+ ISO-* cases proving tenant isolation end-to-end with real Postgres+pgvector+Redis, CI-gated via memory-security-integration job.

Milestone 3.E.1 — Memory Security Integration Test Suite

Status: Complete
Goal: Track E — Exit Gate
Phase: 3 — Core Memory Substrate
Estimated effort: 3 days

ADR: ADR-0060


Why This Milestone Exists

Mirrors Phase 1's 1.5.1 pattern: prove the composed system (real Postgres with RLS, real pgvector HNSW index, real Redis hot cache) enforces tenant isolation end-to-end — not just that each service's unit tests pass in isolation. Individual milestone acceptance criteria only verify their own narrow slice; this gate catches emergent security failures when the pieces are composed together.


Test Matrix

Test IDScenarioExpected
ISO-1.1Org A writes memory, Org B searches with identical query textEmpty result set, not an error
ISO-1.2Org A agent ID used with Org B's PAT/org_id context403, mirrors proxy's AGENT_NOT_AUTHORIZED semantics
ISO-1.3Direct SQL query bypassing service layer, SET LOCAL app.current_org_id to Org AOrg B rows invisible even with malformed WHERE clause (RLS floor)
ISO-1.4memory_relationships edge referencing a memory from a different orgInsert rejected
ISO-1.5HNSW ANN search across a mixed corpus (Org A + Org B rows in same index)Zero cross-org leakage in top-K at every ef_search value
ISO-1.6Redis hot cache: same agent_id UUID in two orgsNo cross-tenant cache read possible ({org_id}:hot_memories:{agent_id} namespacing — org_id is leading segment)
ISO-1.7memory_conflict_escalations row for Org A queried under Org B RLSCount 0
ISO-1.8Hot cache: Org A agent B cannot read Org A agent A hot memoriesEmpty result set
ISO-2.1Hard DELETE of a memories rowmemory_labels, memory_relationships, and memory_conflict_escalations cascade per FK (ON DELETE CASCADE)
ISO-2.2Org-scope GDPR delete + MinIO archive removalDeferred to Phase 4.A.2 (#641) — not Phase 3
ISO-3.1PII write-path bypass via direct DB write (Tier-1 structured PII, e.g. email)Read-path Tier-1 regex re-check blocks return; Tier-2 contextual bypass is a documented residual risk (#642)
ISO-3.2Quarantined memory (status='quarantined')Never returned by search, regardless of similarity score

Test Infrastructure

Reuses the securityTestEnv-equivalent pattern, ported to Python. CI uses GitHub Actions service containers (pinned pgvector/pgvector:0.8.6-pg16@sha256:ccc6e83d6e35e931dc7c5def2022729d5a6c370318d099181995567ff1fb4d6b + redis:7.4.2@sha256:fbdbaea47b9ae4ecc2082ecdb4e1cea81e32176ffb1dcf643d422ad07427e5d9) — same pattern as security-integration and memory-integration, not testcontainers in merge gates.


CI Wiring

New job in .github/workflows/ci.yml, modeled on the existing security-integration job. Must use a pgvector-enabled Postgres image (pgvector/pgvector:0.8.6-pg16@sha256:ccc6e83d6e35e931dc7c5def2022729d5a6c370318d099181995567ff1fb4d6b) and a digest-pinned Redis service (redis:7.4.2@sha256:fbdbaea47b9ae4ecc2082ecdb4e1cea81e32176ffb1dcf643d422ad07427e5d9), not mutable redis:7 tags.


Orientation (indicative)

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

  • open the live tree and follow existing patterns before inventing new ones
  • research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
  • advance the design beyond the sketch where measurement or code reality says so
  • land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

  • Evaluation / benchmarks
  • Memory service / repositories

Success signals

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

  • All 12 ISO-* cases pass with real Postgres+pgvector+Redis (GHA service containers)
  • memory-security-integration added to ci-gate-python / ci-gate-go
  • Tier-1 read-path PII guard on search + hot-cache hydrate
  • ISO-2.2 (org-scope GDPR + MinIO cascade) deferred to Phase 4.A.2 (#641)
  • Tier-2 async PII re-scan deferred (#642) — documented residual risk in SECURITY.md §7.6
  • Repo guards / CI checks pass

Prerequisites

  • Milestones 3.D.1–3.D.3 (read/ranking pipeline complete)
  • Phase 3 Track C milestones (write pipeline)
Edit on GitHub

Last updated on