Phase 4 operator platform

Management API endpoints for reading and overriding per-org/per-agent RPM, backed by Postgres with proxy hot-path reload via Redis pub/sub invalidation and a 30s poll fallback.

Milestone 4.B.2 — Rate Limit Configuration API

Status: Completed
Goal: Track B — Rate Limiting & Quota Enforcement
Phase: 4 — Operator Platform & Multi-Provider
Estimated effort: 2 days
Track: Track B — Rate Limiting & Quota Enforcement


Why This Milestone Exists

The Lua limiter (4.B.1) needs per-org and per-agent limits sourced from configuration, not hard-coded constants. Operators should be able to adjust these limits without a service restart. Configuration is stored in Postgres, read into the proxy's in-memory config on a 30s poll or Redis pub/sub invalidation — never a per-request DB read on the hot path.


Non-Goals

  • Rate-limit metering/billing integration (Phase 4.5)
  • Per-user rate limits
  • Global platform limit configuration (that's an operator-level infrastructure setting, not an API endpoint)
  • Token-spend / month_tokens ceilings (#806)

Shipped surface

GET  /v1/organizations/{org_id}/rate-limits
PATCH /v1/organizations/{org_id}/rate-limits
  • Migration 000028_rate_limit_overrides
  • packages/ratelimit config plane (config_* + mutable HierarchicalLimiter overrides)
  • Proxy bootstrap watcher (pub/sub + 30s poll)
  • Management API publish on ratelimit_config_updates:{org_id} after commit (publish fail does not roll back PG)

Success signals

  • PATCH reflected via pub/sub within ~1s or via ≤30s poll — unit tests record latencies
  • GET current_minute_requests uses Redis org RPM key (or 0 if Redis optional/unavailable)
  • owner/admin PATCH OK; member/viewer PATCH → 403
  • TestAPI_ISO_RATELIMIT_*: foreign org GET/PATCH → 404

Prerequisites

  • M4.A.1 skeleton merged
  • M4.B.1 Lua limiter merged (needed for counter key namespace)
Edit on GitHub

Last updated on