Milestone 4.B.1 — Lua-Based Atomic Hierarchical Rate LimiterCopy Anchor Link

Status: Completed
Goal: Track B — Rate Limiting & Quota Enforcement
Phase: 4 — Operator Platform & Multi-Provider
Estimated effort: 3 days
Track: Track B — Rate Limiting & Quota Enforcement

Why This Milestone ExistsCopy Anchor Link

The Phase 1 limiter was explicitly a placeholder: "org-level token bucket... NOT atomic... Phase 4 will replace with Lua scripts for atomic check-and-decrement," and the RateLimiter interface was deliberately designed to be swappable without touching callers. This milestone cashes in that design decision.

Rate limiting sits in the proxy's hot path and directly gates cost — it should be hardened before multi-provider routing adds more ways to burn budget (each provider has different pricing, so per-org token-spend limits, not just RPM, become necessary once two providers are live).

Non-GoalsCopy Anchor Link

• Rate-limit config API (Milestone 4.B.2)
• Load benchmark (Milestone 4.B.3)
• Per-user (as opposed to per-agent/per-org) rate limits
• Monthly token-spend ceiling / post-response decrement (month_tokens) — deferred to #806

Orientation (indicative)Copy Anchor Link

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

• open the live tree and follow existing patterns before inventing new ones
• research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
• advance the design beyond the sketch where measurement or code reality says so
• land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

• Rate limiting
• Proxy service (HTTP, bootstrap, config)

Suggested naming (provisional)Copy Anchor Link

Rename freely to match the change that actually lands.

• Branch: feature/m4-b-1-lua-hierarchical-limiter
• PR title: feat(proxy): atomic Lua hierarchical rate limiter agent→org→global (m4.B.1)

The Lua Script (Atomic Three-Level Check)Copy Anchor Link

As shipped in packages/ratelimit/hierarchical.go (INCR-then-compare; EXPIRE only on first increment; rollback inner keys on deeper-tier deny):

-- KEYS[1]=agent (unused when disabled), KEYS[2]=org, KEYS[3]=global
-- ARGV[1]=agent_limit, ARGV[2]=org_limit, ARGV[3]=global_limit, ARGV[4]=ttl_sec, ARGV[5]=agent_enabled ("0"|"1")
-- Returns: {allowed (0|1), tier, org_count, org_limit}
local agent_enabled = ARGV[5] == "1"
local agent_limit = tonumber(ARGV[1])
local org_limit = tonumber(ARGV[2])
local global_limit = tonumber(ARGV[3])
local ttl = tonumber(ARGV[4])

local function incr_expire(key)
  local n = redis.call("INCR", key)
  if n == 1 then
    redis.call("EXPIRE", key, ttl)
  end
  return n
end

local function org_get()
  return tonumber(redis.call("GET", KEYS[2]) or "0")
end

if agent_enabled then
  local agent_count = incr_expire(KEYS[1])
  if agent_count > agent_limit then
    return {0, "agent", org_get(), org_limit}
  end
end

local org_count = incr_expire(KEYS[2])
if org_count > org_limit then
  if agent_enabled then
    redis.call("DECR", KEYS[1])
  end
  return {0, "org", org_count, org_limit}
end

local global_count = incr_expire(KEYS[3])
if global_count > global_limit then
  if agent_enabled then
    redis.call("DECR", KEYS[1])
  end
  redis.call("DECR", KEYS[2])
  return {0, "global", org_count - 1, org_limit}
end

return {1, "", org_count, org_limit}

Design notesCopy Anchor Link

1. Check-most-specific-first ordering (agent → org → global): an individual noisy agent is throttled before it can exhaust the org's shared budget, giving better fairness across an org's fleet of agents than a flat org-only counter.

2. EVALSHA with NOSCRIPT fallback to EVAL, script SHA cached at process startup and re-verified on NOSCRIPT error (Redis flushed scripts on restart) — avoids re-sending the script body on every request.

3. Fail-open on Redis error is preserved, unchanged — rate limiting remains "cost control, not a security boundary" per ADR-0015. This redesign makes the allowed path atomic; it does not change the degraded path's philosophy.

4. Token-spend budgets (month_tokens): deferred — see #806. RPM hierarchy only in this milestone.

Redis Key Namespace (as implemented)Copy Anchor Link

ratelimit:{org_id}:agent:{agent_id}:rpm:{unix_minute} → per-agent RPM counter
ratelimit:{org_id}:rpm:{unix_minute} → per-org RPM counter (unchanged from Phase 1)
ratelimit:global:rpm:{unix_minute} → global RPM counter

month_tokens keys are deferred (#806).

Success signalsCopy Anchor Link

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

• Lua script via redis.NewScript + .Run() (EVALSHA→EVAL); hierarchical unit tests cover deny paths
• Limiter.Check signature unchanged — bootstrap swaps RedisSlider for HierarchicalLimiter
• Fail-open on Redis outage verified unchanged (SEC-4.6)
• All three levels (agent, org, global) tested independently
• Token-spend post-response decrement — deferred (see sketch deviations)

PrerequisitesCopy Anchor Link

• Phase 3.5 exit: proxy hot path live
• Redis available (already true since Phase 1)