Limits answer whether a request may consume resources now. Settlement answers what it actually consumed. Pointsman reserves before dispatch because counting only after completion allows concurrent requests to spend the same remaining capacity.
What is being limited#
| Resource | Purpose | Terminal treatment |
|---|---|---|
| Requests | Bound arrival rate | A request count is not an unused token reservation to refund |
| Tokens | Bound estimated prompt and completion demand | Reconcile the applicable estimate with terminal usage |
| Spend | Bound configured cost caps | Preserve incurred spend and release only unused reservation |
| Concurrency | Bound simultaneous work | Release acquired permits on success, refusal, failure and cancellation |
| Session controls | Apply a caller session's scoped rule or halt | Resolve with the request identity; a session label is not authentication |
Rules compose along the resolved scope path, with applicable platform ceilings and qualifiers. Independent rules remain independent counters; a loose child does not erase a parent's cap. Admission uses configured estimation/tokenizer behavior and registry prices. A cost estimate is a control input, not a provider invoice.
Two moments of admission#
Request admission resolves identity, allowed model or group, capabilities, context and output limits, then reserves everything knowable before the exact attempt. The engine can warn, throttle, degrade, block or halt a session. Throttling has a bounded hold and recheck. Degradation re-enters model resolution and admission, so a cheaper substitute cannot bypass the allow set or its own limits.
Attempt admission checks model-dependent rules for group members and fallback hops that the first moment could not cover. The originally admitted concrete model is not reserved twice. An unaffordable fallback is refused rather than treated as already paid for.
Atomicity and its boundary#
Redis Lua operations reserve a counter group atomically. Platform and tenant counters can occupy different hash-tag groups; this is not a distributed transaction over all groups. Platform reservation happens first, and a later refusal must release what was already acquired. Lost releases or counter state require the documented recovery procedures, not an assumption of global atomicity.
The usage ledger, durable reservation records and optional local usage journal serve different purposes. In the qualified profile, journal admission and durable handoff protect terminal accounting under the configured finite bounds. A full or unavailable journal may refuse new obligations. The durable delivery path records what must be replayed; it is not an unbounded queue or a guarantee across arbitrary disk loss.
Settlement and cancellation#
A request may have several physical provider attempts. Each attempt's usage and serving identity must remain attributable, including when fallback succeeds after an earlier failure. Terminal lifecycle guards avoid applying the same settlement twice. Complete, interrupted, cancelled and rejected outcomes remain distinct.
Unused reservations and concurrency permits are released. Provider work already incurred is not erased because an output guardrail blocked delivery or a client disconnected. Missing provider usage may require estimation and uncertainty disclosure; an absent success response does not prove zero provider charge.
Reducing settlement round trips#
The performance candidate batches charge inserts and reservation-state updates, and verifies all charged amounts with one query. Admission transitions update the selected reservation IDs together. An unused read during prepare is removed. These changes reduce database exchanges; they do not defer acknowledgement to an in-memory queue or remove persistence work.
Settlement still locks the reservation rows and budget-trust row, validates every
sibling, and commits the ledger row, charges and settled states in one database
transaction. A conflicting charge or missing sibling aborts the transaction.
Replay retains the original evidence sequence and deferred-operation identity;
it cannot apply a charge twice. Journal file fsync, atomic rename, directory fsync
and durable deletion ordering remain unchanged. The benchmark also verifies
PostgreSQL fsync, synchronous_commit and full_page_writes remain enabled.
The performance results are separate from the accepted long-run qualification of the earlier runtime.
Failure policy#
| Failure | Intended behavior |
|---|---|
| Rate/token counter operation fails | Some non-spend controls fail open and are observable; inspect the configured control |
| Enforcing spend cap cannot be verified | Default deny refuses with budget_unverifiable; unknown applicability also refuses |
| Explicit ledger-fallback mode | Bounded fallback reads only where the oracle can answer; it is not universal Redis replacement |
| Applicable budget exhausted | Budget denial; distinguish it from a transient rate/concurrency 429 |
| Counter state lost | Follow counter-loss recovery and independent spend attestation; a database backup alone cannot prove lost spend |
| Terminal store/journal pressure | Apply the configured bounded admission/delivery behavior; do not silently widen limits |
Why these tradeoffs#
Reservations can conservatively reduce available capacity while work is in flight, but protect against concurrent overspend. Redis gives a low-latency atomic primitive, while durable accounting provides recovery evidence. Failing closed on unverifiable spend prioritizes an owner's financial boundary; allowing selected non-spend controls to degrade favors availability. These choices must be visible to operators.
Implementation anchors: pipeline/RequestAdmission, ratelimit/LimitRuleEngine, RequestBudgetGate, core/limit/LimitResolver, counter Lua scripts, and the usage journal/recording components. See gateway controls, journal recovery, operating notes and operational limits.