# Limits, budgets, and the other controls

Limit rules control requests, tokens and spend before provider dispatch. This guide also explains context admission and provider health. For the underlying flow, see [rate limiting and settlement design](design/rate-limiting.md).

| To configure | Start here |
|---|---|
| Keys and budgets | [Issue keys and set budgets](how-to/issue-keys-and-set-budgets.md) |
| Content inspection | [Guardrails](guardrails.md) |
| Identity and permissions | [Authentication](design/authentication.md) and [authorization](design/authorization.md) |
| Runtime values | `GET /admin/control/effective` |

## One rule shape, not four features

Rate limits, token limits, and spend budgets are the same object here: a **limit rule**. There
is no separate budget system to learn - a monthly dollar cap and a per-minute request cap are
the same JSON with different words in three fields:

```json
{
  "applies_when": {"model": "gpt-4o"},
  "count_by":     ["credential"],
  "amount":       "cost",
  "budget":       25,
  "window":       {"kind": "calendar", "count": 1, "unit": "month",
                   "anchor_zone": "UTC", "anchor_reset_time": "00:00", "anchor_day": 1},
  "action":       "block"
}
```

Read it as a sentence: *when* a request matches (`applies_when`), count its `amount` into a
counter keyed by `count_by`, over `window`; when the counter would pass `budget`, take
`action`. Every part has a closed vocabulary, and an unknown word is refused at the door
rather than stored and ignored:

- **`amount`** - what is counted: `requests`, `tokens`, `input_tokens`, `output_tokens`, or
  `cost` (dollars, priced through the same chokepoint the ledger uses).
- **`applies_when` / `count_by`** - eleven attributes: `scope`, `principal`, `credential`,
  `session`, `agent`, `task`, `model`, `provider`, `endpoint`, `environment`,
  `request_fingerprint`. `applies_when` decides which requests the rule governs; `count_by`
  decides how many counters it fans into. "50 requests a minute per credential per model" is
  `count_by: ["credential", "model"]` - one rule, a counter per pair.
- **`window`** - `rolling` (the last N units, sliding) or `calendar` (resets at an anchor:
  zone, time, day). A monthly budget that resets at midnight UTC on the 1st is a calendar
  window; "no more than X in any 10 minutes" is a rolling one.

Rules attach to the tenancy tree - `POST /admin/organizations/{org}/rules` is the common door -
and compose downward: a parent's rule counts a child's traffic too, so a child cannot escape
what a parent caps - but the resolver does not enforce "only tighten": rules with identical
identities compose tightest-wins, and everything else stacks as independent counters, each
able to refuse on its own.

## The five actions, and what each is for

Each action has a distinct effect when its rule reaches the limit.

| Action | Request behavior | Configuration or observation |
|---|---|---|
| `warn` | Count and emit; continue serving | Advisory event, not a timed state; a duration is refused |
| `block` | Refuse | Refusal includes the deciding counter's limit and remaining capacity |
| `throttle` | Hold and recheck within a bound | Wait derives from counter refill rate |
| `degrade` | Try the model named by `degrade_to` | Response discloses `X-Degraded`; substitute must pass admission |
| `halt_session` | Refuse and persist a session halt | Requires `action_duration_seconds`; other sessions on the key are unaffected |

## What enforcement actually is

Enforcement follows this sequence:

1. Estimate applicable resource demand before calling a provider.
2. Reserve all counters within a Redis slot group atomically, or reserve none of them.
3. If a platform ceiling and tenant rules both apply, reserve the platform group first, then the tenant group.
4. If tenant admission refuses, release the platform reservation best-effort. A lost release can conservatively count high until its window rolls.
5. After work completes, reconcile usage and release unused reservation and concurrency permits.

Atomicity is per slot group, not a distributed transaction across platform and tenant groups. A refusal before dispatch incurs no provider work.

Applied rules expose headroom through `X-RateLimit-Remaining-Requests` and `X-RateLimit-Remaining-Tokens`. An absent header means no rule of that kind matched; check the rule's scope if you expected one to apply.

## When Redis is down

Counter availability and counter trust are separate conditions. Cache Redis has its own clients and failure policy; cache fail-open does not make budgets fail open.

### Counter-store outage

The configured budget failure mode determines how capped traffic behaves when the store does not answer.

| Control or mode | Behavior | Correctness boundary |
|---|---|---|
| Request/token rate limits | Fail open with observable uncounted admission | Can permit unmetered bursts throughout an outage |
| Spend `failure-mode: deny` (default) | Refuse enforcing spend caps or unresolved applicability with `503 budget_unverifiable`; no ledger balance read | No atomic reservation can be established |
| Spend `failure-mode: allow` | Use bounded historical ledger reads where answerable; skip and count unanswerable caps | Reads do not reserve; concurrent admissions can overspend |
| Confirmed absence of an enforcing spend cap | Pass this outage gate | Authentication, policy and other admission checks still apply |

The optional ledger fallback can answer caps keyed by credential, user, application scope or organization scope. It cannot answer authored deep-scope, platform, model or session caps; these increment `pointsman.budget.oracle.unanswerable`.

`pointsman.budget.ledger-fallback-max-concurrent` bounds fallback reads. If no permit is available, the request receives 503. Request-path spend sums have a one-second execution bound. Advisory reads also take permits on the healthy path; fenced requests do not query a balance that cannot change their refusal. Administrative spend reads are a separate operator query.

### Counter state cannot be trusted

An enforcing cost rule returns **`503 budget_recovering`** when its trusted epoch cannot be established. This applies in both failure modes.

The atomic reserve checks detect these conditions:

- The tenant-slot epoch is absent or belongs to another generation.
- The Redis process differs from the one that wrote the epoch, including restart, failover or reload.
- A counter expected inside the epoch is missing.

The fence prevents lost state from appearing as new unspent capacity. Even a clean Redis restart requires recovery evidence. A `warn` cost rule remains advisory; request and token limits are unaffected by this trust fence.

> **Recovery:** restore or independently attest the balance using the [operating procedure](operating-notes.md). Do not clear counters or restart with empty state to bypass the fence. The initial upgrade adopts existing counters once; this is not a recurring recovery mechanism.

### Store checks and recovery signals

The counter store needs `noeviction` and an ACL denying destructive commands such as `RESTORE` and `FLUSHALL` to the gateway user. Check the configured `store-checks` mode: reporting and enforcing a failed check have different effects.

| Check outcome | Behavior |
|---|---|
| Enforced check fails | Capped traffic stays fenced with `budget_recovering` |
| Counter breaker closes | Request an asynchronous fresh check instead of waiting for the next scheduled interval |
| Checks overlap or the breaker flaps | Coalesce work into one check and at most one follow-up per gateway |
| `ACL DRYRUN` errors | Report denial as unverified; an unreadable check is not a pass |
| Genuine counter loss | A successful connectivity/store check alone cannot restore the balance; recovery or attestation is still required |

## The context window

Input and output share a model's context window. Context admission and budget reservation use the same estimator implementation, including tool definitions; the estimate reports its coverage and uncertainty.

### Admission rules

| Request or model condition | Admission behavior |
|---|---|
| Caller supplies an output cap | Require the requested cap in full under the applicable policy; do not silently shrink it |
| Caller omits the output cap | Clamp the assumed output to available room for context admission; the budget reservation still uses its separate flat default |
| Model declares a separate input ceiling | Check that ceiling as well as the combined window |
| Fallback selects another model | Reapply the same context eligibility gate |
| Registry has no declared window | No local window pre-check is possible; the provider can still refuse |

A capless request is checked for input fit, not a guaranteed answer size. The context-admitted output and the flat budget reservation are distinct values.

### Admission policy

Set `pointsman.context.admission-policy` according to the behavior you require.

| Value | Requirement |
|---|---|
| `provider-compatible` (default) | Allow an over-ask only when the selected model row explicitly declares `context_overflow: TRUNCATE` |
| `reserve-output` | Require room for the complete requested output on every model |

Truncation capability comes from the model row, not its adapter. Without that declaration, both policies refuse the over-ask. Anthropic's `model_context_window_exceeded` maps to `finish_reason: "length"` on streaming and non-streaming responses. The caller cannot distinguish a requested-cap stop from a context-window stop through that field alone.

### Counting accuracy

| Mode | Strength | Limit |
|---|---|---|
| Fast estimate | Low-cost approximate counting | Can substantially undercount code, JSON and non-Latin text |
| `TOKENIZER` | Uses a declared, warmed model encoding with bounded concurrency | Does not establish provider-exact chat framing, tool scaffolding or image accounting |

For a routing group, tokenizer mode counts using the candidate rows' encodings and takes the largest result. Every candidate must declare an encoding; a missing encoding produces 400. Select the mode and test representative prompts using the [configuration reference](reference/configuration.md).

## The context-window pre-check

A model whose registry row declares `context_window` refuses an over-long prompt **before the
network call** - a zero-cost 400 naming the limit, instead of a provider round-trip that was
always going to fail. A model with no declared window cannot be pre-checked; the failure then
comes back from the provider and enters the fallback ladder like any upstream error. Declaring
windows on your model rows is the cheap fix.

## Provider health, and what routing does with it

Health is tracked per provider and consumed by routing automatically:

- **Probes** - a one-token call you trigger from the registry
  (`POST /admin/registry/providers/{id}/probe`), advisory, recorded on the row.
- **Circuit breakers** - repeated *retryable* failures open a breaker and take the provider
  out of candidacy until it half-opens. Caller mistakes (4xx-class errors) never open one:
  one malformed request must not take a provider away from everybody.
- **Cooldowns** - a 429 puts the specific key on a cooldown clock rather than tripping the
  breaker: rate pressure heals by waiting, and probing again on the clock is the fastest
  honest recovery. Billing exhaustion also cools the key; inspect the provider account and cooldown reason.

None of this needs configuration to be safe: the fallback ladder re-derives its candidates
against live breaker and cooldown state on every hop. The strategy layer above it -
including what each strategy does badly - is [routing-strategies.md](routing-strategies.md).

### "My requests stopped reaching a provider" - three conditions, three answers

Use the condition and its evidence to choose the response.

| Condition | Gateway behavior | What to inspect |
|---|---|---|
| Provider rate limit (429) | Cool the affected credential; siblings can still serve; do not open the provider breaker | Cooldown log, `pointsman_provider_credential_cooled_total`, `pointsman_provider_credential_live` |
| Tenant budget exhausted | Refuse once no affordable candidate remains; no provider cooldown | `402 budget_exhausted`, cheapest estimate and remaining balance |
| Provider billing exhaustion | Cool the affected key as a billing condition, not a breaker failure | Cooldown reason and provider account balance |
| Provider outage (retryable 5xx, timeout or connection fault) | Record breaker failure; exclude an open provider and follow eligible fallbacks | `resilience4j_circuitbreaker_state`, `X-Served-Provider`, `X-Model-Resolution` |

#### Cooldown timing

The gateway reads provider timing in this order:

1. `Retry-After` as seconds or an HTTP date.
2. `retry-after-ms`.
3. The earliest supported reset header: OpenAI `x-ratelimit-reset-*` durations or Anthropic `anthropic-ratelimit-*-reset` timestamps.
4. `POINTSMAN_ROUTING_COOLDOWN_SECONDS` when no timing is available (default five seconds).

`POINTSMAN_ROUTING_COOLDOWN_MAX_SECONDS` caps the wait (default five minutes). Logs disclose the provider's requested interval and any clamp. A probe that receives another 429 relearns the interval. Rate limiting is a capacity signal and does not teach answer quality.

#### Breaker scope

The provider breaker opens after at least five recorded calls when half or more have failed. It waits ten seconds, then allows three half-open probes. It applies to the provider, while cooldown applies to a credential.

A pre-attempt substitution can report `fallback(<model> -> <served>, pre-attempt, unavailable)`. `unavailable` deliberately does not expose another caller's detailed failure history. Every fallback still passes health, permissions, context and budget eligibility.

## Reading what is in force

Use `GET /admin/control/effective` to inspect control values, provenance and whether a restart is required. The dashboard renders this surface; a configured value still needs request-level verification when validating enforcement.