# Work out what happened to a request

**What you get at the end:** for any request a caller complains about, the model that answered, the
provider it went to, what it cost, how long it took, and whether it was refused - without guessing.

Use the request identifiers to correlate response headers, gateway logs and ledger records. The values shown below are examples.

## Start from the request id

Every response carries one, and every log line that request produced is tagged with it:

```
X-Request-ID: req-82792eaf
```

### Collect request correlation

Ask the complaining caller for this first. Without it you are searching by timestamp, and
timestamps do not distinguish between two users with the same problem.

```bash
docker logs pointsman-gateway-1 --since 30m 2>&1 | grep "req-82792eaf"
```

For one request that gives you the whole story in order:

```
audit - event=llm_request  model=chat-tier  messages=1  streaming=false
ChatCompletionsController - sync request model=chat-tier
strategy=weighted-random chose openai:gpt-4o-mini (decisionId=5924a680-...)
OpenAiProvider - sync call model=gpt-4o-mini messages=1
audit - event=llm_response  model=chat-tier  latency_ms=773  tokens_prompt=11  tokens_completion=5
```

### Read the decision path

Read it as a sequence of decisions. What was asked for, what the router chose and why, which
provider was actually called, and what came back. A request that went wrong is usually a request
where one of those lines is missing, and which one is missing tells you where to look.

## If you have no request id

The gateway keeps a queryable record. It needs an organisation:

```bash
curl -s -b "$J" "localhost:8080/admin/observe/requests?org=$ORG&limit=20"
```

Each row carries `model`, `provider`, `status`, `latency_ms`, `cost_usd`, `cache_hit`,
`error_type`, `total_tokens` and `trace_id`. Sort by latency to find the slow ones, filter by
`error_type` to find the failing ones.

### Troubleshoot administrative authentication

 omitting `org` returns `502` with
`"The upstream provider returned an error: Required request parameter 'org' ... is not present"`.
No provider was involved. It is a missing query parameter wearing a provider error's clothes, and
if you see that message with a parameter name in it, read it as a `400`.

## Reading the headers instead

For a request you can reproduce, the response headers answer most questions without any log at all:

- **Was it cached?** `X-Cache: HIT-L1` means the in-process exact-match tier answered, `HIT-L2` the
  shared semantic one, `MISS` neither. A cache hit has no `X-Served-Provider`, because nobody was
  called - which is also why it was fast.
- **Did it go where you expected?** `X-Provider-Model` is the identifier the provider reported.
  `X-Requested-Model` is what the caller asked for. When they differ, `X-Resolved-Model` says why,
  in words: `gpt-4o-mini (normalized from "GPT-4O-MINI")`.
- **Did it fall back?** `X-Attempted-Models` lists every model tried, in order. Its absence means
  the first choice answered.
- **Was it rate limited?** `X-RateLimit-Remaining-Requests` counts down. Its absence means no rule
  of that kind applied, which is not the same as no limit existing.

The full list, with the condition each one appears under:
[response headers](../reference/response-headers.md).

## Is it the gateway or the provider?

```bash
curl -s localhost:8080/actuator/health
```

The components to read are `db`, `counterStore` and `policyService`. There is no `redis`
component: the Redis auto-configuration and its indicator are excluded, and `counterStore` is the
checked health of the Redis-backed counters. **A degraded
`counterStore` is the one that changes behaviour without failing anything**: rate limits stop being
enforced and requests are admitted, while budgets keep being enforced against the durable ledger.
So "everything got faster and nothing is being limited" is a symptom with a specific cause.

`policyService` down means outcome-learning routing has fallen back to a static strategy. Traffic
keeps flowing on the last published parameters; only learning stops.

## Slow requests

`latency_ms` in the observe rows is the whole request. Compare it against
[benchmarks.md](../performance.md), which records what this gateway's own overhead measures at, so
you can tell a slow provider from a slow gateway. The short version: the gateway's own control
plane is a small number of milliseconds, so a multi-second request is almost always the provider,
and the fix is a routing change rather than a tuning change.

If it is genuinely the gateway, the first thing to check is whether the semantic cache is doing an
embedding on the request path for prompts too short to benefit - `X-Cache-Semantic: skipped-short`
tells you it correctly did not.