Work out what happened to a request

What you get at the end: for any request a caller complains about, the model that answered, the provider it went to, what it cost, how long it took, and whether it was refused - without guessing.

Use the request identifiers to correlate response headers, gateway logs and ledger records. The values shown below are examples.

Start from the request id#

Every response carries one, and every log line that request produced is tagged with it:

X-Request-ID: req-82792eaf

Collect request correlation#

Ask the complaining caller for this first. Without it you are searching by timestamp, and timestamps do not distinguish between two users with the same problem.

docker logs pointsman-gateway-1 --since 30m 2>&1 | grep "req-82792eaf"

For one request that gives you the whole story in order:

audit - event=llm_request  model=chat-tier  messages=1  streaming=false
ChatCompletionsController - sync request model=chat-tier
strategy=weighted-random chose openai:gpt-4o-mini (decisionId=5924a680-...)
OpenAiProvider - sync call model=gpt-4o-mini messages=1
audit - event=llm_response  model=chat-tier  latency_ms=773  tokens_prompt=11  tokens_completion=5

Read the decision path#

Read it as a sequence of decisions. What was asked for, what the router chose and why, which provider was actually called, and what came back. A request that went wrong is usually a request where one of those lines is missing, and which one is missing tells you where to look.

If you have no request id#

The gateway keeps a queryable record. It needs an organisation:

curl -s -b "$J" "localhost:8080/admin/observe/requests?org=$ORG&limit=20"

Each row carries model, provider, status, latency_ms, cost_usd, cache_hit, error_type, total_tokens and trace_id. Sort by latency to find the slow ones, filter by error_type to find the failing ones.

Troubleshoot administrative authentication#

omitting org returns 502 with "The upstream provider returned an error: Required request parameter 'org' ... is not present". No provider was involved. It is a missing query parameter wearing a provider error's clothes, and if you see that message with a parameter name in it, read it as a 400.

Reading the headers instead#

For a request you can reproduce, the response headers answer most questions without any log at all:

• Was it cached? X-Cache: HIT-L1 means the in-process exact-match tier answered, HIT-L2 the shared semantic one, MISS neither. A cache hit has no X-Served-Provider, because nobody was called - which is also why it was fast.
• Did it go where you expected? X-Provider-Model is the identifier the provider reported. X-Requested-Model is what the caller asked for. When they differ, X-Resolved-Model says why, in words: gpt-4o-mini (normalized from "GPT-4O-MINI").
• Did it fall back? X-Attempted-Models lists every model tried, in order. Its absence means the first choice answered.
• Was it rate limited? X-RateLimit-Remaining-Requests counts down. Its absence means no rule of that kind applied, which is not the same as no limit existing.

The full list, with the condition each one appears under: response headers.

Is it the gateway or the provider?#

curl -s localhost:8080/actuator/health

The components to read are db, counterStore and policyService. There is no redis component: the Redis auto-configuration and its indicator are excluded, and counterStore is the checked health of the Redis-backed counters. A degraded counterStore is the one that changes behaviour without failing anything: rate limits stop being enforced and requests are admitted, while budgets keep being enforced against the durable ledger. So "everything got faster and nothing is being limited" is a symptom with a specific cause.

policyService down means outcome-learning routing has fallen back to a static strategy. Traffic keeps flowing on the last published parameters; only learning stops.

Slow requests#

latency_ms in the observe rows is the whole request. Compare it against benchmarks.md, which records what this gateway's own overhead measures at, so you can tell a slow provider from a slow gateway. The short version: the gateway's own control plane is a small number of milliseconds, so a multi-second request is almost always the provider, and the fix is a routing change rather than a tuning change.

If it is genuinely the gateway, the first thing to check is whether the semantic cache is doing an embedding on the request path for prompts too short to benefit - X-Cache-Semantic: skipped-short tells you it correctly did not.