# Chat Completions compatibility

[Documentation](../public-index.md) / Reference / API compatibility

This is the gateway's launch contract, not a claim that every OpenAI client feature
or every upstream model is supported. Adapter forwarding and provider/model support
are separate requirements. Check your actual client and model before rollout.

## Endpoints

| Endpoint | Launch behavior |
|---|---|
| `POST /v1/chat/completions` | Nonstreaming JSON and streaming SSE, subject to the controls below. |
| `GET /v1/models` | Model discovery. |
| `GET /v1/models/{id}` | One caller-available entry; unknown or disallowed entries return 404. |
| `POST /v1/feedback` | Pointsman outcome feedback, with decision ownership/provenance checks. |
| Responses, embeddings, audio, batch, rerank | Not implemented as gateway APIs. |
| MCP or tool execution | Not implemented. The gateway does not execute tools. |

For authentication, curl examples, response fields and errors, see the
[application API guide](client-api.md).

Use a Pointsman virtual key in `Authorization: Bearer ...`. Admin authentication is
a separate surface; see the [admin API](admin-api.md).

## Request controls

| Field or behavior | Support and boundary |
|---|---|
| `model`, `messages` | Core request fields; model resolution and access rules apply. In the shipped Compose profiles an unresolvable `model` returns `400 model_unresolvable` (opt-in fallback: `POINTSMAN_RESOLUTION_UNKNOWN_MODEL=fallback-to-allowed`). Message roles are validated; capitalized roles are refused. In the v1 exact-tokenizer profile the prompt is counted after resolution, with the token encoding of the model or group the request resolves to: an unknown name follows the key's resolution policy exactly as in estimate mode (`fallback-to-allowed` serves an allowed model; `reject` and `resolve-only` refuse with `400 model_unresolvable`; a model outside the key's allow set is `403 model_not_allowed`). An omitted `model` is a different case, governed by `pointsman.resolution.omitted-model`. Under `route` (the default) it is served: the key's `default_model`, else the tenant's default model, else the semantic group match, else the smart tier, else the cheapest allowed model, always inside the key's allowed set and disclosed in `X-Model-Resolution`. Under `refuse` it gets `400 model_required`. A key's policy may override the setting (`omitted_model`). A resolved model, or a group member, whose row declares no `token_encoding` is refused with `400 unsupported_request_control`, and a fallback rung whose encoding the request was not counted with is skipped. |
| Text content | Supported. |
| `image_url` content | **Not supported in the v1 release profile**: its exact tokenizer mode cannot count image tokens and refuses the request with `400 unsupported_request_control`. Outside that profile, estimate mode forwards images to a vision-capable model; its fast estimates do not establish provider-exact image counts. |
| `stream` | Supported, with the streaming caveats below. |
| `max_tokens`, `max_completion_tokens` | One positive output cap under two names. Conflicting values or a value below 1 are rejected. Provider quirks select the outbound field name. |
| Omitted output cap | The gateway supplies a cap, bounded by its default (4,096), model output limit and available context. Set an explicit cap for a different requirement. |
| `temperature` | Supported where the selected provider/model supports it, and sent as given, including 0. **Omitted means omitted.** No temperature is sent, and the provider default applies, unless the operator configured a default: a row's single-valued `temperatureClamp`, or `spring.ai.openai.chat.options.temperature` / `spring.ai.anthropic.chat.options.temperature` for the code-backed adapters ([KI-12](../limitations.md)). Omission is not treated as zero for semantic-cache eligibility. |
| `stop`, `top_p` | Forwarded by the supported adapter families. |
| `seed`, penalties | Adapter-dependent; unsupported candidates are filtered. |
| `logit_bias`, `logprobs`, `top_logprobs` | OpenAI-compatible adapter support. `top_logprobs` requires `logprobs: true`. Logprob requests bypass cache/coalescing. |
| `tools`, `tool_choice`, `parallel_tool_calls`, function `strict` | OpenAI-compatible tool pass-through. Anthropic and Ollama adapters currently refuse tool requests. No tool execution or tool authorization layer. |
| `service_tier` | Carried by the request/provider path where supported; not a guarantee of upstream tier availability. |
| `response_format`, `reasoning_effort`, `thinking` | Refused rather than silently promising structured output or reasoning control. An operator can instead fix one effort per model ROW (`capabilities.reasoning_effort` on an `openai_compatible` row; [admin API](admin-api.md)); a client cannot choose it. |
| `n` | Omitted or `1` only. Other values return 400. |
| `audio`, `modalities`, `prediction`, `web_search_options` | Refused when supplied with a non-null value. |
| File and input-audio content parts | Refused. |
| `cache_control` (Anthropic-style prompt-cache markers on a message or a content block) | **Not forwarded in v1.** Accepted without error and silently dropped: the block's text is forwarded, the marker is not, and the gateway requests no Anthropic cache strategy of its own (`AnthropicCacheControlContractTest`). A client cannot ask Anthropic to write a prompt cache through the gateway. Provider-reported cache usage is still accounted wherever the provider reports it: OpenAI `cached_tokens` and Anthropic cache-read and cache-write counts enter the usage total and are priced with the model row's `cache_read_per1k` / `cache_write_per1k` (`PromptCacheUsageTest`, `CachePricingTest`). |
| `user`, `metadata`, `store`, `prompt_cache_key`, `safety_identifier` | Tolerated and dropped; acceptance does not mean provider forwarding or persistence. |
| `stream_options` | Tolerated and dropped; the gateway requests upstream usage itself: always on the code-backed OpenAI adapter, and on a registry provider row only when the row carries the quirk `"streamUsage": "stream_options"` (see [adding a provider](../how-to/add-a-provider-and-a-model.md)). Without it a row's streamed response has no observable usage, and a completed stream is settled as an ESTIMATE: the admitted prompt estimate plus an output estimate of (streamed content characters + streamed tool-call argument characters) / `pointsman.estimate.chars-per-token`, integer division, so a very short reply can estimate zero output tokens. It is an approximation of what the provider bills, never the provider's count. With upstream usage the provider's reported counts are used (OBSERVED); a stream whose settlement itself fails is a separate, logged path. |

The detailed [adapter matrix](../design/README.md)
is the source for adapter-specific forwarding. Unknown future provider fields are
not an implied pass-through contract.

## Streaming and accounting

SSE supports normal text streaming and cancellation. A streamed tool call is
currently emitted as an assembled chunk when complete, not as progressive argument
fragments. Once a stream has started, an error can arrive in-band on an HTTP 200;
clients must handle SSE error events as well as initial HTTP failures.

A Claude answer through the Anthropic adapter is the response's text blocks, in order,
sync and streamed. Thinking, redacted-thinking and signature content is never part of
`content`, and no reasoning field is exposed (`AnthropicAnswerBlocksTest`). One SDK
boundary remains: the pinned Spring AI 1.1.8 stream decoder refuses a streamed
`redacted_thinking` block, so that stream ends in an in-band error, never as a finished
answer.

That boundary is pinned through the controller for both Claude profiles
(`ClaudeStreamedRedactionTest`). A redacted block that arrives first ends the request with
an explicit error. A redacted block that arrives after a text prefix also ends with an
in-band error event. Some of the prefix may or may not reach the client before that error
arrives, and nothing after the block is delivered.

In both cases:
- there is no `[DONE]` and no successful finish;
- the redacted data never reaches the client;
- the request settles once, as an estimate, never as observed provider usage;
- the failure is not cached.

Native usage is checked before it is mapped (`NativeUsageCapture`). Pointsman checks only
the relationships the provider's declared usage dialect documents, and it checks that every
count is a valid integer.

A provider usage that contradicts itself never becomes observed usage. For example:
- reasoning tokens larger than the completion tokens that should contain them;
- a cache hit/miss split larger than the prompt;
- a cache-creation split that does not sum.

What happens instead:
- **The answer is still delivered.**
- **The usage object is flagged.** It carries `x_pointsman_estimated: true` and
  `x_pointsman_usage_unverified` (a reason code plus the conflicting native counts).
- **The ledger records an estimate.** The usage is booked as `ESTIMATED`: its counts at no less
  than the admitted prompt estimate and output reservation, and its **money** at no less than
  what the serving attempt reserved (trust completion, 20 October 2026; until then only the
  counts were floored, and a reported cache-read discount could settle below the reservation).
  A reported cache-write premium is kept and can take the booking above the reservation. The
  reservation is a conservative liability, not a proven upper bound on the provider's invoice.
- **No cost is learned from it** - neither by the policy-service reward nor by the in-process
  cost sampler of `objective: cost` groups and `selection: cost` quality profiles, which also
  takes no cost from a gateway estimate. The completion still counts for reliability.

Valid usage is unchanged, including an explicit zero, a legitimate cache discount below the
reservation and a valid overrun above it.

### Streaming usage validation

Streamed usage is audited per event, as the SDK reads it. A server-sent event's `data:`
lines are joined (one event can span several lines) before its usage is checked, so a usage
split across lines is checked like a single-line one. An event over 1 MiB, or a data payload
that is not JSON, makes the usage unverified (`native_usage_unaudited`); it is never read as
observed.

### Serving-model identity

The serving identity is checked against the deployment. A model row accepts its own
`model_id` and the extra exact ids it declares in `accepted_response_models` (a documented
alias or a dated snapshot, at most 16; no prefixes or patterns). A response that names another
model (`native_model_mismatch`), names none (`native_model_absent`), or changes its name
mid-stream (`native_model_conflict`) is still delivered, but its usage is unverified: booked as
above at the DISPATCHED deployment's tariff (a response can never select a tariff), excluded
from cost learning, and not published to the response cache. The reported id is kept,
sanitized, on the provider dispatch (`reported_model`) and in `X-Provider-Model`; the usage
object's unverified marker carries the reason code alone, so the body never names it. Providers
such as OpenAI answer with dated snapshot ids; declare them, or their usage is booked as
unverified estimates. Ollama's native responses are checked for identity and count validity.

### Stream terminal outcomes

A stream's terminal outcome is recorded apart from its HTTP status. Once the first chunk is
sent the status stays 200; the usage row's `error_type` then records how the stream ended: the
typed provider error, `upstream_stream_truncated`, `client_cancelled` or `stream_timeout` (null
only for a completed stream), and the serving dispatch's `error_class` records the same (an
error class, `TRUNCATED`, `CANCEL` or `TIMEOUT`) with its accounting provenance unchanged - also
when nothing but a role chunk arrived before the stream ended, whose usage stays unknown. A
truncated stream counts as a failure for learning.

Usage can come from the provider or an estimate when provider usage is absent. A
provider-refused synchronous request records zero gateway cost, and a provider-refused or
interrupted stream keeps its admitted prompt estimate charged. In both cases the provider's
usage is `UNKNOWN`, and neither amount is an observed invoice amount
([settlement contract](../limitations.md)).
If a stream is cancelled or breaks before authoritative usage arrives, recorded usage
and cost can be estimates of the prompt and received output, not the provider's final
bill. Budget counters settle against those gateway accounting values; they are not a
provider-invoice hard cap. Tokenizer mode improves text counts using a declared encoding;
chat framing and tool scaffolding still prevent a provider-exact accounting claim. See
[configuration](configuration.md), [response headers](response-headers.md), and
[launch status](../release-status.md).

## Live provider checks (11 October 2026)

These cases were run once against real providers on the candidate, in small numbers. Each count below is the
provider's own, read from its response before the gateway mapped it. They show that the paths work; they do not certify
a model family.

| Case | Verified | Limits |
|---|---|---|
| Cached input, OpenAI `gpt-4o-mini` | The provider reported 2,560 cached of 2,655 prompt tokens. The request was priced at the cache-read rate. The ledger, the attempt record, the budget counter and the client's `uncached_prompt_tokens` / `cache_read_tokens` all equal the provider's counts. | Two requests, one model. |
| Reasoning model, OpenAI `gpt-5-nano` on a registry row with quirks `{"maxTokensField":"max_completion_tokens","streamUsage":"stream_options"}` | Hidden reasoning tokens are counted once, inside `completion_tokens`, and billed at the output rate. The output cap binds (finish `length`). A stream's usage is OBSERVED. `X-Provider-Model` names the served snapshot. A request without `temperature` succeeds in the 11 October live verification. | Before the fix, an omitted temperature was sent as 0.7 and this model refused it (fixed). `reasoning_effort` is refused. Three requests per run. |
| Ollama v0.40.2 (CPU) with `qwen2.5:0.5b`, code-backed adapter | Sync and SSE responses carry usage. The output cap ends with `length`. `X-Provider-Model` is correct. Prompt and completion counts equal Ollama's native `prompt_eval_count` / `eval_count`. A cancelled stream is recorded as an ESTIMATE. | One small model on CPU, at an operator-declared price. Not a GPU or larger-model qualification. |

## Evidence and source

Automated gateway tests cover these behaviors; the latest Linux test and stub-load
results are in [benchmarks](../performance.md). That evidence is not a live-provider
certification for every SDK, model or streaming workload.

Implementation: request DTO,
control validation,
and token accounting.

### Final stream trust and reuse after a policy edit

A provider can report valid usage early in a stream and then send a conflicting model ID or an unauditable event. The final native trust verdict governs settlement, cache publication and both learning paths, including cancellation and error paths. The internal assembled response keeps the reported counts with an unverified marker; conservative booking separately applies the admitted monetary floor. This does not rewrite bytes already sent to the caller and is not proof of the provider's invoice. An identity failure prevents cache publication and priced learning.

Changing a deployment's `accepted_response_models` partitions response reuse across the registry, including fallback destinations. New requests cannot hit old exact or semantic entries or join work admitted under the old policy. Existing requests finish under their pinned policy. Reordering an unchanged accepted-id set has no effect. The `gen/v3` cache namespace also makes entries created before this enforcement unreachable. It leaves quality qualifications unchanged; on rollback, flush response caches if trust or generation policies changed in the meantime.