Documentation / Reference / API compatibility
This is the gateway's launch contract, not a claim that every OpenAI client feature or every upstream model is supported. Adapter forwarding and provider/model support are separate requirements. Check your actual client and model before rollout.
Endpoints#
| Endpoint | Launch behavior |
|---|---|
POST /v1/chat/completions |
Nonstreaming JSON and streaming SSE, subject to the controls below. |
GET /v1/models |
Model discovery. |
GET /v1/models/{id} |
One caller-available entry; unknown or disallowed entries return 404. |
POST /v1/feedback |
Pointsman outcome feedback, with decision ownership/provenance checks. |
| Responses, embeddings, audio, batch, rerank | Not implemented as gateway APIs. |
| MCP or tool execution | Not implemented. The gateway does not execute tools. |
For authentication, curl examples, response fields and errors, see the application API guide.
Use a Pointsman virtual key in Authorization: Bearer .... Admin authentication is
a separate surface; see the admin API.
Request controls#
| Field or behavior | Support and boundary |
|---|---|
model, messages |
Core request fields; model resolution and access rules apply. In the shipped Compose profiles an unresolvable model returns 400 model_unresolvable (opt-in fallback: POINTSMAN_RESOLUTION_UNKNOWN_MODEL=fallback-to-allowed). Message roles are validated; capitalized roles are refused. In the v1 exact-tokenizer profile the prompt is counted after resolution, with the token encoding of the model or group the request resolves to: an unknown name follows the key's resolution policy exactly as in estimate mode (fallback-to-allowed serves an allowed model; reject and resolve-only refuse with 400 model_unresolvable; a model outside the key's allow set is 403 model_not_allowed). An omitted model is a different case, governed by pointsman.resolution.omitted-model. Under route (the default) it is served: the key's default_model, else the tenant's default model, else the semantic group match, else the smart tier, else the cheapest allowed model, always inside the key's allowed set and disclosed in X-Model-Resolution. Under refuse it gets 400 model_required. A key's policy may override the setting (omitted_model). A resolved model, or a group member, whose row declares no token_encoding is refused with 400 unsupported_request_control, and a fallback rung whose encoding the request was not counted with is skipped. |
| Text content | Supported. |
image_url content |
Not supported in the v1 release profile: its exact tokenizer mode cannot count image tokens and refuses the request with 400 unsupported_request_control. Outside that profile, estimate mode forwards images to a vision-capable model; its fast estimates do not establish provider-exact image counts. |
stream |
Supported, with the streaming caveats below. |
max_tokens, max_completion_tokens |
One positive output cap under two names. Conflicting values or a value below 1 are rejected. Provider quirks select the outbound field name. |
| Omitted output cap | The gateway supplies a cap, bounded by its default (4,096), model output limit and available context. Set an explicit cap for a different requirement. |
temperature |
Supported where the selected provider/model supports it, and sent as given, including 0. Omitted means omitted. No temperature is sent, and the provider default applies, unless the operator configured a default: a row's single-valued temperatureClamp, or spring.ai.openai.chat.options.temperature / spring.ai.anthropic.chat.options.temperature for the code-backed adapters (KI-12). Omission is not treated as zero for semantic-cache eligibility. |
stop, top_p |
Forwarded by the supported adapter families. |
seed, penalties |
Adapter-dependent; unsupported candidates are filtered. |
logit_bias, logprobs, top_logprobs |
OpenAI-compatible adapter support. top_logprobs requires logprobs: true. Logprob requests bypass cache/coalescing. |
tools, tool_choice, parallel_tool_calls, function strict |
OpenAI-compatible tool pass-through. Anthropic and Ollama adapters currently refuse tool requests. No tool execution or tool authorization layer. |
service_tier |
Carried by the request/provider path where supported; not a guarantee of upstream tier availability. |
response_format, reasoning_effort, thinking |
Refused rather than silently promising structured output or reasoning control. An operator can instead fix one effort per model ROW (capabilities.reasoning_effort on an openai_compatible row; admin API); a client cannot choose it. |
n |
Omitted or 1 only. Other values return 400. |
audio, modalities, prediction, web_search_options |
Refused when supplied with a non-null value. |
| File and input-audio content parts | Refused. |
cache_control (Anthropic-style prompt-cache markers on a message or a content block) |
Not forwarded in v1. Accepted without error and silently dropped: the block's text is forwarded, the marker is not, and the gateway requests no Anthropic cache strategy of its own (AnthropicCacheControlContractTest). A client cannot ask Anthropic to write a prompt cache through the gateway. Provider-reported cache usage is still accounted wherever the provider reports it: OpenAI cached_tokens and Anthropic cache-read and cache-write counts enter the usage total and are priced with the model row's cache_read_per1k / cache_write_per1k (PromptCacheUsageTest, CachePricingTest). |
user, metadata, store, prompt_cache_key, safety_identifier |
Tolerated and dropped; acceptance does not mean provider forwarding or persistence. |
stream_options |
Tolerated and dropped; the gateway requests upstream usage itself: always on the code-backed OpenAI adapter, and on a registry provider row only when the row carries the quirk "streamUsage": "stream_options" (see adding a provider). Without it a row's streamed response has no observable usage, and a completed stream is settled as an ESTIMATE: the admitted prompt estimate plus an output estimate of (streamed content characters + streamed tool-call argument characters) / pointsman.estimate.chars-per-token, integer division, so a very short reply can estimate zero output tokens. It is an approximation of what the provider bills, never the provider's count. With upstream usage the provider's reported counts are used (OBSERVED); a stream whose settlement itself fails is a separate, logged path. |
The detailed adapter matrix is the source for adapter-specific forwarding. Unknown future provider fields are not an implied pass-through contract.
Streaming and accounting#
SSE supports normal text streaming and cancellation. A streamed tool call is currently emitted as an assembled chunk when complete, not as progressive argument fragments. Once a stream has started, an error can arrive in-band on an HTTP 200; clients must handle SSE error events as well as initial HTTP failures.
A Claude answer through the Anthropic adapter is the response's text blocks, in order,
sync and streamed. Thinking, redacted-thinking and signature content is never part of
content, and no reasoning field is exposed (AnthropicAnswerBlocksTest). One SDK
boundary remains: the pinned Spring AI 1.1.8 stream decoder refuses a streamed
redacted_thinking block, so that stream ends in an in-band error, never as a finished
answer.
That boundary is pinned through the controller for both Claude profiles
(ClaudeStreamedRedactionTest). A redacted block that arrives first ends the request with
an explicit error. A redacted block that arrives after a text prefix also ends with an
in-band error event. Some of the prefix may or may not reach the client before that error
arrives, and nothing after the block is delivered.
In both cases:
- there is no [DONE] and no successful finish;
- the redacted data never reaches the client;
- the request settles once, as an estimate, never as observed provider usage;
- the failure is not cached.
Native usage is checked before it is mapped (NativeUsageCapture). Pointsman checks only
the relationships the provider's declared usage dialect documents, and it checks that every
count is a valid integer.
A provider usage that contradicts itself never becomes observed usage. For example: - reasoning tokens larger than the completion tokens that should contain them; - a cache hit/miss split larger than the prompt; - a cache-creation split that does not sum.
What happens instead:
- The answer is still delivered.
- The usage object is flagged. It carries x_pointsman_estimated: true and
x_pointsman_usage_unverified (a reason code plus the conflicting native counts).
- The ledger records an estimate. The usage is booked as ESTIMATED: its counts at no less
than the admitted prompt estimate and output reservation, and its money at no less than
what the serving attempt reserved (trust completion, 20 October 2026; until then only the
counts were floored, and a reported cache-read discount could settle below the reservation).
A reported cache-write premium is kept and can take the booking above the reservation. The
reservation is a conservative liability, not a proven upper bound on the provider's invoice.
- No cost is learned from it - neither by the policy-service reward nor by the in-process
cost sampler of objective: cost groups and selection: cost quality profiles, which also
takes no cost from a gateway estimate. The completion still counts for reliability.
Valid usage is unchanged, including an explicit zero, a legitimate cache discount below the reservation and a valid overrun above it.
Streaming usage validation#
Streamed usage is audited per event, as the SDK reads it. A server-sent event's data:
lines are joined (one event can span several lines) before its usage is checked, so a usage
split across lines is checked like a single-line one. An event over 1 MiB, or a data payload
that is not JSON, makes the usage unverified (native_usage_unaudited); it is never read as
observed.
Serving-model identity#
The serving identity is checked against the deployment. A model row accepts its own
model_id and the extra exact ids it declares in accepted_response_models (a documented
alias or a dated snapshot, at most 16; no prefixes or patterns). A response that names another
model (native_model_mismatch), names none (native_model_absent), or changes its name
mid-stream (native_model_conflict) is still delivered, but its usage is unverified: booked as
above at the DISPATCHED deployment's tariff (a response can never select a tariff), excluded
from cost learning, and not published to the response cache. The reported id is kept,
sanitized, on the provider dispatch (reported_model) and in X-Provider-Model; the usage
object's unverified marker carries the reason code alone, so the body never names it. Providers
such as OpenAI answer with dated snapshot ids; declare them, or their usage is booked as
unverified estimates. Ollama's native responses are checked for identity and count validity.
Stream terminal outcomes#
A stream's terminal outcome is recorded apart from its HTTP status. Once the first chunk is
sent the status stays 200; the usage row's error_type then records how the stream ended: the
typed provider error, upstream_stream_truncated, client_cancelled or stream_timeout (null
only for a completed stream), and the serving dispatch's error_class records the same (an
error class, TRUNCATED, CANCEL or TIMEOUT) with its accounting provenance unchanged - also
when nothing but a role chunk arrived before the stream ended, whose usage stays unknown. A
truncated stream counts as a failure for learning.
Usage can come from the provider or an estimate when provider usage is absent. A
provider-refused synchronous request records zero gateway cost, and a provider-refused or
interrupted stream keeps its admitted prompt estimate charged. In both cases the provider's
usage is UNKNOWN, and neither amount is an observed invoice amount
(settlement contract).
If a stream is cancelled or breaks before authoritative usage arrives, recorded usage
and cost can be estimates of the prompt and received output, not the provider's final
bill. Budget counters settle against those gateway accounting values; they are not a
provider-invoice hard cap. Tokenizer mode improves text counts using a declared encoding;
chat framing and tool scaffolding still prevent a provider-exact accounting claim. See
configuration, response headers, and
launch status.
Live provider checks (11 October 2026)#
These cases were run once against real providers on the candidate, in small numbers. Each count below is the provider's own, read from its response before the gateway mapped it. They show that the paths work; they do not certify a model family.
| Case | Verified | Limits |
|---|---|---|
Cached input, OpenAI gpt-4o-mini |
The provider reported 2,560 cached of 2,655 prompt tokens. The request was priced at the cache-read rate. The ledger, the attempt record, the budget counter and the client's uncached_prompt_tokens / cache_read_tokens all equal the provider's counts. |
Two requests, one model. |
Reasoning model, OpenAI gpt-5-nano on a registry row with quirks {"maxTokensField":"max_completion_tokens","streamUsage":"stream_options"} |
Hidden reasoning tokens are counted once, inside completion_tokens, and billed at the output rate. The output cap binds (finish length). A stream's usage is OBSERVED. X-Provider-Model names the served snapshot. A request without temperature succeeds in the 11 October live verification. |
Before the fix, an omitted temperature was sent as 0.7 and this model refused it (fixed). reasoning_effort is refused. Three requests per run. |
Ollama v0.40.2 (CPU) with qwen2.5:0.5b, code-backed adapter |
Sync and SSE responses carry usage. The output cap ends with length. X-Provider-Model is correct. Prompt and completion counts equal Ollama's native prompt_eval_count / eval_count. A cancelled stream is recorded as an ESTIMATE. |
One small model on CPU, at an operator-declared price. Not a GPU or larger-model qualification. |
Evidence and source#
Automated gateway tests cover these behaviors; the latest Linux test and stub-load results are in benchmarks. That evidence is not a live-provider certification for every SDK, model or streaming workload.
Implementation: request DTO, control validation, and token accounting.
Final stream trust and reuse after a policy edit#
A provider can report valid usage early in a stream and then send a conflicting model ID or an unauditable event. The final native trust verdict governs settlement, cache publication and both learning paths, including cancellation and error paths. The internal assembled response keeps the reported counts with an unverified marker; conservative booking separately applies the admitted monetary floor. This does not rewrite bytes already sent to the caller and is not proof of the provider's invoice. An identity failure prevents cache publication and priced learning.
Changing a deployment's accepted_response_models partitions response reuse across the registry, including fallback destinations. New requests cannot hit old exact or semantic entries or join work admitted under the old policy. Existing requests finish under their pinned policy. Reordering an unchanged accepted-id set has no effect. The gen/v3 cache namespace also makes entries created before this enforcement unreachable. It leaves quality qualifications unchanged; on rollback, flush response caches if trust or generation policies changed in the meantime.