Guardrails apply configured input and output policy to the supported request contract. They can detect, flag, redact or refuse content. They do not establish that an answer is true or that every prompt injection will be detected.
Policy composition#
The service builds ordered input and output chains from registered rules. Effective settings combine global policy with tenant/workspace scope and the applicable policy mode. Tightening rules prevent a lower scope from casually weakening a required control. An explicitly permitted tenant-relaxation policy is different from the default restrict-only posture. Request overrides are separately gated and cannot be assumed available.
| Mode | Policy result | Error behavior |
|---|---|---|
| Monitor | Record what would have happened without enforcing the block/redaction | Continue with degraded disclosure when a check fails |
| Enforce | Apply a refusal or redaction to the working request/response | Refuse on rule failure unless the applicable global setting explicitly permits fail-open |
Live configuration changes use snapshots and change notifications. Some unsafe-to-hot-reload parameters, such as regex-bearing settings, have stricter write behavior; consult the supported configuration surface rather than treating every field as dynamic.
Request flow#
- Admission establishes the caller, permissions and reservations before input inspection.
- Resolve the effective policy and allowed request override.
- Run enabled input rules in order. Enforced redaction modifies the working request seen by later rules and cache-key construction. Monitoring does not silently rewrite the prompt.
- On a miss, route and call the provider; on a hit, validate the cached response for this caller.
- Run output policy for a complete response or replay. A blocked response is not delivered as a successful completion, but upstream work already incurred still belongs in settlement.
Native checks examine supported content and tool-call arguments. Redacting a value inside structured tool arguments can require refusal instead of rewriting JSON into a potentially different command. The gateway does not execute those tools.
Streaming is a separate delivery decision#
A live stream cannot retract bytes already sent. When an enforcing output rule needs a full response, the controller buffers the provider result, checks it and replays SSE. This trades time to first byte and memory for enforceable whole-response policy.
A cache replay also has a complete response available for checks. Ordinary live SSE does not acquire a full-response output pass merely because an output rule exists in monitor mode. Do not describe these three paths as equivalent coverage.
Dependencies and trust#
Local rules include bounded text/regex checks, PII/secrets handling and response-shape checks. Optional external moderation, webhook or analyzer integrations add network latency and send the configured content to that service. Their availability and data handling belong to the deployment's threat model.
Evidence recording is asynchronous and separate from enforcement. Losing an audit row does not turn a denied request into an allowed one, and a recorded event is not proof that all content was inspected. Body mirroring of results is opt-in; use the documented response headers and operator metrics.
Failures and operational checks#
An enforced violation returns a guardrail refusal. An enforcing rule that cannot evaluate generally returns guardrail_unavailable unless explicit fail-open policy applies. Monitoring errors remain observable rather than pretending the check passed. External deadlines and regex budgets bound work but cannot make external vendors or heuristic detection perfect.
Test the same rule in monitor and enforce, a deliberate dependency timeout, a redacted input, a tool-call-only response, a cache hit and a streaming request. Verify effective scope settings and whether delivery was live or buffered. Do not use production-sensitive prompts in public documentation examples.
Implementation anchors: guardrail/GuardrailService, GuardrailText, settings/GuardrailResolver, rule implementations, and the controller's stream/replay branches. See rule catalogue, guardrail setup, response headers and security.