View Markdown

Write and enforce a guardrail

On this page

What you get at the end: a rule that refuses requests you never want reaching a provider - tested against sample text before it touches live traffic, watched in monitor mode before it blocks anything, and visible in the ledger afterwards.

The examples configure the shipped contains rule. Replace the sample text and model with values appropriate to your deployment.

The shape of the thing#

Guardrails are a chain of rules that inspect every request on the way in and every response on the way out. The rules are shipped - blocklists, PII redaction, secret detection, length and shape checks, external moderation services - and writing a guardrail means configuring one of them: which words, which PII types, which mode. The full catalog with every parameter is on GET /admin/control/effective, which is also the only honest answer to "what is actually enabled right now".

Every rule has three states, and the path between them is the point of this guide:

  • off - the rule does not act.
  • monitor - the rule evaluates and records, and never refuses. This is how you find out what a rule would do to real traffic before you let it.
  • enforce - the rule acts: blocks, redacts, transforms.

Configure a rule#

The example task: refuse any request that mentions an internal codename. The contains rule is a configurable word check:

bash
curl -s -X PUT localhost:8080/admin/control/guardrails/contains \
  -u "$POINTSMAN_ADMIN_USERNAME:$POINTSMAN_ADMIN_PASSWORD" \
  -H 'Content-Type: application/json' \
  -d '{"enabled":true,"mode":"monitor",
       "params":{"words":["project-hushpuppy"],"operator":"any","not":false}}'

A successful write returns 204 and applies without a restart. Verify the effective parameters before enforcing the rule.

Verify parameter nesting#

A flat body - {"enabled":true,"words":[...]} - is accepted with a 204, the enabled flip takes, and the words are silently ignored, leaving you with an enabled rule configured to match nothing. After any write, read /admin/control/effective back and look at the rule's actual values rather than trusting the status code.

Check the rule condition#

Know which direction your rule fires. contains fires when the condition holds: with operator: any it is a blocklist (fires when a word is present), and not: true inverts it into a require-list (fires when the words are absent - useful for "every prompt must carry the compliance preamble"). Setting not: true on a blocklist, which reads plausibly as "not allowed", silently turns the rule inside out. The dry-run below is how you catch this in ten seconds instead of in production.

Test it against sample text, not traffic#

bash
curl -s -X POST localhost:8080/admin/guardrails/dry-run \
  -u "$POINTSMAN_ADMIN_USERNAME:$POINTSMAN_ADMIN_PASSWORD" \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"summarize project-hushpuppy for me"}]}' | jq

The response walks your sample through the whole chain in order and gives a per-rule verdict:

json
{"name": "contains",                  "verdict": "would block"},
{"name": "prompt-injection-heuristic", "verdict": "not reached"},
{"name": "pii",                        "verdict": "not reached"}

Every verdict is a would - the dry run has no side effects: no audit rows, no metrics, and external services (moderation vendors, webhooks) are never called with your test text, their rows read external - not evaluated instead. Rules downstream of a would-be block read not reached, which is exactly what live traffic would experience.

Test both a matching and a nonmatching sample. Confirm the expected verdict before moving from monitor to enforce.

Watch it in monitor mode#

The rule is live in monitor, so real matching traffic passes and is counted:

bash
curl -s -o /dev/null -w "%{http_code}\n" localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $YOUR_KEY" -H 'Content-Type: application/json' \
  -d '{"model":"demo-model","messages":[{"role":"user","content":"summarize project-hushpuppy for me"}]}'
text
200

Every fire is written to the guardrail audit trail - the guardrail_events table - as a row naming the rule, the stage, and what happened. A monitored match is represented as:

text
guardrail_name | stage | action |  mode
contains       | input | flag   | monitor

flag is monitor's verb: would have acted, did not. Run in monitor until the flag rate matches your expectation of how often real traffic should trip this rule - a rule that fires far more than you expected is usually a rule you misconfigured, and monitor mode is where that discovery costs nothing.

Enforce#

The same write with "mode":"enforce". Then the same request:

json
{"error": {
  "message": "Request rejected by gateway guardrail [contains]: prompt matched the contains check (operator=any)",
  "type": "invalid_request_error",
  "code": "guardrail_violation"}}

400, before any provider is called - a blocked request costs nothing and leaks nothing. The message names the rule and the operator but never echoes the matched text back, deliberately: the words are operator configuration, and a refusal that quotes them teaches a probing caller your blocklist one word at a time.

The refusal is also in the ledger. On GET /admin/observe/requests the blocked request shows as the row it is:

json
{"status": 400, "error_type": "guardrail_violation",
 "provider": null, "total_tokens": 0, "cost_usd": 0.0}

One caveat about caching#

An identical prompt already answered may be served from the semantic cache, and a cached response does not re-run the provider path. While verifying a guardrail change, send X-Cache-Control: no-cache on your test requests - the first version of this page's enforce test "passed" a request it should have blocked because the response came from cache, byte-equal down to the response id.

When the rule itself breaks#

Every rule carries fail_open_on_error, default false. A rule that throws - an external moderation vendor timing out, a malformed regex - fails closed (the request is refused) unless you opt that rule into failing open. Those degradations are counted per rule on /admin/ops/guardrails/health (failed_open / failed_closed), so a silently failing rule shows up as a number rather than a mystery.

Per-tenant tightening#

Everything above is the global chain. A tenant admin can tighten rules for their own organisation - enable a rule that is off globally, or push monitor to enforce - via PUT /admin/control/guardrails/{rule}/tenant?org=..., and the operator can lock a rule against tenant loosening. Loosening below the global setting is refused by the resolver, not by convention. The workspace-level view of what is effective where is GET /admin/control/guardrails/workspace-effective.

Full wire shapes and every refusal for these doors: the admin API reference.

Pointsman v1 candidate documentation Source: 0e09f5e7

Search guides, reference pages and architecture.

Documentation

Open full-size image