Add a second key to a provider, and rotate one

Add a credential-pool member to an OpenAI-compatible provider, verify usage attribution and rotate its credential reference without changing its identity.

What you need#

The code-backed OpenAI, Anthropic and Ollama providers do not support sibling credentials. For a compatible provider, keep one provider row and add members to its pool; do not create duplicate provider rows to represent account keys.

The examples use a provider named deepseek and illustrative IDs/output. Replace them with your registry values. Verification requests use your provider account and can incur charges. In Compose, variables must be passed to the gateway container, not merely set in the caller's shell.

A="$POINTSMAN_ADMIN_USERNAME:$POINTSMAN_ADMIN_PASSWORD"
PID=$(curl -sf -u "$A" localhost:8080/admin/registry/providers \
  | jq -r '.[] | select(.name=="deepseek") | .id')

Add the sibling#

A sibling is a label, an env:NAME pointer, and nothing else required:

curl -sf -u "$A" -X POST localhost:8080/admin/registry/providers/$PID/credentials \
  -H 'Content-Type: application/json' \
  -d '{"label":"account-b","credential_ref":"env:DEEPSEEK_API_KEY_B"}' | jq

{
  "id": "c8b2a32f-91e7-43ff-8f83-dad4c5fba736",
  "provider_id": "6a8fbf6d-097b-4719-b6f1-8f9d968cf941",
  "label": "account-b",
  "credential_ref": "env:DEEPSEEK_API_KEY_B",
  "credential_resolves": true,
  "enabled": true,
  "tpm_limit": null
}

credential_resolves: true means the variable is set on this instance. The value is never returned, here or anywhere. The change takes effect without a restart: the registry reloads and the pool is now two members, the primary from the provider row and this sibling. Keep the id; it is how the ledger and the rotation door refer to this key, and it survives rotation, unlike the label.

GET on the same path lists siblings only. The primary is not repeated here, because it is already on the provider row and two copies of one fact is what this shape avoids.

Prove both keys are serving#

Send a few requests:

for i in 1 2 3 4; do
  curl -sf localhost:8080/v1/chat/completions \
    -H "Authorization: Bearer $YOUR_KEY" -H 'Content-Type: application/json' \
    -H 'X-Cache-Control: no-cache' \
    -d "{\"model\":\"deepseek-chat\",\"max_tokens\":8,\"messages\":[{\"role\":\"user\",\"content\":\"Say $i.\"}]}" \
    -o /dev/null -w '%{http_code} '
done

The responses will all say X-Served-Provider: deepseek, and nothing on a response names the key. The ledger does. Its provider_credential_id column holds the sibling's id from above, or the provider row's id for the primary:

docker exec pointsman-postgres-1 psql -U pointsman -d pointsman -c \
  "select created_at, model, provider_credential_id from usage_events
   where model='deepseek-chat' order by created_at desc limit 4"

 03:03:55 | deepseek-chat | c8b2a32f-91e7-43ff-8f83-dad4c5fba736
 03:03:54 | deepseek-chat | 6a8fbf6d-097b-4719-b6f1-8f9d968cf941
 03:03:53 | deepseek-chat | c8b2a32f-91e7-43ff-8f83-dad4c5fba736
 03:03:52 | deepseek-chat | 6a8fbf6d-097b-4719-b6f1-8f9d968cf941

Without a per-key ceiling, selection is round-robin among live members. The example rows show both member identities. Use the ledger to verify attribution; response headers identify the provider, not its account key.

The pool also shows on the metrics endpoint from the first pooled request on:

curl -s localhost:8080/actuator/prometheus | grep provider_credential

pointsman_provider_credential{deployment="deepseek:deepseek-chat"} 2.0
pointsman_provider_credential_live{deployment="deepseek:deepseek-chat"} 2.0

The first metric is the configured total; the second counts members that are enabled, resolving and not cooling. Use the exported metric names shown here.

Rotate a key#

Rotation changes an existing member's environment reference and preserves its ID. It uses a separate audited endpoint. Ensure the replacement variable is already available to the gateway process before rotating.

Start a long streamed request first, so something is in flight on the pool:

curl -sN localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $YOUR_KEY" -H 'Content-Type: application/json' \
  -H 'X-Cache-Control: no-cache' \
  -d '{"model":"deepseek-chat","stream":true,"max_tokens":300,
       "messages":[{"role":"user","content":"Write a 250-word story about a glacier."}]}' \
  > /tmp/stream.out &
sleep 2

Rotate the sibling while the request is active:

SID=c8b2a32f-91e7-43ff-8f83-dad4c5fba736
curl -sf -u "$A" -X POST localhost:8080/admin/registry/providers/$PID/credentials/$SID/rotate \
  -H 'Content-Type: application/json' \
  -d '{"credential_ref":"env:DEEPSEEK_API_KEY_ROTATED"}' | jq '{label, credential_ref, credential_resolves}'

{"label": "account-b", "credential_ref": "env:DEEPSEEK_API_KEY_ROTATED", "credential_resolves": true}

In-flight requests during rotation#

An in-flight exchange retains its existing client and member identity. The old client closes gracefully after its exchanges finish. Check completion and the ledger attribution after rotation:

wait; tail -c 40 /tmp/stream.out

Pool state after rotation#

Verify the credential readback, live-member metric and rotation audit. A resolving replacement can keep both members live.

The gateway logs when a pool reaches its last live key and when it recovers. An unresolved member can become live on a subsequent registry load once its variable is available to the process. Changing the host shell alone does not change a running container's environment.

curl -sf -u "$A" 'localhost:8080/admin/audit?limit=3' | jq '.[] | {action, detail}'

The audit records the old and new environment-reference names, not the secret values.

A rotation does not clear a cooldown the member is serving. If the key was rate-limited a moment ago, it stays out for the rest of its window (five seconds by default), so a rotation cannot be used to paper over a 429.

Give a key its own ceiling#

If one key is on a smaller plan than the other, declare its tokens-per-minute ceiling on the member. This is an edit, not a rotation:

curl -sf -u "$A" -X PUT localhost:8080/admin/registry/providers/$PID/credentials/$SID \
  -H 'Content-Type: application/json' -d '{"label":"account-b","tpm_limit":120}' > /dev/null

The number is deliberately small so the effect shows inside one page. Eight requests of about 80 tokens each, one second apart, attributed by the same ledger query:

account-b   primary   account-b   primary   primary   primary   primary   primary

The sibling served two, reached 160 tokens in its minute against a usable ceiling of 108 (ninety percent of 120), and received nothing more until the window passed; the primary, with no ceiling, took everything else. The check estimates a request from its prompt length only, so a member is chosen until its tally is within one prompt of the usable ceiling, not before. A PUT without tpm_limit preserves it; explicit "tpm_limit": null clears it. The primary's ceiling is credential_tpm_limit on the provider row itself.

Two limits to keep apart: this one is per key and independent; the provider-total limit in routing configuration is what the gateway sends to the vendor in aggregate, and the two are not halves of each other.

The refusals you will meet#

Use the response status and message to distinguish these refusals.

• 400, "provider 'openai' is code-backed (openai) and cannot hold a pool" - the three code-backed vendors are pools of one. Not a misconfiguration; a limitation with its reason in the message.
• 409, "that would leave enabled provider ... with no live outbound key ... disable the PROVIDER instead" - on a delete, a disable, or a rotation onto an unset variable, when the member is the last live one. Disable the provider first if that is what you mean; then the same call goes through.
• 400, "credential_ref is not updatable here - POST /{id}/rotate replaces the secret" - you sent a changed pointer to the edit door. Rotation has its own.
• 400, "credential_ref is already 'env:...' - a rotation to the same reference changes nothing" - the audit log would otherwise claim a secret was replaced.
• 400, "'primary' is the label the composed primary always carries" - pick another label.
• 400, "credential_ref must be an env reference (env:NAME) - secrets are never stored" - you pasted the key itself. Put it in a variable and point at the variable.
• 400, "tpm_limit must be positive, or omitted for no declared ceiling" - zero is not "no limit" here; omit the field.

Remove a sibling#

curl -sf -u "$A" -X DELETE localhost:8080/admin/registry/providers/$PID/credentials/$SID -w '%{http_code}\n'

204, provided another key stays live. Ledger rows that name the deleted member keep naming it; the column is not a foreign key, on purpose.

What a 429 on one key does now#

A vendor 429 or 402 cools the member that served the request.

See provider health and eligibility to distinguish cooldown, breaker and routing eligibility.