# Add a second key to a provider, and rotate one Add a credential-pool member to an OpenAI-compatible provider, verify usage attribution and rotate its credential reference without changing its identity. ## What you need | Prerequisite | Requirement | |---|---| | Provider | A database-backed row with `adapter_type: openai_compatible` | | Credentials | A primary key and a second key exposed to the gateway as separate environment variables | | Replacement key | For the rotation example, also expose `DEEPSEEK_API_KEY_ROTATED` before starting the gateway | | Access | Authorized administration credentials and a virtual key permitted to call the model | The code-backed OpenAI, Anthropic and Ollama providers do not support sibling credentials. For a compatible provider, keep one provider row and add members to its pool; do not create duplicate provider rows to represent account keys. The examples use a provider named `deepseek` and illustrative IDs/output. Replace them with your registry values. Verification requests use your provider account and can incur charges. In Compose, variables must be passed to the gateway container, not merely set in the caller's shell. ```bash A="$POINTSMAN_ADMIN_USERNAME:$POINTSMAN_ADMIN_PASSWORD" PID=$(curl -sf -u "$A" localhost:8080/admin/registry/providers \ | jq -r '.[] | select(.name=="deepseek") | .id') ``` ## Add the sibling A sibling is a label, an `env:NAME` pointer, and nothing else required: ```bash curl -sf -u "$A" -X POST localhost:8080/admin/registry/providers/$PID/credentials \ -H 'Content-Type: application/json' \ -d '{"label":"account-b","credential_ref":"env:DEEPSEEK_API_KEY_B"}' | jq ``` ```json { "id": "c8b2a32f-91e7-43ff-8f83-dad4c5fba736", "provider_id": "6a8fbf6d-097b-4719-b6f1-8f9d968cf941", "label": "account-b", "credential_ref": "env:DEEPSEEK_API_KEY_B", "credential_resolves": true, "enabled": true, "tpm_limit": null } ``` `credential_resolves: true` means the variable is set on this instance. The value is never returned, here or anywhere. The change takes effect without a restart: the registry reloads and the pool is now two members, the primary from the provider row and this sibling. Keep the `id`; it is how the ledger and the rotation door refer to this key, and it survives rotation, unlike the label. `GET` on the same path lists siblings only. The primary is not repeated here, because it is already on the provider row and two copies of one fact is what this shape avoids. ## Prove both keys are serving Send a few requests: ```bash for i in 1 2 3 4; do curl -sf localhost:8080/v1/chat/completions \ -H "Authorization: Bearer $YOUR_KEY" -H 'Content-Type: application/json' \ -H 'X-Cache-Control: no-cache' \ -d "{\"model\":\"deepseek-chat\",\"max_tokens\":8,\"messages\":[{\"role\":\"user\",\"content\":\"Say $i.\"}]}" \ -o /dev/null -w '%{http_code} ' done ``` The responses will all say `X-Served-Provider: deepseek`, and nothing on a response names the key. The ledger does. Its `provider_credential_id` column holds the sibling's id from above, or the **provider row's** id for the primary: ```bash docker exec pointsman-postgres-1 psql -U pointsman -d pointsman -c \ "select created_at, model, provider_credential_id from usage_events where model='deepseek-chat' order by created_at desc limit 4" ``` ``` 03:03:55 | deepseek-chat | c8b2a32f-91e7-43ff-8f83-dad4c5fba736 03:03:54 | deepseek-chat | 6a8fbf6d-097b-4719-b6f1-8f9d968cf941 03:03:53 | deepseek-chat | c8b2a32f-91e7-43ff-8f83-dad4c5fba736 03:03:52 | deepseek-chat | 6a8fbf6d-097b-4719-b6f1-8f9d968cf941 ``` Without a per-key ceiling, selection is round-robin among live members. The example rows show both member identities. Use the ledger to verify attribution; response headers identify the provider, not its account key. The pool also shows on the metrics endpoint from the first pooled request on: ```bash curl -s localhost:8080/actuator/prometheus | grep provider_credential ``` ``` pointsman_provider_credential{deployment="deepseek:deepseek-chat"} 2.0 pointsman_provider_credential_live{deployment="deepseek:deepseek-chat"} 2.0 ``` The first metric is the configured total; the second counts members that are enabled, resolving and not cooling. Use the exported metric names shown here. ## Rotate a key Rotation changes an existing member's environment reference and preserves its ID. It uses a separate audited endpoint. Ensure the replacement variable is already available to the gateway process before rotating. Start a long streamed request first, so something is in flight on the pool: ```bash curl -sN localhost:8080/v1/chat/completions \ -H "Authorization: Bearer $YOUR_KEY" -H 'Content-Type: application/json' \ -H 'X-Cache-Control: no-cache' \ -d '{"model":"deepseek-chat","stream":true,"max_tokens":300, "messages":[{"role":"user","content":"Write a 250-word story about a glacier."}]}' \ > /tmp/stream.out & sleep 2 ``` Rotate the sibling while the request is active: ```bash SID=c8b2a32f-91e7-43ff-8f83-dad4c5fba736 curl -sf -u "$A" -X POST localhost:8080/admin/registry/providers/$PID/credentials/$SID/rotate \ -H 'Content-Type: application/json' \ -d '{"credential_ref":"env:DEEPSEEK_API_KEY_ROTATED"}' | jq '{label, credential_ref, credential_resolves}' ``` ```json {"label": "account-b", "credential_ref": "env:DEEPSEEK_API_KEY_ROTATED", "credential_resolves": true} ``` ### In-flight requests during rotation An in-flight exchange retains its existing client and member identity. The old client closes gracefully after its exchanges finish. Check completion and the ledger attribution after rotation: ```bash wait; tail -c 40 /tmp/stream.out ``` ### Pool state after rotation Verify the credential readback, live-member metric and rotation audit. A resolving replacement can keep both members live. | Replacement state | Behavior | |---|---| | Variable resolves | Member can serve, subject to enabled state, capacity and cooldown | | Variable is absent, another key remains live | Rotation can succeed; the unresolved member stops receiving new requests | | Variable is absent and this is the last live key | Rotation is refused while the provider is enabled | The gateway logs when a pool reaches its last live key and when it recovers. An unresolved member can become live on a subsequent registry load once its variable is available to the process. Changing the host shell alone does not change a running container's environment. ```bash curl -sf -u "$A" 'localhost:8080/admin/audit?limit=3' | jq '.[] | {action, detail}' ``` The audit records the old and new environment-reference names, not the secret values. A rotation does not clear a cooldown the member is serving. If the key was rate-limited a moment ago, it stays out for the rest of its window (five seconds by default), so a rotation cannot be used to paper over a 429. ## Give a key its own ceiling If one key is on a smaller plan than the other, declare its tokens-per-minute ceiling on the member. This is an edit, not a rotation: ```bash curl -sf -u "$A" -X PUT localhost:8080/admin/registry/providers/$PID/credentials/$SID \ -H 'Content-Type: application/json' -d '{"label":"account-b","tpm_limit":120}' > /dev/null ``` The number is deliberately small so the effect shows inside one page. Eight requests of about 80 tokens each, one second apart, attributed by the same ledger query: ``` account-b primary account-b primary primary primary primary primary ``` The sibling served two, reached 160 tokens in its minute against a usable ceiling of 108 (ninety percent of 120), and received nothing more until the window passed; the primary, with no ceiling, took everything else. The check estimates a request from its prompt length only, so a member is chosen until its tally is within one prompt of the usable ceiling, not before. A `PUT` without `tpm_limit` preserves it; explicit `"tpm_limit": null` clears it. The primary's ceiling is `credential_tpm_limit` on the provider row itself. Two limits to keep apart: this one is per key and independent; the provider-total limit in routing configuration is what the gateway sends to the vendor in aggregate, and the two are not halves of each other. ## The refusals you will meet Use the response status and message to distinguish these refusals. - **`400`, "provider 'openai' is code-backed (openai) and cannot hold a pool"** - the three code-backed vendors are pools of one. Not a misconfiguration; a limitation with its reason in the message. - **`409`, "that would leave enabled provider ... with no live outbound key ... disable the PROVIDER instead"** - on a delete, a disable, or a rotation onto an unset variable, when the member is the last live one. Disable the provider first if that is what you mean; then the same call goes through. - **`400`, "credential_ref is not updatable here - POST /{id}/rotate replaces the secret"** - you sent a changed pointer to the edit door. Rotation has its own. - **`400`, "credential_ref is already 'env:...' - a rotation to the same reference changes nothing"** - the audit log would otherwise claim a secret was replaced. - **`400`, "'primary' is the label the composed primary always carries"** - pick another label. - **`400`, "credential_ref must be an env reference (env:NAME) - secrets are never stored"** - you pasted the key itself. Put it in a variable and point at the variable. - **`400`, "tpm_limit must be positive, or omitted for no declared ceiling"** - zero is not "no limit" here; omit the field. ## Remove a sibling ```bash curl -sf -u "$A" -X DELETE localhost:8080/admin/registry/providers/$PID/credentials/$SID -w '%{http_code}\n' ``` `204`, provided another key stays live. Ledger rows that name the deleted member keep naming it; the column is not a foreign key, on purpose. ## What a 429 on one key does now A vendor `429` or `402` cools the member that served the request. | Event | Pool behavior | Request or routing effect | |---|---|---| | Vendor supplies `Retry-After` | Cool that key for the requested duration, capped at five minutes | Subsequent eligible traffic can use another live member | | No retry duration | Default cooldown is five seconds | Rotation does not clear the cooldown | | Current attempt is rate-limited | Do not retry that attempt on a sibling | Follow the configured fallback ladder; inspect `X-Model-Resolution` | | Account-key rate limit | Do not count it as a provider-breaker failure | A healthy sibling can continue serving | | Provider 5xx or timeout | Normal breaker accounting applies | A broader outage can open the provider breaker | See [provider health and eligibility](../gateway-controls.md#my-requests-stopped-reaching-a-provider---three-conditions-three-answers) to distinguish cooldown, breaker and routing eligibility.