Prepare a deployment

Documentation / How-to / Deployment

The quickstart is a local evaluation setup. Before exposing a deployment, work through the items that affect your intended workload. The launch status separates implemented features from completed release evidence.

Choose the deployment profile#

The evaluation quickstart and qualified v1 profile use different accounting and storage settings.

On the quickstart, request detail explains that per-attempt recording is not enabled. An empty view is not evidence that no provider work occurred.

Credentials and network boundary#

Use real credentials for every provider you enable. Keep the key-hash pepper stable, backed up and separate from the repository. Set the key environment explicitly and replace sample database/admin credentials. Bootstrap admin settings seed an empty admin table; they are not a password-reset mechanism.

The base Compose stack publishes the gateway and dashboard ports on the Docker host. Put the application behind authenticated administrative access and your TLS ingress. Keep Postgres, Redis, the learner and observability services private. The learner has no public authentication boundary of its own. Read the security posture for the exact published surfaces.

Persistence and upgrades#

Preserve the database, Redis and learner volumes according to the behavior you need to recover. Test restoring a backup with the same pepper. Apply migrations to a representative database copy before a version upgrade; see operating notes.

Supported upgrade scope#

V1 is the initial supported installation. Repository v0.x tags are development markers; upgrading a pre-release deployment into v1 is not a supported or tested path. Earlier schema migrations are development history, not upgrade evidence. Later releases must state their supported upgrade paths.

Migration failures#

On the v1 profile (tested 11 October 2026 with disposable data), At startup the gateway validates the schema history and applies any pending migration before it serves anything. Two failures were induced on a populated v1 installation: - a validation failure: the recorded checksum of an applied migration no longer matched the shipped script; - an application failure: a migration re-applied onto an existing table, so its CREATE TABLE failed.

In both cases the gateway never became ready and answered no request. The container exited and restarted, failing the same way each time, and the log named the cause (Migration checksum mismatch for migration version 64; relation "budget_held_inclusion" already exists). All 43 application tables stayed identical, compared row by row: the failed statement was rolled back by PostgreSQL's transactional DDL.

Not tested: - a migration that cannot run inside one transaction; - a failure while traffic is being served; - a large database; - running an older gateway on a newer schema.

Recovery contract#

There is no in-place downgrade. Running an older gateway on a newer schema, or rolling a migration back, is not supported and has not been tested. The supported recovery from a failed or unwanted upgrade is to restore the verified stopped-write backup taken before it. A database restore alone does not make spending safe again, so follow the budget-recovery procedure in operating notes: declare the restore with POINTSMAN_BUDGET_RECOVERY_RESTORE_ID, then attest each held tag before reopening traffic.

The tested sequence below was run on the drill's own disposable data. It stops the gateway, takes a custom-format dump, then recreates the database and restores it in one transaction. The restored tables and schema history matched the backup exactly. Budgeted requests were refused (503) with no provider call until the tag was attested, then served.

# take the backup with the gateway stopped (no writer)
docker compose ... stop gateway
docker compose ... exec -T postgres pg_dump -U "$POSTGRES_USER" -d pointsman -Fc > pointsman-backup.dump
# restore: an empty database, then the dump in a single transaction
docker compose ... exec -T postgres psql -U "$POSTGRES_USER" -d postgres -c 'drop database pointsman'
docker compose ... exec -T postgres psql -U "$POSTGRES_USER" -d postgres -c 'create database pointsman'
docker compose ... exec -T postgres pg_restore -U "$POSTGRES_USER" -d pointsman --no-owner --exit-on-error --single-transaction < pointsman-backup.dump
# start the gateway with a NEW restore id, then attest (operating notes, steps 3-6)

Use the same -f files and --env-file as your deployment in place of .... Keep the backup, the counter-store persistence and the journal volume until recovery is complete.

For durable usage delivery on one Compose gateway, set POINTSMAN_USAGE_JOURNAL_DIRECTORY=/data/usage. That directory is already mounted by the stack. Each replica needs its own stable persistent journal volume; do not scale multiple processes onto the same journal. Empty means legacy best-effort async recording. Durable accounting design and limits explain unresolved provider-outcome reconciliation after interruption. The recovery drill proves explicit UNKNOWN recovery after a killed gateway and OBSERVED replay when PostgreSQL fails after provider dispatch. It also proves the first uncached key request returns retryable 503 in the configured database-connection budget while PostgreSQL is down, before any provider dispatch. It does not cover provider reconciliation or a backup-restore drill.

Select and record the operating profile#

The qualification runs use settings beyond the evaluation defaults. For the documented budget/accounting profile, configure these explicitly and verify their startup checks:

POINTSMAN_USAGE_JOURNAL_DIRECTORY=/data/usage
POINTSMAN_BUDGET_FAILURE_MODE=deny
POINTSMAN_BUDGET_FENCE_STORE_CHECKS=enforce
POINTSMAN_CACHE_L2_STORE_CHECKS=enforce

Run Compose with --env-file docker/.env. REDIS_COUNTER_HOST, REDIS_CACHE_HOST, POSTGRES_HOST, POSTGRES_PORT, POSTGRES_DB, POSTGRES_USER and POLICY_SERVICE_URL override the bundled endpoint defaults. Passwords and other settings in the environment file remain supported. An external endpoint does not remove the bundled service or its dependency checks; use a deployment-specific Compose overlay/topology for that. Changing PostgreSQL database/user settings does not migrate an existing database volume.

Use separate counter and cache Redis instances: the counter store must preserve spend state, while the cache store has a memory limit and may evict. Follow the ACL and persistence requirements reported by the store checks; enforce is not a substitute for configuring Redis correctly. The evaluation defaults use report for those checks, and an empty journal directory selects best-effort usage recording. Those settings are not equivalent to the tested durable/enforcing profile.

Start the v1 overlay#

docker compose --env-file docker/.env \
  -f docker/docker-compose.yml \
  -f docker/docker-compose.v1.yml up -d

The overlay fixes these settings for the qualified profile:

The qualified runtime passed this profile on 28 October 2026. Match the image/source to release status; later performance candidates require separate qualification.

Profile restrictions#

• Image input: tokenizer mode refuses image_url content with 400 unsupported_request_control.
• Cost rules: use tenant-scoped rules. Root-scope cost rules, platform cost ceilings (including warn-only) and enforcing per-model cost rules cannot be represented by this reservation manifest. Applicable requests receive 503 accounting_unavailable, counted by pointsman_limits_engine_errors_total{stage="manifest_unsupported"}.
• Counter-trust loss: capped requests are deliberately refused. Before reopening, manual attestation must cover unresolved spend and the backup gap.

Outbound network restriction is the operator's prerequisite: the profile does not install any firewall or egress rule. The application refuses admin-set destinations that resolve to link-local, metadata or unlisted private addresses, but it cannot stop a connection-time DNS change. The one network control this repository has tested, its limits, and the pattern that is not supported (an internal Docker network with an HTTP egress proxy) are in outbound destinations.

For literal text-request caching, set pointsman.cache.modalities.text.enabled: false in application configuration; L1 stays available. Semantic reuse is approximate even with its lexical verifier. temperature-mode: exact-only alone does not disable it at zero temperature. Use X-Cache-Control: no-store when a request must be recomputed.

Capacity and observability#

Size total process memory, including heap, native allocations and kernel memory. POINTSMAN_GATEWAY_MEMORY_LIMIT selects the Compose cgroup limit; changing it can also affect JVM heap ergonomics and requires matching workload validation.

Reclaimable cache can keep the cgroup near its limit. The qualification footprint is anonymous plus shared plus unreclaimable kernel memory. The shipped memory alerts use a broader working-set estimate as an early warning; neither raw usage nor heap alone explains pressure.

Validate your actual container limits, providers and traffic shape before routing production traffic.

PostgreSQL disk#

At a sustained 50 RPS, a 90-minute run on 25 September 2026 grew the database by about 0.55 GiB per hour: about 13 GiB per day and about 400 GiB per 30 days. The billing ledger and its provider-dispatch evidence are kept 730 days (POINTSMAN_RETENTION_USAGE_EVENTS_DAYS, operator-only, floor 365), reservation manifest rows until 35 days after their period ends, and the routing decision log 30 days. The v1 profile keeps those defaults, so the database keeps growing for up to two years before the nightly purge bounds it: size the disk for your request rate and retention, and watch free space. The retention knobs are listed in the configuration reference.

Startup network dependencies#

Observed on the v1 profile, 11 October 2026. On a fresh v1 installation, from the first container start through the first requests and a gateway restart, every connection from the stack to an address outside its own network was recorded. The provider was a local fake:

No other container connected outside. The MiniLM embedding model is part of the image and is not downloaded. The gateway no longer reads the cloud instance-metadata service. Before 11 October, DJL's call-home read 169.254.169.254 at startup; the image now sets OPT_OUT_TRACKING=true. Building images (--build) also downloads Maven, npm and base-image content. An offline or air-gapped installation is not claimed or tested; what happens when publish.djl.ai is unreachable was not tested. Allow these destinations, or pre-populate the model_cache volume, when you restrict egress. Provider and operator-configured destinations come on top, as described in security.

Startup and first requests#

The following observations were measured on the v1 profile on 25 September 2026.

• Time to readiness. The gateway reports ready about 18 s after its container starts. Readiness waits for the MiniLM model probe, which passes about 11 s in, and for both tokenizer encodings, which are warmed at boot. So the first counted request never pays a tokenizer or embedder load.
• First requests. A tenant's first requests activate its budget epoch; a few concurrent ones may be refused 503 budget_recovering for about a second and succeed on retry (known issues).
• First cache misses. In the first seconds, cacheable prompts miss until they have been seen once, so their latency is the provider's.
• Measure after warmup. The qualification harness labels the first two minutes as warmup and excludes them from steady-state figures. Measure your own steady state the same way.

Choose intake, cache, stream-assembly and tokenizer limits from the operational limits reference. The overload follow-up met the refusal-latency gate after the TCP backlog fix; the original cold-process heap-delta gate and final ingress/resource envelope remain open. Do not assume a configured permit count alone proves fast shedding.

Use INFO logging for an operational baseline. Keep tracing explicitly disabled or configure a working collector; do not compare an unavailable-collector run with a healthy tracing deployment as if they were the same profile. Monitor request errors and latency, budget-store failures, cache skips/errors/dropped stores, intake refusals, usage-delivery backlog and process memory.

Validate and roll out#

Exercise the compatibility subset with your clients and provider credentials, including streaming cancellation and usage tails. Run load tests against disposable stores, not an existing tenant database. Include a separate load generator for capacity claims and test your real ingress and connection behavior. Benchmark method covers the current proof.

Start with traffic you can inspect, define rollback criteria, and test recovery from provider and store outages. For replicas, read multi-instance operation first. If enabling learning, use a bounded workload and meaningful feedback; the learning guide explains cold-start exploration.