Quickstart: your first Chat Completion

Run Pointsman locally, issue a virtual key, and make a real provider request. This is an evaluation setup. Read the deployment guide before exposing the stack to other users.

Prefer a trial without model-provider credentials? Use the mock-provider walkthrough.

1. Prepare the environment#

Install Docker with Compose 2.24 or newer, curl, jq and openssl. You need an OpenAI API key for this walkthrough; requests incur provider charges. Leave time for the first image build and the roughly 90-MB embedding-model download.

git clone https://github.com/srinivas-reddy-gouru/pointsman.git
cd pointsman
cp docker/.env.example docker/.env
openssl rand -hex 32

Edit docker/.env, setting these existing entries. Paste the generated hex string as the pepper and choose your own admin password:

POINTSMAN_AUTH_PEPPER=<paste-the-generated-value>
POINTSMAN_KEY_ENVIRONMENT=test
POINTSMAN_ADMIN_USERNAME=admin
POINTSMAN_ADMIN_PASSWORD=<choose-a-long-password>
OPENAI_API_KEY=<your-openai-api-key>
ANTHROPIC_API_KEY=dummy-key

The current bootstrap initializes both provider clients. dummy-key satisfies the unused Anthropic client's startup check for this OpenAI-only local trial; Anthropic requests will fail until you configure a real credential. Keep requests explicitly on gpt-4o-mini for this walkthrough. Placeholder credentials are not a production configuration.

Keep the pepper stable across restarts and back it up with your secrets: changing it invalidates existing key hashes. Do not commit docker/.env. Shell-exported variables override values in the file during Compose interpolation; unset stale exports if the running settings differ from what you edited.

2. Start and check readiness#

From the repository root:

docker compose --env-file docker/.env -f docker/docker-compose.yml up -d --build
docker compose --env-file docker/.env -f docker/docker-compose.yml ps
curl --fail --show-error http://localhost:8080/actuator/health/readiness

Repeat the readiness request until it returns a successful HTTP response. If startup fails, inspect the logs instead of waiting indefinitely:

docker compose --env-file docker/.env -f docker/docker-compose.yml logs --tail=100 gateway

The gateway uses port 8080 and the dashboard uses 3001. The base Compose file publishes these application ports on the Docker host; it keeps the backing-service ports inside the Compose network. Use a trusted local machine for this walkthrough. The dev overlay is only needed to access backing services from the host, and publishes those additional ports on loopback.

3. Create an organization and a virtual key#

These commands use the bootstrap admin admin. curl prompts for the password you chose in step 1. The provider key stays in the gateway; your application receives a separate revocable virtual key.

ORG_ID=$(curl --fail-with-body --silent --show-error --user admin \
  -H 'Content-Type: application/json' \
  -d '{"name":"my-team"}' \
  http://localhost:8080/admin/organizations | jq -er '.id')

KEY=$(curl --fail-with-body --silent --show-error --user admin \
  -H 'Content-Type: application/json' \
  -d '{"label":"first-app","kind":"service"}' \
  "http://localhost:8080/admin/organizations/$ORG_ID/keys" | jq -er '.key')

Check each command succeeded before continuing. The complete key is returned only once; the server stores a keyed hash. kind: service means the key belongs to the organization. Key ownership and budgets →

4. Send a completion#

curl --fail-with-body -i http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","max_tokens":64,"temperature":0,"messages":[{"role":"user","content":"Explain what an AI gateway does in one sentence."}]}'

Expect an OpenAI-shaped Chat Completion response. Inspect X-Request-ID, X-Served-Provider, X-Provider-Model, X-Cache and X-Cache-Semantic to see what happened. Their exact values depend on the selected model and cache state; the header reference explains the conditions.

Repeat the identical request to try the exact cache. An eligible stored response can produce X-Cache: HIT-L1, avoiding another provider call. Cache eligibility, store completion and output guardrails still apply; a paraphrase is not guaranteed to hit the semantic tier. temperature: 0 explicitly requests deterministic sampling. Cache behavior and tuning →

5. Open the dashboard and choose a next step#

Open localhost:3001 and log in with the same admin account. Follow the dashboard tour with screenshots to understand providers, routing groups, Configuration Studio, Canvas and request investigation. Inspect the provider/model catalog and request usage, then:

• Issue a key with a budget.
• Add another provider.
• Split traffic between models.
• Turn on outcome learning when you have feedback to provide.

Troubleshooting and stopping#

docker compose --env-file docker/.env -f docker/docker-compose.yml down

This stops the stack while retaining named volumes. Keep those volumes and the same pepper if you want existing keys and data to survive the next start.

See routing learn on a controlled demo#

Once the quickstart works, the included demo can show the mechanism with three fixture providers: fast/good, slow/good and fast/poor. It sends requests and feedback, then reads the gateway's own learned beliefs. It is not a benchmark of real-model answer quality or savings. First-build/download time is separate: one recorded clean rehearsal took about seven minutes to become ready, so a ten-minute total setup is a goal to test, not a launch promise.

Run this on your disposable evaluation stack. The script temporarily restarts the gateway with disclosed demo settings. Set the same admin credentials you chose in docker/.env (use your shell's private input for the password), then run:

export POINTSMAN_ADMIN_USERNAME=admin
read -r -s -p 'Pointsman admin password: ' POINTSMAN_ADMIN_PASSWORD; printf '\n'
export POINTSMAN_ADMIN_PASSWORD
bash scripts/demo-learning.sh
unset POINTSMAN_ADMIN_PASSWORD

The input example is for Bash. The script reads Compose settings from docker/.env. Keep its per-checkpoint output and final verdict: seeing traffic pass is not the same as seeing the arms separate. This demo is an operator-run traffic test. The packaged demo passed on the qualified release candidate; its evidence is recorded in launch status. Running it yourself illustrates mechanics on your evaluation stack and does not establish real-model quality or savings.

The dashboard setup recipes explain role boundaries, credential types, exact recorded-request verification, shared budgets and optional adaptive routing.