# Send some of your traffic to a different model

**What you get at the end:** nine requests in ten go to the model you trust, one goes to the model
you are evaluating, and your application code does not change - it keeps asking for one name.

This is the same mechanism as a canary deploy, an A/B test, and a gradual migration. They differ
only in the numbers you write.

Run the Compose commands below from the repository root after the
[quickstart](../tutorial/README.md).

## The idea

A **group** is a name your callers ask for that stands in front of several real models. When a
request arrives for the group, something has to pick one of them - and which something you choose
is the strategy.

For a rollout you want `weighted-random`: it picks at random, but loaded. A candidate with weight 9
wins about nine times in ten against one with weight 1. The randomness is the point. A rule that
sent the first ten percent of requests to the new model would give you the ten percent that arrive
first, which is not a sample of anything.

## Define the group

Groups are boot configuration, supplied as a JSON blob:

```bash
SPRING_APPLICATION_JSON='{"pointsman":{"routing":{"groups":{
  "chat-tier":{"strategy":"weighted-random","candidates":[
    {"provider":"openai","model":"gpt-4o-mini","weight":9},
    {"provider":"openai","model":"gpt-4o","weight":1}]}}}}}' \
  docker compose --env-file docker/.env -f docker/docker-compose.yml up -d gateway
```

### Persist the configuration

This is boot-scoped and leaves on the next plain restart. That is a real constraint and it cuts
both ways: an experiment cannot outlive its container by accident, and a split you meant to keep
disappears the next time somebody runs `docker compose up` without the variable. Put it in your
`.env` if you mean to keep it.

Wait for readiness before sending anything, or the first requests land between the old container
dying and the new one answering:

```bash
until curl -sf -o /dev/null localhost:8080/actuator/health/readiness; do sleep 2; done
```

## Send traffic to the name, not the model

```bash
curl -s localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $YOUR_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"chat-tier","messages":[{"role":"user","content":"hello"}]}'
```

Your callers only ever say `chat-tier`. Changing the split is a config change on the gateway, not a
deploy of every application that calls it. That is most of the value.

## Check which model actually answered

`X-Provider-Model` on the response is the identifier the provider itself reported:

```
X-Provider-Model: gpt-4o-mini-2024-07-18
```

Count them across a few dozen requests and you will see roughly your weights.

### Interpret the observed split

A small sample does not establish the configured proportions. With a 90/10 split, the probability that twenty independent choices all select the 90% model is about 12%. Verify the resolved strategy and collect a larger sample before interpreting the split. For example:

```
strategy=weighted-random chose openai:gpt-4o-mini (decisionId=5924a680-...)
```

If you want to know that the group is live rather than that the dice were kind, read that line
instead of counting. It appears once per request in the gateway log, names the strategy, and would
be absent entirely if the group had not resolved.

## What you will not see

**There is no `X-Decision-Id` header on a weighted-random group.** That header exists so a caller
can report how an answer turned out, and a random draw has nothing to learn from feedback. It
appears only when the outcome-learning router made the choice. Its absence here is correct, not a
missing feature - see [response headers](../reference/response-headers.md).

## Moving the dial

Change the weights and restart. There is nothing else to do: no draining, no warm-up, no
coordination between instances, because each request is an independent draw. Going from 9:1 to 1:1
to 0:1 is three restarts and no application changes.

**When you want the gateway to move the dial for you** based on how the answers actually turn out
rather than on numbers you chose, that is the bandit strategy, and it is a different guide.