The policy service is the background learner for the gateway's balanced Thompson/LinTS paths. It accepts attributed observations and publishes parameter snapshots. Model selection happens in the gateway; the learner is not a mandatory network hop for each routing decision. The separate in-process cost objective does not use this learning loop.
For administrator grants, scope inheritance and request model permissions, see policy-based authorization. Those decisions live in the Java gateway; this Python service does not authorize administrative actions.
The learning loop#
- The gateway records the decision context and actual outcome. Eligible completion, failure or late quality signals become reward events.
- A bounded reporter sends events asynchronously. Retries preserve event identity and payload; exhausted delivery is counted rather than hidden.
- The learner validates the payload, learning semantics, composition and retained-history boundary.
- It computes a proposed transition and commits the event record, affected learner state and version together in SQLite.
- Only after commit does in-memory state move. A failed write must not leave a new posterior visible only in memory.
- The gateway periodically polls
/policy/parametersand uses its local snapshot for later decisions.
The separation protects request latency from synchronous learner work. It also means feedback is eventually reflected in routing: a gateway acknowledgement, an enqueued reward and an applied learner update are different events.
Event identity and revisions#
An event identity includes decision_id, signal_kind and revision. A matching retry returns the prior acknowledgement without applying the event twice. Reusing the identity with a different payload is a conflict. Supported newer revisions replace the contribution of the prior revision instead of counting both as independent evidence.
The learner retains the evidence needed to make these decisions. History compaction has a committed time boundary; an event outside retained history is refused rather than replayed as new. Storage-capacity exhaustion also produces a typed refusal instead of silently evicting deduplication identities that are still needed.
Publicly described, privately reachable endpoints#
| Endpoint | Purpose |
|---|---|
GET /health |
Process health |
GET /policy/parameters |
Published versioned learner state |
POST /policy/reward |
Attributed event ingestion and acknowledgement |
GET /policy/receipts |
Applied, duplicate, stale and conflicting event observations |
GET /policy/history |
Retained-history budget and maintenance state |
POST /policy/seed |
Explicit operator-controlled prior seeding |
POST /context/process |
Optional cache-context processing, separate from routing selection |
The retained /policy/decide API is not the gateway's normal selection path. The optional context endpoint can be on the request path for selected cache modes; saying “background learner” must not hide that separate dependency.
The service has no application authentication boundary of its own in this deployment. Keep it on the private service network; its port is not published by the base Compose file. Anyone able to reach mutation endpoints could affect learned state. Do not expose it at pointsman.dev or alongside the public documentation.
Persistence and recovery#
SQLite persists posteriors, applicable LinTS state, reward events and the version counter. One worker owns the in-memory registries; startup refuses an incompatible multiple-worker configuration. Scaling HTTP workers is not a supported way to scale the learner.
A missing/empty volume starts from priors with a warning. A store that fails its integrity/readability checks refuses startup and is preserved. A healthy restart loads persisted state. Restoring an older or empty store may publish a lower version; gateways adopt the authoritative snapshot and record the regression rather than indefinitely training one state while sampling another.
Keep the history window compatible with the gateway's feedback validity, retry horizon and clock allowance. A shorter window is not merely a disk-space optimization; it changes which events can still be applied safely.
Failure behavior and tradeoffs#
| Condition | Result |
|---|---|
| Learner unavailable | Gateway continues with a last snapshot or priors as applicable; learning delivery/poll failures are observable |
| Same event retried | Original acknowledgement, no extra training |
| Same event identity, changed payload | Conflict refusal |
| History expired or invalid time | Typed refusal; no reconstruction from missing history |
| SQLite unavailable/full | No committed state transition; bounded reporter retry/drop behavior applies |
| Valid quality revision | Recompose using the recorded decision semantics; do not mix incompatible reward definitions |
This design favors attributable, replay-safe learning over accepting every signal. It cannot correct biased labels, establish causality from observational traffic or guarantee a business outcome. Changes to objectives, rubrics and model identity require explicit policy/evidence handling.
Implementation anchors: policy_service/main.py, schemas.py, bandits/thompson_registry.py, bandits/store.py, bandits/ts2.py; gateway RewardReporter, PolicyClient and BanditParametersRegistry. See learning-history operations, adaptive routing and deployment.