The Sentinel page (AI control plane)

The Sentinel page

Sentinel (left nav) is the AI control plane for monitoring your AI agents in production. Where the Workbook AI-quality nodes gate AI in CI, Sentinel continuously watches live agent runs: your agent reports each run, Sentinel evaluates it, scores it, and feeds back guidance the agent can use to improve.

The tabs

Tab	Shows
Agents	Registered AI agents, their health and connection key.
Evaluations	Per-run scores against your rubric, filterable by score band.
Guidance / Backlog	Improvement guidance and learned behaviors derived from evaluations.

Scores are banded: ≥ 70 good, 40–69 warn, < 40 poor — so regressions are obvious at a glance.

Cookbook: connect an agent

1. Go to Sentinel → Agents → Add agent; set an Agent name.
2. Copy the Agent ingest key (sent as the X-Sentinel-Key header).
3. From your agent, report each run and read back its evaluation/guidance:

# report a run
POST https://app.verops.io/api/sentinel/report_run
Headers: X-Sentinel-Key: <agent-ingest-key>
Body: { "agent": "support-bot", "input": "...", "output": "...", "tools": ["kb"], "metadata": {...} }

# then read its evaluation (score + guidance)
GET https://app.verops.io/api/sentinel/evaluation/<runId>

4. Watch the run appear under Evaluations with its score and rationale.
5. Use Guidance to apply recommended improvements; over time Sentinel tracks behaviors learned.

How it relates to Workbooks

Sentinel reuses the same evaluator engine as the Workbook AI-quality nodes (correctness, faithfulness, relevance, completeness, safety — with the deterministic safety backstop). The difference is where it runs: Workbooks evaluate in CI and synthetics; Sentinel evaluates production traffic continuously. It also exposes an MCP surface (tools, resources, prompts) so agents can self-critique and apply guidance.

Define what 'good' means in your rubric; Sentinel scores every run against it and surfaces the worst so you fix the systemic issues, not one bad answer.