Explorer, dashboards and health
The Explorer's dimensions and measures, saved views and CSV export; the six stock dashboards and the sentinel.* metrics; compare views that state a delta and refuse to over-interpret; the Sentinel Health page and the change digest.
Explorer
Sentinel → Explorer is an aggregate view over every run in a time range. Pick what to group by — agent, agent version, campaign, environment, fleet, judge model, evaluation version, run state, health — optionally a second level (then by), and which measures to show:
| Measure | What it is |
|---|---|
runs / scoredRuns | How many runs, and how many of them have actually been graded. Averages below are over scored runs only — a version half-way through evaluation must not read as half as good. |
avgScore / minScore | Mean and worst score. |
breaches / criticalBreaches | Runs that breached, and those at critical severity. |
errors | Runs whose events carried an error. |
avgDurationMs, avgToolCalls | How long a run took and how many tools it called. |
avgTokens, totalTokens, judgeTokens | The agent's own token use, and what the judge spent grading it — the evaluation bill, separately. |
The vocabulary is curated: field names are lookup keys, not SQL, so any combination is safe to run and nothing here can be used to read outside your organization. Save view keeps a question with a name and, if you tick it, shares it with your organization; Export CSV downloads the current table (cells are escaped against spreadsheet formula injection). Every row links to the runs behind it.
Dashboards
Sentinel → Dashboards ships six stock dashboards, each defined in the Explorer's own terms, so every widget has an Explore button that opens the exact query behind it:
| Dashboard | Answers |
|---|---|
| Fleet Health | Are my agents healthy right now? Runs, breaches, critical breaches, by agent. |
| Agent Quality | Which agents and versions are scoring well, and which are sliding? |
| Evaluation Pipeline | Is Sentinel keeping up? Runs by state, backlog, partial and failed evaluations. |
| AI Cost & Efficiency | What is each agent spending, per run and in total, and what does grading it cost? |
| Quality Detail | The dimension heatmap (agent × dimension — a per-agent average says an agent is weak, this says which dimension), provider latency percentiles, consensus amplification (judge tokens against run count — the multiplier on the evaluation bill), and pipeline gate usage. |
| Safety & Automation | Safety breaches, recovery deliveries and refusals, connector health. |
Sentinel also publishes its series into the platform's metrics store as sentinel.* metrics — sentinel.run.count, sentinel.run.breached, sentinel.agent.avg_score (by agent), sentinel.judge.tokens, sentinel.evaluation.backlog_age_seconds, sentinel.connectors.failing and the rest — so they can be charted on any platform dashboard, queried in Data Explorer and used in alert rules next to your infrastructure metrics. A set of platform dashboards over these series can be provisioned into the dashboard list (folder AI Agents) with POST /api/v1.0/sentinel/platform-dashboards/provision; provisioning is reconciling, so running it twice creates nothing twice.
Compare
Two rows in a table are not a comparison — they make the reader do the subtraction and remember which column was the baseline. Sentinel's compare views (an agent's Versions tab; version-to-version, agent-to-fleet and cohort comparisons through the API) state the delta, declare which direction is better per measure (higher is better for score, worse for breaches — never inferred from the sign), weight averages by run count rather than averaging averages, and refuse to interpret below ten runs a side: the numbers are shown, but a difference between them is not called evidence.
Sentinel Health
Sentinel → Sentinel Health is Sentinel watching itself, because a monitoring system that fails quietly is worse than none. Each panel is a sparkline over the last hour to the last week, with a note on why it matters:
- Evaluation backlog and backlog age — how many runs await the judge and how long the oldest has waited. Age is the one that distinguishes busy from stuck.
- Evaluations in flight, failed, partially evaluated — the judge's own error budget. Failures rising is where a provider outage or a model reaching end of life shows first.
- Sentinel's own token cost and agent token spend — what grading costs against what the agents cost.
- Runs by judge model and average score by judge — because identical runs score differently under different judges, and a judge change looks exactly like a quality change until you look here.
- Monitors firing, recovery deliveries failed, unproven connectors, failing connectors.
- Query latency p95 — the responsiveness of these screens themselves.
The change digest
At the top of the Control Plane, the change digest answers "what moved since I last looked?" from structured data only — no model writes it. It reports quality moves, new agents, agents gone silent, cost moves, safety breaches, monitors firing or blind, connectors failing or unproven, and judge disagreement, and it distinguishes quiet (nothing happened) from no data (nothing could be measured), which are not the same reassurance.