VerOpsDocs
Get started
Docs / Sentinel

Troubleshooting Sentinel

Runs that never appear, runs that never score, partial evaluations, 'judged without knowledge-base grounding', scores that moved when the judge did, monitors that never or always fire, approved actions that went nowhere, and MCP clients that see no tools.

sentineltroubleshootingingestjudgeprovidergroundingembeddingsmonitorsconnectorsmcp

Where to look first

Two places answer most questions: the Runs list, filtered to the agent, and Sentinel Health. If the run is not in the list, the problem is upstream of Sentinel; if it is in the list but never scored, the problem is the judge.

My run never appears

  • Wrong host. Events go to https://sentinel.verops.io/api/v1.0/sentinel/events — the dedicated Sentinel host — not to app.verops.io. The response to a good request is {"accepted": 1}.
  • Wrong or rotated key. A 401 means the X-Sentinel-Key header is missing, mistyped, belongs to a deleted agent, or was rotated after the agent was configured. Rotation invalidates the old key immediately; copy the current one from the agent's page.
  • The body was rejected. A 400 names the field. event_type is required; everything else has a default.
  • It is there — under a different name. Search by execution_id in the header search. If your agent generates a new id per event, each event became its own one-event run; fix the id and the next run will be coherent.

The run is there but has no scores

  1. It is still quiescing. Sentinel waits about 90 seconds after the last event before judging. A run that keeps receiving events keeps waiting. Use Evaluate now to force it.
  2. The judge cannot run. Open Sentinel Health: Evaluations failed rising, or Backlog age climbing while In flight stays at zero, means the judge provider is not answering. The usual causes, in order: the model configured for Sentinel has reached end of life at the provider (a 410 Gone in the evaluation error), the provider key was revoked or its quota exhausted (401/429), or the provider endpoint has changed (404). Fix it under Settings → AI providers for the Sentinel feature. Runs that failed are retried with backoff up to five times; after that they stay Evaluation failed and must be re-queued with Evaluate now once the provider is fixed.
  3. Only some dimensions are scored. The run is Partially evaluated and each skipped dimension carries a reason — a timeout, an out-of-range answer, a provider error on that call. Occasional skips are normal; a steady rate is what the Skip rate monitor is for.

"Judged without knowledge-base grounding"

This note on an evaluation means factual_grounding (and the RAG dimensions) were scored on the run's own context only, not checked against your reference material. It is honest, not broken. Two causes, and the evaluation names which:

  • No documents. Add reference material under Sentinel → Configuration → Knowledge base. Grounding starts on the next evaluation.
  • No embedding provider. Documents are matched by meaning, which needs an embedding model. If the AI provider configured for your organization does not offer embeddings — some providers serve only chat models — grounding cannot run even with documents present. Configure a provider that does, or one dedicated to embeddings, under Settings → AI providers; the Configuration page shows the current grounding status and a Set up grounding link when this is the reason.

Scores changed but nothing about the agent did

Check Sentinel Health → Runs by judge model. Identical runs score differently under different judges — a change of judge model, or a provider silently swapping one, moves every score at once. The evaluation's provenance names the judge prompt and policy versions it ran under; two evaluations with different provenance are not comparable, and the compare views will say so.

A monitor never fires, or fires constantly

  • Preview before saving. The preview replays the condition over recent history and projects alerts per day. Tune the bar and window until the projection is a number a person will still read in a fortnight.
  • No data is not OK. A monitor scoped to an agent that has no runs, or to a measure the events do not carry (cost without token fields, latency without latency_ms), reports no data. Adopt the schema 2.0 envelope for those fields.
  • Muted or cooling down. The monitor's page shows both; a muted monitor says who muted it and why.

A recommendation was approved but nothing happened

  1. Halt is on. Halt all recovery actions under Configuration stops every delivery. It is meant to be turned on in a hurry and forgotten; check it first.
  2. No connector in scope carries that action. The delivery log on the Actions page says so explicitly. Add the action to a connector whose scope includes the agent.
  3. The connector is unproven or failing. Both are counted on Sentinel Health. An unproven connector has never completed a delivery — usually an endpoint that rejects the signature or a private address that Sentinel refuses to call. The delivery log records each attempt and the response.
  4. The policy refused it. An action above the autonomy level, or outside the catalogue, is recorded as rejected — and counted by the Prohibited-action monitor, because repeated refusals are a signal.

MCP: the agent sees no tools

The server URL is https://sentinel.verops.io/api/v1.0/sentinel/mcp with X-Sentinel-Key in the headers of the MCP configuration. A client that connects but lists no tools usually sent no key, or sent it as a query parameter. The MCP page shows the audit of every call that reached Sentinel, including failed ones, so you can see whether the client is arriving at all.