Model compare and KB grounding
A/B models with an independent judge; semantic knowledge-base grounding.
Model compare and knowledge-base grounding
AI Model Compare
A/B two providers/models on the same prompt, scored by one independent judge (never each candidate grading itself), with a winner and per-criterion deltas — so you pick the better or cheaper model with evidence.
In the node's config panel, fill in the Prompt, pick Model A and Model B from your registered providers, tick the criteria to score, and optionally add a Reference (the expected answer, for correctness) and a Context (the grounding source, for faithfulness). The outputs include the winner, each model's mean and per-criterion scores, the score gap, and each model's response latency.
Knowledge-base grounding (semantic)
Ground faithfulness/correctness against your curated facts instead of a pasted blob. Documents are indexed when you save them and matched by meaning, not just keywords — e.g. "can I get my money back on my card?" finds a "refund to the original payment method" document with no shared words. If semantic matching is temporarily unavailable it falls back to keyword retrieval, so grounding never hard-fails.
Manage the knowledge base
- Open Sentinel → Configuration → Knowledge base to add, organize and clear your curated documents. Group related facts into a collection (e.g.
support-policies). - In an AI Evaluate node, tick Ground against the knowledge base and (optionally) name the collection — the retrieved facts become the judge's context automatically.
VerOps Sentinel
The same evaluator engine, pointed at production traffic, becomes continuous AI quality and safety monitoring — VerOps Sentinel.