New evaluation scores: a clearer view of how your AI performs

Good news. We gave our evaluation scores a serious upgrade. You now get a much better picture of how your AI performs and where it may need attention.

Lisette Vredenburg Product owner

3 December 2025 2 min read

Evaluation of an answer with scores for reliability, relevance and source relevance.

What changed?

From 0-1 to 0-100

Evaluation scores now appear as percentages (0-100) instead of decimals (0-1). That makes a score clear at a glance. A score of 85 lands right away, where 0.85 makes you stop and think.

Judges tuned to vragen.ai

We started with ARES as the basis for our evaluations, but vragen.ai has grown enormously since. So we tuned our AI-as-judge systems specifically to what vragen.ai needs. The upshot of all that attention? Evaluations that fit better, so you see immediately which questions need work. From there you can improve your content or your AI configuration for better answers.

Transparent reasoning

This is probably the best part (Joris's words, not ours): every score now comes with detailed reasoning. No more guessing why an answer got a particular score. You see exactly what the AI judge saw and why it reached that conclusion.

How does it work?

Curious about the details behind a score? Just click an evaluation score on an answer in your dashboard and the explanation appears. You see what the evaluation measures, and more importantly why the AI judge arrived at this specific score.

Reasoning behind the reliability score: for each claim, which source supports the answer.

Take the reliability score of 100 in the screenshot above. Click it and you see that the answer is well supported by relevant source data and that every claim can be verified. No vague sense of where the score came from, just a clear view.

Why does this matter?

Good evaluations are crucial for improving your AI assistant over time. With these new scores you can:

Spot problems quickly: see at a glance which answers need attention
Improve where it counts: understand exactly what needs fixing
Build trust: show stakeholders that you take AI quality seriously

Getting started

The new evaluation scores are live for all users. There is nothing you need to do: every new answer is evaluated automatically with the updated system.

Curious about your own AI scores? Check your dashboard and click a score to see the reasoning. We'd like to hear what you think.