What has changed?

From 0-1 to 0-100

Evaluation scores are now displayed as percentages (0-100) instead of decimals (0-1). This makes scores instantly clearer. A score of 85 is immediately compelling, whereas 0.85 requires a moment of thought.

Judges tailored to Vragen.ai

We started with ARES as the foundation for our evaluations, but Vragen.ai has grown tremendously. Therefore, we have tailored our AI-as-a-judge systems specifically to the unique requirements of Vragen.ai. The result of this well-deserved care? More aligned evaluations, making it immediately visible which questions require attention. This allows you to specifically improve your content or AI configuration for better answers.

Transparent justification

This might be the coolest part (literal words from Joris): every score now comes with a detailed justification. No more guesswork about why an answer got a certain score. You see exactly what the AI judge saw and why it arrived at this conclusion.

How does it work?

Curious about the details behind a score? Simply click on an evaluation score next to an answer in your dashboard, and you will instantly see an explanation. You see what the evaluation currently measures, and more importantly: why the AI judge arrived at this specific score (the justification).

Take, for example, the reliability score of 100 shown in the screenshot above. Clicking on it reveals that the answer is well-supported by relevant source data and that all claims can be verified. No more vague gut feelings about where the score came from – just crystal-clear insight.

Why is this important?

Excellent evaluations are crucial for the continuous improvement of your AI assistant. With these renewed scores, you can:

• Quickly identify issues: see at a glance which answers need attention

• Optimize in a targeted way: understand exactly what needs to be improved

• Build trust: show stakeholders that you take AI quality seriously

Get started

The new evaluation scores are now live for all users. You do not need to do anything; all new answers will be automatically evaluated using the updated system.

Curious about your AI scores? Check your dashboard and click on a score to view the justification. We look forward to your feedback!