New evaluation scores: more insight into the performance of your AI
Great news! We've significantly upgraded our evaluation scores. You'll now get much better insight into how your AI is performing and where some attention might be needed.
Lisette Vredenburg
Product Owner
What has changed?
From 0-1 to 0-100
Evaluation scores are now displayed as percentages (0-100) instead of decimals (0-1). This makes scores clearer at a glance. A score of 85 is immediately understandable, while a 0.85 requires a moment of thought.
Judges tuned to Vragen.ai
We started with ARES as the basis for our evaluations, but Vragen.ai has grown tremendously. Therefore, we have tailored our AI-as-judge systems to the unique requirements of Vragen.ai. The result of this well-deserved attention? Evaluations that align better, allowing you to immediately see which questions need attention. This helps you improve your content or AI configuration for better answers.
Transparent Justification
This might be the coolest part (in Joris's own words): every score now comes with a detailed justification. No more guessing why an answer receives a certain score. You see exactly what the AI judge has seen and why it came to this conclusion.
How does it work?
Curious about the details behind a score? Just click on an evaluation score next to an answer in your dashboard, and you'll immediately see an explanation. You'll see what the evaluation now measures, and more importantly, why the AI judge has arrived at this specific score (the justification).
Take, for example, the reliability score of 100 that you see in the screenshot above. When you click on it, you'll see that the answer is well-substantiated with relevant source data and that all claims can be verified. No more vague feelings about where the score comes from – just crystal clear insight.
Why is this important?
Good evaluations are crucial for continuously improving your AI assistant. With these renewed scores, you can:
Quickly identify issues: see at a glance which answers need attention
Optimize precisely: understand exactly what needs improvement
Build trust: show stakeholders that you take AI quality seriously
Get started
The new evaluation scores are now live for all users. You don't need to do anything; all new answers are automatically evaluated with the updated system.
Curious about your AI scores? Check your dashboard and click on a score to view the justification. We're eager to hear your feedback!
