Metric Calculation

How are Benchmarking Metrics calculated?

There are two main scores that we calculate for Benchmarking Metrics.

1. Relevance Score

2. Performance Score

Detailed information about the calculation of these scores is given below.

RELEVANCE SCORE CALCULATION

Relevance Score is calculated based on Precision Score and Recall of a Query,

Precision Score

Currently, we are calculating a Precision score based on the following 5 metrics:

Accuracy

R-Precision

MRR (Mean Reciprocal Rank)

NDCG (Normalised Discounted Cumulative Gain)

Recall

PERFORMANCE SCORE CALCULATION

Benchmarking Metric Calculation Parameters

In the above calculations, different weights are used to balance the Relevancy scores. The user can provide these by clicking on the Settings button located on the top-right corner of the Model Analyzer screen.

Accuracy Trends

Accuracy trends are calculated by categorizing results in four categories, as given below. By looking at each category, the user can get an idea on how to tune the Relevancy Model to get more results in the true category.

RESULT CLASSIFICATION