Understanding Bias Rankings: Score Dials vs Graph and Ranking Tables

The Bias dashboard contains several visualizations that use the same underlying ranking data but display it in different ways:

1. Score Dials (Mean Bias)
2. Details Page (Grouped by AI Platform)
3. Scatter Graph + Ranking Tables

Because these components represent different stages of the ranking calculation, the numbers may appear different. This page explains how the rankings are calculated and how each visualization uses that data.

How Rankings Are Calculated

The system evaluates how different Large Language Models (LLMs) rank products within a peer set.

The process occurs in two main steps:

1. Calculate the average ranking across all LLMs
2. Sort products by that average to determine their relative position

Step 1: Each LLM Produces Its Own Ranking

Each AI model ranks products among their peers for a customer segment.

For every LLM × Customer Segment combination, the model produces a whole-number ranking within the peer set.

From the Details Page:

Because rankings at the LLM × Customer Segment level are ordinal rankings within a peer set, they are always whole numbers (1, 2, 3, etc.).

You can see these whole-number rankings when viewing the details page for a specific customer segment.

Step 2: Calculate the Average Rank

The system calculates the average rank across all models.

Example calculation:

(3.9 + 4.4 + 2.7 + 3.4 + 4.6) / 5 = 3.8

This average rank value is what appears in:

• Global Average on the Details Page
• Score Dials at the top of the dashboard

Example:

The score dial showing "3.8 Mean Bias (GenAI)" comes directly from this average.

Step 3: Convert Average Rankings into Positions

After calculating the average rank for every destination, the system then:

1. Sorts products by their average rank
2. Assigns positional rankings (1st, 2nd, 3rd, etc.)

Lower average ranks indicate better performance.

MY BRAND Peer Data:

Here:

• averageRankScore = average across all LLM rankings
• relativeRank = positional ranking after sorting by the average

How Each Dashboard Component Uses This Data

1. Score Dials

Score dials display the average ranking value across all LLMs.

Example:

MY BRAND average rank = 3.8

Displayed as:

3.8 Mean Bias (GenAI)

Score dials do not show positional rank.

2. Details Page (Grouped by AI Platform)

The details page shows:

• Individual rankings from each LLM
• The calculated Global Average

Example:

Global Average:

3.8

This is the same value shown in the Score Dial.

3. Scatter Graph and Ranking Tables

The scatter graph and ranking tables display relative positions, not average values.

The system sorts all products by their average rank score and assigns positions.

Example:

Because MY BRAND has the lowest average rank, it receives:

Relative Rank = 1

This is why the scatter graph places MY BRAND at position (1,1).

Why the Numbers Look Different

Customers may notice that:

• The score dial shows 3.8
• The graph shows #1

This is expected.

The difference occurs because:

Both values are derived from the same underlying data, but they represent different ways of displaying the results.