Trustvocate Blog

Bayesian vs. Arithmetic Mean - What we chose for our star rating calculation and why?

If you are here, it’s probably because you love math or are trying to decide which mean to use for your average rating calculator. Let me walk you through why we at Trustvocate chose to stick with the arithmetic mean over Bayesian average. I will keep the real math to the minimum for my own sake and instead walk you through our thought process. 

What is Arithmetic mean and Bayesian average?

Arithmetic mean – This is what we learnt at school. Simply put, it is the summation of observations divided by the number of observations. For example, if one business has two reviews of 5 and 4 stars each, the average would be 4.5 stars (i.e. 5+4/ 2). Easy-peasy innit?

Bayesian average – hold tight! Let’s say you have to choose between the following two businesses to get your suit tailored: 
1. Tailor store 1 – rated 5 stars based on 1 review
2. Tailor store 2 – rated 4.5 stars based on 50 reviews
Which one should you see first when you sort by highest rated? It probably should be Store 2 since it has a bigger number of reviewers and more 5-star reviews compared to Store 1. This is what Bayesian’s average helps us achieve. With all the math you can find here, using Bayesian formula would give us 3.3 stars for Store 1 and 4.6 stars for Store 2.

You seem to be convinced that the Bayesian average should be the way forward for all review algorithms. But that is not true. Let me explain why.

Complications of using Bayesian average:

  • The formula is not absolute. It requires one to assume the value of a variable based on the data collected in the past. Using the variable helps us predict what the average score of the reviewed business is likely to be when it receives n number of reviews. At Trustvocate, we are just starting out and basing our formula on the limited data we have would be a wrong move. 
  • Bayesian average, its derivation and its results are difficult for a commoner to decipher. Seeing a business with one 5 star review with an average of 3.3 stars is unsettling to those who do not know Bayesian average. Most of the users on our website would fall under that category. Internally, we believe that instead of confusing our users, it is sometimes better to show things as they are and leaving it to the user to interpret and decide.

Why arithmetic means works just fine for us:

  • We are in the market for the greater good. When people make a decision looking at reviews, they are very likely to come back to voice their own opinions. Hence a business that performs bad and is rated 5 stars based on 1 review will probably see a drop in the review average when people who make a decision looking at the review come back. Though arithmetic average might give an unclear picture in the short run, no business will be at an unfair advantage/ disadvantage in the longer run. 
  • If newer businesses get an unfair advantage in the short run because of the lesser number of reviews, so be it! Trustvocate is built to enable new and emerging online brands to compete with the big-fishes that have massive marketing budgets through genuine customer feedback. Seems our algorithm is helping us what we set out to do.

We chose to go with arithmetic mean for now to rank businesses on Trustvocate. It’s definite and a lot easier to understand. If you are really a mathematician and want to debate me on this, hit me up at hriday@trustvocate.com. But before you do that, Google too moved from Bayesian to Arithmetic mean for businesses on Google. Just saying…

Hriday Agarwal

Add comment