Shopping at LEGO or Amazon?
Please use our links: LEGO.com Amazon
As an Amazon Associate we earn from qualifying purchases.

How are the review ratings calculated?

BooTheMightyHamsterBooTheMightyHamster Northern edge of London, just before the dragons...Member Posts: 1,296
edited January 2013 in Brickset.com
I'm a bit baffled, so if someone could clarify, I'd be grateful.

In the review database, each model has a 'score', of (presumably) somewhere between 0 and 5.
Each reviewer gives a 1-5 brick score for 5 categories (Overall, Building experience, Parts, Playability and Value for Money), and I'd assumed that these were combined to give the overall score.

However, if we look at #42006, it only has one review. For the five categories, it scored 5,5,5,5,4.

I'd therefore expect that 24/5 would give it a 4.8.

And yet it has a score of 3.8.

Can anyone explain how the scores are determined, please?

Thanks!

Boo

Comments

  • Cam_n_StuCam_n_Stu UKMember Posts: 368
    Looking at http://www.brickset.com/detail/?Set=79003-1 the review summary says 3.8 (1 Review), but when you click on the review it's Huw 5* review!
  • leemcgleemcg Member Posts: 607
    I've seen Huw discuss this before (but I'd never be able to find the post).

    He uses a clever method to calculate the average, which gives a set with 20 5* reviews a better average than a set with only 1 5* review. He adds in another virtual review (I think with 3 stars), which will gradually get less and less weight as there are more reviews.
  • LostInTranslationLostInTranslation UKMember Posts: 5,524
    I think this has been queried before - I believe it's because there's only one review and some kind of weighting is applied. I'll try and find the previous discussion.
  • JosephJoseph Member Posts: 651
    edited November 2012
    ^ & ^^ Here you go: http://www.bricksetforum.com/discussion/comment/27119/#Comment_27119
    Huw said:

    There's a weighting score, of 2.5, added which causes the number of ratings as well as the rating values themselves to influence the overall rating.

    So, 1 score of 4

    4 + 2.5 / 2 = 3.2

    1 score of 5

    5 + 2.5 / 2 = 3.75

    5 scores of 5

    5*5 + 2.5 / 6 = 4.5

    100 scores of 5

    100*5 + 2.5 / 101 = 4.97

    It stops a single score of 5 giving the same rating as a set that has 100 scores of 5.

    Clever eh? :)

  • Cam_n_StuCam_n_Stu UKMember Posts: 368
    OK. I guess it is one way to do it, but it does result in misleading summary scores in these sort of instances. Also, as I wasn't aware of this weighting I've been taking summary scores at face value for a few sets. Thinking it was disappointing and not bothering to read the actual reviews!

    I think I prefer IMDB.com's approach which doesn't show a summary score at all until it has an specified number of reviews, 10 in their case I think.

    Surely it would be better to only apply the weighting when sorting sets on score rather than for each individual on-page summary score where you can see it only has 1 or 2 reviews and hence should treat the score with caution?
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,470
    Thanks everyone. It may have some disadvantages but I think they are outweighed by the advantages and its simplicity.
  • BooTheMightyHamsterBooTheMightyHamster Northern edge of London, just before the dragons...Member Posts: 1,296
    @Huw
    @atkinsar

    Sorry chaps, this probably ought to be in 'The Database', rather than 'The Forum'.

    If somebody could shift it, that would be great.

    Thanks,

    Boo
  • CCCCCC UKMember Posts: 17,947
    edited January 2013
  • BooTheMightyHamsterBooTheMightyHamster Northern edge of London, just before the dragons...Member Posts: 1,296
    @CCC

    Ah... Clever stuff!
    Thanks for the explanation.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,470
    edited January 2013
    This method was copied from the way it was done back in the day at Lugnet.

    It's very simple, but works very well in ensuring that sets with one rating don't skew the top rated sets table
Sign In or Register to comment.

Shopping at LEGO.com or Amazon?

Please use our links: LEGO.com Amazon

Recent discussions Categories Privacy Policy

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Brickset.com is a participant in the Amazon Services LLC Associates Program, the Amazon.com.ca, Inc. Associates Program and the Amazon EU Associates Programme, which are affiliate advertising programs designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.

As an Amazon Associate we earn from qualifying purchases.