Ranking Ranking Scales

I love ranking things. But how do the ways to rank things, rank? Read on in this rank post to climb the ranks of rank ranking rankers.

1. MAL Ratings

   MAL ("My Anime List") ratings range from 0-10, where 7.5 is "pretty good", anything above 8 is a must watch, anything below 7 is garbage, below 6 is horrific. The best of all time hover around 9.

   Pros:

   • Funny
   • Result of real aggregated opinions; implicitly encodes bias and both ends of the bell curve in terms of rating scales for optimal universal understanding
   • Always leaves room for something better and something worse
   • Accepts that most everything is garbage (0-7)
   • Easy cross comparisons between various foods and anime

   Cons:

   • People always say, "6/10? That's not bad!"
   • You have to explain what MAL is

   Rating: 8.1

2. Likert-ish Scale

   Basically, pick five words and assign them those values (awful, bad, average, good, great). Pretty simple. Removes the need for the rater to decide what a "3/5" is. However, removes a lot of freedom of choice, ensures a collision for rating >5 items.

   Rating: 7.2

3. Pure 1-5000 (Overwatch SR style)

   Funny idea.

   Pros:

   • Can rate tea as GM
   • Works well with any normal distributions

   Cons:

   • Hard to rate things below 3000 unless you really hate them
   • Most food ratings are a skewed normal distribution

   Rating: 7.2

4. Inverse-MAL scale

   If everything is garbage, why would you spend the most number-space resolution on those? Invert the MAL spectrum and spend 1-3 on garbage and spread out all the good ones between 4-10. (I forgot who gave me this idea, but it's pretty good).

   Rating: 7.15

5. Pure 1-10

   Good luck actually getting people to rate 1-10 with 5 being average.

   Rating: 7.1

6. Pure 1-100

   Fundamentally the same as 1-10 with a decimal, but less cool. Reminds people of percentages and academic grades, which also creates a bias of <60 being the "really bad" cutoff.

   Rating: 7

7. Pure 1-5

   It's just 1-10 but worse. Only suitable for cruddy online star systems where you need to force the unwashed masses into picking "good" or "bad" while still giving them the illusion of choice.

   Rating: 6.8

8. Japanese 1-5

   Pretty close to what the 1-5 "should" be in a normal distribution. Anything above a 3 is probably good, Anything below a 3 is bad. A 3 is average. Feels stupid writing that out, but it's certainly not common to see 1-5 used in that way. Which disturbs me.

   Rating: 6.7

9. Forced Ranking

   No scores, everything is rated implicitly in relation to everything else on the list by its ordering. There's no objective assigned value.

   Pros:

   • Don't have to worry about a scale

   Cons:

   • Sucks
   • Doesn't show how much better or worse one entry is vs another

   Rating: 6.3

10. Fixed Sum

    You have X points. Distribute them.

    Pros:

    • Can't be so generous with the points, point boy

    Cons:

    • Cursed
    • Impossible to rank more than X things

    Rating: 6

11. Airbnb 1-5

    Anything below a 4 means you were sleeping in a vat of goo and were mugged at 3AM. Bad, mediocre, good, and stellar are all between 4 and 5.

    Rating: 5.3

12. Uber 1-5

    Anything below a 4.9 means you died in the process of attempting to rate the thing. Everything else is a 5.

    Pros:

    • Funny
    • They know where you live

    Rating: 4.2