Give labels to the ratings to improve consistency and open up the full rating range

Description of your request or bug report:

Rating stuff is hard, and everybody treats ratings differently. If you see someone rating a book as a 3, what does that mean? Do they use the full range of ratings, and 3 is still something they enjoyed? Is it their “this was okay”? Or is this their minimal rating, because they just use 3-5 on everything?

I really like the official Goodreads rating system for that. It uses the full range available, it gives the scores meaning, and I think I barely ever have to ponder how many stars to give the book:

  • 1 star - didn’t like it
  • 2 stars – it was OK
  • 3 stars – liked it
  • 4 stars – really liked it
  • 5 stars – it was amazing

EDIT: Updated suggestion for labels

They display it as a tooltip when hovering over the stars. (In some views, at least; sadly, they are a bit inconsistent.)

Alternatively (and maybe additionally?), I think it would be be great to give half-stars. Sometimes you have a book which feels like it’s better than e.g. your usually “3”, but still not quite a “4”, so I’d like to give it a 3.5 instead of feeling like i’m under- or overrating the book.

Trello link: (leave in blank)

I agree with your general concern (I saw someone on Natively giving nearly all 5/5 ratings and thought “well that’s not helpful”), but I want to mention that there is another case for when people generally rating 3-5 out of 5: they are choosing books to read that they are likely to enjoy. For comparison, my average rating on AniList for anime is 8/10. Some people might see my 8/10 average and think that I’m not using the whole scale, but that’s not the case. I’m simply only watching anime I expect to enjoy, so I rarely have to give a series less than 6/10.

Personally, I have two problems with these descriptions.

  1. The voicing is inconsistent. So much so that I’m surprised a company like Goodreads hasn’t made it consistent by now. Ideally it shouldn’t jump between feelings by the individual (e.g. “liked it”) and descriptions about the quality (e.g. “it was amazing”). Personally, I’d say ratings are always subjective, so I think subjective descriptors such as “liked it” are better. (I’d replace “it was amazing” with “loved it”.)
  2. I feel like 2/5 should be “didn’t like it” and 1/5 should be “hated it”. To me, “it was OK” is way too positive of a statement for 2/5. For me, 2/5 books have some merit of course (otherwise it would get 1/5), but usually I think 2/5 books are less than “okay”.

Anyway, all that is just my subjective opinion. As you said, rating stuff is hard and everyone does it differently. And honestly, even if there were descriptions I wouldn’t change how I rate books. I think as Natively gets more users, inconsistent rating methods will balance out.

8 Likes

Hugely seconded. Goodreads is my favorite place to rate stuff because I don’t have to hem and haw and try to figure out if I’m being internally consistent with a book.

1 Like

I get that, but and that might just be completely me, but:

  • I’m not sure I would ever rate a book with a “I hate it” label. Half of it because a) I also only pick books that I think I might like, and b) to be honest, it feels unnecessarily strong. Thinking about it, I don’t think I have ever consumed any media that I hated. There were plenty of media that I “didn’t like” tho.
  • I regularely wish that I had more “good” options than just 4 or 5.

I get your point, but something being so bad to deserve the lowest rating should be pretty rare, no? I think if you’re only picking books you think you might like, it would be perfectly reasonable if you never gave 1/5 stars. Another thing to consider is that just because you aren’t likely to give 1/5 = “hated it” ratings (and neither am I for that matter), that doesn’t mean no one would. So I’m not sure the rating system should be skewed for people like us who happen to be unlikely to give 1/5 ratings.

(For reference, I just checked my Goodreads account, and I apparently gave one the worst books I have ever read a 2/5. And while I’ve given several Japanese books 2/5, I’ve never even considered giving a 1/5 so far. So clearly I really don’t like giving out 1/5 scores!)

2 Likes

Yup, I think I have rated two things 1 star. One was a ‘short story’ off Aozora (it turned out to actually be a suicide note by the author) and the other was a short story with graphic descriptions of animal cruelty. I feel pretty comfy saying I ‘hated’ both of those.

I walk into a lot of books and stories completely blind. Cool title, pretty cover, had vaguely heard of the author, whatever. My ratings span 1 through 5 stars, with most probably being a 3 or 4. I am the person who uses all the star options :joy:

5 Likes

Haha. Well, that was a quick demonstration.

Looking at the ratings here, many people might actually consider 3 to be “it was OK”. That’s actually another case for keeping it that way if labeled; possible consistency with existing ratings.

Thinking about it, I mainly want two things actually:

  • Having to think less about whether I’m internally consistent and what ratings mean.
  • More room for nuance in the positive ratings.

Originally I thought of the Goodreads labels because they open up a new positive rating space, but I think both could also be reached by e.g. the following labels:

  1. hated it
  2. didn’t like it
  3. it was OK
  4. really liked it (“liked it” feels a bit weak, tbh, but I’m not married to the exact wording)
  5. loved it

…and being able to set half-stars.

2 Likes

I checked my IMDb account, and I have given four shows/movies a 1/10. But even having watched 300+ anime, I haven’t given an anime 1/10 (I did give a 2/10, which I would have rounded down to 1/5 if that was the rating scale though).

I have occasionally bought random manga based on the cover, but I did read the summary and knew the genre before buying, so it was still a case of being very unlikely I’d give it 1/10.

3 Likes

I definitely feel the difficulty here… I think “really liked it” and “loved it” for 4/5 and 5/5 respectively accurately describe how I give out those ratings. I think “hated it” and “didn’t like it” would also be accurate descriptions for how I would give out 1/5 and 2/5 ratings. 3/5 is a little more difficult though. Sometimes “liked it” feels accurate. Sometimes “okay” or “fine” would be more accurate. Perhaps I could say 3/5 often means “I liked it, but have some reservations”, but at that point “liked it” is accurate enough (and I like the phrasing being consistent).

(I don’t know why I keep commenting on the descriptors though, since as I said in my first reply, none of this will change how I rate things anyway. :joy:)

1 Like

i’m one of those people who have rated almost everything with 5 stars. i’d find textual descriptions of what the stars mean much more useful than just the numeric value, and might well go through and reevaluate my ratings if the system were changed accordingly.

when it comes to the language-learning rating, additional guidelines on what the stars mean would be even more useful.

5 Likes

This raises some added complexity specific to Natively… because of the “entertainment” and “language learning” rating breakdowns, some people may incorporate those into the overall rating too. For those people, terms like “liked it” wouldn’t necessarily be sufficient. But I wouldn’t want the overall rating descriptions to include “language learning” aspects either since that could overwhelm people who just want to rate based on enjoyment. (I for one, never consider “language learning” in my overall rating.)

Hmm… :thinking:

2 Likes

I don’t either. I barely even use those extra ratings anymore. Pretty much only comes up when I’m full on ‘meh’ about a book but I can see it being a useful learning tool. Most recent example I can think of for that is モテ薬 | L33 which fell super flat for me, but I could see being super helpful for someone trying to expand their science vocab and get used to various different types of speech.

If it didn’t exist I don’t think I’d miss it, though. I write reviews more often than not. Do people look at those extra ratings?

4 Likes

I try to give a “language learning” rating for the first volume in any series I read, but it’s not like I deeply analyze it. I mostly rate it based on whether it uses reasonably normal language (so not some super niche setting with weird words) and if it seems like something that might be good for expanding one’s knowledge of grammar, kanji, or vocab. That said, since I used WaniKani and skewed my learning heavily towards kanji and reading, I’m not sure my “language learning” ratings are accurate for people coming from a more balanced learning approach.

I don’t ever look at the extra ratings since they are inconsistently filled in and at my level I read for entertainment, not language learning. Perhaps people with less reading experience do look though, I’m not sure.

3 Likes

The only books where I add a “language learning” rating is graded readers, and thinking about it, so far I’ve only used the 5 star rating there. A prime example where this goes into my overall rating is Japanese Short Stories for Beginners: 20 Captivating Short Stories to Learn Japanese & Grow Your Vocabulary the Fun Way! | L15, which is severely lacking on the entertainment front, but was a fantastic starter book with a nice difficulty curve and good “tooling” (dictionary, summary, translation, quiz).

I wonder how other people use it, especially on non-graded readers. I guess some books are better for certain stages or goals. I wonder what kind of books are generally bad for language learning.

In general, I feel like I see “5 star language learning” more often than anything else, and usually it just means that a star gets added to the overall score (when people do “(entertainment+language learning)/2”). I wonder how the range is there. (And yeah, it kinda makes sense; you learnt stuff while reading the book, because it was written in normal, useful Japanese. But does that really deserve an extra star on every book?)

I feel like a way to highlight why this is good for language learning or for whom it is good would be more useful.

Apart from that, not sure how you’d “solve” this, if it needs to be solved, and if it should be part of this request (because it complicates things a lot more).

(I think I’ve also seen books which were like “language learning 4, entertainment 4, overall 5”, or the other way around, and I’m like… I sure would love to know what went into that rating process.)

4 Likes

tbh most of the Aozora Bunko short stories I read and the occasional Very Old :tm: book are probably not great for most people’s reading goals which are usually:

  • Consume modern media
  • Pass the JLPT
  • Survive in everyday Japan

but for me, to whom reading that stuff is part of my reason for learning anyways, they’re great! I commented on some story forever ago that most people don’t need to know words like 阿片窟 あへんくつ, opium den :stuck_out_tongue:

I agree it’s kind of outside the request, but discussing it is fun. I like hearing how other people approach rating and reviewing from different angles.

5 Likes

The reason I write reviews for the 世にも奇妙な商品カタログ (series) | L25 series is because I think the rating alone doesn’t make justice of what I want to express.

While I don’t think I’ll do it for Manga aside from the first volume sometimes, I think I’ll keep doing it for the rest of the light novels I read.


As for the plenty of 5/5 ratings, I think allowing half starts or increasing the scale to 10 would solve it. The thing with the full scales is that 3/5 is a 6/10 and then you jump to either a 8/10 or 10/10 if you base the scores in that scale. It gives you less flexibility.

While giving stars from 1 to 5 without halves might be fine to a customer rep that replied to your email, a book is more complex than rate it from 1 to 5, and way more subjective.

2 Likes

I quite like this scale, too. GR veteran here, too. Most of my ratings are 3 stars… a decent book, but nothing special for me. I might still recommend it, if someone is looking for a specific type of book. 4 stars were books I really liked and 5 stars are basically books I absolutely adore. 2 stars is literally an OK book. Something I didn’t like but didn’t dislike either. Just something I feel meh about for whatever reason. 1 star is usually if the book is extremely bad (rare) or has problematic content (my typical reason).
I would very much like half-star ratings, though…
Regarding the entertainment/learning ratings: I am pretty sure I have given a book 3 stars before but gave higher ratings on entertainment and learning. I don’t think a good book is necessarily entertaining… nor is an entertaining book necessarily good. :sweat_smile: I don’t know if these ratings are needed but I don’t think they hurt.

2 Likes

I haven’t used Natively ratings much yet, but I don’t think adding descriptions to reach number would change how I rate anything, or how I view other’s ratings.

Expanding the ratings to half-values or out of 10 would be nice, though.

That said, I definitely err on the side of rating high/middle. I checked my anime ratings on anilist.co and of the 206 anime I’ve marked as watched I only rated 7 below a 4. However, I also generally feel its unfair of me to rate an anime with out finishing it or at least giving it a decent chance, since there a lot that just don’t fit my tastes, so the ones left on my list with low ratings are ones that started out promising and then I ended up disliking after watching at least 5-6 episodes.

1 Like

There has been some interesting suggestions in this thread.

Maybe we can get the input of what @brandon thinks.

Thanks for the ping @Megumin and for the topic @Legato.

I get the desire for consistency in ratings… but I do think it’s a bit difficult. We could have guidelines like GoodReads has, but would they actually be abided by? For example, if we specify 3 stars = ‘liked’ vs imo the more commonly 3 stars = ‘OK’, would people actually do it?

Honestly, I somewhat doubt it. I’m totally fine with implementing a soft guidelines like GoodReads though (surface the guidelines as a hover over in some places, maybe a FAQ, stipulate it’s a soft suggestion… etc), but I don’t think we should expect much impact from it, it’d just be a nicety.

As for the guidelines, here’s my humble suggestion, mostly in the context of ‘recommendation’ :slight_smile:

  1. hated it (do not recommend)
  2. didn’t like it (wouldn’t recommend, unless you are really interested)
  3. it was OK (wouldn’t proactively recommend, but if asked wouldn’t recommend against)
  4. really liked it (would proactively recommend)
  5. loved it (would highly recommend, and would keep telling you you HAVE to read it)

For your absolute favorites, I think having favorite functionalities distinguishes those books from other 5 stars.

Half stars
I guess this is just a matter of preference. I don’t like half stars (it’s essentially just a 10 pt rating system) and I think 10 pt systems are less common & more stressful and potentially more noisy. I think people mostly like it to distinguish their favorites vs their loved books, but I think a favorites functionality is best for that, not complicating the rating system.

But, it’s a somewhat arbitrary decision. I don’t particularly imagine I’ll change it soon :sweat_smile:

Extra Ratings (Language Learning / Entertainment)
Yeah I don’t imagine everyone will use these or use these consistently. However, when reading a review I think it provides helpful quick context - there are some books (especially easier books) which people find boring but very useful. The extra ratings simply allow you to give a quick clarifier on your overall rating.

They aren’t particularly useful, but I thought they were a nice, interesting addition :slight_smile:

Happy to hear reactions to all that :upside_down_face:

8 Likes