I generally trust natively levels, though I find that some of the more veteran readers of the site seem to rank books a lot lower than where I put them. I forgot who it was, but someone in the コップクラフト | L35 book club who had read it before said that they thought it was closer to an L30 than the L35 it was and is still rated as.
I think that the scale of around 1-40, which becomes closer to 19-35 in practice, is probably too dense to accurately make a distinction between a lot of content. I’m not sure if the backend has more fine grained ELO rankings, but from the user perspective I feel like the discrete steps between the levels lose a lot of nuance and ends up grouping together content that can feel noticeably different. If there is more accurate rankings on the backend, it’d be cool to see that reflected in something like an option/toggle to see decimal number rankings.
I noticed this as well back when starting to read light novels; the ratings between manga and actual novels has a very large gap between them, to the point that I don’t hold any stock in how the two are rated against each other. Something like orange | L23 for example is rated L23, but is significantly more difficult than an L23 manga. Reading in general is obviously more difficult, but it doesn’t seem to be reflected in most book ratings.
There is a big difference, but I feel like that stems from the fact that the drawings make guesswork a lot easier. You can even completely miss a couple sentences and still be able to follow along, because you can see what’s happening. That’s what makes reading manga (or anything with a substantial visual support) a lot easier. If you just look at the text, the difficulty of the grammar, the vocab, that’s what I tend to grade the difficulty of manga on. It can be objectively harder than novels, but taken together with the visual support, it’s like having a sort of crutch. So it enables people to read above their level, because they don’t need to understand every single sentence anymore.
The pictures are definitely a big help to comprehension, but also novels tend to use very different grammatical structures and vocabulary in Japanese that takes a long while to get used to. Often comprehension becomes a matter of not only unknown vocab, but also parsing longer sentences and looking up less common grammar.
I feel you and I actually think it’s more difficult than 狼と香辛料 1, as I needed more look-ups per page for it. So I am happy that it’s at least back at L35.
That’s me!
I stand by that too. Somehow, Natively is crashing for me right now so I can’t check, but I’d say it’s level ~32 (slightly harder than 本好き), which is closer to 30 than 35
To the original question, this thread has reinforced my trust in the levels. The fact that we’re basically discussing edge cases, and a few design/structural issues with room for improvement, seems to me like an indication that the levels work well.
It can definitely help, but I find sentence complexity and range/level of vocab more relevant, in this regard (furigana can also make a huge difference here, but that’s just a side effect of vocab complexity)
I think as well, the visual format of manga at least discourages having too many very long sentences, simply because the result can look quite ugly and they need to balance the space for art too.
I do have some problems with books that have a lot of gradings, but I feel like that’s a me thing. I mentioned it already somewhere, but I thought 夏と花火と私の死体 | L27 was way harder (the grading was 26 when I read it) than コーヒーが冷めないうちに and most other books I’ve read in that range (I just specifically read those books in close relation to each other). While I think that コーヒーが冷めないうちに is just a matter of fact description with lots of repetitive narration (which I find to be easier), 夏と花火と私の死体 had a more long flowery sentences that took me longer to parse. I am still really surprised of many of the gradings it has, for example being easier than あなたも殺人犯になれる! | L26 あなたも殺人犯になれる! is “just” dialogue and very little descriptive scenes and I find that to be much easier. But since there are so many people that grade it differently from me, I feel like it’s more of a me thing and am really hesitant to grade at all
I’m guessing you’re referring to the second story? If so, I agree. But the first story took up the bulk of the book, and it really was extremely simple language (fittingly, as the narrator was a child). Books with uneven difficulty like this one are especially hard to grade.
Please don’t! Your gradings are as important as everyone else’s. While not everyone agrees on what is difficult and how difficult it is, it’s important for all voices to be heard.
And let’s not forget that readers don’t decide the level directly. I don’t look at a book and say, ah yes, a level 27. Well, I might, but what I input in Natively are comparisons, and specifically comparisons I’m prompted to do. I’ve often graded a book to the best of my memory only to see it end up at a level very different to the one I expected.
I do think the first story is underrated too! It’s not flowery, but often includes really long winded sentences (not in the grand scheme of things, but in the grand scheme of other books around that level ), which can be (in my opinion) very hard to parse for a beginner. While the other 2 books I mentioned only have very short, very descriptive sentences.
Here is some example I quickly picked up when browsing through : 純白の服と白い肌が村の女の人には珍しい清潔さを感じさせ、外の明るさをそのまま持ってきたかのように、少し薄暗い家の中でも光って見える。
To illustrate how the difference felt for me in English:
This would be a scene in コーヒーが冷めないうちに and あなたも殺人犯になれる!
Someone is coming in. They sit down. Oh no, they are sad.
“Don’t be sad!” the other character says.
While 夏と花火と私の死体 felt like:
Someone was coming in and sitting down, they had a sad look on their face while hearing the words “Don’t be sad!” coming from a close direction.
It’s basically the same vocab and not that hard, but still, noticeably more effort for me to parse To be fair, there were lots of easy and short sentences in 夏と花火と私の死体 too, but it had (in my perception) way more of those more difficult long winded ones. But maybe only I have issues with those kind of sentences?
No, of course it’s not only you. That’s why I say it’s important for everyone to grade, even if their opinion seems different.
These are all good examples that I didn’t remember at all. For me, there were I think 2 years or so of reading experience between the two books. コーヒーが冷めないうちに was hard but doable for me at the time (first novel), while 夏と花火と私の死体 felt almost boringly easy as a general feel, even if some sentences were more flowery than others, as you now reminded me. Even though I know that コーヒーが冷めないうちに wasn’t objectively hard, it was much harder for me at the time I read it than 夏と花火と私の死体 two years later, so even though I try to adjust my comparison to that I may still be wrong, or only subjectively right.
Which is to say, please grade as you see fit instead of hesitating to grade because you disagree with previous gradings. You may be fixing skewed gradings after all.
Those seem pretty typical at that level range, from what I’ve read. Interesting that the other books you mention seem to do without them
I second this! Also personally I look at grading as a ご褒美 (prize/reward) for finishing a book (tho it occasionally feels like a chore)… “You finished the book, and earned the right to give your feedback” sorta thing…
So definitely take your well earned prize!! (assuming this metaphor resonates at all)
That’s why I felt it might be a me-thing ! Although I only quickly went through the book to grab a few sentences, if I would’ve noted down sentences that gave me trouble while I was reading the book it would’ve been probably more representative.
I take Natively gradings as a very rough guideline and usually I can only tell from the first 20 pages if something is easy or hard for me anyway.
Oh really, for me gradings are more like a chore because I feel like it is very hard…! I only do it because I hope people will profit from it.
Agreed. The only time you should feel the need to not grade something is when you truly cannot remember how difficult it was compared to something else. Despite all the complaining me and others have done in this thread of levels being off and gradings being wrong, that is exactly what makes the site work. It’s not one person’s opinion, but an aggregate of what everyone thinks, which is what makes it work so well.
Gradings used to feel like a reward for me. But since I stopped being able to grade extra it’s gotten harder and feels more like a chore now. (It was a bug at first, but now the button is completely gone so I think the ability is removed from the site.)
I used to be able to remember the relative difficulty of things pretty easily because I was constantly doing extra gradings which would cause me to continually have to remember the difficulty. Now that I don’t have that constant reminder, I quickly forget how difficult something was and end up skipping half of the gradings until I find something that I do remember.
This makes it feel like more work, and also makes me more unsure of the gradings that I do manage to give.
This is probably a bug, particularly since Our Grading System | Natively still mentions extra grading. @brandon not sure if this is infrastructure-change related or not. (If a separate bug report would be helpful, I can make one)
Funny enough I actually stopped doing the extra gradings cuz that started feeling like a chore
Mm, sometimes I’ll remember things from months ago clearly, and other times I can’t remember the thing I read last week. Sometimes I try to make notes about the language, but it gets harder over time, due to plateauing and over-familiarity (like everything with in a genre/subgenre becomes “yeah that’s typical”, unless it really stands out for a specific reason)
Everyone has different standards for their own reading level. There are some people getting into their first books, just happy to be there. Some read with paper dictionaries where lookups are expensive, some read with hover dictionaries where lookups take 2 seconds. Some read extensively without looking things up at all. I think these groups all have different tolerances for what they understand, and that often kills the gradings.
I’d bet that the majority of users are probably reading digitally with a hover or tap dictionary. So, when they go to rate a book’s difficulty against another book’s, this hover dictionary user might not even consider that one book had way more vocab, more niche vocab, more spread vocab, and just rate them the same because the reader’s own level wasn’t even a factor in the difficulty. One book may have been linguistically easier but conceptually difficult and so was rated harder. This is a problem, I think.
Another big problem shows up a lot more in the tv/movie ratings. Since ratings are based on how hard it felt rather than how hard it was, stories that are more visual/plot based are generally rated as easier than stories that are character/dialogue driven, even when the language used in the former was significantly harder. I saw a review the other day while browsing where someone said Terrace House and Neon Genesis Evangelion were similar difficulties, and I think the only way that could be true is if you watched both on mute and didn’t pay attention to the words. NGE is chock full of military, sci-fi, keigo, in-universe terms, while Terrace House is largely slice of life, more or less natural reality tv that biases toward more common vocab. But since NGE has a strong visual element and Terrace house does not, it gets rated (artificially) as closer in difficulty than it should.
And the entire catalog is like this. I’ve seen shows on there that I’ve watched before with just absurdly off ratings (especially on the shows that get recommended to beginners most like Yuru Camp. They’re relatively easy, but they’re not that easy). The whole system is extremely susceptible to white noise, and while beginners-low intermediates should be encouraged to immerse and participate, they just throw off the accuracy of it all because they aren’t good enough judges of the difficulty of content that is significantly above their level.
What makes the grading system good is that it’s spread over a lot of users. We’re all using different methods to read, so it all averages out in the end. I feel like there’s also probably less people doing purely digital lookups than you think, or maybe I’m just the outlier here. But yeah, I hardly ever do lookups in Spanish, and for Korean for some books I do quick digital lookups, for other books I have paper books so I do a couple manual lookups per page and for others I do physical lookups for every single new word.
And I do think that the fact that a book is conceptually difficult should be taken into account for the grading. If the story is super straightforward, you have more room for misunderstanding things. Take a book where the narrator switches its point of view, from one character, one place and one time to another constantly, now if you misunderstand just one paragraph you can end up being completely lost. That’s also why children’s books are usually a lot easier than novels aimed at adults, it’s not just a matter of vocab, but also the concepts being more straightforward.
あなたも殺人犯になれる! is “just” dialogue and very little descriptive scenes and I find that to be much easier. But since there are so many people that grade it differently from me, I feel like it’s more of a me thing and am really hesitant to grade at all 
To be fair, there were lots of easy and short sentences in 夏と花火と私の死体 too, but it had (in my perception) way more of those more difficult long winded ones. But maybe only I have issues with those kind of sentences?
! Although I only quickly went through the book to grab a few sentences, if I would’ve noted down sentences that gave me trouble while I was reading the book it would’ve been probably more representative.
