I was just rating a manga that already had a level of 25. I was provided with six other manga to rate against, four of which were level 24 and two of which were level 25. I ended up rating the manga harder than all six of them (which bumped the level to 26), but I feel like this doesn’t let me really nail down the difficulty of the manga. For example, what if I had been given some level 26-28 manga to compare against as well and still thought this manga was harder? That would tell you more than only grading against similar level manga. The same can of course occur on the easier side.
In short, I’d like to grade against manga similar in level, a bit below in level, and a bit above in level. Or perhaps it should be more dynamic. Since I graded the manga as harder than the first two similarly leveled manga, subsequent gradings should be against progressively higher level manga as long as I keep saying the manga is harder.
I think some context is needed. I assume it was about your gradings on 毎月庭つき大家つき 1 which has 4 different people grading it now (including yourself), so the system probably has already a good idea of where about it should be and was more narrowly trying to fit and jumping a bracket (27 begins N2 level) is more difficult when 20~ other grades point to being mid-to-high N3.
I’m not someone who runs the site or anything, but I know the system will not ask for grades way outside of expected bounds when multiple users have placed it already. Extra grading exists, but unless something is seriously wrong with the existing gradings then Upper N3 is about right?
To your next point, grading it against harder or easier works when the grading is pretty accurate is not really helpful for everyone - the grading system is supposed to get a gauge on how difficult all users perceive it to be and not take random guesses outside of an expected range because those gradings should be 1. obvious. and 2. unnecessary.
I don’t know how the system selects which books to ask you to grade against, but I’d also like to see more variation in what’s asked, or have the ability to choose the most apt comparisons myself when grading. Two books the system chooses to compare could be the most useful comparison from a technical standpoint (i.e. close enough levels to make the grading certain), but from the reader standpoint they might be too different in genre or something for that comparison to feel as relevant as another possible comparison.
Interesting - I will sometimes see that it will check for extra gradings and will either let me do it or just check and reject it. I know that beginner (level 0-20) will not be asked for level 30+ books. I don’t know exactly where the cut off is, but I know that if I grading a work that hasn’t been graded before it usually does somewhere +/- 6 and then from the responses I give it will adjust downward. I haven’t really come across a work that was massively off expectation.
You do make a good point of genre and such and I know nothing of how it factors in, but many things have been added without content tags or reviews - I think more than 50% of all works have not received a single grading even for even a single volume/season. The entire system runs on user input and it’s tough to sort it out with no input. Comparisons to genres and such may be possible, but how would you track that before gradings are done? Manually fixing listings is tough and time consuming - I’m not sure if such a thing is practical.
I’ve rambled on enough, but some insight into the process would be helpful.
That was the book, yeah. To be clear, I was just using this book as an example and I’m not even saying the site should use gradings against books with larger level gaps in a specific way. I just feel like being able to give more variety of gradings allows me to express my opinion better and the site could (doesn’t have to) use this information to influence the level. Also, for what it’s worth, the separation of JLPT levels is arbitrary. There’s no difference between going from L25 to L26 and going from L26 to L27.
Regarding extra gradings, those aren’t actually used right now in determining levels. I believe this is because the current system favors recent gradings and if extra gradings were considered (naively) it would give too much influence to the last grader.
Kinda to this point, sometimes I have a specific book (usually one of the last that I’ve graded) that I know how to compare against the current book, but that book never comes up. I wish you could start the grading with a specific book and then the algorithm would go from there.
4 Likes
As an Amazon Associate, Natively earns from qualifying purchases through any Amazon links on the site.
