Do you trust Natively levels?

It definitely is. Reading your first proper no-pictures novel is like slamming up against a brick wall, followed by a 100 kilometer trudge through a tar pit. In fact, the next two or three are also pretty rough unless the content is on the easier side.

I mainly read on desktop using ttsu and yomichan, so lookups are near immediate. I tend to do very fast checks on new words to see the meaning and frequency, and then I will use the built-in “add to anki” feature if it’s within my target frequency range (or if it’s an obviously useful word).

When I’m in transit or out of the house I read on a kindle paperwhite, which has a built-in pop-up dictionary. It’s nice but since it’s an older e-ink device it’s much slower and clunkier, so I tend to try and avoid lookups unless I really don’t know it, and I’ll highlight the word.

I don’t read often on paper, but when I do I am almost guaranteed to know the kanji, so I’ll take a guess as to the reading, and if I get it wrong I’ll just write a word I do know with each character. I really hate trying to use OCR on my phone, it’s often slower for me than the way I do it currently, and same for those “construct it from the radicals” search boxes

8 Likes

Indeed. I think DuoUnderscore’s concerns about really low level users are legit, but the opposite is a problem too. Advanced readers can’t easily differentiate between levels of easiness and will often grade books that should be 5+ levels apart as being the same. That’s why it’s important to have gradings by a variety of users.

10 Likes

I’m curious: what are you looking to get out of Natively?

As an example, I love that I get exposed to books especially, but also other media that I wouldn’t know about otherwise, and that I have a vague guide as to the amount of effort that material would be be for me to read.

Additionally, book clubs have been really instrumental in me pushing my level as I apparently am motivated to read with a group, but that’s less about Natively specifically.

I really don’t want this to be a pile on thread bc I feel like that’s where we’re maybe going, but two things here:

  1. Do you think you would have gotten to your current level having not read far above your level with tools?

  2. What about the situation where you “know” all the words but don’t understand the sentence? That requires zero tools but I think we can agree that that would be above someone’s level.

6 Likes

I had hoped for a way to find content at a very similar level to other content I was very comfy with so I could gradually see the rise, and it’s close but a bit too hit-or-miss so now I just find longer shows/book series and err into too-easy stuff when I branch out.

Book clubs here look fun! Haven’t tried any yet, but they look pretty nice to participate in.

I do (and maybe would have been better off for it), but my stance on the levels thing is unrelated to learning strategy. If you want to read hard books, go for it, I’ll encourage it. I’m just acknowledging that tool assistance and community influence can upset the gradings.

I try my best to avoid this scenario; I aim for 100% of the sentences I read to be understood even if I don’t know all the words. I’m a very particular case in that regard, hence why I wish (and I seem to stand alone on this) that the levels would be pretty consistent, and that when I open a 26 I wouldn’t get something harder than other 28s and when I go to watch a 29 rated series, I don’t get something that’s probably more like a 24. Again, it’s sometimes pretty good but a lot of the time it’s just like… way off, and then you look at the written reviews and you can see why.

4 Likes

@brandon

With the risk of making the system more complicated and not sure how useful in the end it would be:

Maybe aside from grading, having a way to vote if you agree/disagree/strongly disagree with a book level, as to gather data and further tweak the algorithm?

6 Likes

The thing is, gradings are always going to be somewhat subjective. If you compare a book with slightly more difficult grammar to one with slightly more difficult vocab, should they be the same level or should one be higher than the other, and which? There’s no right answer to things like this, it’s all personal perspectives. So something that might seem easier than something else to you might seem the opposite for someone else.

Another factor could be what type of vocab you’ve exposed yourself to. For example, I’ve started watching Single’s inferno | L25?? and while there’s technically a very small range of vocab, it’s also a lot of vocab that I’m not used to, so at times it feels harder to follow along than your regular romance drama to me, because I’ve watched a lot of those and specifically sat down to learn a lot of the vocab. (I’m also really bad at konglish, because, again, I’m just not used to hearing it) Someone that has watched mostly reality tv, on the other hand, would likewise struggle more with dramas.

9 Likes

Perhaps. It’s possible fodder for a request.

In all honesty though, I think the best way to improve the algorithm is to simply look at the grading data and see how self-consistent it is. Some books & tv shows are bound to have more variance for all the reasons @DuoUnderscore and others have stated and it’d be nice to surface that information. I can’t find the request, but I thought there was a feature request to surface historical rating level changes and compute how large the variant / how consistent something is.

I also agree with a lot of @DuoUnderscore says around Video especially. I think since a lot of books have a narrator that speaks in one style, that can make books feel a bit more consistent? Not true for manga of course. I also agree with @DuoUnderscore that people are watching stuff that’s above their level in Video more consistently, especially with tools like language reactor built into Netflix.

I really do like @bibliothecary 's Detailed language evaluations suggestion too. I think that’d illuminate the ‘why’ questions you’re asking on how difficulty is being assesed.

Ultimately though, it comes down:

  • is it helpful for tracking your progress?
  • is it helpful for discovering new content?

I think you’re right that it’s not 10/10 for each of those, especially for video. And there is a lot of work that can be done to analyze possible systematic biases. And I like to hear opinions from people who don’t find it useful and prefer not to use them. I must admit though, that while I do get frustrated it’s not totally accurate, even if flawed, I find it very nice.

Granted, I did start it, so hopefully I think that :joy:

15 Likes

I’m not sure what error you’re talking about but referring to words per page wouldn’t work out since word count per page varies quite a bit.

If i’m looking up 2 or 3 words per 100 words, that means I should understand 97-98% of what I’m reading, if we can ignore writing style for a moment. So a very tolerable number in my opinion.

Sometime last year I remember coming across some research saying the 98%+ is the ideal for extensive reading.

I hope that made that sense

That’s why I only stay in that 97-98% range. I lack the perseverance that some people have on this forum :sweat_smile: (how do they do it?)

Found this as I was writing this post:

3 Likes

But on the other hand you voted above that you “hardly ever“ make look-ups.

Sorry, there is probably some misunderstanding on my side: you probably count 2-3% of the words as hardly ever, while for me, when I look up 2-3% of the words which are 12-15 words/page in a standard novel, that is identical to looking up every new word (in a mid level 35 novel). For myself I think that this is a hard thing to do, but for you - with a pop-up dict - this probably does not feel that hard.

2 Likes

This is kinda fun, but the made up words don’t follow language rules and thus, are actually harder to understand than if they did, imo.

“Of borgle”, which seems to replace “alone” would have been better if it was just “borgle”.
I have not the slightest idea what “Is shnooling no one. Sometimes you see don’t to shnool all day.” is supposed to be. The context should make it easy to understand, but this doesn’t.

And this is true for the other examples. So, unless you are reading badly written stuff or stuff with lots of mistakes, even 90% would still be enough to follow a story and 95% is basically full understanding. 90-95% is my usual range for understanding. I find 98% too easy (for learning - it’s nice for just mindless consumption).

98% is good for a beginner, I think, who is still building up a basic core vocabulary but the more you advance, the more those 2% would just be words you rarely need. (i.e. specialised vocab, etc.) Nice to have, but not necessary… whereas at 90-95% I get more flowery words, synonyms, etc. that are well worth encountering.

But maybe that’s just me. :sweat_smile:

4 Likes

That’s why I said at most 2 or 3 per 100, in reality I can understand the words from context or the kanji, so I’m not always looking up unknown words I come across while I read, I will usually save that for SRS most of the time. I love being absorbed in reading, so i will do my best to not break out of that.

And yes, you’re right, kindle makes it super easy for me to look up words, usually 5-10 seconds and then I’m straight back into reading.

It is subjective, though 2-3% feels right for "Hardly Ever"in my opinion.

3 Likes

I agree it is flawed, but I think it captures the feeling of the reading experience well. I could maybe tolerate 95% but I’d prefer to not to.

Like you said, its an individual thing, I don’t want to feel like l’m learning while I read stories, so I always go for 98%+.

6 Likes

I think generally accurate give or take 5 levels.

1 Like