[IMPORTANT UPDATE] Come help the grading system! Overhauled with Benchmark Books, Movies & TV Shows

It is not recalculating, no. I will try to put in more benchmarks tomorrow, but it’s a bit of work to figure it all out and I’m trying to develop processes to make this analysis quicker for future grading systems.

The grading system will be a bit chaotic this week, but hopefully by the end it will be better.

We just need a lot of benchmarks to get it started.

1 Like

For the 다락원 graded readers, will you put them as benchmarks?

I’d say:

  • 토끼전 is not TOPIK 1 but actually TOPIK 2, so somewhere between L8-10

  • 콩쥐팥쥐전 should be TOPIK 2-3

  • 흥부전 is high TOPIK 3

  • 전우치전 and 춘향전 are TOPIK 4, but easier than 또다시 같은 꿈을 꾸었어요 | L25 imo. Those two graded readers should be at the same level I think, 전우치전 has harder grammar and 춘향전 harder vocab, which kinda evens out in the end.

1 Like

Oh great, that will help a lot! I’ll get that worked in today.

1 Like

@HopeWaterfall so those 다락원 reader benchmarks made a massive difference! Just those alone triggered essentially the entire 두루책방 series to become ‘graded’ which led to most of those other lower level books becoming graded (as @bibliothecary has put in work grading there!)

Hopefully you will get a lot less silly grading prompts for books now that a lot more things are graded. I still am trying to put some more work into the videos today, along with maybe figuring out a few more good books benchmarks.

5 Likes

I’ve been going through my finished books and extra grading the ones that still had temporary ratings. I noticed that some I’ve already graded a lot were still temporary, for example https://learnnatively.com/book/cdb7c7a388/ :

Before grading - 202 gradings but still temporary rating

After grading - 212 gradings, non-temporary rating

3 Likes

I noticed a weird level for Korean Stories for Language Learners: Traditional Folktales in Korean and English (Free Online Audio) | L2. I wouldn’t be able to tell exactly what level it should be at, since I haven’t finished it and apparently the later stories are the most difficult, but it’s definitely not L2.

I thought that was weird until I had a look back at the first story - a dialogue of four lines between two characters saying hello, my name is…, and goodbye. L2 seems right.

The later stories might be about L20, but it’s kind of irrelevant since with learning materials it’s only the starting level that’s important.

Oh whoops! Yeah the benchmark was set incorrectly for that… i meant to set at lvl 12 but it was put at lvl zero! Recalculating now. Also updated the current list of benchmarks in the top message.

While I do agree that you could set at lvl 2 and that these graded readers are confusing to grade, I do think I wouldn’t recommend the graded reader to a lvl 2 person if the stories get that hard.

I think a lvl 12 rating is probably a better recommendation… but there’s no hard and fast rule here :thinking:

1 Like

@brandon , currently the further you go down theHarry Potter series , the lower the levels get.

Also, this entire bogus Yes24 series is set at the same level.

It seems a bit misleading to grade it as L12 if half the text is below that level, especially since it will put off those who would benefit from reading the early texts; the higher level essentially renders the first half of the book useless as a learning resource.

I think it would make more sense to set it at the starting level and explain in the description that unlike other graded readers that are set at one level, this is more akin to a textbook that gradually increases in difficulty and takes time to work through.

2 Likes

set those as benchmarks

I marked that as a variant series.

Unfortunately, there’s not a terribly good way to handle these sorts of books. I guess I could not put it as a ‘benchmark’, but even when I didn’t it came out to lvl 11, so not really different.

I think we’ll just have to accept whatever people grade it as. Reviews & Lists can somewhat help mitigate the issue you’re describing.

3 Likes

So yeah, this is a little wacky I know. The reason is that I still am not sure how to incorporate your ‘additional’ gradings. I essentially need to look at a book, see if it’s temporary, and if it is, convert additional gradings to become ‘standard’ gradings.

However, in general it seems like there were enough standard gradings so that it was unnecessary to add that process. In the normal course of affairs, this would never happen - those gradings would’ve never entered the ‘additional’ gradings state. This is only happening for Korean because we’re recalculating it after we’ve had it around for a while.

3 Likes

Korean books and their Japanese equivalent.

Korean Japanese
곰 곰 곰 베어 시리즈 (series) | L30?? くまクマ熊ベアー (series) | L23
키노의 여행 01권 시리즈 (series) | L30?? キノの旅 (series) | L29
약사의 혼잣말 1 | L30?? 薬屋のひとりごと (series) | L38
늑대아이 아메와 유키 | L30?? おおかみこどもの雨と雪 | L29
S노벨 플러스 S novel + (series) | L30?? ティアムーン帝国物語~断頭台から始まる、姫の転生逆転ストーリー~ (series) | L32
후궁의 까마귀 (series) | L35?? 後宮の烏 (series) | L35
청춘 돼지는 바니걸 선배의 꿈을 꾸지 않는다 | L30?? 『青春ブタ野郎』シリーズ (series) | L28
86 에이티식스 1 | L30?? 86―エイティシックス― (series) | L37
NHK에 어서 오세요 | L30?? NHKにようこそ! | L32
널 오타쿠로 만들어줄 테니까, 날 리얼충으로 만들어줘! 1 | L30?? おまえをオタクにしてやるから、俺をリア充にしてくれ! (series) | L26
미소년 탐정단 | L30?? 美少年 (series) | L35
FLESH & BLOOD 플래쉬 & 블러드 1 | L36?? FLESH & BLOOD (series) | L35
마리아님이 보고 계셔 01권 시리즈 (series) | L30?? マリア様がみてる (series) | L27
책벌레 공주 (series) | L30?? 虫かぶり姫 (series) | L32
S노벨 플러스 S novel + (series) | L30?? わたしの幸せな結婚 (series) | L30
이 멋진 세계에 축복을! 1 | L30?? この素晴らしい世界に祝福を! (series) | L30
J novel next (series) | L30?? 幻獣調査員 (series) | L33??
여성향 게임의 파멸 플래그밖에 없는 악역 영애로 환생해버렸다... (series) | L25?? 乙女ゲームの破滅フラグしかない悪役令嬢に転生してしまった... (series) | L26
수염을 깎다. 그리고 여고생을 줍다. 시리즈 (series) | L25?? ひげを剃る。そして女子高生を拾う。 (series) | L26
이세계 피크닉 1 | L30?? 裏世界ピクニック (series) | L32
밤의 괴물 | L30?? よるのばけもの | L29
십이국기 (series) | L35?? 十二国記シリーズ (series) | L32-37
하카타 돈코츠 라멘즈 | L30?? 博多豚骨ラーメンズ (series) | L30??
5 Likes

Dang bibliothecary. :open_mouth: What a huge list!

3 Likes

German: D.Gray-man 1 | L24??
Spanish: D.Gray-Man 1 | L24??

(Putting these here since bibliothecary already has a giant list.)

4 Likes

Just gonna split the table since the links aren’t rendering.

Korean Japanese
너의 췌장을 먹고 싶어 (일반판) | L30?? https://learnnatively.com/book/7ab5587407/
울고 싶은 나는 고양이 가면을 쓴다 | L30?? 泣きたい私は猫をかぶる | L25
얼음요괴 이야기 (series) | L24?? 氷の魔物の物語 1 | L25
우리들의 식탁 | L24?? https://learnnatively.com/book/596dc80c01/
코믹 책벌레의 하극상 시리즈 (series) | L24?? 【マンガ】本好きの下剋上 (series) | L25
니콜라의 느긋한 마계 기행 시리즈 (series) | L24?? ニコラのおゆるり魔界紀行 (series) | L21
비블리아 고서당 사건수첩 시리즈 (series) | L35?? ビブリア古書堂の事件手帖 (series) | L30
디 그레이 맨 시리즈 (series) | L24?? D.Gray-man (series) | L25
아저씨와 고양이 1 | L17 おじさまと猫 (series) | L18
소녀 동지여 적을 쏴라 | L30?? 同志少女よ、敵を撃て | L38
죽이기 시리즈 (series) | L30?? 〈メルヘン殺し〉シリーズ (series) | L29
나미야 잡화점의 기적 | L30?? ナミヤ雑貨店の奇蹟 | L31
마법의 시간여행 (series) | L15?? マジックツリーハウス (series) | L20
별의 커비 스타 얼라이즈 시리즈 (series) | L20?? 星のカービィ (series) | L21
마녀 배달부 키키 시리즈 (series) | L25?? 魔女の宅急便 (series) | L26
요괴의 아이를 키우고 있습니다 시리즈 (series) | L15?? 妖怪の子、育てます (series) | L26
청의 왕 : 탑의 소녀 | L22?? ナルマーン年代記 (series) | L29
카드캡터 사쿠라 | L30?? カードキャプターさくら | L21
실버 다이아몬드 (series) | L24?? SILVER DIAMOND (series) | L24??
얼음요괴 이야기 (series) | L24?? 氷の魔物の物語 1 | L25
종점 unknown 언노운 (series) | L24?? 終点unknown 1 | L24??
아저씨와 고양이와 소녀 (series) | L20?? おじさんと猫と少女 (series) | L24??
비의도적 연애담 시리즈 (series) | L28?? 非意図的恋愛談〜俺は恋愛なんか求めてない!~ | L24??
디앤씨웹툰비즈 (series) | L24?? 俺だけレベルアップな件 (series) | L27
[페어리] 형이 좋아! | L24?? アニツウ! | L24??
3 Likes

Hit the link limit again… :melting_face:

Korean Japanese
환상의 에덴 | L24?? まほろば絵伝【電子限定描き下ろし付き】 | L24??
원 위크 패밀리 One Week Family | L24?? ワンウィークファミリー | L24??
쇼타 형아 1 | L24?? ショタおに (series) | L22
카니발 시리즈 (series) | L24?? カーニヴァル (series) | L24??
니콜라의 느긋한 마계 기행 시리즈 (series) | L24?? ニコラのおゆるり魔界紀行 (series) | L21
바로크 (series) | L24?? BAROQUE~バロック~ 1 | L24??
뉴 루비코믹스 (series) | L24?? 坂の上の魔法使い (series) | L24??
[고화질] 러브리스 (LOVELESS) 시리즈 (series) | L24?? LOVELESS (series) | L24??
마왕님과 슈운 시리즈 (series) | L24?? 魔王様としゅんくん 1 | L24??
클레이와 핀과 꿈꾸는 편지 1 | L30?? https://learnnatively.com/book/3e6aa809e1/
NT-novel 엔티 노벨 (series) | L30?? バチカン奇跡調査官 黒の学院 | L30??
노블엔진 팝(NE pop) (series) | L30?? ペットショップ夢幻楼の事件帳 思い出はいつもとなりに | L30??
빈 점포 있습니다 | L30?? 空き店舗 幽霊つき あります | L30??
파트너는 초등학생 | L30?? 相棒は小学生 図書館の少女は新米刑事と謎を解く | L30??
악역 영애입니다만 공략 대상의 상태가 이상합니다 1 | L30?? 悪役令嬢ですが攻略対象の様子が異常すぎる (series) | L30??
수속성의 마법사 1 | L30?? 水属性の魔法使い 第一部 中央諸国編I | L35??
만국과자점 마음 가는 대로 : 머나먼 약속과 꿀색 달의 웨딩케이크 | L30?? 万国菓子舗 お気に召すまま (series) | L30??
비극의 원흉이 되는 최강악역 최종보스 여왕은 국민을 위해 헌신합니다 1 | L30?? 悲劇の元凶となる最強外道ラスボス女王は民の為に尽くします。 (series) | L30??
슬레이어즈 1 | L30?? スレイヤーズ (series) | L37
영매탐정 조즈카 | L30?? https://learnnatively.com/book/eb3732b766/
앨버트가의 영애는 몰락을 소망합니다 | L30?? アルバート家の令嬢は没落をご所望です (series) | L30??
나「」만「」의「」비「」밀「 (일반판) | L30?? か「」く「」し「」ご「」と「 | L25??
무기모토 산포는 오늘이 좋아 | L30?? 麦本三歩の好きなもの (series) | L29
어리고 아리고 여려서 | L30?? 青くて痛くて脆い | L31
배를 가르면 피가 나올 뿐이야 | L30?? 腹を割ったら血が出るだけさ | L30??
3 Likes

Last one, I hope! :sweat_smile:

Korean Japanese
[대여] 오늘 밤, 세계에서 이 사랑이 사라진다 해도 | L28 https://learnnatively.com/series/ee3e64730d/
바다가 들리는 편의점 | L30?? コンビニ兄弟 (series) | L30??
편의점 인간 | L27?? https://learnnatively.com/book/2f20d58780/
네가 마지막으로 남긴 노래 | L25 君が最後に遺した歌 | L30??
수상한 목욕탕 | L30?? 嵐の湯へようこそ! | L30??
녹슨 도르래 | L30?? 錆びた滑車 (文春文庫 わ 10-5) | L30??
비블리아 고서당 사건수첩 시리즈 (series) | L35?? ビブリア古書堂の事件手帖 (series) | L24
세상의 마지막 기차역 | L35?? 西由比ヶ浜駅の神様 | L30??
슬로모션 | L30?? スローモーション 偕成社コレクション | L30??
아름다운 흉기 | L30?? 美しき凶器 | L30??
모모 | L30?? モモ | L30??
쓰쿠모 서점 지하에는 비밀의 바가 있다 | L30?? 九十九書店の地下には秘密のバーがある (ハルキ文庫 お 20-1) | L30??
기억 파단자 | L30 殺人鬼にまつわる備忘録 | L30??
기묘한 괴담하우스 | L35??
이상한 그림 | L32 変な絵 | L28
오늘 밤, 거짓말의 세계에서 잊을 수 없는 사랑을 | L28?? 嘘の世界で、忘れられない恋をした | L30??
고백 | L28?? https://learnnatively.com/book/39de0a8b67/
대망 (series) | L30?? 徳川家康 1 出世乱離の巻 | L30??
공허한 십자가 (개정판) | L30?? 虚ろな十字架 (光文社文庫 ひ 6-14) | L30??
13번째 인격 | L35?? 十三番目の人格 ペルソナ ―ISOLA | L32??
절대정의 | L36?? 絶対正義 | L30??
해피엔딩에서 너를 기다릴게 | L30?? 太陽のシズク ~大好きな君との最低で最高の12ヶ月~ | L30??
수상한 이웃집 시노다 시리즈 (series) | L22?? シノダ!チビ竜と魔法の実 | L22??
3 Likes

You are amazing, thank you !! :laughing:

I’m not if I will make all of these benchmarks, but it’s an extremely good reference.

(FYI for some reason the japanese links aren’t showing up on the last table :scream:)

Edit: Nevermind

2 Likes

I’m still working on them! :laughing:
These are books with Japanese authors, so I’m still finding the relevant books.

2 Likes