Come help the grading system! Request for Benchmark Books, Movies & TV Shows

German
21

Thanks for restarting the discussion!

@Brandon has said it would be helpful to find some more benchmark books, so let’s do that :slight_smile: Also - perhaps he could chime in with suggestions for high impact books to either benchmark or get reading :wink:

So far the Natively levels are set like this for German:

  • L0-5 ( ~A0 )
  • L6-12 ( ~A1 )
  • L13-19 ( ~A2 )
  • L20-26 ( ~B1 )
  • L27-33 ( ~B2 )
  • L34-40 ( ~C1 )
  • L41+ ( ~C2 )

A1-A2 benchmarks

In Japanese graded readers are generally fixed, and I think your proposal to do that sounds really helpful.

What do you think about those Hueber A1 books, are they relatively uniformly difficult or does it vary a lot? I’m wondering since you read them if you think we should we set the level at L6 (bottom of A1), L12 (top of A1), or just set, for example, one and then grade the others in relation to that one?

Kind of a similar question here - could you propose a couple for setting the level? That would get us some good benchmarks at the A1 and A2 levels!

from B1+

For the B1 level and up, I have made some suggestions below and would appreciate any comments on setting more benchmark books:

So far (already graded books, mostly for my own reference)

Here are books I have read that already have known gradings/levels:

I think the levels of those look good.

Just about everything else I’ve read doesn’t have a fixed grading yet, though. I wonder if anything in the follow list might be useful to set as benchmarks?

I kept track of the relative difficulty of these as I read them so at least for me personally, I’m fairly confident about at least their relative difficulty for me:

  • 42 (C2) Ä Kolumnen This will probably never be a high impact book and I don’t really recommend it. I guess fixing it would only be helpful in that anything else I read near the top end could be graded against it. I’m proposing a C2 grading as I have a very firm German knowledge, yet would have missed so much in this if it hadn’t been for reading it with my native SO. This book requires lived experience in Germany (90s) and near native understanding of language nuance to “get” it.
  • 35 (lower C1) Die Straße nach Isfahan. Again, this will never be a high impact book and tbh I thought it was terrible. I think perhaps if fixing just the above book is sufficient for getting my books a known grading, then we could leave this one out. It was flowery with period and niche vocab, which added to the difficulty, which is why I think C1 is probably more appropriate than B2. But if it helped as a benchmark for getting other books I read at the top level graded, then perhaps it’s worth considering using as a benchmark.
  • 28 (lower B2) Does lower B2 sound like about the right level for a “normal” novel level? In this category I would put: Die Ladenhüterin, Medusa, Handbuch Ödland, and Das Jesus Video. Any of these have the possibility of being popular German novels on here, so perhaps fixing one could be helpful.
  • 26 (upper B1) Does upper B1 sound like the right level for an easy novel? In this category I would put: Alles außer irdisch, Anderwelt, Der Pfau, Tintenherz. Any of these have the possibility of being popular German novels on here, so perhaps fixing one could be helpful.
  • 24 (lower B1) Verborgenes Japan. This is slightly superficially written and fairly breezy. I’m not as sure where to place it since my level is much higher than this, but it feels easier to me than the 26s, and it’s harder than Der Alchimist which is currently 22. But, it won’t be a high impact book, so it’s not necessarily one I would recommend as a benchmark if we don’t need it.

So in summary - my top recommendations for benchmarking would be choosing one or more of the L26 and L28 novels I’ve read, such as Tintenherz and Der Jesus Video, as they have already been read by other people. And then it may well be helpful to set something on the high end like the L42 I proposed. Any thoughts?

3 Likes
22

The Hueber Lese-Novelas | L17?? are all at the same level: they’re all written by one author, they have a very similar narrative structure, and similar language use. I’d be tempted to put it at A6 as it’s written very plainly, with short sentences and chapters, and I only had to look up a handful of words (having never studied the language).

I’ve only read Lesen und Uben: Frankenstein + CD (Lesen und üben) | L17?? from the series, but it was definitly more difficult than the Hueber, so I’d put this at the high end of A1. I’d imagine these would be of varying difficulty given that they’re adaptions of novels and presumably have different writers adapting them. I may try a few more of these and report back my findings. :smiley:

3 Likes
23

@bibliothecary is always up for the challenge! :smile:

You mean A1 (not A6)? Like level 8?

I will compile the high value target list for potential benchmarks that would have the most impact this week.

3 Likes
24

Sorry, I meant L6 (since A1 is L6-12)! :confounded_face: Or just the lower end of that range since it’s easy for English-speakers to understand without (many) lookups.

:nerd_face:

3 Likes
25

Ok! I actually found a pretty awesome utility I built for finding potential benchmarks. Here are the results for books :smile:

Maybe take a look @mitrac & @bibliothecary & everyone if you have a chance. I will too, just a few things to get to!

Items between benchmarks

Items between benchmarks

Items that have been graded both harder and easier than current benchmarks (or equal to one)—good candidates to fill level gaps.

Items with benchmark gradings

Items with benchmark gradings

Non-benchmark items that have been compared to at least one current benchmark.

Other highly graded items

Other highly graded items

Most-graded non-benchmark items (may not yet have benchmark comparisons).

3 Likes
26

That is very useful, thanks! I’ll have a look :smiling_face:

2 Likes
27

Ok, putting the little prince and the alchemist on my reading list.

For watching, I am currently working my way through anime (post Nicos Weg) hitting Avatar, princess Mononoko, and soon Bloom into you which I think already has a rating. Is that helpful?

3 Likes
28

We have very few german video gradings, so anything is super valuable! We also don’t have any benchmarks for video. Once we get a 1000 - 2000 video gradings it starts to make a lot more sense to do. We only have 200 right now! :sweat_smile:

3 Likes
29

Oh gosh then probably half are from me! I will try to be more thoughtful with my video ratings rather than just vibes then.

On another note, I think I requested Sailor moon manga to German books. Could that be added as I am about to finish the second one and move onto other books so it is a useful comparison for me.

3 Likes
30

Ah that book hit a perplexing, somewhat rare Amazon error that i’ve been meaning to get to. This is the encouragement i needed. Will fix!

Edit:

Anything is appreciated! But yes, please bring along your best German-learning friends to grade stuff too! :sweat_smile:

3 Likes
31

I’ve just finished a couple of graded readers:

I’ll try to make my way through some of the other ones mentioned as potential benchmarks. Looks like it’s gonna be a German-heavy week! :smiley:

3 Likes
32

Here is a list of as many of those @brandon as I could get through, and also making it consistent with @bibliothecary’s experience.

This is the suggestion for fixing potential high impact books / useful anchors so far:

For the first 3, @bibliothecary, what do you think about this scaffolding:
L0-L5 single sentence drills
L10 earliest A1 narrative reader
L18 harder end of graded reader

That is why I suggested to put the Hueber Lese Novelas starting from L10 but if you think we should knock them back and allow more space between these let me know

  • L10 Anna, Berlin / Hueber Lese-Novelas. The whole series suggests it is A1 so we could fix the whole thing, or just fix volume 1 and wait for bibliothecary to make a judgment on that
  • L12 Learn German With Stories: Café in Berlin - 10 Short Stories For Beginners. Based on being slightly harder than the previous book by bibliothecary’s assessment and 2 other reviewers
    • @brandon can you check this one - if I’m not going crazy 7 people have rated it easier than the same exact book
  • L16 Short Stories for beginners - upper A2 graded reader that has gradings harder than Cafe in Berlin and easier than the Mord am Morgen
    • It’s currently at L19 so my proposal is fix it lower because that is way too close to Der Alchimist L22
    • The so-called intermediate book in the series is very unlikely to be L26, so I would suggest unfixing that one and allowing it to be graded.
  • L18 harder end of graded readers: Mord Am Morgen. Marked consistently as harder than the one before, and L18 makes sense to me as an upper level graded reader in the way they are marketing it.
  • L22 (B1) Der Alchimist - already fixed
  • L24 Die Kleine Hexe
  • L26 Percy Jackson 1: Diebe im Olymp
  • L26 TIntenherz
  • L26 Tschick
  • L26 Die Ladenhüterin
  • L27 Harry Potter (may have to move it) - coming out consistently in gradings and in relative books as easier than Momo, or move Momo and upper range up a bit more.
  • L28 Momo - I propose moving Momo up to 28. I started reading it and am now halfway through
  • L30 Das Jesus Video - I’ve revised my rating and have pushed it up to account for additional academic and scientific vocabulary
  • Die Verwandlung - currently L31, I’m really not sure where this should land. If it is not currently fixed I would let it float.
  • I need to read some more books in a targeted way at the top end before I form any strong opinions on that.
3 Likes
33

This seems good based on the series mentioned, but I don’t think I’d make them rules, as I’m sure there are graded readers that would fall outside the L10-18 range. :slightly_smiling_face:

2 Likes
34

Definitely agree not as rules, thanks for pointing that out. I was trying to figure out a kind of rational way to choose some numbers for those easiest books that felt like it also made sense for the way they lead into the later ones. But since you’ve actually read them I wanted to see if it actually felt right!

2 Likes
35

@mitrac thank you so much for going through all of that! :star_struck:

Strange Gradings

It seems like we originally had that Café in Berlin series graded by volume but switched later to grading the locked for the whole series… so the UI is bucketing all the 7 gradings to other volumes together.

It probably makes sense to grade that one by volume?

I must admit though, since I’m quite busy right now with a lot of other data issues, can we streamline this a little for me? @mitrac maybe edit the top post here with proposed benchmarks for me to add once you and @bibliothecary (and others) have agreed on benchmarks. If you have a short reasoning note or a link to a discourse comment for why it’s a benchmark, leave that too (i have a notes field for Benchmarks for later admin reference).

Once you’re ready to go, just ping me and I can add any benchmarks. We can also remove or alter benchmarks and recalculate the whole entire grading system easily.. so no worries on messing things up! Also, if you need me to run my potential benchmark script for movies, tv shows, books, happy to do that too!

3 Likes
36

FIxed that bug, you should be all good to go! :smile:

4 Likes
37

Thank you!

1 Like
38

sounds good, I’ll do a bit more comparing and wait for more comments, and then tag you when I update the list!

3 Likes
39

What do we think everyone, any more thoughts on those benchmarks? :upside_down_face:

2 Likes
40

Since we haven’t attracted any more comments, I’m tempted to just put them in place. How about I update the top post as you suggested and ping you when I’ve done that.

Next up I’ll revive the German book club :sun_with_face:

4 Likes