Improve library page speed when "Collapse series" is "yes"

Description of your request or bug report:
[FROM: BRANDON]
We will be focusing on speeding up the server response when ‘collapse series’ is ‘yes’. It is surprisingly slow in some cases imo. We will not be addressing the mobile sluggishness which @seanblue addressed in his comments as we cannot personally reproduce.

[FROM: SEANBLUE]
I have my library page bookmarked on both desktop and on mobile. On desktop, clicking the bookmark loads my library relatively quickly. However, on mobile after the page frame loads and the spinning is shown, it takes an additional 5+ seconds to load the content of my library. It’s possible this is simply the difference between the performance of my laptop vs. my phone, I don’t know.

Additionally, the performance is significantly worse when “Collapse series” is set to “yes”.

Desktop “no” collapse: 200-250ms
Desktop “yes” collapse: 700-1000ms

I don’t know how to get to the performance times on my phone. However, it seems to take 1-2 seconds when collapse is “no” and about 5 seconds when collapse is “yes”.

Trello link:

Hmm, those mobile times are pretty concerning and a bit confusing.

‘collapse’ should absolutely take a little longer, but its performance has been a head-scratcher for me. It’s a built-in elastic search function (elastic search is the search software I use) and I do see people complain a bit about its performance, but my suspicion has been it’s more sluggish than it should be.

Regardless though, the only difference between mobile and desktop should be network speed. Since the size of the content for your library is only around 100kB, there’s not a whole lot I can do. I can only control server processing time and content download size… the latter of which may be a differentiator between desktop & mobile times, but its not an issue here.

FWIW I tried your library on 4G data and I don’t see an issue. I’d encourage you to try again and see if you experience the same thing. Perhaps there was a temporary slowdown on the search server too, i don’t know :man_shrugging:

Definitely appreciate the feedback on site speed, very helpful. Please continue to monitor.

STATUS: PENDING FOR MORE FEEDBACK

1 Like

It’s pretty consistently slow. I noticed it a couple weeks ago and it happens every time I load the page from scratch. By the way, I’m on WiFi, so I don’t think 4G comes into it.

Maybe we should ignore the mobile aspect and make this ticket exclusively about improving performance of “collapse” = “yes”.

1 Like

Well that’s odd and concerning then :confused:

FWIW I don’t have any issues over wifi, so I’m having hard time reproducing. I’m not sure what could be the issue there… mobile & desktop should be quite close in speed over wifi, I think.

Sounds good. I’ll edit the description then. Approved.

1 Like

Any plans to work on this in the near future?

1 Like

Potentially deserves a look after the audiovisual update.

I will say, I probably should update my elasticsearch anyway and maybe that will help this issue. I’ve noted some optimizations they’ve done in this area.

I know it can get pretty slow right now, ugh. I do have to look at it.

1 Like

Ohh, you’re using elastic search for this? I’ve only used it for aggregating log data, not as a primary data source.

As someone who primarily uses SQL, I would have just loaded the data ungrouped and naively just done a GroupBy (C# Linq lingo) and called it a day. :joy: I have no idea why the query you’re using is so slow, but doing a grouping in-code on a few hundred entries should take virtually no time at all.


This is a bit of a tangent, but in the long run might it make sense to do all filtering, sorting, and grouping on the front end? That way you only have to query the back end for data on initial page load and the page updates near-instantaneously for the user.

2 Likes

Yeah, it’s easiest to just throw up a search index and allow queries to it as the data needed is spread across a few tables. It’s much faster and less complicated than constructing the equivalent db queries.

Right, I have no idea why the collapse is taking so long.

Regardless, you are right that I could simply do the grouping afterwards for the user libraries as the number of items in any given user library is quite small. I won’t be able to do that for the book / video search because there are too many items, so we will still have to improve the collapse functionality.

But yeah, I should just preload and filter on the front end. Maybe we should just change this ticket to that. Then it would be lightening fast outside of text search :slight_smile:

I think I mostly did it this way because it was closest to how I did the global video / book search, but there’s no real reason for that.

2 Likes

Yeah. Upon further review, I think i’'m going to simply change this ticket to ‘Improve library speed’ and move all the filtering and grouping to the front end.

I think it should be relatively high priority, as the speed does seem slow!

1 Like

I think a quick fix (if you consider server-side non-elasticsearch grouping quick) is still worth doing independently. Presumably changing the page to do all the filtering, etc. on the front end would be a lot more work.

Ultimately up to you of course! If it’s truly made high priority then it should be fine.

1 Like

You’re right, a two step release is probably warranted. :slight_smile:

1 Like

I’m claiming this as completed. Let me know if you still have issues!

2 Likes

Will be closing topic to release votes if not issues or follow-ups are reported in the next 24h.