• 0 Posts
  • 13 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle



  • It’s not the tech here. Postgres can scale both vertically and horizontally (yes there are others that can scale easier or in different factors of CAP).

    The problem is how the data is being stored and accessed. Lemmy is doing some really inefficient data access and it’s causing bottlenecks under load.

    Lemmy (unfortunately) just wasn’t ready for this level of primetime yet… It has a number of issues that are going to be quite tricky to fix now that it’s seen such wide adoption (database migrations are tricky on their own, doing so on a production site even harder, doing so on 8k+ independent production sites… Sounds like a nightmare)


  • Idk. I’m conflicted on this. While I agree… For bigger/broader topics, I can definitely see that the quality of discussion and the repetition of topic is already really bad for smaller/niche communities.

    When the subreddit had maybe 2k subs worldwide, and now is comprised of 50 subs spread across 3 instances… It’s rough. That community is just dead and that sucks.

    I guess I’d rather have one centralized community on one big (yet open source) instance where I know we can leave and move again, than have the community just die entirely


  • It’s not that they’re for this specifically… It’s that they are self centered. They’re the same 75% of the population that is willing to cross the picket line at Starbucks cuz they want their coffee. They don’t think about the workers rights, they only care about coffee.

    The same people just want memes and football and porn. They don’t care about what’s behind the scenes unless it directly impacts them. And let’s be honest, the reddit changes (for now) impact like 10% of reddits user base. That’s not enough for them to give up some dumb memes for


  • It’s just amazing really. Had they rolled out a reasonable rated plan, and maybe even a discount to highly known apps, and even set a “the price will go up each 6 months for the next 2 years till we reach this higher, but still reasonable price”… All the apps would have added a subscription model for like $0.99 a month and none of us would have really complained.

    Like, honestly most of us would have paid and maybe grumbled slightly but said “that’s the cost of maintaining this huge community, and I get more than $0.99 in value from it” and just kept shit posting on Reddit all day.

    If they wanted to block AI models, limit the API keys to only well known apps or those that are manually verified to be not-an-AI by reddit admins.

    It’s amazing how dumb a corporation can be sometimes (or has some nefarious endgame ala Twitter)



  • Yeah, apparently I was wrong about this (still learning lemmy and fediverse stuff…). Text content of posts and comments are “synced” to your server and stored in your database there. Then future requests for that content are served from your instance. So its not as bad as I thought it was (the network load should be lower since you aren’t acting as entirely a proxy, more like a cache), but database bloat will be a huge problem (its already a big problem in other federated things like mastodon and matrix, where every server ends up saving everything they want into their own database).

    I’m not sure what happens when the original server goes down, does the federated servers discard that data? Or do we each maintain a forever copy until we want to get rid of it ourselves? There’s also some notes I’ve seen about how servers only incrementally cache federated content (only posts and comments that are viewed by someone are fetched, and new comments may not be fetched until someone wants to see it)… so not everybody has a “pure and full” copy of posts necessarily.

    But overall I wonder how all the various sysadmins hosting these lemmy instances will deal with the expotential growth they’re going to see, or if smaller instances will start defederating to save on hardware costs (no reason for my tiny instance that only talks about blue shiny rocks to federate with lemmy.world and store all that content)


  • Yes. If you run the server, then you are the source of truth of that community. All other servers that federate your community query your server to access the community and show it to their users.

    So if you run a server and a community explodes there, you might only have 500 users on your instance, but you might have 50k users reading that community and interacting with it from other Lemmy instances, thus your server needs to scale to 50k users worth.

    And ever more essential, your server is the source of truth of that community. So if your server is hacked or corrupted or deleted, that community is gone. Other instances don’t mirror it (except for temporary caching), so the Lemmy network essentially is a trust network of other people maintaining servers long term (and each inventing a monetary system to pay for it). I still think the network might be better than a centralized system like reddit, but it definitely has a lot of growing and policies that need to be sorted out very soon


  • A much better use of resources, but you shard the data amongst potentially untrusted hosts (ie, anybody can stand up a lemmy instance and start hosting posts/comments, and then get sick of hosting it and delete their instance and all the uploaded data).

    Federation only allows access to the network of servers, it doesn’t protect the data at all, which means at any moment an entire community of useful historical information could just be wiped away (especially since there’s currently no monetary incentive to continue hosting, its only being done out of desire to be part of the network).

    I guess I’d rather see the blockchain (or simpler caching/mirroring) approach, something like the torrent network, so that no single person has access to delete content. We can all choose to not view or not mirror content we don’t agree with, but nobody can single-handedly own or modify the data


  • I think they’re saying that they like using the local view as a /r/all type replacement (a view of the highest voted content across all communities). But they’d like to be able to selectively add an entire other instance to their local view.

    The only way to do this today is to subscribe to every community on your main instance and every community on the other instance, and use Subscribed as a mega view… But then that ruins the Subscribed view as a selected subset.

    To put it another way, I want to view all the best content across N many instances at once, so I can discover new communities.