We’ve upgraded lemmy.world to 0.18.1-rc.1 and rolled back that upgrade because of issues.
(If you had posted anything in those 10 minutes between upgrade and rollback, that post is gone. Sorry!)
The main issue we saw is that users can’t login anymore. Existing sessions still worked, but new logins failed (from macos, ios and android. From linux and windows it worked)
Also new account creation didn’t work.
I’ll create an issue for the devs and retry once it’s fixed.
Edit Contacted the devs, they tell me to try again with lemmy-ui at version 0.18.0. Will try again, brace for some downtime!
Edit 2 So we upgraded again, and it seemed to work nicely! But then it slowed down so much it was unuseable. There were many locks in the database. People reported many JSON errors. Sorry, we won’t be on 0.18.1 any time soon I’m afraid…
Thanks for all your hard work @ruud@lemmy.world. It’s not easy!
Thank you so much for all the hard work here, I know how frustrating attempting an upgrade like this can be.
Both Dessalines and Nutomic have been working their butts off to get 0.18.x ready for the Reddit API changes. Huge hopes they can pull through!
Dessalines:
Nutomic:
As any engineer who does ops can tell you - you did the right thing - the solution is always to roll back, never force a roll forward, ever.
We should totally do pre and post update parties though. Even if the update fails we can have an excuse for drinks and a fun thread.
Thank you for the transparancy!
I love post like this, another reason of why this instance is so good
Sucks that lemmy.world won’t be working with Jerboa when the reddit apps quit working in a couple days.
On the plus side, I’m about to open the beta that handles both just fine (plus a lot of features no one else is doing yet). The deadline I for myself is Saturday when the Reddit clients go down.
Unfortunately it was built by someone who lurks and comments, and I realized even though merging feeds from accounts on different servers is going to be awesome, other people like to post and I need to stop getting off track
The stress of the timeline is starting to get to me and I’m terrified no one is going to give it a chance, but I’ve fixed most of the things that drove me nuts in jerboa… That should count for something, right?
Hol up, are you soft announcing the development of a new Jerboa-based app? What’s it called? Good luck to you! I hope you achieve your goals with the project and that is well received. ☺️
Hey, so it’s not jerboa based and I ended up pushing it back a few weeks, but I’m now looking for beta testers on Android if you’re interested
As for the name, I think I’m going with Luna for several reasons, a big one being that someone suggested it and Flemmy was the only name I could think of beforehand
I’m posting updates to !flemmy@lemmy.world for now, if you’re interested I’ll be posting instructions for the beta soon.
Whether you’re interested or not I’d love to get feedback. I’m good with data and UX flow, but I mostly just comment - I need to learn how other people want to use Lemmy
Sucks that you couldnt get it working yet, but there are bound to be problems at this scale.
Anyone have any resources for learning what running a lemmy server is like ? Seems pretty interesting
If you’re running a small server it’s pretty straightforward and hands-off. It’s only when you get to the scale of these larger instances that you run into issues. I just rerun ansible every time there’s an update and it otherwise just manages itself.
was there a test environment? why rc in production?
We are the test environment. You get what you pay for.
@ruud@lemmy.world just wondering if you have considered setting up a second, beta, instance of lemmy.world open to the public?
With all the performance issues with 0.18.1, it’s highlighted that there needs to be a way to stress test these updates before applying them to the main instance.
Yes, considering that. But we’ll need people to use that when we will do testing…
Well, you have at least one person who would be willing to use it ;D
One more. You find a stable way to notify anout upgrades and get a test sheet to run through and we can generate posts and activity to help test with.
Light the beacons! lemmy.world calls for aid!
