Server outage

On the evening of the 23rd of august I wanted to migrate this server to Ubuntu 24.04, as it was still on Ubuntu 20.04. The offline backup worked great, but the restore not so much. I wasnโ€™t able to get the system to boot again afterwards.

At 2am, I just gave up and went to bed. Next day at 1pm, after a fresh reinstall and about three hours of work, Iโ€™ve got the website back up, and the public quake servers are running again. But there is still a lot of work ahead restoring other systems.

For instance, some subdomains (quake.froschgrosch.de, df.quakenewbies.com) have not yet been restored. As annoying as the situation is, it is a good opportunity for me to restructure. Iโ€™m thinking about a subdomain like files.froschgrosch.de, where all files used by various services are hosted. Also, the TLS certificates that I had automated with certbot โ€“ while convenient โ€“ were kind of difficult from a security standpoint. If the DNS API token were to be compromised, bad actors could have messed up the entire froschgrosch.de domain.

In case something is not working, give me a few days and Iโ€™ll hopefully have fixed it.