Today the Jabber/XMPP services were offline for some hours because of an attack on our server. Sorry for the inconvenience.
Wow, the migration of the offline storage from file to database did take very long. I thought one or two hours are enough. No, the scripts ran for 10 hours. Therefore the Jabber/XMPP services were offline for about 10 hours, from ~10:30 pm CEST yesterday to 8:50 am CEST today. We are very sorry about that. Good news: No more big downtimes are planned in the near future.
We need to convert the Prosody offline storage to the new database format. Therefore we need to take the server down. This will happen at between 10 and 11 pm CEST today.
Everything is done, the server runs on Prosody 0.10 and is up for over an hour now. Thanks to MattJ for his help solving the small issues when upgrading.
No new features like MAM and WebSocket are activated, yet. This will follow. But I had to disable mod_privacy_lists and mod_blocking and enable mod_blocklist. This means that we drop support for XEP-0016 which only older clients used nowadays as XEP-0191 is the newer standard which we still support with mod_blocklist.
Prosody 0.10 was finally released yesterday. So today at 8:30 pm CEST (~3 hours from now) we plan to upgrade our Prosody server from 0.9 to 0.10. The XMPP/Jabber services will be offline for about 1 to 2 hours. At least this is our plan. ;-) Please stay tuned!
This will affect Jabber itself and all transport services, and also this website. And it will take place during the next 24 to 48 hours. Please be patient and expect some service outages. We changed the TTL of our DNS zones to 1 minute but nevertheless it could be that your client still looks for the old server. You might need to relogin to the Internet and/or restart your client.
We will be back here when everything is moved. Until then you also can reach us on Twitter, where a link to this post will also be posted.
The Spectrum transport services are down at the moment. It will take one to three days until these services are up again. Mostly because we are about to move all Jabber related services to a new machine. So please stay tuned.
There were some outages in the past weeks.
A big outage was from yesterday to today, almost one day. Coming back just two hours ago. But everything is fine again. We had some server hardware problems which are solved now. No data loss and no need to restore our backups.
The other outages are much harder to detect. There is a annoying bug in Prosody at the moment which causes database connections to fail at some point. Then a restart is needed. We need the information then (because most of the time it only affects users who want to login, not already logged in users) to restart the server. Another bug which is unknown at the moment is that connections to jabber.ccc.de are failing from time to time. A restart solves it – or causes it. You choose… ;-) But these problems are Prosody related (the first for sure, the second maybe) and we work on it.
So we need your help. If you cannot login, tell us. Here or Twitter, does not matter. If you see your buddies from other servers offline at once, tell us. At the moment we just know about this jabber.ccc.de problem which happened two times.
PS: We are not thinking about cancelling our Jabber services here, do not worry! Some people were afraid.