There were some outages in the past weeks.
A big outage was from yesterday to today, almost one day. Coming back just two hours ago. But everything is fine again. We had some server hardware problems which are solved now. No data loss and no need to restore our backups.
The other outages are much harder to detect. There is a annoying bug in Prosody at the moment which causes database connections to fail at some point. Then a restart is needed. We need the information then (because most of the time it only affects users who want to login, not already logged in users) to restart the server. Another bug which is unknown at the moment is that connections to jabber.ccc.de are failing from time to time. A restart solves it – or causes it. You choose… 😉 But these problems are Prosody related (the first for sure, the second maybe) and we work on it.
So we need your help. If you cannot login, tell us. Here or Twitter, does not matter. If you see your buddies from other servers offline at once, tell us. At the moment we just know about this jabber.ccc.de problem which happened two times.
PS: We are not thinking about cancelling our Jabber services here, do not worry! Some people were afraid.
We updated our old expired certificates a while ago. And now I just updated the website. You can find the new fingerprints here.
Also we had some problems with certificates, especially on this website. Sometimes the webserver delivered the old and expired certificate. This was because some old tasks of Apache were still in the memory. Problem solved three days ago.
Because there are problems/bugs with ejabberd 13.x connecting to our transports we just replaced the certificate for our main virtual host jabber.hot-chilli.net. This host also does service our transports.
Until then we had used a certificate with the common name *.hot-chilli.net, now we have one with jabber.hot-chilli.net and *.jabber.hot-chilli.net common names which does not lead to the problem.
Some days ago we already replaced the *.hot-chilli.net due to the OpenSSL Heartbleed bug.
The captchas of our registration forms (account and transport pre-registration) work again.
There was a session management problem which now is solved. Sorry for the time it did not work.
The web forms for changing passwords and removing accounts are not available, yet. Also there will be forms to reset passwords by sending out a mail. Therefore there will also be a web form to bind mail addresses to Jabber accounts. Please stay tuned.
Ok, the main virtual host jabber.hot-chilli.net and all private virtual hosts moved to the new server today. Also the website and the transport data.
“Transport data” because the transports itself do not run, yet. We need some help from Spectrum maintainers – maybe there is something basicly wrong with the new config files or Spectrum 2 is still too buggy. We will see. Until then please stay tuned and be patient – we do not want to have the transports offline for a long time, but it can take some days.
Also Prosody is having some issues on which we are working hard, especially the maintainer of Prosody, MattJ. Our server is having a lot of load, also “crazy” users 😉 and a Debian system which seems to create more problems with Prosody than other Linux distributions. So if the Jabber server itself is offline from time to time in the next days, we are working on it.
Our jabber server certificate is currently expired.
Unfortunately this expiration date was not correctly set in our schedule and hits us now in a time where Roi is in his well-deserved vacation.
He will update the certificate next week once he is back.
Sorry for all the inconvenience.
At the moment we don’t have any idea why. As I am in Italy right now I will investigate this after my return home in the middle of the week. Please stay tuned.
Is anybody aware of any current exploits for ejabberd?
We experienced problems with ICQ transport the last days. According to the developers of Spectrum this is a “glib poll” issue. epoll unfortunally does not work with glib (yet).
Therefore Spectrum ICQ transport now uses libev3 and epoll.
It looks good at the moment, I hope it stays that way. We will see what happens during the next days as our server is the only server which runs this configuration. Not even the Spectrum developers were able to test it, yet. 😉