Categories: DowntimesProblems

Server flatlined four times in not even two hours…

Due to until now unknown reasons the server which holds the Jabber services crashed four times in the last not even two hours. From one second to the other the ejabberd processes took every resource they could get, and even more. 8 gigs of RAM and 8 gigs of swap, everything gone. Plus a lot of CPU load. The machine was loaded that “top” refreshed just every 5 minutes and in the end just a hardware reset helped to reboot the machine.

For the tech geeks:

top – 19:56:21 up 31 min,  1 user,  load average: 22.86, 13.11, 8.71
Tasks: 240 total,   3 running, 231 sleeping,   0 stopped,   6 zombie
Cpu(s):  1.4%us,  5.8%sy,  0.0%ni, 12.4%id, 80.3%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:   8190900k total,  8138972k used,    51928k free,      796k buffers
Swap:  8393848k total,  7276916k used,  1116932k free,    42404k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
3239 ejabberd  20   0 15.7g 6.0g 460 S   23 76.7   3:06.65 beam.smp

We are looking into this issue. Maybe a severe bug with ejabberd, maybe a DoS attack. We don’t know, yet.

Roi

Recent Posts

XMPP Housekeeping now active for openim Domains!

We just activated XMPP Housekeeping for the openim.de and openim.eu domains. These two domains joined…

3 months ago

openim.de

Yesterday we took over the domain openim.de and the XMPP/Jabber users running on it. The…

2 years ago

Upgrade to Prosody 0.12

We upgraded the Jabber server to Prosody 0.12 (from 0.11) yesterday, see the changelog.

2 years ago

Hot-Chilli Matrix server

If anyone is looking for a Matrix server besides matrix.org or matrix.im, please visit us…

3 years ago

Converse.js finally works here

We did a lot of digging into the issues that the browser windows froze when…

3 years ago

New public domain: hot-chilli.im

We just added another public domain to our XMPP/Jabber server: hot-chilli.im For the future we…

3 years ago