SMP: What is it and why do you care?

SMP stands for Symmetric Multiprocessing, a feature of most modern multi-core and multi-processor computers that allows multiple CPUs to work together in running a single properly-coded application. This can allow an application to work faster and more efficiently, such as by allowing it to process one piece of data while it is receiving the next.

Most programs don't need this functionality. Evergreen, for example, is split into a large number of smaller programs. The entire system can thus be spread not only across multiple CPUs, but across a large number of servers.

Where SMP comes in handy for Evergreen is in the communication between all of these smaller programs. They all communicate with each other over XMPP, which is currently handled by ejabberd.


Ejabberd is built on Erlang, and current versions of Erlang and ejabber both support SMP. The problem is that by default in some packaged versions this functionality is turned off in ejabberd. This means it will run a single primary thread and be constrained by a single processor's capabilities. On a single core server this is likely fine, but on a multi-core or multi-processor system this creates a bottleneck.

For example, in our production environment we would get complaints of slowness and errors. If we were already looking we would sometimes see ejabberd's main process, beam, pegged at 100% CPU usage. This indicated that it was struggling to handle the load placed upon it at the time. At the same time we would find that other parts of the system were failing to stay connected, or to initially connect, to ejabberd. After enabling SMP we found that ejabberd would jump to nearly 200% CPU usage, but we have not seen any of the issues we thought were related that we were seeing previously. Or at least we haven't yet.

Note you may already be using SMP mode, and thus have nothing to do. To determine this you can do a very simple check with the "ps" command while ejabberd is running:

ps -U ejabberd

If you see a process named "beam.smp" running then SMP mode is already enabled. If you only see "beam" then SMP mode is not enabled, or your system doesn't support it.

Luckily, ejabberd's SMP support is easy to turn on if it is available. On Debian and Ubuntu there is a "defaults" file for ejabberd in /etc/default/ejabberd. Redhat-based systems likely have an /etc/ejabberd/ejabberdctl.cfg to serve the same purpose. If when you open your system's file you find a line like this:

#SMP=disable

You can enable SMP support by changing it to this:

SMP=auto

That is, remove the leading # and change "disable" to "auto". This ensures that even if you don't have SMP support for some reason ejabberd will still run. If you want to force it on because you know you have support you can use "enable" instead of "auto", but "auto" is safer and appears to be the default when installing from source manually.

To make the change take effect you will need to stop all of your OpenSRF/Evergreen services, stop ejabberd, then start ejabberd. A restart of ejabberd may not work to change the setting (although I just plain never have that work anyway, I always end up stopping, waiting, and starting it again). Once ejabberd is back up you can repeat the ps trick above to see if SMP mode activated before restarting OpenSRF/Evergreen services.