Wednesday, February 23, 2011

JMS speed test: ActiveMQ vs HornetQ

A while ago I was asked by a client to evaluate different open source JMS providers. The ultimate goal was to set up a highly available messaging system that can manage high throughput.

Me and my colleagues dug deep in our memories and on the internet to find all of the open source JMS providers. And there are plenty. In the end we looked at the following:
  • ActiveMQ
  • OpenMQ
  • RabbitMQ
  • OpenJMS
  • HornetQ
Besides the requirements of high availability and throughput there were some other requirements:
  • must be JMS 1.1 compliant
  • must be easy to set up and administer
  • vibrant community for support
Because of these reasons we quickly abandoned OpenJMS, which seems to have stopped evolving somewhere in 2006.
RabbitMQ is not JMS compliant, which we really need.
OpenMQ was dropped a little later, since in the first performance tests we found that it was noticably slower than ActiveMQ and HornetQ.

In a next phase we did quite extensive load tests on HornetQ and ActiveMQ, which I'll summarize below.

For our tests we used the following setup:
3 similar machines with 2 quad cores, 8GB of RAM, RHEL 5, Java Hotspot VM 64 bit (1.6.0_21-b06). 2 machines were used for hosting the JMS providers. One machine contains a master, the other a backup instance. The third machine is used for generating load.

The load generation is done using the Sonic test harness. This framework allowed us to generate load on the the JMS providers with different producers and consumers of the JMS queues.

These are some of the results we got (based on size of message, whether messages are persisted, if a transaction is used, number of concurrent producers and receivers):


As you can see both JMS providers are about equal when you look at non peristent messaging.BUT when looking at persistent messaging, HornetQ is just amazing. The throughput you get there is just mindblowing. This has without a doubt to do with the Asynchronous IO feature of HornetQ. This is only available on *NIX based systems but it is well worth it when you are looking for a performant open source JMS provider.

There are other sources of comparisons like these, but they were in our opinion either biased or did not give us enough insight for our situation:
Authored by: Jeroen

7 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Multiple Senders and Receivers on a single JMS Queue is not a really good test IMO.

    You are measuring more the capacity of a single queue to receive multiple sends simultaneously, than the capacity of the server to handle throughput.

    It may be a valid case, but most use cases I think of involving multiple Producers will involve multiple queues.

    ReplyDelete
  3. Hi Clebert, thank you for your valuable feedback. To get a view on the throughput of the entire JMS server it would indeed make more sense to have producers and consumers on multiple queues. In our architecture the throughput of a single queue is however very important. We are using a single queue per message type. One of these types is rather dominant when it comes to load on the system. It takes like 90% of the total message traffic. In case a single queue would not be able to cope with this load, we would need to rethink some of the architecture. So this was an important use case for us.
    Kind regards,
    Jeroen

    ReplyDelete
  4. Can you please let me know where you got the activemq and hornetQ client jars? I am getting the following error messages
    Exception in thread "main" java.lang.RuntimeException: [TestParams] Unable to initialize vendor-specific parameters for vendor extension lib com.progress.perf.harness.ActiveMQExtensions
    at com.progress.perf.harness.TestParams.parse(TestParams.java:336)
    at TestHarness.main(TestHarness.java:294)
    Caused by: java.lang.ClassNotFoundException: com.progress.perf.harness.ActiveMQExtensions
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:169)
    at com.progress.perf.harness.TestParams.parse(TestParams.java:325)
    ... 1 more

    ReplyDelete
  5. Great post!

    Can I pls let you check my book on HornetQ?

    http://www.amazon.com/dp/1849518408/?tag=packtpubli-20

    Piero

    ReplyDelete
  6. Something wrong with the design, 25 senders to one queue? and 25 consumers and hence the benchmarks. There should be more number for producers and consumers. Another test with different sizes of messages and messages per commit.

    http://blog.x-aeon.com/2013/04/10/a-quick-message-queue-benchmark-activemq-rabbitmq-hornetq-qpid-apollo/ will show HornetQ has the worst performance.

    ReplyDelete
    Replies
    1. I meant less number of producers, consumers in most cases.

      I have quite many times seen load balancing to separate queues consumed by one consumer especially when you want XA and also processing takes time which is almost always and to make sure you don't ever loose a single message when you have not yet acked or your consumer went down.

      Delete