This benchmark attempts to quantify the overheads introduced by the MIF container as messages are passed between components in a pipeline. We want the MIF to be as frictionless as possible, allowing the MIF 'plumbing' to deliver messages to components as quickly as possible so that application throughput is maximized. The diagram below shows the test application configuration.
This diagram illustrates that the MIF pipeline for this benchmark test consists of a simple program to generate a synthetic load – the “Load Generator” – and two separate processing branches. The top branch is referred to as the “Splitter/Aggregator Branch” and the bottom branch is the “Cache Branch.”
Load Generator. Sends messages to both JMS queues at a known rate (measured in messages per second). Message size will be set to a small value (e.g., 1 KB) for initial tests, then increased until the maximum data throughput is observed. The goal of the tests is to find the highest throughput in terms of rate and data volume. In other words, benchmark results are evaluated in terms of Messages per second for each message size, and data throughput in MB per second for the best performing test.
Splitter/Aggregator Branch. The module labeled “Splitter” receives string messages from a JMS topic and splits each message in half, resulting in two new messages. Then it forwards one message to “Module1” and the other to “Module2.” The modules relay the messages to the aggregator, which combines the Strings contained in the messages into the original string.
Cache Branch. The modules in this branch perform read-only operations. Each component exchanges only cache references to the actual data. The module labeled “Input” receives data from a JMS topic and inserts it into the cache. Then, it passes a cache reference to “Process”, which in turn passes the reference to “Output.” Lastly, the message is read from the cache and sent to a JMS topic which represents the end of the branch.
Benchmark tests were preformed in March 2008 using MIF version 0.8.1 on 2 sets of hardware:
- Grove: A Cluster of dual Xeon processors @3.0 GHz with 4 GB of RAM
- Marmot: A single quad core Xeon E5345 machine @ 2.33 GHz with 16 GB of RAM
The following is a graph representing message copy latency between MIF modules. Here, a message copy is the transfer of a message from one module to another. Latency is measured by averaging the time taken to copy messages between all modules in the benchmark pipeline over a period of time.
The benchmark test suite is available in the MIF distribution (Download MIF).