PTS on AWS with m1.medium EC2 and EBS with Provisioned IOPS

More results running PTS with PIOPS 1000 and 2000

I took a look at AWS m1.medium EC2 instances with Provisioned IOPS. This post is light on instructions and is basically just graphs of performance results.

test configurations & results

• medium-attached-ebs: m1.medium EC2 instance, standard EBS volume (ext4)

  • 1309262-SO-JOKEYRHYM90

• medium-attached-ebs-piops1000: m1.medium EC2 instance, EBS volume (ext4) provisioned for 1000 IOPS

  • 1309258-SO-JOKEYRHYM10

• medium-attached-ebs-piops2000: m1.medium EC2 instance, EBS volume (ext4) provisioned for 2000 IOPS

  • 1309253-SO-JOKEYRHYM39

• small-attached-ebs: m1.small EC2 instance, standard EBS volume (ext4)

  • 1308270-SO-JOKEYRHYM25

• small-attached-ebs-piops1000: m1.small EC2 instance, EBS volume (ext4) provisioned for 1000 IOPS

  • 1309230-SO-JOKEYRHYM39

• small-attached-ebs-piops2000: m1.small EC2 instance, EBS volume (ext4) provisioned for 2000 IOPS

  • 1309233-SO-JOKEYRHYM49

comparison

Note: each test is actually run several times, with the below scores representing the average. The “error” column is the maximum deviation from the average by one or more test runs.

AIO-STRESS

FLEXIBLE IO TESTER

This is one of the tests where m1.medium without PIOPS fared dramatically worse than m1.small without PIOPS. I re-ran these tests with the same results. If your application is similar to this benchmark, you may want to consider sticking with m1.small.

SQLITE

I didn’t expect these results, but PIOPS 1000 and 2000 appear to test better on the m1.small than they do on the m1.medium.

FS-MARK

Here’s another benchmark where m1.small no-PIOPS beats m1.medium no-PIOPS. If your application deals with loads of small files, you’d be better off sticking with m1.small. PIOPS 1000 and PIOPS 2000 perform roughly the same, regardless of instance size.

DBENCH

The combination of the m1.medium’s CPU and Provisioned IOPS allows the 128-client test to produce the best results here. Without either of these, the 48-client and 12-client tests tend to be the sweet spot.

The m1.small results are fairly similar regardless of PIOPS. Meanwhile, the m1.medium results show performance scalining linearly with the level of PIOPS. This is a benchmark where PIOPS didn’t really shine on the m1.small, but clearly does on the m1.medium.

As usual, the results for 1-client are very different to the multi-user tests. As such, it’s only a mere curiousity that the m1.small beats m1.medium (with matching PIOPS).

IOZONE

The behaviour of m1.medium without PIOPS is very suspicious here, but I did run the PTS twice on different m1.medium EC2 instances with similar results. Of course, I provisioned them one after the other, so there is a chance I got the same aberant hardware twice in a row. :P

The CPU difference gives m1.medium the advantage in the read test.

All combinations perform the write test equally well here. The exception is m1.medium without PIOPS, which loses by a large margin.

THREADED I/O TESTER

The m1.medium’s CPU destroys the m1.small on the 32-thread read test.

Writing data is a different story. m1.small with no-PIOPS actually beats m1.medium by a descent margin.

COMPILE BENCH

For these tests, PIOPS 2000 doesn’t seem to have much advantage over PIOPS 1000. Predictably, m1.medium with PIOPS rules the roost here.

For the compile and create tests, m1.small matches m1.medium. That is, until you turn on PIOPS.

UNPACKING THE LINUX KERNEL

POSTMARK

GZIP COMPRESSION

POSTGRESQL PGBENCH

APACHE BENCHMARK

observations

Now that I have some results from both m1.small and m1.medium EC2 instances, it’s clear exactly which tests are affected by CPU performance (although CPUs are not the focus of this series of articles).

These are the tests that were very CPU-bound, with little variation in results when only PIOPS differed:

• Compile Bench: Read Compiled Tree
• Unpacking the Linux Kernel
• PostMark
• Gzip Compression
• Apache Benchmark

There were a few tests where performance scaled with PIOPS rather than CPU horsepower:

• Flexible IO Tester
• FSMark
• IOZone
• PostgreSQL PGBench

PIOPS 2000 was over 10% better than PIOPS 1000 in the following cases:

• Flexible IO Tester
• SQLite
• Threaded IO Tester
• Compile Bench: Create & Read
• PostMark

It’s difficult to making more observations on the value of PIOPS with the following benchmarks producing anomolous results on m1.medium without PIOPS (i.e. m1.small being faster):

• Flexible IO Tester
• FSMark
• DBench: 1-client
• IOZone: write
• Threaded IO Tester: write

conclusions

It seems as though m1.medium EC2 instances have a regression (compared to m1.smalls) for cases where many smaller writes to EBS (without PIOPS) are being requested. Superficially, this seems illogical: m1.medium instances enjoy a better network interface, more RAM and a faster CPU.

Perhaps basic EBS becomes a major write bottleneck with a faster CPU?

In many cases, PIOPS 1000 was faster than no-PIOPS, particularly the cases with strange no-PIOPS results. If you are using m1.medium EC2 instances with applications that favour storage IO, then you really ought to consider PIOPS 1000 at least.

series

1. Phoronix Test Suite on AWS
2. PTS Results on Provisioned IOPS on AWS EBS
3. PTS Results on Provisioned IOPS on AWS EBS, Part 2
4. PTS on AWS with m1.medium EC2 and EBS with Provisioned IOPS

Ron - 28 Sep 2013