Comparing Amazon EC2 to Joyent SmartOS

Recently, I’ve been using Amazon web services (EC2, especially) quite a bit more at work. At home, I still use OpenIndiana, so I’ve been really interested in comparing Joyent’s offerings against Amazons first hand. In particular, my tasks I have in Amazon’s cloud always feel CPU bound, so I’ve decided to do a comparison of just CPU performance, giving some context to Amazon’s jargon ECU (Elastic Compute Unit) by comparing it with a Joyent SmartOS instance, as well as my MacBook Pro, iMac and OpenIndiana server.

So I spun up a Joyent Micro SmartOS instance and an Amazon EC2 linux Micro and small instances.

Joyent startup is impressive. The workflow is simple and easy to understand. I chose the smartosplus64 machine just because it was near the top of the list.

Amazon startup is about what I’ve learned to expect. Many more pages of settings later we’re up and running.

Installing ruby 1.9.3 with RVM

Ubuntu linux has fantastic community support, and many packages just work out of the box. Following the RVM instructions was easy to get it installed.

SmartOS, like OpenIndiana often requires a bit more work.

I made this patch to get ruby to compile: https://gist.github.com/4104287
Thanks to this article: http://www.hiawatha-webserver.org/forum/topic/1177

A Simple Benchmark

Here’s a really quick ruby benchmark, that will sort 5 million random numbers in a single thread:

require 'benchmark'

array = (1..5000000).map { rand }
Benchmark.bmbm do |x|
  x.report("sort!") { array.dup.sort! }
  x.report("sort") { array.dup.sort }
end

I also tested my MacBook Pro, my iMac and my Xeon E3 OpenIndiana server to get some perspective.

Here’s the results:

Machine	Benchmark (sec)
MacBook Pro 2.66gHz core i7 (2010)	86.99
iMac 24″ 2.5GHz core i5 (2012)	19.30
Xeon E3-1230 3.2GHz OpenIndiana server	35.57
Joyent EXTRA SMALL SmartOS 64-bit	55.10
Amazon MICRO Ubuntu 64-bit	361.42
Amazon SMALL Ubuntu 64-bit	123.69

Snap. Amazon is *SLOW*! And iMac the surprise winner!

And so what is this Elastic Compute Unit (ECU) jargon that Amazon have created? Since the Amazon Small instance is 1 ECU, we can reverse measure the others into compute units. And by converting their hourly price to a monthly price (* 24 hours * 365.25 days / 12 months), we can also determine the price per ECU:

Machine	Benchmark (sec)	$/hour	ECUs	$/month/ECU
MacBook Pro 2.66gHz core i7 (2010)	86.99		1.422
iMac 24″ 2.5GHz core i5 (2012)	19.30		6.409
Xeon E3-1230 3.2GHz OpenIndiana server	35.57		3.477
Joyent EXTRA SMALL SmartOS 64-bit ruby	55.10	$0.03	2.245	$9.76
Amazon MICRO Ubuntu 64-bit	361.42	$0.02	0.342	$42.69
Amazon SMALL Ubuntu 64-bit	123.69	$0.07	1.000	$47.48

Snap. Amazon is *EXPENSIVE*!

My laptop with 4 threads could do the CPU work of 5.7 small amazon EC2 instances, worth $270/month. And my Xeon box with 8 threads could do the work of 27.8 small instances, worth $1320/month. (I built the whole machine for $1200!!). Mind you, these comparisons are on the native operating system, but if you’re running a machine in house this is an option, so might be worth consideration.

I’ve read that comparing SmartOS to Linux in a virtual machine isn’t a fair comparison because you’re not comparing apples with apples; one is operating system level virtualisation (Solaris Zones), the other is a full virtual machine (Xen Hypervisor). Well tough. All I need to do is install tools and my code and get work done. And if I can do that faster then that is a fair comparison.

Conclusion

Joyent CPU comes in more than 4 times cheaper than Amazon EC2.

Amazon need to lift their game in terms of CPU performance. They offer a great service that obviously extends far beyond a simple CPU benchmark. But when you can get the same work done in Joyent significantly faster for the comparable price, you’ll get far more mileage per instance, which is ultimately going to save the dollars.

 

EDIT: 19/11/12: Joyent’s machine is called “Extra Small”, not Micro as I originally had it.