Matt Connolly's Blog

my brain dumps here…

How to get “Getting Things Done” eBook in Australia

A few friends / colleagues have told me about David Allen’s book “Getting Things Done”. So I thought I’d check it out.

Turns out, it’s hard to get in Australia:

The Australian title of Getting Things Done is “How to Get Things Done”
see: http://www.davidco.com/getting-things-done/international-editions

Amazon now sell the kindle version of that name for 50% more than the price of the US version.
US Kindle Edition $12.99 USD
AUS Kindle Edition $19.29 AUD (~20.23USD)
http://www.amazon.com/How-Get-Things-Done-ebook/dp/B00B5KPVQM/ref=tmm_kin_title_0?ie=UTF8&qid=1365542478&sr=1-1

It isn’t available through Apples iBook store.

You can purchase the ebook directly through penguin Australia for AUD $17.99 (~18.89 USD)
http://www.penguin.com.au/products/9781742538709/how-get-things-done

I bought this one (not wanting to pay the highest price). The DRM protected version is hard to get going. You need to register with Adobe digital editions and find a computer or ipad app that can log in to Adobe’s service so that you can then read it.
http://www.adobe.com/au/products/digital-editions.html

You need to install the Adobe Editions software on Mac / Windows PC, then register the purchase and convert the downloaded file into a DRM signed eBook. (Penguin Books don’t tell you any of this.) Then you need to find a reader that will open the eBook:

List of devices:
http://blogs.adobe.com/digitalpublishing/supported-devices

I already had the “Readmill” reader app on my iPad, and even though it isn’t listed in the supported devices, it could open the eBook after signing in to Adobe’s servers with my new Adobe account.

I’m very technically proficient and I found this a very difficult process to go through. It’s a shame that a legitimate purchase is **so hard**.

I would not recommend going down the Penguin books eBook path. It is painful.

If the publishers just made it available on Amazon for less than ridiculous price conversion and iBook store, it would be much easier. While trying to figure out how to do all this, I found several sites in Russia where I could download a pirated PDF of the book.

My preferred option would the Apple iBook store at a reasonable price. (ie: parity pricing with US product + 10% GST would be ideal). But I guess that’s in the hands of the Australian publisher isn’t it?

Building netatalk in SmartOS

I’m looking at switching my home backup server from OpenIndiana to SmartOS. (there’s a few reasons, and that’s another post).

One of the main functions of my box is to be a Time Machine backup for my macs (my laptop and my wife’s iMac). I found this excellent post about building netatalk 3.0.1 in SmartOS, but it skipped a few of the dependencies, and did the patch after configure, which means if you change you reconfigure netatalk, then you need to reapply the patch.

Based on that article, I came up with a patch for netatalk, and here’s a gist of it: https://gist.github.com/mattconnolly/5230461

Prerequisites:

SmartOS already has most of the useful bits installed, but these are the ones I needed to install to allow netatalk to build:

$ sudo pkgin install gcc47 gmake libgcrypt

Build netatalk:

Download the latest stable netatalk. The netatalk home page has a handy link on the left.

$ cd netatalk-3.0.2
$ curl 'https://gist.github.com/mattconnolly/5230461/raw/27c02a276e7c2ec851766025a706b24e8e3db377/netatalk-3.0.2-smartos.patch' > netatalk-smartos.patch
$ patch -p1 < netatalk-smartos.patch
$ ./configure --with-bdb=/opt/local --with-init-style=solaris --with-init-dir=/var/svc/manifest/network/ --prefix=/opt/local
$ make
$ sudo make install

With the prefix of ‘/opt/local’ netatalk’s configuration file will be at ‘/opt/local/etc/afp.conf’

Enjoy.

[UPDATE]

There is a very recent commit in the netatalk source for an `init-dir` option to configure which means that in the future this patch won’t be necessary, and adding `--with-init-dir=/var/svc/manifest/network/` will do the job. Thanks HAT!

[UPDATE 2]

Netatalk 3.0.3 was just released, which includes the –init-dir option, so the patch is no longer necessary. Code above is updated.

Building ruby 2.0.0 for Mac

After downloading ruby source code, use this:

CC=/usr/bin/clang ./configure ...

This also works with RVM

CC=/usr/bin/clang rvm install ruby-2.0.0

Some quotes from Mike Lee on Wednesday at YOW Brisbane

Here’s a selection of my favourite quotes from Mike Lee’s workshop at Yow conference in Brisbane this Wednesday:

  • “Porting from photoshop to Objective-C” (regarding making apps)
  • “Backward Compatibility is voodoo bullshit”
  • “Cut features, not quality”
  • “Speculation is a great evil”
  • “Never underestimate the power of doing something well over the power of doing something first”

In a word, the workshop was awesome.

It’s rare to meet someone so intense and passionate about what they do, and experience such brutal honesty in an intimate setting.

Thanks YOW, and thanks Mike Lee!

Comparing Amazon EC2 to Joyent SmartOS

Recently, I’ve been using Amazon web services (EC2, especially) quite a bit more at work. At home, I still use OpenIndiana, so I’ve been really interested in comparing Joyent’s offerings against Amazons first hand. In particular, my tasks I have in Amazon’s cloud always feel CPU bound, so I’ve decided to do a comparison of just CPU performance, giving some context to Amazon’s jargon ECU (Elastic Compute Unit) by comparing it with a Joyent SmartOS instance, as well as my MacBook Pro, iMac and OpenIndiana server.

So I spun up a Joyent Micro SmartOS instance and an Amazon EC2 linux Micro and small instances.

Joyent startup is impressive. The workflow is simple and easy to understand. I chose the smartosplus64 machine just because it was near the top of the list.

Amazon startup is about what I’ve learned to expect. Many more pages of settings later we’re up and running.

Installing ruby 1.9.3 with RVM

Ubuntu linux has fantastic community support, and many packages just work out of the box. Following the RVM instructions was easy to get it installed.

SmartOS, like OpenIndiana often requires a bit more work.

I made this patch to get ruby to compile: https://gist.github.com/4104287
Thanks to this article: http://www.hiawatha-webserver.org/forum/topic/1177

A Simple Benchmark

Here’s a really quick ruby benchmark, that will sort 5 million random numbers in a single thread:

require 'benchmark'

array = (1..5000000).map { rand }
Benchmark.bmbm do |x|
  x.report("sort!") { array.dup.sort! }
  x.report("sort") { array.dup.sort }
end

I also tested my MacBook Pro, my iMac and my Xeon E3 OpenIndiana server to get some perspective.

Here’s the results:

Machine Benchmark (sec)
MacBook Pro 2.66gHz core i7 (2010) 86.99
iMac 24″ 2.5GHz core i5 (2012) 19.30
Xeon E3-1230 3.2GHz OpenIndiana server 35.57
Joyent EXTRA SMALL SmartOS 64-bit 55.10
Amazon MICRO Ubuntu 64-bit 361.42
Amazon SMALL Ubuntu 64-bit 123.69

Snap. Amazon is *SLOW*! And iMac the surprise winner!

And so what is this Elastic Compute Unit (ECU) jargon that Amazon have created? Since the Amazon Small instance is 1 ECU, we can reverse measure the others into compute units. And by converting their hourly price to a monthly price (* 24 hours * 365.25 days / 12 months), we can also determine the price per ECU:

Machine Benchmark (sec) $/hour ECUs $/month/ECU
MacBook Pro 2.66gHz core i7 (2010) 86.99 1.422
iMac 24″ 2.5GHz core i5 (2012) 19.30 6.409
Xeon E3-1230 3.2GHz OpenIndiana server 35.57 3.477
Joyent EXTRA SMALL SmartOS 64-bit ruby 55.10 $0.03 2.245 $9.76
Amazon MICRO Ubuntu 64-bit 361.42 $0.02 0.342 $42.69
Amazon SMALL Ubuntu 64-bit 123.69 $0.07 1.000 $47.48

Snap. Amazon is *EXPENSIVE*!

My laptop with 4 threads could do the CPU work of 5.7 small amazon EC2 instances, worth $270/month. And my Xeon box with 8 threads could do the work of 27.8 small instances, worth $1320/month. (I built the whole machine for $1200!!). Mind you, these comparisons are on the native operating system, but if you’re running a machine in house this is an option, so might be worth consideration.

I’ve read that comparing SmartOS to Linux in a virtual machine isn’t a fair comparison because you’re not comparing apples with apples; one is operating system level virtualisation (Solaris Zones), the other is a full virtual machine (Xen Hypervisor). Well tough. All I need to do is install tools and my code and get work done. And if I can do that faster then that is a fair comparison.

Conclusion

Joyent CPU comes in more than 4 times cheaper than Amazon EC2.

Amazon need to lift their game in terms of CPU performance. They offer a great service that obviously extends far beyond a simple CPU benchmark. But when you can get the same work done in Joyent significantly faster for the comparable price, you’ll get far more mileage per instance, which is ultimately going to save the dollars.

 

EDIT: 19/11/12: Joyent’s machine is called “Extra Small”, not Micro as I originally had it.

iOS Core Data: Group by and count results

I looked in the usual places and couldn’t find any decent examples of using core data to do a group by and count. So I’m making one here.

Let’s suppose you have a bunch of Records, and each Record has a Status attribute among others, and you want a break down of how many have each Status. The SQL would be:

SELECT `Status`, COUNT(*) FROM `Records` GROUP BY `Status`

How do we do this with Core Data?

start with a fetch request:

NSFetchRequest* fetch = [NSFetchRequest fetchRequestWithEntityName:@"Record"];

set up an attribute description and expression description for the two values we want in the results:

NSEntityDescription* entity = [NSEntityDescription entityForName:@"Record"
                                          inManagedObjectContext:myManagedObjectContext];
NSAttributeDescription* statusDesc = [entity.attributesByName objectForKey:@"status"];
NSExpression *keyPathExpression = [NSExpression expressionForKeyPath: @"url"]; // Does not really matter
NSExpression *countExpression = [NSExpression expressionForFunction: @"count:"
                                                          arguments: [NSArray arrayWithObject:keyPathExpression]];
NSExpressionDescription *expressionDescription = [[NSExpressionDescription alloc] init];
[expressionDescription setName: @"count"];
[expressionDescription setExpression: countExpression];
[expressionDescription setExpressionResultType: NSInteger32AttributeType];

tell the fetch request to only fetch these, and group by the status attribute description

[fetch setPropertiesToFetch:[NSArray arrayWithObjects:statusDesc, expressionDescription, nil]];
[fetch setPropertiesToGroupBy:[NSArray arrayWithObject:statusDesc]];
[fetch setResultType:NSDictionaryResultType];
NSError* error = nil;
NSArray *results = [myManagedObjectContext executeFetchRequest:fetch
                                                         error:&error];

And voila. The result is an array of dictionaries, each with “status” and “count” keys and corresponding values. Even though the group by guarantees the grouped values be unique, there could be more than one grouped column, so it does make sense.
If you think that’s a lot of code, it is. Let’s compare that with the Rails solution:

Record.count(:group => :status)

And that returns a dictionary (Hash), keyed by status, with the counts in the values

Rails: beware a column named ‘type’

It’s been said many times to not use the column name ‘type’, because rails uses this to define the subclass of a model that should be loaded. (great feature, by the way).

But if you have an existing database with a column called “type” and you want to interface to that database with active record, this is a great tip (thanks to this post):

class MyModel < ActiveRecord::Base
  self.inheritance_column = "inheritance_type"
end
Ahh… back to work.

Xcode testing AFNetwork Operation callback blocks

Just recently, I was writing some tests in Xcode for some HTTP requests using the AFNetwork library. Previously I’ve used the ASIHTTPRequest library, but in this particular project, I’ve chosen to use AFNetworking for its JSON support.

Since the requests run asynchronously we need a way to wait for the operation to complete. This is easy:

- (void)testRequest
{
    MyHTTPClient* api = [MyHTTPClient sharedInstance]; // subclass of AFHTTPClient
    NSDictionary* parameters = [NSDictionary dictionary]; // add query parameters to this dict.
    __block int status = 0;
    AFJSONRequestOperation* request = [api getPath:@"path/to/test"
                                        parameters:parameters
                                           success:^(AFHTTPRequestOperation *operation, id responseObject) {
                                               // success code
                                               status = 1;
                                               NSLog(@"succeeded");
                                           } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
                                               // failure
                                               status = 2;
                                               NSLog(@"failed");
                                           }];
    [api enqueueHTTPRequestOperation:request];
    [api.operationQueue waitUntilAllOperationsAreFinished];

    STAssertTrue([request isFinished], @"request finished");
    STAssertEquals(request.response.statusCode, 200, @"request returned 200 OK");
    STAssertEquals(status, 1, @"success block was executed");
}

This is great for testing that the request completes, and verifying its status. But if we need to test anything in the success or failure callbacks, the last test will fail with `status == 0`.

This is because AFNetwork processes its response in a background thread, and the final success or failure block callback is dispatched asynchronously from there to a specific queue, which unless provided is the main queue. This means that the block won’t get called until *AFTER* the test code has completed.

Putting in some kind of a lock causes a deadlock, since the test is running on the main thread, and the block callback never gets an opportunity to run. The solution is to manually run the main threads runloop until the callbacks have been processed.

Here’s my solution:

- (void)testRequest
{
    MyHTTPClient* api = [MyHTTPClient sharedInstance]; // subclass of AFHTTPClient
    NSDictionary* parameters = [NSDictionary dictionary]; // add query parameters to this dict.
    __block int status = 0;
    AFJSONRequestOperation* request = [api getPath:@"path/to/test"
                                        parameters:parameters
                                           success:^(AFHTTPRequestOperation *operation, id responseObject) {
                                               // success code
                                               status = 1;
                                               NSLog(@"succeeded");
                                           } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
                                               // failure
                                               status = 2;
                                               NSLog(@"failed");
                                           }];
    [api enqueueHTTPRequestOperation:request];
    [api.operationQueue waitUntilAllOperationsAreFinished];

    while (status == 0)
    {
        // run runloop so that async dispatch can be handled on main thread AFTER the operation has 
        // been marked as finished (even though the call backs haven't finished yet).
        [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
                                 beforeDate:[NSDate date]];
    }

    STAssertTrue([request isFinished], @"request finished");
    STAssertEquals(request.response.statusCode, 200, @"request returned 200 OK");
    STAssertEquals(status, 1, @"success block was executed");
}

This addition will continually pump that run loop which allows AFNetwork’s async dispatch of the block to the main queue to execute, and hey presto! We now have a test that can also verify code in the success (or failure) completion blocks of an AFNetwork request operation.

OpenIndiana – installing ImageMagick and Rmagick gem

I found that the Rmagic gem wouldn’t install with the standard OpenIndiana package for ImageMagick because it was too old, and the one installed from the SFE repository didn’t seem to work. But installing ImageMagick from source (version 6.7.6) was pretty straight forward.

Only catch here was that because I installed it in /opt/local, the Magick-config tool couldn’t find its package config (.pc) files. i.e., I was getting this:

$ /opt/local/bin/Magick-config --cflags
Package MagickCore was not found in the pkg-config search path.
Perhaps you should add the directory containing `MagickCore.pc'
to the PKG_CONFIG_PATH environment variable
No package 'MagickCore' found

The Rmagick extension needs to find `Magick-config` in PATH, and that needs to find it’s package config files. So:

$ export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/opt/local/lib/pkgconfig
$ export PATH=$PATH:/opt/local/bin
$ gem install rmagick -v '2.13.1'
Building native extensions. This could take a while...
Successfully installed rmagick-2.13.1
1 gem installed
Installing ri documentation for rmagick-2.13.1...
Installing RDoc documentation for rmagick-2.13.1...

Done.

OpenIndiana – running openvpn as a service

Here’s a gist for the XML manifest to run openvpn as a service:

https://gist.github.com/2484917

It expects that there is an openvpn config file at /etc/openvpn/config which, you’ll need to configure with your settings, certificates, etc.

If you configure it to run a tap interface then bonjour advertising will work over the link, which is great if you want time machine or other bonjour services to work to an OpenIndiana server from a mac connecting from anywhere with openvpn.