RVM in OpenIndiana revisited

I just went through the process of setting up RVM from scratch in a zone in OpenIndiana. I wrote previously about the gnu path problem, but didn’t write down any of the OpenIndiana dependencies. They are:

Installing RVM

To install RVM, you will need:

1. curl certificates installed into /etc/curl/curlCA (not installed by default in a new zone)
2. The following packages are required to install rvm:
   `# pkg install archiver/gnu-tar text/gawk text/gnu-grep pkg:/sfe.openindiana.org/developer/versioning/git`
   Note: This requires adding the “SFE” (Spec Files Extra) repository as a known publisher. If this is already set up in your global zone then the new non-global zone should inherit it.
3. Set the PATH environment variable to include `/usr/gnu/bin` at the front, i.e.: in .bashrc
   `export PATH=/usr/gnu/bin:$PATH`

Building Ruby 1.8.7

You will additionally need the following packages installed to build ruby 1.8.7 using rvm:

`# pkg install runtime/gcc text/gnu-patch developer/library/lint system/header system/library/math/header-math file/gnu-coreutils`

Et voilá!

Installing Ruby Enterprise Edition also worked for me. But it’s installer requires another ruby to be installed first.

Problems

So far the following rubies / gems have had problems:

• Ruby 1.9.3 does not build correctly. (1.9.2 does, though) See the illumos bug tracker: https://www.illumos.org/issues/1587
• Passenger gem installs, but apache module fails to compile.

 

So it’s not yet a perfect world

TTCP, in Ruby

I’ve used the TTCP tcp test program from time to time, and am at present looking at some networking in Ruby. So why not have a look at porting that to ruby? So I did.

This has been built as a gem, which an executable ‘ttcp’ that will install in your gem’s bin folder. You can get the gem from here: http://rubygems.org/gems/ttcp

Or type: `gem install ttcp` at your terminal.

Source code is released under MIT license, and available on github: https://github.com/mattconnolly/ttcp

So far, I’ve tested it out on Mac and OpenIndiana in ruby 1.8.7, 1.9.3 and JRuby 1.6.5. I can’t seem to run the tests in JRuby, but it appears to work anyway.

Enjoy.

Testing with cucumber – session state

I was recently at the Brisbane Ruby on Rails meetup where one discussion was about the speed of testing. I’ve also read in many places that rails apps are terribly slow at testing.

In a recent rails app I’m working on, I wrote several features describing how I want the user sign up sign in and sign out to function. However following this, my next features about a user’s data also required me to repeat the user sign in step in every scenario. While it’s not too hard for each scenario to start with “given I am signed in (as a role)”, or put that step in. “Background:” section, it does mean that the capybara steps are repeated for every single scenario.

So, I started to explore how a scenario can skip the sign in web steps, and it appears it cannot be done (easily). This stack overflow discussion is very relevant: http://stackoverflow.com/questions/1271788/session-variables-with-cucumber-stories

I do find it very odd that cucumber steps can access and modify the rails database (eg load fixtures / create objects with factories and save them) but cannot access the session state. The same applies to rspec request specs.

I guess that’s just the way it is, but in the interest of speeding things up, I’d like to know if there’s any special reasons for this limitation that prevents us from speeding up these tests.

Using RVM with Jenkins

I’ve recently set up RVM running with Jenkins, and finally have it all working just the way I want.

I’m running on OpenIndiana and have Jenkins running as its own user. I ssh in as that user and install rvm for that user and verified that I can run ruby tests with rvm from the terminal.

There’s a “RVM plugin” (version 0.1) for Jenkins which has great promise – I like the idea of running all build steps with RVM setup. However, I couldn’t get it to find my rvm installation no matter how much stuffing around with environment variables I did. So I’m not using it for now.

The trick was to use a “Execute shell” build step and use a script like so:

#!/bin/bash -e
export TERM=xterm
[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"
# Use the correct ruby
rvm use 1.9.2@gemset --create
# Do any setup
# e.g. possibly do 'rake db:migrate db:test:prepare' here
bundle install
# Finally, run your tests
rake test
# and build the gem
rake build
# and install the gem locally
rake install

This was for a ruby gem project which installs that gem in the named rvm gemset. Other projects can use this gem simply by using that same gemset.

Note also the line: “export TERM=xterm” – rvm is designed to run in a terminal where the TERM environment variable is set. Apparently there was a change to gracefully ignore its absence or be able to use TERM=dumb for no colour, (github issue 698), but that doesn’t appear to be working for me in rvm 1.10.2.

Good luck.

sudo with askpass

Normally I connect to a remote server with ssh then at the command prompt execute a command using sudo if I require the additional privileges. Just today, I had occasion to run sudo over ssh (something broken with the login prompt). Without having preconfigured anything, I got this to work:

ssh (server) SUDO_ASKPASS=/usr/lib/ssh/ssh-askpass sudo -A (command)

In my case, I had an appropriate command at ‘/usr/lib/ssh/ssh-askpass’. Your system might be different.

Telstra Bigpond pays up for router to make up for crappy Netgear cable modem!

After a frustrating series of unsuccessful attempts to get Telstra Bigpond to replace their crappy Netgear Cable modem with something that would actually work, and resolving the issue by supplying my own router, I made a formal complaint to Telstra documenting all of the trouble shooting that I’ve done, the dates that I had rang technical support, etc, etc.

The crux of my complaint was this:

I have provided enough technical analysis of the problem to unequivocally identify that the Netgear CGD24N cable modem is incapable of performing the required functionality for the “Telstra Complete Home Saver 200GB Bundle” service.

I have given Telstra Bigpond a fair opportunity to resolve the issue and it has not been resolved.

I have taken further steps and resolved the issue myself by disabling a significant portion of the Netgear CGD24N’s functionality and installing an additional wireless router (Linksys model E2000) at my own expense.

I therefore make the following demands:

1. I demand that the cost of this Linksys E2000 router ($115.00) be reimbursed to me as a credit to my Bigpond account. (Original invoice attached.)
2. I demand acknowledgement from Telstra that no suitable customer premises equipment has been delivered and subsequently the conditions for the contract for service have not been met. I will then continue to use Bigpond as my Internet service provider just as if a contract period had expired naturally, subject to no cancellation clauses, and continuing on a rolling monthly basis.
3. I demand a written response to this complaint with 14 days. 

And today, I received a phone call from their complaints resolution people, offering me a refund for the $115.00 which I happily accepted and a guarantee that I’ll be released from any contract-period cancellation charges should I cancel my service. The guy was polite and gave me a reference number, his Telstra employee number and the phone number for his office (call centre I suppose). I’ll take what I can get.

A refund of $115.00 certainly costs Telstra less than a complaint escalated to the TIO, is less than what they’ve already spent on technical support that couldn’t deal with the issue, and is worth way less than the amount of time I’ve spent on this issue, but I’ll take it as a win.

I just hope they’ve really learnt something from this and can give Netgear a boot up the proverbial for making such second rate equipment (and themselves for supplying it).

Happy days.

Telstra Bigpond Cable crappy Netgear Router part 7

In my last post, I wrote that after too many unsuccessful calls to Bigpond technical support to replace my crappy Netgear CGD24N cable modem, I spoke with someone in sales who said they would send me a replacement modem and that it would be a Thomson cable modem. Well the modem arrived in the mail today, and guess what? They sent me another Netgear CGD24N. Fools.

Enough is enough. I went to my local computer store and bought a Linksys wireless router, disabled NAT on the CGD24N (effectively disabling all of its router functions and using it just as a modem), and connected the Linksys router. Boom. Fast, snappy perfect internet.

It just goes to show that the router part of the CGD24N is total rubbish.

I’ve been on the phone to Telstra Bigpond about a dozen times now, have had a field technician check the line 3 times (it’s fine, there’s nothing wrong with the line) and replace the modem with the same model 3 times (it’s still rubbish) and have no solution. Except for me to spend more money on another piece of gear to do the job of the router that Telstra provided.

Now that I have a perfectly working connection, my motivation to speak to Telstra call centres that can’t do anything about this problem is massively reduced.

However, I think other potential customers should know about these issues. And likewise, so does the TIO.

Telstra Bigpond Cable crappy Netgear Router part 6

So I got home from work and my wife was downloading something onto her computer, I plugged in my laptop and boom… Web pages timing out. Massive SYN_SENT in my netstat output. No problem connecting to the router, I could see that the router had a good signal strength, and the line definitely hadn’t dropped out because the other computer was downloading. I could connect to my nas and do a backup there.

The only piece of the network that wasn’t working correctly was the Netgear router, with the symptoms described by others when the NAT (Network Address Translation) table is full and the router simply cannot handle any more connections.

So, on the phone to Telstra Bigpond technical support again to see what can be done about it. Nothing, it turns out, because all they can do is send out a Field Technician to check the line (which has happened 3 times now), or replace the modem with the same model. Not good enough.

They referred me to sales. Odd, but at least I’m going somewhere else because the people on the tech support line clearly don’t have any capability to deal with my issue. Logically this makes some sense – the sales team have provisioned substandard equipment, what can support do about it?

So I got on the phone to sales to continue complaining about this. After explaining the situation yet again – I need to make a recording to play down the line to them – they decided to send me a replacement modem. I insisted that they needed to provide a “better” modem, because another Netgear would simply result in the same problems (we’ve already been down that path).

The girl tells me that she’s going to send me a Thomson modem. I’d only heard of Thomson making ADSL modems, and she *couldn’t* tell me the model number which is quite concerning. I’ve since looked up Thomson (bloody hard to find anything on Technicolor’s crappy website) and they do indeed make cable modems.

So fingers crossed, this replacement is a cable modem and that it’s router works properly.

Failing that, I can feel a letter to the TIO coming on.

Oh, and if anyone reading this has used a Thomson Cable modem, especially if provisioned by Telstra, I’d love to hear how it went!

When rate limiting your server more than doubles your server output…

At work, we’ve had a few customers mentioning to us that they’ve experienced slow downloads of data from our servers. When I’ve tested it at home, I’ve experienced the same thing, albeit not quite as bad. The best data rate I could get was about 30% of our server’s bandwidth.

In the last few days I’ve had several conversations with the network engineers at our ISP trying to identify exactly what the problem is. (Thank goodness we’re not with Telstra, if we had to wait for 3 times for a field technician to check if it was plugged in ok, we’d lose our business!)

After having the ISP’s network engineer change a few settings on their equipment, and doing some speed tests to a mini speed test site on their servers, we were still only able to utilise about 30% of our output bandwidth. Crapola.

He explained to me that our rate limiting was done by traffic policing at the switch on the other end of our link. After some reading about what traffic policing was, I’m led to understand that when your data rate is exceeded, packets are dropped. Shouldn’t be too much of a drama, TCP is designed to recover from packet loss, and it does a great job of it, right?. But, what does this packet loss mean to our actual throughput rates?

After making numerous other changes, none of which helped our bandwidth problem, I decided to try something else: rate limiting our server.

Our web files are served by apache running on a Mac, and luckily the Mac OS includes rate limiting controls in its built in firewall. (Great little tutorial here).

So with the `ipfw` command at the ready, I limited outgoing traffic on port 80 (http) to 80% of our bandwidth. And viola! Download rates rose more than double from 30% to 80% of our output limit!!

I never expected that rate limiting our server would cause our outgoing data rate to increase! Especially, more than double!

I’m sure there is a time and place for dropping packets (traffic policing), but it appears to be not working well for us. If anyone has more input on where this is appropriate or for suggestions of other alternatives, please let me know!

Telstra Bigpond Cable crappy Netgear Router part 5

When I previously rang Telstra, continuing my hunt for a replacement of Telstra Bigpond’s crappy Netgear CGD24N cable modem, I was told a field technician was required to replace the modem THREE TIMES before they would escalate the issue beyond the call centre to someone higher up in Telstra. Well guess what? The guy never showed up.

I guess this is some ploy of theirs to never get to the THREE TIMES so that they never have to escalate the issue. Bad form.

Last night I rang again, and at least this time was escalated to a senior person in the call centre. This guy, Ed, was pretty bright. But he still didn’t know anything about Network Address Translation as insisted that noise on the line was causing dropouts because of dynamic IP configuration. Yeah, right.

So now I have an appointment for another technician, this will be visit #2 on Friday morning. I hope this guy turns up.