At work, we’ve had a few customers mentioning to us that they’ve experienced slow downloads of data from our servers. When I’ve tested it at home, I’ve experienced the same thing, albeit not quite as bad. The best data rate I could get was about 30% of our server’s bandwidth.
In the last few days I’ve had several conversations with the network engineers at our ISP trying to identify exactly what the problem is. (Thank goodness we’re not with Telstra, if we had to wait for 3 times for a field technician to check if it was plugged in ok, we’d lose our business!)
After having the ISP’s network engineer change a few settings on their equipment, and doing some speed tests to a mini speed test site on their servers, we were still only able to utilise about 30% of our output bandwidth. Crapola.
He explained to me that our rate limiting was done by traffic policing at the switch on the other end of our link. After some reading about what traffic policing was, I’m led to understand that when your data rate is exceeded, packets are dropped. Shouldn’t be too much of a drama, TCP is designed to recover from packet loss, and it does a great job of it, right?. But, what does this packet loss mean to our actual throughput rates?
After making numerous other changes, none of which helped our bandwidth problem, I decided to try something else: rate limiting our server.
Our web files are served by apache running on a Mac, and luckily the Mac OS includes rate limiting controls in its built in firewall. (Great little tutorial here).
So with the `ipfw` command at the ready, I limited outgoing traffic on port 80 (http) to 80% of our bandwidth. And viola! Download rates rose more than double from 30% to 80% of our output limit!!
I never expected that rate limiting our server would cause our outgoing data rate to increase! Especially, more than double!
I’m sure there is a time and place for dropping packets (traffic policing), but it appears to be not working well for us. If anyone has more input on where this is appropriate or for suggestions of other alternatives, please let me know!