yukes.com >> Lazy Jay

Let’s get technical again…

I’m often surprised at how little depth some sysadmins have.  And even more so, when they have no ambition to learn how to code.  The best sysadmins I know can grok code — even complex code — and better when they can produce it too.

If you don’t code, you’ll never get deep enough in the stack — and understand the whole loop — start to finish.

I remember a situation where we needed to transfer large amounts of data… 100GB – 1TB database backups.  The original solution involved a huge tarred file and SSH/SCP… far from optimal.  SSH requires TCP, which is problematic for high performance… even with super dialed-in OS-level tuning.  With SSH and TCP we were seeing 100-150Mbps performance on the local network.  Turning encryption off helps a little, but I think there is a window / buffering issue that limits overall performance.

Many suggest rsync.  But in my experience, rsync never performed much better…

Searching for alternatives, I hit up some buddies…  one guided me to “tar pipe” which is tar plus netcat. This solution destroyed SSH/SCP — ringing in TCP results of 850Mbps and UDP transfers up to 940Mbps.  But it wasn’t perfect, netcat isn’t super fault tolerant… and there exist some reliability issues.

Another buddy suggested UDT “UDP-based Data Transfer” which was more reliable than tarpipe, and performed the best… reaching 950Mbps.  Basically Gigabit thoroughly maxed out.  Not much more to optimize!

UDT was a huge winner… 80-90% speedups.  Effectively reducing the amount of time to restore a database backup in our organization from hours to minutes…

And time is money… check it out!