2009
07.30

On Wednesday, I rolled out of bed late and had a voicemail from my friend. My buddy got comp’d a fat cabana over at Red Rock casino for their Wednesday pool party. Nice! Here’s how the day was gonna look:

Before heading to the pool, I figured I should put in a couple minutes of work. I logged in to quickly check some stats, and noticed one of my older web servers was in trouble.  I logged in and noticed that Apache httpd was using all the CPU — I was receiving a LOT of traffic!

Poking around – I got netstat telling me there’s a shit load of connections… Sweet!

# netstat -ntu | grep :80 | wc -l
318

Did I make front page on Digg?
Am I getting Slashdot’d?
Hmm… Looking deeper at the traffic source:

# netstat -ntu | sed -e 's/::ffff://g' | awk '{print $5}' | cut -d : -f1 | sort | uniq -c | sort -n
...
33 77.220.186.142
35 77.220.186.143
40 77.220.185.143
53 188.40.47.196
...

Fuck… some asshole was hitting my website from as many as 10 hosts across Germany and Russia! And they’re winning! My server had slowed to a crawl — rendering the website useless.

I’m supposed to be at a cabana at Cherry pool in Vegas — throwing down cocktails with friends! I don’t have time for some Euro trash hackers!

I quickly add some firewall rules to drop the heaviest offenders:

# /sbin/iptables -I INPUT -s 77.220.180.140 -j DROP
# /sbin/iptables -I INPUT -s 188.40.47.240 -j DROP
# /sbin/iptables -I INPUT -s 77.220.185.142 -j DROP
# /sbin/iptables -I INPUT -s 77.220.186.140 -j DROP
...

All said and done was back up in just 15 minutes… and off to the pool!


For more reading on the subject: http://deflate.medialayer.com/

And when I have a minute, I’m definitely gonna install mod_security http://www.modsecurity.org/