Tonight at 8:45 our main server load alarm tripped, on Nagios, at Beyond Hosting. By the time I was able to ssh in, load had exceeded 220.64 and the box was headed straight for kernel panic, this is a “dual core”. Quickly throwing the IPTables firewall onto drop all I was able to prevent the box from locking up, after a few minutes (takes a little to recover from the load) of checking netstat and Apache logs we were able to narrow the attack down to 2 separate ip.
Simply added blocks for the 2 addresses with:
iptables -I INPUT -s x.x.x.x -j DROP
I played with apache mod_evasive a little bit during the attack, pretty solid addon and I highly recommend it. We tried out (D)Dos deflate but no matter what settings you put it on, it seems to think you have 2500+ connections from 1 IP and black list everyone.
You can view how many connections you have on your server by running:
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n