Gettin’ hacked

I had an email this morning only five days after my billing cycle started saying that I was close to using my 15GB of data. Normally we spend about 5-10GB, but 15GB in five days is exceptional.

I started digging and after ruling out intentional upload/download from inside my network (using wireshark on my LAN segment) I then started looking at the wireless side of things. I have a server on my LAN that is connected via wireless, and this is also a public facing server with sshd running on it.

I had a quick page through the /var/log/auth.log file and to my surprise I found repeated attempts to log in with various usernames (491 different ones so far) from various locations. These were coming in at the rate of one every four seconds. I can’t see how this would account for 15GB (or 1.5GB for that matter) but when I called Orcon they said they saw a definite increase in traffic over the last two days, which also corresponds to the first entry in the auth.log file.

Here’s what the log entries look like:
Sep 27 13:29:59 pandora sshd[17366]: Failed password for invalid user oracle from 60.190.133.228 port 45662 ssh2
Sep 27 13:30:03 pandora sshd[17368]: Failed password for invalid user michael from 60.190.133.228 port 45857 ssh2
Sep 27 13:30:07 pandora sshd[17374]: Failed password for invalid user ftp from 60.190.133.228 port 46079 ssh2
Sep 27 13:30:12 pandora sshd[17376]: Failed password for invalid user test from 60.190.133.228 port 46301 ssh2
Sep 27 13:30:15 pandora sshd[17379]: Failed password for invalid user webmaster from 60.190.133.228 port 46553 ssh2
and so on …

The requests have come from a range of IP addresses in China, Europe, Canada and Bangladesh. It’s likely to be a distributed and targeted attack.

Interesting. I did install fail2ban some time ago which I was told was supposed to prevent this kind of thing but it was an install-and-forget excercise. I’ll have to research it a bit to find out how it actually works.

categories Uncategorized | GuruBob | datetime September 27, 2008 3:07 pm

5 Comments

  • By Scott, September 27, 2008 @ 4:01 pm

    I thought by default fail2ban would ban after 4 failed attempts for 10 minutes. You could try setting some iptable rules yourself. Seems crazy that ssh login attempts used that much bandwidth.

  • By Guru, September 27, 2008 @ 7:38 pm

    I’m suspecting something else as well – you’re right, SSH shouldn’t use that much, unless there’s some kind of weird attack under way that isn’t resulting in actual SSH failed login attempts.

    I’ve hopefully got a quiet evening at home alone tonight (baby sleep willing) so will have to look closer. Have had another gig eaten up since lunchtime (~6 hours).

    Will have to look at some kind of application bandwidth reporting as well for future interest. Would have been nice to get on top of this sooner.

  • By Guru, September 28, 2008 @ 1:24 am

    I think I’ve found and fixed the problem. The SSH hacking was coincidental and resulted in me finding out a bit more about fail2ban (will have to run ipchains alongside it I think – not currently doing that).

    The bandwidth was eaten up because someone sent me a 16MB email, and my custom delivery solution (home made, probably already exists as part of exim) was choking because the PHP CLI was configured to allow a maximum script size of 32MB. I bumped it up to 64MB and the email came through. I guess my script is holding a copy of the whole message in memory or something (it does some mail header parsing).

    So, it had been trying to download it pretty much constantly for the last three days and failing, then trying again. The bugger about this is that as far as my ISP is concerned this will be chargable bandwidth $#*#@!

    Who sends images as BMP’s anyway?!

    Will keep an eye on it but I think it’s fixed.

  • By Nathan, October 2, 2008 @ 2:21 pm

    In regards to the SSH attempts, you could always try running the SSH daemon on a custom port.

    I used to have heaps of login attempts, and changing the port to something else has eliminated these completely.

  • By oz, January 10, 2009 @ 12:32 am

    fail2ban worked good as gold for me, but yeah you need iptables/ipchains running
    blocks malformed apache attacks too

Other Links to this Post

RSS feed for comments on this post. TrackBack URI

Leave a comment

WordPress Themes