It was a bad start to a Monday morning: I arrived at work to find the intrusion detection system so bogged down in alerts that it was barely responsive.
Something bad had happened over the weekend. The IDS — in this case, a couple of snort sensors logging to a postgresql database — had been extremely busy logging alerts over pretty much the whole weekend. To review the alerts, I used the BASE front-end, and it was this latter that was taking such a long time to tell me anything, since it was querying a database which was around ten times as large as I had originally envisaged using in production.
A few minutes digging in the BASE console suggested that most of the 200,000 alerts had been generated by the potential SSH scan rule from Bleeding Threats. Since the usual daily load was nearer 20,000 alerts, it was a fair guess that a lot of malicious activity had been going on over the weekend. The snort rules that were firing were mainly the latter out of the two shown below, but this would depend on the location of your snort sensor and how you have $HOME_NET and $EXTERNAL_NET defined.
#Submitted by Matt Jonkman
alert tcp $EXTERNAL_NET any -> $HOME_NET 22 \
(msg: "BLEEDING-EDGE Potential SSH Scan"; \
flags: S; flowbits: set,ssh.brute.attempt; \
threshold: type threshold, track by_src, count 5, seconds 120; \
classtype: attempted-recon; reference:url,en.wikipedia.org/wiki/\
Brute_force_attack; sid: 2001219; rev:14;)
alert tcp $HOME_NET any -> $EXTERNAL_NET 22 \
(msg: "BLEEDING-EDGE Potential SSH Scan OUTBOUND"; \
flags: S; flowbits: set,ssh.brute.attempt; \
threshold: type threshold, track by_src, count 5, seconds 120; \
classtype: attempted-recon; reference:url,en.wikipedia.org/wiki/\
Brute_force_attack; sid: 2003068; rev:2;)
For those of you who don't speak snort, this means fire an alert when the rate of connections to port 22/tcp from a particular source goes over a certain threshold. In other words, one of the hosts on site was port-scanning other machines all over the Internet.
This particular site's network had a collapsed backbone topology, which makes traffic monitoring a fairly straightforward affair, as long as you can remember a few IOS commands to set up your SPAN port and have a server capable of dealing with a couple of hundred megabits per second of IP packets. I had also taken the precaution of loading the entire Bleeding Threats snort ruleset onto the IDS and gradually tuned it over the previous few months, so I was fairly certain that I had a real problem at this stage.
Incident Response
One thing that helped immensely at this point was my database of machines, consisting of IP and MAC addresses, host names and the people responsible for each resource. Since we know that a TCP handshake cannot, in general, complete if the source address is forged, I could simply look up the name of the owner from the IP address in the alerts, and then look up the phone number of that person in the internal telephone directory.
The owner was surprised to hear of the activity, so I immediately disconnected the machine from the network. Here, again, it was great to have a database mapping the MAC address of the machine to the particular Ethernet switch it was connected to. This information was updated nightly by a script which walked all our Cisco switches and noted which MAC addresses had been seen on which network interfaces. (If you would like something similar, netdisco is a more polished solution than our home-brewed scripts.)
I then logged into the upstream switch and typed in my favorite, and most used, IOS command:
conf t
interface fastEthernet 0/15
shutdown
I can only recommend you try to build similar databases so you can look up equipment details quickly and easily during an incident. That way, you can find computers which are not where they're meant to be, are spoofing their IP address, or have decided to become an authoritative DHCP server for your whole network.
ไม่มีความคิดเห็น:
แสดงความคิดเห็น