Combating bots: Anti-Botnet software versus IDS, flows, and other methods

Ryan Naraine's Eweek article titled "Growth of Anti-Botnet Startups Points to AV Deficiencies" got me thinking about how I and my peers handle botnets. Naraine cites Andrew Jaquith from the Yankee Group who said "It's not a good thing that security products are failing and not catching all the threats. The fact that there's a perceived market need [for anti-botnet protection] is an indictment of anti-virus companies in general".

Most security professionals I know don't expect our antivirus to catch botnets - we expect antivirus to catch viruses, and some malware. We expect some of the nastier pieces of spyware to be caught, but not all of it, particularly things that have some level of usefulness to users. Do we truly expect our antivirus to catch botnets reliably? I don't think so. Antivirus doesn't have the network-centric viewpoint that can help catch bots, and it doesn't have the wide ranging global data to identify botnet controllers. Heuristics might catch bot-like behavior, but bots aren't viruses. They're malware, they're often packed in with trojans and remote control tools and all sorts of other things - and they may have their path opened by a virus or other nasty, but at the end of the day, they don't behave like a virus does.

I've seen AV regularly detect chunks of the toolkit dropped onto a bot'ed host. That's where central AV management and awareness of what the threat looks like is crucial. Is that something AV can handle? Not in a traditional AV environment. In a unified security reporting environment, with links into the network, endpoint hosts, and the smarts to put the data from multiple inputs together it may work.

So are AV companies double dipping? Many already charge for spyware definitions on top of AV definitions. Naraine again cites Jaquith - "Ultimately, it's hard for an enterprise to justify paying twice for botnet protection when they're already paying for anti-malware protection". The question becomes - is the expectation that anti-malware will protect your hosts from all host based threats?

I don't think that front line analysts expect that - yet. Will we at some point? Yes, as AV suites and anti-malware software grow into a coherent product, and as they become more aware of their environment, our understanding of what a competent, complete tool is will change. For now, we'll see the same things that we did with anti-spam and anti-spyware tools. They'll remain separate, and can be deployed in a mix and match mode for those who need them. Eventually - as we've seen with anti-spam technologies - they will enter most mainstream products, and specialized tools will remain available for those with specific needs or more power.

Knowing that botnet detection tools are just starting to be commercialized, what options do you have right now? The good news is that a lot of the tools you need are likely already in place, or are easily accessible. Here's a quick overview:

Traditional AV

May detect some bots, or components of botnets, but central reporting is necessary to get a big picture view. Some AV also includes the ability to block some outbound traffic such as outbound IRC traffic. This can help stop systems from joining the botnet - you may have a compromised machine that just won't phone home.

Traditional AV is a great first step if you are getting useful data from it. If it only protects endpoints and doesn't contribute to your overall awareness, you're missing out on functionality, and you'll miss out on chances to see when something new hits you.

IDS/IPS

Installing an IDS on your outbound link can be a great way to detect botnet traffic. Knowing what you expect to send out, and watching for traffic that doesn't match - IRC traffic from a server, or http traffic to many hosts in quick succession, or any of a host of other things that you're used to seeing coming in as attacks can be a good indicator of a compromised host. As botnets move to encrypted HTTP communication, you may not be able to see what the traffic is - but the attacks and other actions are likely to still trip your sensors.

Flows

Flows are a great tool when combating botnets. A simple filter can help catch new outbound flows, and watching for flow patterns associated with DDoS attacks and other outbound traffic can help you pin bots down quickly.

Flows are also useful when looking for other compromised hosts. Often identifying a single host and matching what it does to other can quickly show you all the hosts in your network that have been compromised with the same package.

External Reporting

Reports from third parties and organizations such as ISACs can be invaluable. While it is poor practice to rely on third party notices as your sole source of information, ignoring reports is not only bad net citizen ship, it can be outright dangerous. Check to see if your organization has access to an ISAC or other peer group that might feed useful data to you from an external perspective.

Future Issues and Direction

Much as we have seen in the market as the major antivirus companies have added anti-spyware capabilities, we will likely see the major vendors acquire anti-botnet technologies to add to their stable. For now, those products are likely to be stand alone, but progress should lead to the capabilities being added to edge devices and security appliances. We may even see anti-botnet capabilities added to enterprise class desktop security suites - monitoring of outbound traffic via host IDS/IPS and firewall capabilities pushes extrusion detection to the endpoint, and will provide a more granular security environment.

Will we see the smaller independent vendors with good products acquired? Will they lose their edge if they are? Time will tell, but my feeling is that botnet detection technology growth will continue to mirror the development cycle of other security products in the market.