Archive for 29 May, 2009

Collection is King, part one

Posted in NSM on 29 May, 2009 by Alec Waters

So we’re now at the point where a SPAN port is spewing mirrored traffic at our sensor. What do we do with it?

Well, we’re going to apply a methodology to it (I’m deliberately avoiding mentioning any specific products here – security is a process, not a product). The methodology in question is called “Network Security Monitoring”, or just NSM for short.

NSM wasn’t my idea, nor do I claim to be any kind of authority on the subject. I do, however, practice it every day, and I can tell you that it works. One of the foremost NSM authorities is Richard Bejtlich, who has an excellent blog and has written some fine books. I would strongly recommend reading them.

Richard defines NSM as the “collection, analysis, and escalation of indications and warnings to detect and respond to intrusions.” It sets out to deliver to you, the analyst, the following four categories of network-based information:

  1. Alert data. These are the alerts raised by a detection device, such as an IDS. Alert data by itself doesn’t really go far enough though – it isn’t possible to validate a security incident based solely on an IDS alert. We need more.
  2. Session data. This is the networking equivalent of an itemised telephone bill. It shows you who spoke to whom, at what times, and for how long (expressed in terms of IP addresses, protocols, ports, byte volumes, etc). It won’t tell you what was said, but it will at least tell you that a conversation took place. By mining session data, you can find out which hosts are busy sending and receiving lots of traffic, and also what kind of traffic that was based upon the protocol and port (FTP, HTTP, etc).
  3. Statistical data. Given our collection of session data, we can build baselines of “normal” usage and report on anything that deviates from this baseline.
  4. Full-content data. These are the actual packets themselves, and their content. The full-content data can be mined to extract things like URLs requested, emails sent/received, IM chats, FTP sessions. etc.

Given this lot, you can go a great deal further than you could if you only had an IDS. A trigger for an investigation could be:

  • An IDS alert. If you’re practicing NSM, you can:
    • See the alert in the context of the session that carried it. For example, if an IDS alert fires because it has spotted an exploit URL being requested of your web server, an NSM practitioner will be able to see the web server’s response to the potentially harmful request. If the web server responded with a 404 “not found” error, for example, then the attack was unsuccessful.
    • Investigate the attacker’s precursor activities by leveraging the session data. Has the attacker been here before? Has he conducted reconnaissance? Has he tried many different attacks? Or perhaps the same attack over and over?
    • See if the victim machine exhibited any unexpected behaviour during the time after the IDS alert was raised. Post compromise, an attacker will often download tools to consolidate their position – traces of this will be present in our session and full-content data.
  • A statistical alarm. Has our mailserver started sending more mail than usual? Are there an unusual number of connections to or from our public web server?
  • A report based on session data. For example, a public facing web server should be busy serving web pages to the public. If we can filter these out of our session data, we are left with things like:
    • FTP sessions where new content is uploaded
    • The server’s own hits on auto-update services, like Windows Update.
    • Anything else! This is where you’ll see traces of an attacker downloading their tools, or malware communicating with a command and control server.

An analyst practicing NSM is clearly well equipped to detect and investigate the Badness. Next time, I’ll show a practical application of the NSM methodology.

Alec Waters is responsible for all things security at Dataline Software, and can be emailed at alec.waters(at)