glTail parsers for Snort, net-entropy and viewssld

glTail is a tool for realtime log visualisation, which according to the website allows you to “view real-time data and statistics from any logfile on any server with SSH, in an intuitive and entertaining way.”

glTail can read from any text logfile you like, and via a set of parsers can extract information such as IP addresses for graphical display. Each row from the logfile may trigger several blobs, e.g. source IP, dest IP, etc, as you can see in the video below:

I’ve written some parsers for Snort, net-entropy and viewssld. A screenshot of them all in action is shown below (click for full size view):

The red blobs are related to Snort, cyan ones to net-entropy, and the yellow shades are from viewssld. The numeric columns show the rate at which each item is appearing, and the length of the coloured highlight bars show the proportion of occurences of a given item relative to the others.

The parser files and a sample config.yaml file that uses them can be found here (snort.rb, net-entropy.rb, viewssld.rb and config.yaml).

Useful?

So, it’s a pretty visualisation of interesting stuff, but is it useful and actionable? It’s certainly hopeless for correlation – when a signature fires, it’s more or less impossible to tell the associated IP addresses and ports even if you have a very quiet sensor. At the other end of the scale, if you’re inundated with blobs you can alter the regexes in snort.rb to match on a specific IP/protocol/signature etc to be a little more selective.

Where I think this may prove most useful is when you’re learning from an incident. If you’ve investigated an incident where someone compromised your webserver, you could pull all the relevant log entries that show:

  • Snort alerts (when the attacker was probing for vulnerabilities)
  • Apache/IIS log entries (showing everything else they did to your server)
  • net-entropy logs (showing the attacker’s outbound backdoor SSH tunnel).

If you were to pump all of these logs through gltail you’d have an effective visualisation of the attack. For inspiration, check this out:


Alec Waters is responsible for all things security at Dataline Software, and can be emailed at alec.waters@dataline.co.uk

About these ads

7 Responses to “glTail parsers for Snort, net-entropy and viewssld”

  1. Would you mind if I included your parsers with glTail? The more the merrier. :-)

    glTail was originally made on a challenge from my boss at the time, that it would be pretty much impossible to visualize what actually happened on our servers.

    It’s mostly useful as a quick overview of what kind of traffic is passing through all your servers ‘right now’, and if something out of the ordinary is going on. For instance a lack of blobs. Or mostly really large ones when you’re used to small ones (we had response time as blob size).

    Hooking it into the sales log was always fun. Automatic bestseller lists, automatic $’s / hour, etc. Management was usually glued to that display whenever we did a change to see how it worked out. :-)

    • Hi Erlend,

      It’s fine to include the parsers with glTail :) Like you say – the more the merrier!

      glTail is a really nice piece of work, flexible and extensible enough to allow you to use your imagination. It’s a lot of fun, too :)

      alec

  2. [...] glTail parsers for Snort, net-entropy and viewssld (wirewatcher.wordpress.com) [...]

  3. Nicholas Borror Says:

    I realize its been a over a year but since this is the only Google result for gltail and snort I’m giving it a shot. I really want to thank you for writing a snort parser for gltail!

    I seem to be having issues getting it to work. Can you tell me what snort log output are you using? Alert_fast seems to be closest to your regex but the beginning doesn’t seem to match what I have for mine. Perhaps we are on different versions of snort, do you know what version you were using at the time? A line from your snort alert log would be very helpful so I can see if it matches. Here is one from mine.

    01/04-15:55:30.980293 [**] [1:1000003:1] Unauthorized Traffic on Port 23 Custom Rule [**] [Priority: 0] {TCP} 12.119.62.233:57128 -> 12.121.123.44:23

    • Hi Nicholas,

      Have you tried experimenting with uncommenting the sigid and priority lines in the snort.rb file? The regex is also looking for the string ‘snort’, which isn’t present in your output.

      Take a look in the config.yaml file on the Supplemental Files page (same place you downloaded the .rb files). This shows that I’m taking the somewhat blunt approach of dumping everything into /var/log/syslog and parsing that.

      hth,
      alec

      • Nicholas Borror Says:

        Thank you for the timely reply. Looks like someone will be brushing up on the regex! Thanks for your assistance!

  4. Hi Nicholas,
    I have the same log output. Can you post your snort regex here ?
    Thank you!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 28 other followers

%d bloggers like this: