Sidestepping inline URL content filters

Picture the scene – you have a small office network with a single gateway device hooking you up to the Internet. The gateway device performs your basic firewall and NAT tasks, and also hooks into some kind of URL filter server like Websense or McAfee SmartFilter. The URL filter will categorise all websites that people ask for, and will permit or deny access to them according to some defined policy. The trigger for this categorisation is the border device – it must look out for HTTP requests passing through it and act on them like this:

  1. The border device sees an HTTP request passing through it, heading outbound from the office network.
  2. The border device allows the request to carry on to its destination.
  3. When the response arrives, the border device will hold onto it and not release it back to the requester.
  4. Whilst the request is heading off to its destination, the border device parses out the URL and passes it off to the URL filter server (Websense/McAfee/whatever).
  5. The URL filter server will categorise the requested URL, apply a policy to it, and send a yes-or-no response back to the border device.
  6. If the response is a ‘yes’, the border device releases the response held in step 3 and the user will see their webpage.
  7. If the response is a ‘no’, the border device throws away the held response and instead returns an HTTP redirect to the user which will take them to some kind of blockpage telling them why their request was denied.

Organisations usually use this technology for applying some kind of HR-sanctioned policy (no porn, no warez, etc.), but there’s a strong security angle to its use, too.

McAfee SmartFilter has the option to ‘block’ certain categories you choose. It can also ‘warn’ instead of block – the difference is that the user can bypass a warn page, whereas a blockpage is a total dead end. Consider a security-only policy that looks like this and is designed to defend against common web threats like injected hidden iframes etc:

  • The policy will ‘block’ on anything that is categorised as being in any way malicious. Hopefully, if the URL filter vendor is doing their job properly, commonly injected URLs will fall into this category pretty quickly.
  • The policy will ‘warn’ on anything that is uncategorised. New websites turn up every nanosecond, so clearly the URL filter vendors are always playing catchup. In the case of something like conficker, frequent new URLs are part of its C&C strategy, so we have to exercise caution when allowing users to see uncategorised websites.

Why is this an effective strategy? Well, if an injected hidden iframe is categorised as Malicious, the hidden iframe will contain our URL filter’s blockpage instead of the malicious content and the user is saved. Likewise, if the iframe’s content is uncategorised, it will contain the warn page instead of the mischief – again, the user is saved.

Warning on uncategorised as opposed to outright blocking has the happy side effect that the user can proceed to view uncategorised sites that they have explicitly asked for. There’s no way a user can bypass a warn page that’s in a hidden iframe!

Clearly, this strategy isn’t 100% effective – it’s a preventative measure, after all, and prevention eventually fails. You’re primarily at the mercy of the URL filter vendor’s accuracy and timeliness of their categorisations, which can of course let you down. Having said that, from personal experience this is an extremely effective technique of keeping out the majority of today’s web-based attacks on desktop machines.

At this point, let’s think back to step four above:

“Whilst the request is heading off to its destination, the border device parses out the URL and passes it off to the URL filter server”

The key phrase here is “parses out the URL” – the border device actually needs to inspect the traffic passing through it and decide that something is or is not an HTTP request. Clearly, each vendor will have their own ways of doing this, and I can only really speak for Cisco kit here because that’s where my experience lies.

I’ve been finding that certain classes of web-based attacks seem to take measures that sidestep URL content filtering that may otherwise prevent delivery of the attack code to the browser. Whether they do this sidestepping deliberately isn’t something that I can tell you, but it does happen even if it’s just by accident.

The first sidestepping technique is to use a nonstandard port. There have been lots of .cn-targetted iframes injected recently, and their latest trick is to use port 8080 instead of 80. Why is this effective? It’s effective because (for Cisco at least) in order to parse out a URL the border device must recognise the traffic as HTTP, a step typically taken by looking at destination port numbers. A Cisco will, by default, only inspect traffic on port 80, so our iframes targetted at 8080 will not be vetted by the URL filter at all and will get delivered to the user’s browser:

router#show ip port-map http
Default mapping:  http   tcp port 80   system defined

Ooops. To fix this, you can either:

  • Apply some kind of egress filtering so that users can only see a subset of destination ports, or
  • Explicitly tell your Cisco to inspect more ports for HTTP:
router#show ip port-map http
Default mapping:  http   tcp port 80     system defined
Default mapping:  http   tcp port 8080   user defined
Default mapping:  http   tcp port 8000   user defined
Default mapping:  http   tcp port 8001   user defined
Default mapping:  http   tcp port 8801   user defined

Clearly the problem here is that there are over 65000 ports for the baddies to choose from…

The second sidestepping technique is to modify the HTTP verb that is used to fetch the mischief. For example, Waledac will use HTTP POSTs instead of GETs when communicating with its C&C servers. Waledac’s authors probably chose this verb due to the amount of data they want to upload, but it has an unpleasant side effect. Sadly, a Cisco will only pass GET URLs to the URL filter server – POSTs are ignored and are passed through, just like traffic on a non-http port. I don’t think it would take much for some injected javascript on a legitimate page to output a form, POST it to a malicious server and put the response into a hidden iframe…

I have no idea if other inline URL filter devices are susceptible to the same sidesteps, but Cisco ones definitely are. Whether they intended to or not, the bad guys are bypassing yet another layer of preventative defensive measures – it’s yet another thing for us all to look out for!

Comments welcome.

There is an update to this post here.

Alec Waters is responsible for all things security at Dataline Software, and can be emailed at alec.waters(at)

4 Responses to “Sidestepping inline URL content filters”

  1. The strategy we use is Layer 7 filtering. This positively identifies the traffic regardless of what port the traffic is using, then passes it to our URL filtering service (ufbguard with squid), then determines if it’s okay to pass or not.

    Typically we block all outgoing http destined to ports other than 80 unless the site is purposely whitelisted.

    That blocks all of these :80XX and others. We use that for a variety of protocols like IRC, FTP ,etc.

    That’s just my opinion of how it should be done. I could be wrong.

  2. Hi Tom,

    Thanks for the comment, and I completely concur with the soundness of your strategy 🙂 Have you tried the little tests in the followup post, “Test your inline URL content filter”? I’m guessing you got a a urldbfilter hit for all four of them!

    What do you use for classification? It’s clearly a lot better than Cisco’s offering!


  3. We use the open source l7-filter, iptables, some custom rules and a sound strategy.

    We use squid with some finely tuned acl’s that only lets in binaries from whitelisted sources.

    We tried marketing it, but by focusing on IT consultants, it seems they don’t want the lost revenue from cleaning viruses to ever stop!

    That’s just our opinion, but our strategy does work.

  4. Joe Uncool Says:

    Personally, I like the idea of routing NOTHING (well, nothing web-ish) on to the internet, using DHCP option 252 for auto proxy detect & use a good proxy & filter (Squid with a 3rd party filter is good). Those iframes just don’t work if they are not on a port that Squid deems to be safe.

    Likewise I ave also used I7 filter with IPTABLES to trap http traffic on non-standard ports, which works nicely too. I prefer my “foot on the throat” approach, though.

    Of course they also call me the Network Nazi…dunno what that’s about! 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: