How to ban traffic from certain referrers from your website

We all know the situation – you have a high content website that is getting pounded day in and day out either by somebodys ‘bot or because somebody is trying to cheat your referrer mechanism/top list or put you out of business. Often you find your page is in an invisible iframe someplace. Its a difficult one because you dont want to preclude genuine traffic but you cant afford the 5 gigs of bandwidth these guys are costing you.

Well here is a simple solution for those whose sites are on apache servers.

Step 1:

First Make a little list (or not so little) of the cheaters/link spammers refwerring url’s (you can get this from your stats)

Step 2: if you dont already know it find out the absolute path to your home directory – you can do this by creating a php file
with just four lines in it and running it from your browser:
$ path=$ _SERVER['DOCUMENT_ROOT'];
echo “($ path)”;
?>

You should see something like (/var/www/yoursitename/home) which is the ‘real’ path to your files.

Step 3 open your existing htaccess file or make a new one (must be named .htacces thats dot or period htaccess with no extension)

Options +FollowSymlinks
RewriteEngine on
RewriteMap deflector txt:/path/to/deflector.map

RewriteCond %{HTTP_REFERER} !=”"
RewriteCond $ {deflector:%{HTTP_REFERER}} ^-$
RewriteRule ^.* %{HTTP_REFERER} [R,L]

RewriteCond %{HTTP_REFERER} !=”"
RewriteCond $ {deflector:%{HTTP_REFERER}|NOT-FOUND} !=NOT-FOUND
RewriteRule ^.* $ {deflector:%{HTTP_REFERER}} [R,L]

If you already have the first two lines then you dont need to add them again. Replace path/to/deflector.map with your absolute path plus the name of a text file we are just about to create. you can call it anything you like but if it were called ‘shitlist.map’ then that part of the line might look like:

RewriteMap deflector txt:/var/www/somesitename/home/shitlist.map

Step4: Now we create our final file – this decides who gets to be deflected elsewhere and never see our site or drain our bandwidth. this file will look like this:

##
## shitlist bad request deflector
##

http://www.bandwidthleech.com/index.html -
http://www.cheater.com/index2.html -
http://www.havingajoke.com/index.html http://eclectech.co.uk/mindcontrol.php

Any line that starts ## is a comment. Then you have a list of the URL’s you want to block traffic from. If you put in a dash (-) as the second place then the page gets redirected to itself then you then itself….until the surfers browser detects it. In other word it kills his bandwidth not yours! If you put a new url instead of the dash then the traffic goes there instead. I like redirecting to the tin foil hat song at eclectech because when the music starts I can imagine puzzled surfers on pages with embedded iframes wondering what the hell is going on!

Now you might say that you can use a framebreaker script on iframes to avoid framing, however many traffic purchase schemes use frames (not iframes) to present your content and wont allow them. Google does not seem to be fond of them either because it breaks their ‘cached’ page view. In addition in Explorer there is a javascript about that stops framebreakers from working!

I should mention there is another use for this. You may want to present different content depending on the referrer – this allows you to implement referrer based redirect just put your own pages on the rhs in the map file.

Thats all folks!