Reporting bit.ly URLs

One of the emmergant email spam problems is URL shorteners. One of the biggest shorteners on the market today it http://bit.ly. At $WORK we receive continuous reports of spam messages, and many of the spam complaints contain these url shorteners in order to bypass many blacklisted URLs.

Bitly allows registered users to submit up to 500 malicious bitly links at a go, but before we submit our links let’s check those links against two major domain name based block lists.

First, we resolve the bitly links into target URLs:

perl -lane 'print $1 if /(http:\/\/bit\.ly\/.*)/' bitly-complaints | while read link; do echo "$link => $(wget -S -O /dev/null $link 2>&1 | grep Location: -m1)"; done | tee bitly-complaints.resolved

This will generate output like:

http://bit.ly/hglCco => Location: http://eknl.elezuniz.com
http://bit.ly/gm5mH3 => Location: http://xXhL.ubavoram.com
http://bit.ly/i9dOPg => Location: http://wptiz.elupudah.com
http://bit.ly/gEB3zU => Location: http://R5yfk.ubavoram.com
http://bit.ly/feecBB => Location: http://itt.exqfevur.com
http://bit.ly/i82F1i => Location: http://wodg.igqcybap.com
http://bit.ly/f211P2 => Location: http://rhv.qlapynqd.com
http://bit.ly/hlMwTD => Location: http://cuyl.elupudah.com

Now take the domain name on the right hand side, and hit it against the blocklists:


perl -lane 'print $1 if /Location:\s+http:\/\/(.*)/' bitly-complaints.resolved | while read x; do echo "$x => $(dig $x.multi.surbl.org +short a)"; echo "$x => $(dig $x.dbl.spamhaus.org +short a)"; done | grep 127 | awk '{print $1}' | sort | uniq > bitly-complaints.bad

This will check the domains against both SURBL and Spamhaus’ DBL. If the address is on either of those blacklists, then it gets sent to the stdout.

We now grep against bitly-complaints.bad:

fgrep -f bitly-complaints.bad bitly-complaints.resolved | awk '{print $1}' | sort | uniq

Take the list of links (up to 500 at a time) and paste them into Bit.ly’s report spam url.