Just blocked a new bot/crawler called Topsy. On their webpage, they pretend to be cool and tell you just exclude them with robots.txt. But this is how they crawl you:
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1"
Does that look like a bot to you? Nah, me neither. Uncool. I blocked their IP (208.74.66.43) and will add the subnet if necessary.
No comments:
Post a Comment