Just been writing a Perl script to leverage the Google engine. When I do a request to Google using the URI http://www.google.com/search?q=searchTerm with searchTerm equal to whatever, it fails with a 403 Fobidden error.

For me, this was actually due to the user agent. LWP identifies as libwww-perl/#.##”, where “#.##” is substituted with the version number of the library. Google doesn’t seem to like this. I guess it’s a simple way of detecting scripted attacks.

Anyway, the simple work-around is to change, or even just remove, the user agent string:

$ua->agent(‘Mozilla/5.0′); # Identify as Mozilla
$ua->agent(“”); # don’t identify

Either of these work for me. I personally did not identify it, so that Google does not try anything browser-specific in the response

On a side note, I change the logo for the site in the top right last night…looks much nicer hehe