Googlebot

The Googlebot is Google's webcrawler that is used to add new sites to Google's index. Read how to verify a visit really came from Googlebot. Google Webmaster Tools can fetch pages as if Googlebot had crawled the website.

Google App Engine
Google App Engine applications can use the infrastructure Googlebot uses by using the URLfetch service.