A guide to Google's different spiders

[mandy]

From maryrussel's iwriteforu blog: article covers googlebot, mediabot, imagebot, and adsbot:

Google currently indexes over 8 billion web pages. However, before these pages were placed in the index, they were each crawled by a special spider known as the GoogleBot. Unfortunately, many web masters do not know about the internal workings of this virtual robot.

In fact, Google actually uses a number of spiders to crawl the Web. You can catch these spiders by examining your log files.

This article will attempt to reveal some of the most important Google spiders, their function, and how they affect you as a web master. We'll start with the well-known GoogleBot...

full article: http://blogs.allmusicsearch.com/allmusic...ey_do.html