Robot

A Robot scans (or crawls) through all the web pages on the world wide web and complies an index for a search engine. From this index, the search engine (such as Google) delivers search results (SERPs) when people search for something on the Internet. Google's Robot is known as Googlebot. Other examples of search engine Robots include Yahoo bot, and msnbot.

Robots or spiders are programs and/or automated scripts used by search engines to scan the website pages on the Internet. A spider will download a copy of the HTML content (not including graphics) from web sites. The copies are stored and later processed and indexed by the search engine to provide fast searches and up-to-date data.

A Robot is also called a web spider, web scutter, a bot, a crawler, or a web robot.