Search Engine Spiders
When you submit your site to the search engines, the search engine server – commonly referred to as a “robot” or a “spider” – will send a request to your website either immediately or within a few weeks to index your website content. This indexing of the website is also commonly known as “spidering”. Therefore, after you submit your site to search engines, a search engine robot will spider your site.
The indexed data will get stored in a database and the data will be displayed when searches are made on the search engines. Most robots will only index text or HTML based pages. Robots cannot read text in graphics. Therefore, it is very important to have some text on your page, which includes your keywords, if you want your page to be found in the search engines.
For those that have their own servers or access to log files on their site statistics, you can easily detect if a robot has indexed your site. The search engine robots have unique IP addresses that can be detected.