In general terms, when a search engine has been made aware of the existence of a website, they will often send their “robot” to “crawl the site. The robot follows links on the site (usually from the home page) to lower level pages on the site. It catalogues keywords and other information on the site and stores this information in the search engine’s database. This information then determines the relevancy of the website in the event particular terms are used in the search engine’s search function.