Different types of navigation can complicate or completely stop the indexing of the site, in particular the materials of your Web site. Bots search engines ignore the Internet, relying on the link structure of sites to find new pages and rescan those that can be changed. Subject link structure and a large nesting level navigation of the site with a small amount of unique content, can be a problem for a good indexing site. Possible problems for the bots in indexing site: * Url address with 2 or more dynamic parameters, ie (spiders can not crawl not simple url address, it is because they sometimes lead to problems with automated crawling) * Pages with more than 100 unique links to various resources of the web site (spiders may not get around them all) * Pages in more than 3 clicks from the root site (if there are many other outbound links, spiders often miss internal page) * The resources from the "Session ID" or Cookie navigation structure * Resources Framed * Resources are available only through the form and pressing * Pages that require a drop down menu (HTML attribute) to access * Pages only available through the search * Page locked (prohibited to indexation) specifically (using the meta tag or file robots) * Documents requiring a login (the entrance to the closed part of the site) * Pages with redirection (call it cloaking the search engines, for this happens and sanctions) to index the site and its contents were completely scanned should organize direct links to each page to be indexed, search engine bot. If a page is not available from the home page (where most of the robots can start indexing), it is likely that these pages will not be indexed in search engines. And yet, sitemap will definitely help in indexing the site.