Deep web

:Not to be confused with Dark web
The deep web,http://www.iadisportal.org/digital-library/the-mechanics-of-a-deep-net-metasearch-engine invisible web,https://doi.org/10.1016%2Fj.acalib.2004.04.010 or hidden web are parts of the World Wide Web whose contents are not indexed by standard web search-engine programs.
==Overview==
The deep web sites are not commonly indexed in the search engines. They can be accessed by a direct URL or IP address, but may or may not require entering a password or other security information to access actual content.https://web.archive.org/web/20200301174912/https://www.questia.com/article/1G1-370513892/how-do-you-want-me-to-do-it-does-it-have-to-look (June 8, 2014) Uses of deep web sites include web mail, online banking, cloud storage, restricted-access social-media pages and profiles, and web forums that require registration for viewing content. It also includes paywalled services such as video on demand and some online magazines and newspapers.
==Methods==
Methods that prevent web pages from being indexed by traditional search engines may be categorized as one or more of the following:
*Contextual web: pages with content varying for different access contexts (e.g., ranges of client IP addresses or previous navigation sequence).
*Dynamic content: dynamic pages, which are returned in response to a submitted query or accessed only through a form, especially if open-domain input elements (such as text fields) are used; such fields are hard to navigate without domain knowledge.
*Limited access content: sites that limit access to their pages in a technical manner. Sites may feature an internal search engine for exploring such pages.
*Non-HTML/text content: textual content encoded in multimedia (image or video) files or specific file formats not recognised by search engines.
*Private web: sites that require registration and login (password-protected resources).
*Scripted content: pages that are accessible only by links produced by JavaScript as well as content dynamically downloaded from Web servers via Flash or Ajax solutions.
*Software: certain content is hidden intentionally from the regular Internet, accessible only with special software, such as Tor, I2P, or other darknet software. For example, Tor allows users to access websites using the .onion server address anonymously, hiding their IP address.
*Unlinked content: pages which are not linked to by other pages, which may prevent web crawling programs from accessing the content. This content is referred to as pages without backlinks (also known as inlinks). Also, search engines do not always detect all backlinks from searched web pages.
*Web archives: Web archival services such as the Wayback Machine enable users to see archived versions of web pages across time, including websites that have become inaccessible and are not indexed by search engines such as Google. The Wayback Machine may be termed a program for viewing the deep web, as web archives that are not from the present cannot be indexed, as past versions of websites are impossible to view by a search. All websites are updated at some time, which is why web archives are considered Deep Web content.
Some of the websites categorized here use "robots.txt" to prevent web crawlers from indexing specific pages, media files, and directories.
==References==
Category:Websites