PDQ Library:  Deep Web

Deep Web - what is it?

Deep Web sites store their content in searchable databases that only display results in response to a request - then a Web page is generated on-the-fly to display the results of your request. Ordinary "spider" indexing of surface Web sites, by search engines like Google or Altavista, misses Deep Web information, which far exceeds the surface Web sites we are more familiar with.

Public information on the Deep Web is currently 400 to 550 times larger than the commonly defined World Wide Web.

Completeplanet.com has information about Deep Web sites which indicates that just 60 of the known 100,000 Deep Web sites contain about 750 terabytes of data (including HTML code) in about 85 billion records or documents, or roughly 40 times the size of the known surface Web. These sites appear in a broad array of domains from science to law to images and commerce.

On average, Deep Web sites receive about 50% greater monthly traffic than surface sites and are more highly linked to than surface sites; however, the typical (median) Deep Web site is not well known to the Internet search public.

The Deep Web is the largest growing category of new information on the Internet. Deep Web sites tend to be narrower in focus, but with deeper content than conventional surface sites. Total quality content of the Deep Web is much greater than that of the surface Web, and many Deep Web sites require a fee for access. Deep Web content is highly relevant to every information need, market and domain, with most sites containing topic specific databases.

TOP back