Sunday, January 4, 2009

Useful Tips and Tools to Research the Deep Web

Alisa Miller's recently wrote on Online College Blog that "experts say that typical search engines like Yahoo! and Google only pick up about 1% of the information available on the Internet. The rest of that information is considered to be hidden in the deep web, also referred to as the invisible web." According to, a study by Bright Planet indicates that the deep web is estimated to be up to 550 times larger than the 'surface web' accessible through traditional search engines and over 200,000 database-driven websites are affected by the problem.

Alisa provides a list of 100 tips and tools to help get the most out of searching the Internet. I have regularly suggested to my students that they use a meta-search engine such as Dogpile or Metacrawler. This is Alisa's first suggestion. She provides a list of 12 different engines to try. UC-Berkley in Finding Information on the Internet: A Tutorial suggests that you create your own custom search engine. Google Custom Search Engines (CSEs) focus on selected websites within the Google database. They are easy to make at Google Coop. You will need a Google account or Gmail account. Make specialized search engines instead of using giant meta-searchers or huge search engine databases. Use them to focus on pages on a subject. For more details, see their Getting Started Creating a Custom Search Engine (PDF).

Of course, as a Ph.D. student, I've found that using specialized databases such as ProQuest, EBSCOhost, and more to be helpful. One of my favorite (fee charged) database is
Questia where you can not only access journal, magazine, and newspaper articles but also access whole books.

Unfortunately, many of the databases are just that--fee based. However, Google has worked with many publishers to gain access to some material that wouldn't ordinarily be accessible to search spiders because it is locked behind subscription barriers. This information is available through Google Scholar. Google also offers Google Books. If you want to check out a book before you buy, this is the way. One drawback--in order to comply with copyright laws, Google can't display the whole book. Whole sections of the book will be unavailable. Joel at BizzNTech provided 15 Handy Google Search Tricks.

1 comment:

  1. Thank you for writing a very helpful guide for newbies like me. While I’m not yet ready to buy something from the darknet, do you know if I can securely browse through a site? Can I be flagged, or monitored, just by visiting a darknet site? Dark Web vpn for torrenting


Thanks for your comments.