Sunday, January 4, 2009

Useful Tips and Tools to Research the Deep Web

Alisa Miller's recently wrote on Online College Blog that "experts say that typical search engines like Yahoo! and Google only pick up about 1% of the information available on the Internet. The rest of that information is considered to be hidden in the deep web, also referred to as the invisible web." According to DeepWeb.com, a study by Bright Planet indicates that the deep web is estimated to be up to 550 times larger than the 'surface web' accessible through traditional search engines and over 200,000 database-driven websites are affected by the problem.



Alisa provides a list of 100 tips and tools to help get the most out of searching the Internet. I have regularly suggested to my students that they use a meta-search engine such as Dogpile or Metacrawler. This is Alisa's first suggestion. She provides a list of 12 different engines to try. UC-Berkley in Finding Information on the Internet: A Tutorial suggests that you create your own custom search engine. Google Custom Search Engines (CSEs) focus on selected websites within the Google database. They are easy to make at Google Coop. You will need a Google account or Gmail account. Make specialized search engines instead of using giant meta-searchers or huge search engine databases. Use them to focus on pages on a subject. For more details, see their Getting Started Creating a Custom Search Engine (PDF).




Of course, as a Ph.D. student, I've found that using specialized databases such as ProQuest, EBSCOhost, and more to be helpful. One of my favorite (fee charged) database is
Questia where you can not only access journal, magazine, and newspaper articles but also access whole books.

Unfortunately, many of the databases are just that--fee based. However, Google has worked with many publishers to gain access to some material that wouldn't ordinarily be accessible to search spiders because it is locked behind subscription barriers. This information is available through Google Scholar. Google also offers Google Books. If you want to check out a book before you buy, this is the way. One drawback--in order to comply with copyright laws, Google can't display the whole book. Whole sections of the book will be unavailable. Joel at BizzNTech provided 15 Handy Google Search Tricks.

6 comments:

  1. Thank you for writing a very helpful guide for newbies like me. While I’m not yet ready to buy something from the darknet, do you know if I can securely browse through a site? Can I be flagged, or monitored, just by visiting a darknet site? Dark Web vpn for torrenting

    ReplyDelete
  2. "Really interesting! I've heard of the dark net but thought that only really savvy computer people could access it. Thank you so much for the information. I'm going to have a look, even though I don't think I will be able to find the interesting stuff like subversive political writing from people in repressed countries. I'll have fun looking though!
    "

    ReplyDelete
  3. I thought this was interesting. At first I thought it was about something like witchcraft, etc. I would like to find out more about it. I wonder why it is not accessed on the regular web.
    Joseph Donahue

    ReplyDelete
  4. I am a nooblet to the darknet too, have only just found out about it by reading this hub. Thank you for your explanation of what the darknet is and how to access it. Will have to check it out.
    CrazyAsk

    ReplyDelete
  5. I've never heard of it, but can understand why it is needed by certain factors. This is interesting information.
    Kelly Hubbard

    ReplyDelete
  6. I am a nooblet to the darknet too, have only just found out about it by reading this hub. Thank you for your explanation of what the darknet is and how to access it. Will have to check it out ;)

    Harold Burton

    ReplyDelete

Thanks for your comments.