The term "invisible web" or "deep web" refers to the vast repository of information that search engines and directories don' have direct access to, like databases at university libraries, sites that require passwords to view, or sites that for some reason don’t want search engines to crawl them. Unlike pages on the visible Web (that is, the Web that you can access from search engines and directories), information in databases is generally inaccessible to the software spiders and crawlers that create search engine indexes.

I knew that there was a very large amount of information out there that was not covered by Google or any other search engine, but I was very surprised at just how much!

Think of it this way: Google, considered by most people in the know to have the largest search database, has about eight billion pages in its index. Those eight billion pages seem like a lot until you consider that the Deep Web is estimated to be 500 times bigger than the searchable Web. Multiply 500 by the 8 billion in Google's index - plus add in the fact that Google is only indexing a fraction of the searchable Web (around 250 billion pages are on the Web today)...

WOW! That just blows me away! But the next question, is, of course, how do we access that hidden web...just because it's hidden from the search engine algorithms does mean it's not out there for us to discover:

There are sites that serve as invisible Web "gateways" that will help you do this. Here are just a few:

• Invisible Web: Gary Price and Chris Sherman have put together a searchable directory of various databases on the Invisible Web.
• Librarians Index to the Internet: A directory of various sites on both the visible and invisible Web put together by librarians; all are reviewed before inclusion and have the Librarian Stamp Of Approval.
• GPO Access: An amazing site. You can search hundreds of US Government databases at the same time.
• WebLens Scholarly and Academic Research Resources: Includes databases as diverse as the Stanford University High Wire Press Archives, a searchable index of past issues of the New England Journal of Medicine, and the Smithsonian Institution Libraries, a “galaxy of knowledge.”

As an information junkie, it really makes me ponder how my academic life would have different if I had had such easy access to this information when I was younger. I was the way-too-smart-for-my-own-good kid who spent countless hours at the public library. Reading anything I could get my hands on, turning the filmstrip cubbies into an early Mystery Science Theater 3000 with fellow smarty-pants friends, and begging the staff (who grew to know me quite well) to please let me check out foreign language tapes. In my nine-year-old mind, it was very important that I learn to speak Italian. Because there are so many Italians in North Dallas, I guess.

It was a wonderful way to pass the time, and back in those days my mother thought nothing of dropping me off to spend an entire day there. And she almost never argued about the trunk full of books I inevitably carried home, or the countless fines that I owed on a regular basis for inevitably returning things late.

All good preparation for a history degree from the finest such program in the country, especially when I finally obtained a coveted stack pass for the main collection at Doe. It didn't happen until my senior year at Cal, when I was writing my thesis under the supervision of the incredible Lawrence Levine.

But it was worth the wait...library geek nirvana! That semester I spent incredible amounts of time researching primary sources in the dusty and claustrophobia-inducing stacks, playing with years of special-order microfiche, and discovering the other treasures of the campus library system, like the Bancroft Library.

It's fun to think about what I could have done if I had had these resources at my fingertips as a child...but I wouldn't trade my all-day library marathons or Doe-stack-induced asthma attacks for the world. Those experiences gave me the appreciation for get-your-hands dirty research that I can carry on as I dig through the Deep Web with a primal library geek joy.

deep web, invisible web, cloaked web, library geek