Now that we’ve looked at the basics of searching, we need to talk about where to apply these searching skills.
According to one estimate, there are more than 1500 search services on the World Wide Web, and it seems there is a new one announced every day. If you count the number of sites which compile directories or lists of links to web sites there would be tens of thousands of web site finding aids.
Most of these search services are free to the user. They make their money through selling ads or offering additional premium services. Some charge web sites which want to be listed in their directory or database, and web site administrators pay because they want people to find their sites.
These tens of thousands of web site finding aids break down into eight types.
- Human indexed — omni guides
- Human indexed — subject specific
- Spider / robot indexed — omni guides
- Spider / robot indexed — subject specific
- Non-web resource search sites
- Meta search – omni and subject specific
- Web rings
- New types of search sites
The lines between these slices can be pretty fuzzy. Some robot sites for web searching also have human indexed guides (i.e. Go: http://www.go.com). Some robot sites will search for web pages and for other Internet protocol items, like newsgroup messages or ftp files, or search in archives of articles (i.e. Northern Light: http://www.northernlight.com).
You need to understand how the different types of search sites were created, where the database of information about web pages and sites comes from, the range, and limitations of the different databases.
You also need to understand that it is impossible to understand the ins and outs of all of the major search services. My advice is to think of search services like I think of friends. It is better to know and understand a few really well so you can depend on them than to know many but only superficially.
So, go visit a number from each of the following categories, but select just a few you really want to get to know.
In this chapter we will look at the eight different types of web search sites and finding aids and discuss their general characteristics. We will also look at the kinds of differences to look for between the sites that fall into that category. I will not try to profile individual sites. There are some great books and web sites that do that (see bibliography in the margins). I will list 5-10 of the sites that fall into that category so you can make some visits and look them over yourself.
For each of the search site categories we’ll look at:
- What they are and how they are created: An overview of the process for getting information about web sites into this kind of service and how the service works
- How much of the web they index and turn-around time: A very rough idea of the scope of their indexing of the 320 million or so indexable pages on the web (number of indexable pages is from a Science magazine article, April 1998) and of the frequency of updating the database with new entries or validity checks of entries currently in the database. Most of the figures used in this section of the profiles comes from Search Engine Watch, the most comprehensive guide to the who, what, when, where, whys and hows of search engines: http://www.searchenginewatch.com This type of information is the hardest to be exact about because the competitive nature of the search site business makes this strategic information.
- When to use them: A note about the usefulness of this particular type of search site for different research needs.
- What you are searching in the database: What are the actual contents of the database you are searching on the site, how much and what type of information is there on the entries about web sites or pages in the database.
- Strengths: What is particularly good about this type of search site.
- Weaknesses: What to look out for when using this type of search site.
- Other features commonly offered: What else you can do on these kinds of sites.
- Examples of sites: Names and web addresses of 5-10 of these types of sites.
- Exercises: A few sample questions to try your hand at using these types of sites (refer back to the searching basics chapter for more about specifics of searching).
