Share to facebook

You can search for Web sites at many places. Literally thousands of sites, in
fact, provide the ability to search the Web. (What you may not realize, how­
ever, is that all these sites search only a small subset of the World Wide Web.)

However, most searches are carried out at just a small number of search sites.
How do the world’s most popular search sites rank? That depends on how you
measure popularity: the percentage of Internet users who visit a site (audience
reach); the total number of visitors; the total number of searches carried out at
a site; or the total number of hours that visitors spend searching at the site.
Each measurement provides a slightly different ranking, though all provide a
similar picture, with the same sites appearing on the list, though some in

slightly different positions.

The following list runs down the world’s most popular search sites, based on the total search hours at each site during a one-month period, as compiled in a 2003 Nielsen/NetRatings study:

Google.com                           18,700,000 hours

AOL.com                               15,500,500 hours

Yahoo.com                            7,100,000 hours

MSN.com                              5,400,000 hours

AskJeeves.com                     2,300,000 hours

InfoSpace.com                     1,100,000 hours

AltaVista.com                      800,000 hours

Overture.com                       800,000 hours

Netscape.com                      700,000 hours

EarthLink.com                    400,000 hours

LookSmart.com                   200,000 hours

Lycos.com                             200,000 hours

Remember, this is a list of search sites, not search systems. In some cases, the
sites have own their own systems. Google provides its own search results, but
AOL and MSN do not. (AOL gets its results from Google, and MSN’s results

come from Inktomi, a company owned by Yahoo! — at least at the time of this writing.)

The fact that some sites get results from other search systems means two
things. First, the numbers in the preceding list are somewhat misleading. They
suggest that Google has around a third of all the search hours. But Google

also feeds AOL its results — add AOL’s hours to Google’s, and you’ve got
almost two thirds of all search hours. Clearly the Google search system is far
more important than the Google search site. In fact, the Google search system
also feeds four more systems on this list — Yahoo!, Ask Jeeves, Netscape, and
EarthLink — and many smaller sites that don’t appear on this list. Some esti­
mates put Google’s share of the Web’s search results as high as 75 or 80 per­
cent. (That statistic will change soon, perhaps even by the time you read this,
as you find out a little later in this chapter — Yahoo will stop using Google
results soon.)

The second thing to understand is that you can ignore some of these systems. At present, for example, and for the foreseeable future, you don’t need to worry about AOL.com. Even though it’s probably the world’s second most important search site, you can forget about it. Sure, keep it in the back of your mind, but as long as you remember that Google feeds AOL, you need to worry about Google only.

When you get to the search sites that appear below Lycos in the preceding list,
the sites become dramatically less important. Google, according to this chart,
has almost 100 times the search hours spent at Lycos. And the first 11 sites on
this list combined have 265 times the search hours of Lycos. (However, as I
explain in a moment, this list doesn’t include some important search systems.)

Now reexamine the list of the world’s most important search sites and see what you remove so you can get closer to a list of sites you care about. Check out Table 1-1 for the details.

Table 1-1                                          Big-Time Search Sites

Search Site                  On the List?          Description

Google.com               Yes                    Google’s the big kid on the block. Lots of people

search the Google index on its own search site, and it feeds many sites. Obviously Google has to stay on the list.

AOL.com                    No                     Fuggetaboutit — AOL gets search results from

Google (although it manipulates them slightly, presenting them in a different way from the same search at Google itself) and from the Open Directory Project.

Yahoo.com                Yes                    Yahoo! gets results from Google, Inktomi, and

Overture, and although it doesn’t feed any other
sites, it does have its own search system (a
search directory in fact), so keep it on the list.

MSN.com                   No                     At the time of writing, MSN gets results from

Inktomi; remove it from the list.

AskJeeves.com           Yes                    Ask Jeeves gets its search results from Teoma,

but because it owns Teoma, I refer to the search
system throughout the book as Teoma/Ask
Jeeves (and keep it on the list of important
search systems). It also gets search results
from Google and the Open Directory Project.
Teoma/Ask Jeeves feeds results to many other
search sites, too.

InfoSpace.com            No                     InfoSpace gets its results from FAST/AlltheWeb

and doesn’t feed any other systems, so it’s out.

AltaVista.com              Yes                    AltaVista is owned by Overture, which is now

owned by Yahoo! AltaVista has its own search
system. Although AltaVista doesn’t feed any
other sites currently, it seems likely that it may
be feeding results to Yahoo! at some point in
the future, so keep it on the list.

Overture.com              No                     Overture is primarily a pay-per-click (PPC)

system, without its own non-advertising search
system, so it gets its non-ad results from Inktomi.

Search Site                 On the List?    Description

17

Netscape.com             No                     Netscape gets results from Google and the

Open Directory Project (Netscape owns the Open Directory Project, though). Netscape is pretty much a clone of Google, so there’s no need to keep it on the list.

EarthLink.com             No                     Another Google clone, EarthLink gets all its

results from Google and the Open Directory Project, so it’s out too.

LookSmart.com    No                             LookSmart is another PPC system. It gets non-

ad results from Inktomi and Zeal (and maybe, soon, from Grub).

Lycos.com                 No                     Lycos gets results from FAST/AlltheWeb and

the Open Directory Project, so you can remove it from the list.

Based on the information in Table 1-1, you can whittle down your list of sites to four: Google, Yahoo!, Ask Jeeves, and AltaVista. These four search sites are all important, and Google is also an important search system, feeding three quarters of the world’s search results to AOL, Yahoo!, Netscape, EarthLink, and many other search sites. Teoma/Ask Jeeves is an important search-system feeder, too, providing results to many smaller search sites.

Okay, so you visited one or two of the sites that you just crossed off and found
that you can submit your Web site to the index at that site. What’s going on
here is that the search site is selling paid inclusion into the search system that
feeds it. (I talk about paid inclusion in Chapter 9.) When you pay Lycos to sub­
mit your site, for example, Lycos takes your money and then places your site
into FAST/AlltheWeb — which isn’t a Lycos search system. Lycos is simply
acting as a reseller.

Some important systems are not important sites. For example, MSN, one of the
world’s most important search sites, gets its search results from Inktomi and
LookSmart. To take this into account, make the following changes to your list:

Add Inktomi to the list. It’s not a search site itself — you can’t search Inktomi’s index at Inktomi.com — but it’s an important search system, feeding not only MSN but also Overture and LookSmart.

Leave LookSmart off the list because it gets its results from Inktomi and Zeal. However, you better add Zeal to the list.

Now add the most important feeder systems to a “new and improved” chart. The chart becomes a combination of the four important search sites that main­ tain their own search systems — and, in a couple of cases, feed others — and the most important feeder systems: Inktomi, the Open Directory Project, FAST/ AlltheWeb, and Zeal. Table 1-2 shows the chart in all its glory.

Table 1-2                                  The Search Systems to Watch

Search Site/System               Description

Google.com                        This is the world’s most important search site and the

most important feeder system.

Yahoo.com                         Yahoo.com doesn’t feed anyone, but it’s still the world’s

second most important search site.

Teoma/Ask Jeeves               Not only is this an important search site, but the search

system also feeds sites such as About.com, Mamma.com, and many others.

AltaVista.com                       It doesn’t feed anyone right now but may feed Yahoo!

soon. It is still an important search site in its own right.

Inktomi                               Inktomi feeds MSN — an important site, of course —

Overture, and LookSmart. It also feeds sites such as About,

HotBot, goo, and many sites you may never have heard of.

The Open Directory              This is not much of a search site — few people have even

Project                                heard of it — but it’s a helluva search system, feeding

Google, AOL, Ask Jeeves, Netscape, EarthLink, Lycos, and
almost 400 other sites. (You can find it at www.dmoz.org.)

FAST/AlltheWeb                   This system feeds InfoSpace and Lycos, but it also feeds

many other systems, such as Excite, HotBot, Terra.com, Overture, and many more. AlltheWeb is a search site, and it owns the technology known as FAST, so I refer to it as FAST/AlltheWeb and don’t get into the distinction that
FAST actually feeds AlltheWeb.

FAST/AlltheWeb is owned by Overture, which in turn is
owned by Yahoo! And the FAST/AlltheWeb index is huge,
rivaling that of Google — with billions of pages — so it’s
likely to be part of Yahoo!’s strategy to dump Google.

Zeal                                    Zeal feeds LookSmart (in fact LookSmart owns Zeal), but

only for noncommercial sites. If you have a commercial
site, the only way into LookSmart is through Inktomi or by
buying PPC placement