The Major Search Engines
Why are the services below considered to be the
Major Search Engines? They are all either well-known or well-used.
For webmasters, these services are the most important
places to be listed, because they can potentially generate so much traffic.
For searchers, these well-known, commercially-backed
search engines generally mean more dependable results. These search engines
are more likely to be well-maintained and upgraded when necessary, to keep
pace with the growing web.
Not all of the services below are "true" search engines
that crawl the web. For instance, Yahoo and the Open Directory
both are "directories" that depend on humans to compile their listings.
In fact, most of the services below offer both search engine and directory
information, though they will predominately feature one type of results
over the other.
AOL Search
AOL Search allows its members to search across the
web and AOL's own content from one place. The "external" version, listed
above, does not list AOL content. The main listings
for categories and web sites come from the Open Directory (see below).
Inktomi (see below) also provides crawler-based results, as backup to the
directory information. Before the launch of AOL Search in October 1999,
the AOL search service was Excite-powered AOL NetFind.
AltaVista
AltaVista is consistently one of the largest search
engines on the web, in terms of pages indexed. Its comprehensive coverage
and wide range of power searching commands makes it a particular favorite
among researchers. It also offers a number of features
designed to appeal to basic users, such as "Ask
AltaVista" results, which come from Ask Jeeves (see below), and directory
listings primarily from the Open Directory. AltaVista opened in December
1995. It was owned by Digital, then run by Compaq (which purchased Digital
in 1998), then spun off into a separate company which is now controlled
by CMGI.
Ask Jeeves
Ask Jeeves is a human-powered search service that
aims to direct you to the exact page that answers your question. If it
fails to find a match within its own database, then it will provide matching
web pages from various search engines. The service went into beta in mid-April
1997 and opened fully on June 1, 1997. Results from Ask Jeeves also appear
within AltaVista.
Direct Hit
Direct Hit is a company that works with other search
engines to refine their results. It does this by monitoring what users
click on
from the results they see. Sites that get clicked
on more than others rise higher in Direct Hit's rankings. Thus, the service
dubs itself a "popularity engine." Direct Hit's technology is currently
best seen at HotBot. It also refines results at Lycos and is available
as an option at LookSmart and MSN Search. The company also crawls the weband
refines this database, which can be viewed via the link above.
Excite
Excite is one of the most popular search services
on the web. It offers a medium-sized index and integrates non-web material
such as company information and sports scores into its results, when appropriate.
Excite was launched in late 1995. It grew quickly in prominence and consumed
two of its competitors, Magellan in July 1996, and WebCrawler in November
1996. These continue to run as separate services.
FAST Search
Formerly called All The Web, FAST Search aims to
index the entire web. It was the first search engine to break the 200 million
web page index milestone. The Norwegian company behind FAST Search also
powers the Lycos MP3 search engine. FAST Search launched in May 1999.
Go / Infoseek
Go is a portal site produced by Infoseek and Disney.
It offers portal features such as personalization and free e-mail, plus
the search capabilities of the former Infoseek search service, which has
now been folded into Go. Searchers will find that Go consistently provides
quality results in response to many general and broad searches, thanks
to its ESP search algorithm. It also has an impressive human-compiled directory
of web sites. Go officially launched in January 1999. It is not related
to GoTo, below. The former Infoseek service launched in early 1995.
GoTo
Unlike the other search engines, GoTo sells its
listings. Companies can pay money to be placed higher in the search results,
which
GoTo feels improves relevancy. Non-paid results
come from Inktomi. GoTo launched in 1997 and incorporated the former University
of Colorado-based World Wide Web Worm. In February 1998, it shifted to
its current pay-for-placement model and soon after replaced the WWW Worm
with Inktomi for its non-paid listings. GoTo is not related to Go, above.
Google
Google is a search engine that makes heavy use of
link popularity as a primary way to rank web sites. This can be especially
helpful in finding good sites in response to general searches such as "cars"
and "travel," because users across the web have in essence voted for good
sites by linking to them.
HotBot
Like AltaVista, HotBot is another favorite among
researchers due to its large index of the web and many power searching
features. Inmost cases, HotBot's first page of results comes from the Direct
Hit service (see above), and then secondary results come from theInktomi
search engine, which is also used by other services. It gets its directory
information from the Open Directory project (see below). HotBot launched
in May 1996 as Wired Digital's entry into the search engine market. Lycos
purchased Wired Digital in October 1998 and continues to run HotBot as
a separate search service.
Inktomi
Originally, there was an Inktomi search engine at
UC Berkeley. The creators then formed their own company with the same name
and created a new Inktomi index, which was first used to power HotBot.
Now the Inktomi index also powers several other services. All of themtap
into the same index, though results may be slightly different. This is
because Inktomi provides ways for its partners to use a common index and
still distinguish themselves. There is no way to query the Inktomi index
directly, as it is only made available through Inktomi's partners with
whatever filters and ranking tweaks they may apply.
LookSmart
LookSmart is a human-compiled directory of web sites.
In addition to being a stand-alone service, LookSmart provides directory
results to MSN Search, Excite and many other partners. AltaVista provides
LookSmart with search results when a search fails to find a match from
among LookSmart's reviews. LookSmart launched independently in October
1996, was backed by Reader's Digest for about a year, and then company
executives bought back control of the service.
Lycos
Lycos started out as a search engine, depending
on listings that came from spidering the web. In April 1999, it shifted
to a directory model similar to Yahoo. Its main listings come from the
Open Directory project, and then secondary results come from either Direct
Hit or Lycos' own spidering of the web. In October 1998, Lycos acquired
the competing HotBot search service, which continues to be run separately.
MSN Search
Microsoft's MSN Search service is a LookSmart-powered
directory of web sites, with secondary results that come from AltaVista.
RealNames and Direct Hit data is also made available. MSN Search also offers
a unique way for Internet Explorer 5 users to save past searches.
Netscape Search
Netscape Search's results come primarily from the
Open Directory and Netscape's own "Smart Browsing" database, which does
an excellent job of listing "official" web sites. Secondary results come
from Google. At the Netscape Netcenter portal site, other search engines
are also featured.
Northern Light
Northern Light is another favorite search engine
among researchers. It features one of the largest indexes of the web, along
with the ability to cluster documents by topic. Northern Light also has
a set of "special collection" documents that are not readily accessible
to search engine spiders. There are documents from thousands of sources,
including newswires, magazines and databases. Searching these documents
is free, but there is a charge of up to $4 to view them. There is
no charge to view documents on the public web— only for those within the
special collection. Northern Light opened to general use in August 1997.
Open Directory
The Open Directory uses volunteer editors to catalog
the web.Formerly known as NewHoo, it was launched in June 1998. It was
acquired by Netscape in November 1998, and the company
pledged that anyone would be able to use information from the directory
through an open license arrangement. Netscape itself was the first licensee.
Lycos and AOL Search also make heavy use of Open Directory data, while
AltaVista and HotBot prominently feature Open Directory categories within
their results pages.
RealNames
The RealNames system is meant to be an easier-to-use
alternative to the current web site addressing system. Those with
RealNames-enabled browsers can enter a word like
"Nike" to reach the Nike web site. To date, RealNames has had its biggest
success through search engine partnerships. In particular, it is strongly
featured in results at AltaVista, Go and MSN Search.
Snap
Snap is a human-compiled directory of web sites,
supplemented by search results from Inktomi. Like LookSmart, it aims to
challenge Yahoo as the champion of categorizing the web. Snap launched
in late 1997 and is backed by Cnet and NBC.
WebCrawler
WebCrawler has the smallest index of any major search
engine on the web—think of it as Excite Lite. The small index means
WebCrawler is not the place to go when seeking obscure
or unusual material. However, some people may feel that by having indexed
fewer pages, WebCrawler provides less overwhelming results in response
to general searches. WebCrawler opened to the public on April 20, 1994.
It was started as a research project at the University of Washington. America
Online purchased it in March 1995 and was the online service's preferred
search engine until Nov. 1996. That was when Excite, a WebCrawler competitor,
acquired the service. Excite continues to run WebCrawler as an independent
search engine.
Yahoo
Yahoo is the web's most popular search service and
has a well-deserved reputation for helping people find information easily.
The secret to Yahoo's success is human beings. It
is the largest human-compiled guide to the web, employing about 150 editors
in an effort to categorize the web. Yahoo has over 1 million sites listed.Yahoo
also supplements its results with those from Inktomi. If a search fails
to find a match within Yahoo's own listings, then matches from Inktomi
are displayed. Inktomi matches also appear after all Yahoo matches have
first been shown. Yahoo is the oldest major web site directory, having
launched in late 1994.
Major Metacrawlers
Below are especially well-known, well-used or long-established
meta searchservices.
Go2Net / MetaCrawler
http://www.go2net.com/
One of the oldest meta search services, MetaCrawler
began in July 1995 at the University of Washington. MetaCrawler was purchased
by go2net, an online content provider, in Feb. 97. The commercial backing
has helped improve the responsiveness of the service.
MetaCrawler
http://www.metacrawler.com
now powers searches at the Go2Net portal site. (more about Go2Net)
SavvySearch
http://www.savvysearch.com/
Another one of the older metasearch services, around
since May 1995 and formerly based at Colorado State University. It is highly
customizable and covers a huge-range of general and specialty search sites.
Dogpile
http://www.dogpile.com/
Popular metasearch site that sends a search to a
customizable list of search engines, directories and specialty search sites.
Dogpile also runs the MetaFind metasearch site that sends searches only
to crawler-based search engines.
Inference Find
http://www.infind.com/
An alternative
to typical metacrawlers, Inference lists results grouped by subject, rather
than by search engine or in one giant list. For example, a search for "Uma
Thurman" groups results into "Uma Thurman" and "Pulp Fiction," among other
categories. It taps into Alta Vista, Excite, Infoseek, Lycos, WebCrawler
and Yahoo. The service began in May 1995, moving to its present domain
in Oct. 1996.
ProFusion
http://www.profusion.com/
Customizable, with broken link detection available.
Formerly based at the University of Kansas.
Mamma
http://www.mamma.com/
Sends search requests to major search services.
The Big Hub
http://www.thebighub.com/
Allows you to search many major search engines or
a huge number of specialty sites, all from the same place. Formerly the
Internet Sleuth.
Major
Children's Guides