- The Washington Times - Thursday, October 18, 2001

Finding something on the World Wide Web should be like finding a needle in a haystack, given the Web's seemingly endless array of virtual pathways.

The search is fairly easy, however. Today's search engines, from bare-bone machines such as Google.com to more venerable Web portals such as Yahoo.com, give Web surfers a significant boost toward finding exactly what they want online.
It wasn't always this way.
In the Web's earlier days, the mid-1990s, Web "portal" sites pages that offer searches, news, online chats and other services were where most Internet dwellers flocked for search purposes. Portals, Web sites that users could make their home page when they started up their Internet connections, were one-stop destinations that experts once thought would attract surfers and advertising revenues alike for years to come.
Those rudimentary search engines often examined Web pages by looking for key words buried within the pages. Search technology employed by portals such as Excite.com and Lycos.com also scanned for "meta-tags," words buried in a page's HTML (hypertext markup language) coding, the language that creates the pages.
Jupiter Media Metrix Vice President Ross Rubin says such portals found that although the search technology attracted consumers, it also quickly sent them away without retaining them or persuading them to use a portal's other services. The content wasn't compelling enough to detain them, so they simply used the search engine and left, leaving portals with no revenue.
"Search has become a specialized business," says Mr. Rubin, whose New York-based company measures Web pages viewed and offers Internet consulting work. "It returned to its roots. It's more of a technological business."

Search engines typically crawl the Web looking for pages to add to their databases, or indexes. Then they conduct their future searches from this copious collection of sites.
The owners of some sites take a hands-on approach and register their sites with various engines to make sure they make their presence felt.
Then, when a person goes to a search engine and punches in a key phrase or word, the engine scans its indexes to find relevant matches.
Today, many sites with a search capacity, such as AOL.com, rely on outside companies to supply the search technology. Google, a Mountain View, Calif., company that runs one of the Web's most visited search-only sites, and Inktomi, a Foster City, Calif., company, license such technology. The latter firm supplies search help for Msn.com and Hotbot.com.
Mr. Rubin says consumers are more satisfied with Internet searches than in the past.
"Google has done a much better job than anything before it," he says.
Google, which began as a research project at Stanford University in the mid-1990s, is ranked the 15th-most-visited site by Jupiter Media Metrix, making it the highest-ranked search-only Web site. Sites run by AOL, Microsoft and Yahoo are the most visited pages, respectively.
Google searches the Web via key words inside Web pages, as other search engines do, but also through link analysis.
If Page A is linked to Page B, it considers such a link a vote by Page A for Page B. If a page links to another, that is like saying the first page considers the other page worthy of its, and the surfer's, attention. The more such votes a page gets, the higher it appears on the search results pages.
Northernlight.com,another search-only site, also uses link analysis.
Google.com spokeswoman Eileen Rodriguez says the users of search engines look for, in order: ease of use, relevant search results and speed when they punch in their search terms. She adds that her company's research shows that users go online for searches more than for any other activity, save using e-mail.
Google uses a network of about 10,000 computers to tap into an index of more than 1.6 billion Web pages, up 60 percent from a year ago.
Some information, no matter how effective modern search engines prove to be, remains out of their reach, says University of Maryland computer science professor William Pugh.
"There's a lot of information available which is only available if you enter specific data," Mr. Pugh says of sites requiring personal information. Search engines will inspect such pages, and "if they see a form, they turn away."
Internet users who maintain Web pages but want to keep their contents such as snapshots of friends relatively private can use the search engines' quirks to their advantage. Because engines such as Google.com rely on links for part of their searches, for instance, leaving links off a personal Web page assures at least some privacy, Mr. Pugh says. Friends can access the pages by learning and typing in their exact Web address.
Danny Sullivan, a Shrewton, England, journalist who writes Search Engine Watch (www.searchenginewatch.com), says search engines quickly have become intertwined in our daily lives.
"Some surveys find that search engines are the top way people seek information, over asking a friend or going to the library," he says. "We depend upon them so utterly."

Mr. Sullivan says the Internet implosion of the past year or two meant the end for many established search engines, including Go.com, formerly known as Infoseek.com.
"They spent a lot of money trying to build traffic but didn't know how to make money from that traffic," he says. Sites such as Google.com earn money by licensing out their search technology. Portal sites depended largely on banner advertising, which has yet to provide a vital revenue stream for many Web sites because companies aren't convinced how effective they are in drawing in consumers and are unwilling to spend advertising dollars in such an unproven medium.
That's bad news for consumers.
"You like to have diversity in search engines," he says. "Different search engines can view the Web in different ways. Google doesn't always work for everything you want."
Because each engine indexes the Web individually, a Web site in Google.com's memory banks may not be in Northernlight.com's memory banks, and vice versa.
Portals may be fading in importance, but a few other wrinkles have emerged on the search-engine scene. Iwon.com provides a search service with an incentive every time a registered user surfs its site, he or she is entered in a variety of sweepstakes. Prizes range from $1,000 to $35 million on April 15, its annual promotion on Tax Day.
Another new trend is sponsored searches. If a consumer searches for "shoes" on a sponsored site, a link to Payless.com may appear at the top of the search result list if that company paid the search engine the requisite fee.
Overture.com, a search engine formerly known as Goto.com, provides such searches.
One search engine, Northernlight.com, doesn't seek out the average consumer, but instead wants those looking for specific information with business implications.
"We're not the search engine for people who are just browsing the Web," says Northern Light CEO David Seuss. "We've always had subject classification as one of our leading consumer benefits."
Northern Light, whose Web site offers a dense listing of search options geared toward the serious surfer, indexes more than 60 million pages of business information not readily available through other engines from a database of more than 7,100 business and information sources.
The engine also combines a regular search engine with the potential to browse a database of periodicals, for a fee.
The Web is ripe for such specialized searches, he says.
"Consumers are not a monolithic group," he says. "Segmentation has always existed on the Web."
Mr. Rubin says despite all the innovations in search technology and those yet to come most engines will suffice for the average consumer.
"If content is abundant on the topic, any search engine will do a good job," Mr. Rubin says.

"Some surveys find that search engines are the top way people seek information, over asking a friend or going to the library."
Danny Sullivan, Search Engine Watch

Sign up for Daily Newsletters

Manage Newsletters

Copyright © 2021 The Washington Times, LLC. Click here for reprint permission.

Please read our comment policy before commenting.


Click to Read More and View Comments

Click to Hide