You are here
INTERIM UPDATE ON SEARCH ENGINES
Abstract:
Almost every one is aware that there has been a high shake-up in the dot.com world. Many technology and Internet companies have cut staff or even gone into bankruptcy. According to a story being passed around on the Internet, "If you bought $1,000 worth of Nortel stock one year ago, it would now be worth $49. If you bought $1,000 worth of Budweiser (the beer, not the stock) one year ago, drank all the beer, and traded in the cans for the nickel deposit, you would have $79. My advice to you is to start drinking heavily." For those of us who did not invest heavily in the tech bubble, it is important to know what effect these changes have had on the WWW search engines.
Part of The Alchemist's Lair Web Site
Maintained by Harry E. Pence, Professor of Chemistry, SUNY Oneonta, for the use of his students. Any opinions are totally coincidental and have no official endorsement, including the people who sign my pay checks. Comments and suggestions are welcome (pencehe@oneonta.edu).
INTRODUCTION
Almost every one is aware that there has been a high shake-up in the dot.com world. Many technology and Internet companies have cut staff or even gone into bankruptcy. According to a story being passed around on the Internet, "If you bought $1,000 worth of Nortel stock one year ago, it would now be worth $49. If you bought $1,000 worth of Budweiser (the beer, not the stock) one year ago, drank all the beer, and traded in the cans for the nickel deposit, you would have $79. My advice to you is to start drinking heavily." For those of us who did not invest heavily in the tech bubble, it is important to know what effect these changes have had on the WWW search engines. Even though my students will not do their annual review of search engines for chemistry until next semester, Brian Pankuch and I have agreed that an interim report on recent developments might be helpful.
Search Engine Changes
Surprising enough, despite all the doom and gloom about the tech industry, new search engines continue to come on line. The most interesting of the recent engines is Wisenut, which some writers are claiming will be competitive with Google. Wisenut made its debut in early September of 2001, with a relatively large index (about 850 million pages) and a search algorithm similar to that used by Google to produce more accurate searches. The Wisenut white paper is a PDF file that may be downloaded to obtain more information about this new engine. Perhaps one of the most interesting aspects of Wisenut is the claim that it is designed to handle up to 1 trillion URLs. Since the current estimates of the size of the searchable web are only about two to four billion pages, this suggests that Wisenut plans to stay competitive for some time.
As those who have read my previous reports may remember, I consider Danny Sullivan to be an excellent search engine analyst, so I place considerable weight on the evaluation of Wisenut by Sullivan, which suggests that this new engine will not really challenge Google because there is more money to be made by providing search services to large sites. Sullivan quotes one expert who recently estimated that the annual growth rate for search technology sector will be over 50% per year. Nevertheless, the large index size, interesting features, and plans for continued growth suggest that this is an engine to watch in the future.
Another promising new search engine that has recently started operation is Teoma, which also claims to compete with Google in terms of focused search algorithms. Teoma was created by computer scientists at Rutgers university and uses an algorithm that bundles search results in terms of natural Web communities to create more relevant results. Before Teoma was even out of beta test, however, it was purchased by Ask Jeeves, which is itself a well-known search engine. According to a press release, Ask Jeeves plans to continue to maintain a separate Teoma site and also introduce the Teoma technology into the existing Ask Jeeves engine. Both Wisenut and Teoma will be included in the annual update of search engines for chemistry that will appear in the Newsletter next spring.
[Author's comment: Although there was not time to do an extensive comparision of these two new engines, I did do a search on the phrase "ionic solvents" on Google, Wisenut, and Teoma. Wisenut found 38 documents(but claimed 47); Teoma found 24; and Google found 89 (and claimed about 116). It was the opinion of the author that the first ten Google Links seemed more generally useful than the first ten links returned by Teoma, which in turn seemed more useful than the first ten from Wisenut. Finally, it was clear (at least on the day that this test was run) that the Wisenut search was much slower than the other two.
On the negative side, two of the oldest search engines in existence reported serious financial problems. The parent company of the Excite search engine has filed for bankruptcy, and is trying to sell the Excite.com portal. The engine is still in operating at this time, but it is questionable how long it will continue to be kept up-to-date. Meanwhile, Altavista, another well-known search engine has made another major staff cut (30%) and appointed a new chief executive officer. The Altavista engine continues to operate, but the reports from search engines analysts (see below) suggest that the index has stopped growing in size. It has also been reported that in April Altavista stopped updating the regional sites that it maintains with links for sites in specific countries, mainly in Europe but also countries like Australia, Canada, New Zealand, India, and Korea. And finally, About.com, which is notable for providing human guides who organize and search the web, is reported to be eliminating 300 of its 750 guides.
Changes in the Size of Search Engine Indexes.
The latest report from Danny Sullivan on Search Engine Sizes, dated Aug. 15, 2001, indicates that Google has by far the largest index, followed by FAST, Altavista, and Inktomi, in that order. The graph of search engine size over time indicates that Google is also growing much faster than the competition, including some, like Altavista, that are static. On Aug. 14, 2001, Greg Notess did a similar study on Changes in Search Engine Size that shows that Google, FAST, and Altavista are the three largest engine indexes (in that order)and since April, Google and FAST have increased in size; Altavista and Northern Light have remained about the same size: and MSN, which is powered by Inktomi, has actually decreased in size. Recently, Greg Notess has compared the results of the major search engines using 25 small, single-word terms. He identified Google as the leading search engine, as might be expected from the size of its index, because it returned the greatest number of hits overall and also found the greatest number of hits on 19 of the 25 individual searches. Based on his results, he ranked Fast in second place and Wisenut, which was mentioned above as a potential competitor to Google, was third. Notess ranked Altavista at sixth but he noted that in two of the searches it did give the best results. Teoma, which is also mentioned earlier, can only display 200 results, and so in most cases it could not be compared with the other engines. See Total Hits from 25 Engines for more complete results.
Acknowledgments
As noted in the earlier reports in this series, the Search Engine Watch web site, maintained by Danny Sullivan, and the Search Engine Showdownweb site, maintained by Greg Notess are excellent resources for anyone who wishes to keep up with current developments in WWW search engines. These sites are a major source for the information in this report.
Date:
10/20/01 to 10/24/01
- Log in to post comments