Before AI Search

Between 1994 and 2010, people found information on the web through directories that humans edited and through search engines that matched typed keywords to pages and returned ranked lists of links. Businesses learned that a high position on those lists brought customers, and an industry formed around earning that position.¹ Along the way a few products tried to answer questions directly, and Google began to blend images, news, and video into its results.²

January 1994: Yahoo Starts as a Hand-Built Directory

Yahoo began in January 1994, when Jerry Yang and David Filo, two electrical engineering graduate students at Stanford University, created a website called "Jerry and David's guide to the World Wide Web."³ The guide grew into the Yahoo! Directory, which was Yahoo's first offering and was edited by people.⁴ Visitors found websites by choosing a topic heading and clicking through narrower categories.⁵

Yahoo became a public company through an initial public offering in April 1996, and its stock price rose 600% within two years.³ In 1998, Yahoo replaced AltaVista with Inktomi as the crawler-based search engine underneath the directory.⁴ Yahoo later paid $280 million to acquire Inktomi in 2002.³

The directory lost importance when Yahoo changed its main results to crawler-based listings under Yahoo! Search in October 2002, and it had closed by December 26, 2014.⁴

Yahoo in October 1996

1994 to 1996: Full-Text Search Engines Appear

Search engines that indexed the words on web pages arrived in the same years. Brian Pinkerton began work on WebCrawler on January 27, 1994, at the University of Washington. It launched on April 21, 1994, with more than 4,000 websites in its database, and it was the first web search engine to provide full-text search.⁶ Lycos began in May 1994 as a research project by Michael Loren Mauldin at Carnegie Mellon University.⁷

Researchers at Digital Equipment Corporation (DEC) created AltaVista to showcase the company's hardware by making it easier to find files on its public network, and DEC launched it on December 15, 1995.⁸ AltaVista's traffic grew from 300,000 hits on the first day to more than 80 million hits per day by 1997.⁸ AltaVista's own home page in October 1996 told visitors that the service gave them access to the largest web index, with 30 million pages found on 275,600 servers, and that it was accessed over 20 million times per weekday.⁹

Query volume rose just as quickly. Brin and Page noted in 1998 that the World Wide Web Worm, an early search engine, received an average of about 1,500 queries per day in March and April 1994, and that AltaVista claimed to handle roughly 20 million queries per day in November 1997. They expected the top search engines to handle hundreds of millions of queries per day by the year 2000.¹⁰

AltaVista in October 1996

Ranking on these engines depended heavily on how a page's own words matched a query. Brin and Page wrote in 1998 that automated search engines relying on keyword matching usually return too many low-quality matches, and that some advertisers tried to mislead them.¹⁰ Webmasters and content providers began optimizing their sites for search engines in the mid-1990s, as the first engines cataloged the early web.¹

1997: The Phrase Search Engine Optimization Comes Into Use

A 2004 article by Danny Sullivan, then an industry analyst, dates the phrase "search engine optimization" to 1997 and credits the practitioner Bruce Clay as one of the first people to popularize it.¹ As early as 1997, search engine providers began adjusting their algorithms to prevent ranking manipulation.¹

June 1997: Ask Jeeves Takes Questions in Plain English

Ask Jeeves launched on June 1, 1997, after a beta version in mid-April. Garrett Gruener and David Warthen created it, and the company was incorporated in Berkeley, California, in 1996.¹¹ The original platform let users submit questions in natural language, which set it apart from competitors that relied on keywords.¹¹ Its branding centered on Jeeves, a valet inspired by the fictional servant created by the British author P. G. Wodehouse.¹¹

Ask Jeeves stock traded on the Nasdaq from July 1999 to July 2005, and on September 18, 2001, the company acquired the search engine Teoma.¹¹ InterActiveCorp (IAC) bought the company in July 2005, and in February 2006 "Jeeves" was dropped from the name and the site became Ask.¹¹ Ask.com shut down on May 1, 2026.¹¹

Ask Jeeves in 2000

1998: Brin and Page Describe Google

Larry Page and Sergey Brin developed PageRank at Stanford University in 1996 as a research project for a new kind of search engine, different from the leading engines of the time such as AltaVista.¹³ In a 1998 paper, they described Google as a prototype with a full-text and hyperlink database of at least 24 million pages.¹⁰ They chose the name Google because it is a common spelling of googol, or 10 to the power of 100, which fit their goal of building very large-scale search engines.¹⁰

Page and Brin were not the first to rank sites by links. A search engine called RankDex, designed by Robin Li at IDD Information Services in 1996, ranked the popularity of a website by how many other sites linked to it.¹³ The patent that Page filed for PageRank in 1998 cites Li's earlier patent.¹⁴

The Anatomy of a Large-Scale Hypertextual Web Search Engine

The paper's approach to ranking was PageRank, which scores a page using the links that point to it. The authors had built maps containing as many as 518 million hyperlinks, which allowed rapid calculation of a page's PageRank, an "objective measure of its citation importance."¹⁰

The paper also described a second use of links. Most search engines associated the text of a link with the page that the link sat on. Google also associated that text with the page the link pointed to, because anchors often provide more accurate descriptions of web pages than the pages themselves, and because anchors can exist for documents a text-based engine cannot index, such as images, programs, and databases.¹⁰

Appendix A of the paper, titled "Advertising and Mixed Motives," is direct about the risks of paid search. The authors wrote that they "expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers."¹⁰ They described OpenText, which was reported to be selling companies the right to be listed at the top of results for particular queries, a business model that "resulted in an uproar."¹⁰

Google in Beta, December 1998

1998 to 2000: Paid Placement and AdWords

GoTo, a spin-off from the incubator Idealab, was the first company to successfully provide a pay-for-placement search service.¹⁶ In February 1998, GoTo offered advertisers the option of bidding on how much they would pay to appear at the top of results for specific searches, and by July 1998 advertisers were paying up to a dollar per click.¹⁶ On October 8, 2001, GoTo renamed itself Overture Services. On October 7, 2003, Yahoo bought Overture, its biggest customer, for $1.63 billion.¹⁶ In February 2003, Overture had also acquired AltaVista for $140 million.⁸ AltaVista closed on July 8, 2013.⁸

Google entered search advertising on October 23, 2000, when it announced AdWords, a program that let any advertiser buy keyword advertising that appeared on the google.com results page. About 350 businesses and advertising agencies had joined during the beta earlier that month.¹⁷ Text ads appeared to the right of search results, highlighted as sponsored links and kept separate from the results, while premium sponsorship ads stayed at the top.¹⁷

AdWords priced ads by impression, at CPMs of $15, $12, and $10 for the top, middle, and bottom ad positions, which Google expressed as 1.5 cents, 1.2 cents, and 1 cent per impression. At the time, people ran more than 20 million searches per day on google.com.¹⁷

The AdWords announcement listed the controls advertisers received: ads that appeared instantly after an order, previews, an unlimited number of ad versions, and reporting on click-through rates and impressions. Targeting options included keywords, negative keywords, phrase matches, and exact matches. An ad appeared only when a user entered the same keywords or phrase that the advertiser had bought.¹⁷ Larry Page, then Google's co-founder and CEO, said AdWords was built to "address the needs of any business" with a program that was "affordable and easy to use."¹⁷

2000 to 2009: Keywords, Links, and Spam

As rankings came to carry commercial value, site owners tried to manipulate them. Keyword stuffing loads keywords into a page's meta tags, visible content, or link anchor text to gain an unfair advantage in search rankings, and it was used in the past to win top positions for particular phrases.¹⁸ A link farm is any group of websites that all link to one another to raise search rankings.¹⁹

Matt Cutts of Google wrote on September 21, 2009, that Google did not use the keywords meta tag in web ranking. He explained that about a decade earlier, search engines judged pages only on the content of the pages themselves, and the keywords tag quickly became a place to stuff often irrelevant keywords that visitors never saw. Because the tag was so often abused, Google began disregarding it many years before the post.²⁰

In early 2005, Google's Matt Cutts and Jason Shellen of Blogger proposed the rel="nofollow" link value to address comment spam across blogs. Google announced that hyperlinks with the value would not influence the link target's PageRank, and Yahoo and Bing also respect the attribute.²¹

2001 to 2006: Questions Go to People

Several services sent questions to human answerers. Google Questions and Answers launched in June 2001, with Google staffers answering questions by e-mail for a flat fee of US$3.00.²² Its successor, Google Answers, was an online knowledge market active from April 2002 until December 2006, and it accepted no new questions after November 30, 2006.²²

Yahoo! Answers released its beta version to the general public on December 8, 2005. The site ceased operations on May 4, 2021.²³

May 2007: Universal Search Blends Content Types

On May 16, 2007, Google announced its first steps toward a universal search model. Beginning that day, the company incorporated information from sources that had been separate, including videos, images, news, maps, books, and websites, into a single set of results.²

Marissa Mayer, then vice president of search products and user experience at Google, said that "the ultimate goal of universal search is to break down the silos of information that exist on the web and provide the very best answer every time a user enters a query."² Google's example was a search for Darth Vader, which would return a blended set that included a parody of the movie, images of the character, news reports about the latest Lucas film, and websites about the actor James Earl Jones, all ranked by relevance.²

Udi Manber, then Google's vice president of engineering, said at the time that most of Google's innovation addressed basic ranking algorithms and was often not obvious to users, who saw "more accurate results, more often, in more languages."² The release also added navigation links above the results that suggested other areas of Google to search. A search for "python," for example, generated links to Google Blog Search, Book Search, Groups, and Code.²

May and June 2009: WolframAlpha and Bing

WolframAlpha is an online knowledge engine developed by Wolfram Research. It computes answers and visualizations from a knowledge base of structured data that comes from other sites and books, and it officially launched on May 18, 2009, to mixed reviews.²⁴

Microsoft CEO Steve Ballmer announced the transition from Live Search to Bing on May 28, 2009, at the All Things Digital conference in San Diego, California. Bing officially replaced Live Search on June 3, 2009.²⁵

September 2010: Google Instant Predicts the Query

On September 8, 2010, Marissa Mayer, by then vice president of search products and user experience, announced Google Instant. She called it search-before-you-type: Instant takes what a person has typed, predicts the most likely completion, and streams results in real time for those predictions.²⁶

Google had built search-as-you-type demos earlier, by Amit Patel in 1999 and Nikhil Bhatla in 2003, but decided against releasing them because nobody wants results for the partial query [bike h] while searching for [bike helmets]. Predicting the likely completion was the change that made the idea workable.²⁶

Google's testing showed that Instant saved the average searcher two to five seconds per search. It became the core search experience on Google.com for Chrome, Firefox, Safari, and Internet Explorer 8, and Google offered it to signed-in users in France, Germany, Italy, Russia, Spain, and the United Kingdom.²⁶

By the end of 2010, a Google results page combined ranked web links, paid text ads, and blended results from several content types.^17,2

Key Organizations