Market Research / GEO/AEO History
The Citation Economy
Chapter 9, 2023 to 2026
How citations became the currency of AI answers from 2023 to 2026: licensing deals, publisher programs, crawler controls, studies of which sources get cited, and the tactics brands adopted to appear in them.
By TGA EditorsOctober 30, 20266 min read
A citation in an AI answer is a source the system names or links alongside the text it generates. The first chatbot search products attached sources from the start, as The ChatGPT Shock describes for Microsoft's new Bing and for Perplexity, and Google's AI Overviews, covered in Google Answers Back, include links. A citation is the point at which a reader of a generated answer can reach the source, and publishers, platforms, and brands began to treat it as something with a value. This chapter follows the deals, programs, tools, and studies that built that market from 2023 to 2026. The research that gave the practice its name and the vendors that sold it are in The Origins of GEO.

July 2023 to May 2024: Publishers License Content to AI Companies
On July 13, 2023, The Associated Press and OpenAI announced an agreement under which OpenAI would license part of AP's text archive and AP would draw on OpenAI's technology and product expertise. The press release said AP "does not use" generative AI in its news stories.1
On December 13, 2023, Axel Springer and OpenAI announced a partnership under which ChatGPT users would receive summaries of selected news from POLITICO, BUSINESS INSIDER, BILD, and WELT, including content that is otherwise behind a paywall. The announcement said ChatGPT's answers would include attribution and links to the full articles, and that the arrangement also covered use of the publisher's content to train OpenAI's models. It called Axel Springer the first publishing house globally to partner with OpenAI on a deeper integration of journalism in AI technologies.2
Other kinds of publishers made similar agreements. On February 22, 2024, Reddit announced an expanded partnership with Google under which Google would receive programmatic access to Reddit's posts and comments through its Data API, and the Associated Press reported that the arrangement was valued at roughly $60 million.3,4 On May 16, 2024, Reddit and OpenAI announced a partnership under which OpenAI would use the same API to bring Reddit content into ChatGPT.5 On May 22, 2024, News Corp and OpenAI announced a multi-year agreement giving OpenAI permission to display content from News Corp mastheads, including The Wall Street Journal, Barron's, MarketWatch, the New York Post, The Times, and The Sun, in response to user questions.6
July 2024: The Perplexity Publishers' Program
On July 30, 2024, Perplexity announced a program for publishers, with TIME, Der Spiegel, Fortune, Entrepreneur, The Texas Tribune, and WordPress.com as its first partners. The post said that "from day one" Perplexity had included citations in each answer. Perplexity planned to sell advertising on its related-questions feature and to share revenue with a publisher when the company earned money from an interaction that referenced that publisher's content. Partners also received free access to Perplexity's APIs, which let a publisher build a question-and-answer feature on its own site that cites only its own content, and the company called ScalePost.ai as a platform it would work with to connect publishers and AI companies.7
Perplexity Announces Its Publishers' Program

September 2024: The llms.txt Proposal
On September 3, 2024, Jeremy Howard of Answer.AI proposed that websites add a file called /llms.txt, a markdown document with brief background information and links to more detailed pages that a language model can use when it needs context about the site. The proposal also suggested that pages offer a clean markdown version at the same address with .md added. The proposal gave coding assistants and software documentation as its main examples. It said the FastHTML project follows it for its documentation and that all nbdev projects now create .md versions of their pages by default.8
Answer.AI Proposes llms.txt

Google's documentation on AI features, read on October 30, 2026, does not mention llms.txt.9
October 2024: ChatGPT Adds Web Search
On October 31, 2024, OpenAI launched ChatGPT search for paid subscribers, with free, enterprise, and education users to follow. The Verge reported that responses included clickable citations to original sources and a sources sidebar, and that the feature was built with "a mix of search technologies," including Microsoft's Bing.10 TechCrunch reported that the feature ran on a fine-tuned version of GPT-4o, descended from the SearchGPT prototype shown that summer, and that responses carried in-line and sidebar attribution to the news publishers and data sources with which OpenAI had licensing deals.11 The attribution to licensed publishers made the earlier agreements visible in the product.
March 2025: The Tow Center Tests Citations
On March 6, 2025, the Tow Center for Digital Journalism at Columbia University published the results of a test of eight generative search tools: ChatGPT Search, Perplexity, Perplexity Pro, DeepSeek Search, Microsoft's Copilot, Grok-2, Grok-3 (beta), and Gemini. Researchers chose 20 publishers, some that permit AI search crawlers and some that block them, selected ten articles from each, and asked each tool to identify the headline, publisher, publication date, and URL of an excerpt. They ran 1,600 queries. Collectively the tools answered more than 60 percent of queries incorrectly. Perplexity was wrong on 37 percent and Grok 3 on 94 percent. The authors also reported that the tools fabricated links and cited syndicated and copied versions of articles, and that content licensing deals "provided no guarantee of accurate citation."12
May 2025: Ahrefs Studies Brand Mentions in AI Overviews
On May 26, 2025, Ahrefs published a study of 75,000 brands that looked at which search factors correlated with brand mentions in AI Overviews. Brand web mentions had the highest correlation, 0.664 on the Spearman scale, followed by brand anchors at 0.527, brand search volume at 0.392, the number of backlinks at 0.218, and branded ad traffic at 0.216.13

June and July 2025: Which Sites Get Cited
A Semrush analysis of more than 150,000 citations in AI answers from June 2025, charted by Visual Capitalist, found reddit.com cited most often at 40.1 percent, ahead of wikipedia.org at 26.3 percent, youtube.com at 23.5 percent, and google.com at 23.3 percent.14 The Pew Research Center study of Google AI summaries described in Google Answers Back reached a similar result for one product: Wikipedia, YouTube, and Reddit were the three most frequently cited sources in AI summaries, together accounting for 15 percent of the sources listed.15

July 2025: Cloudflare Blocks AI Crawlers by Default
On July 1, 2025, Cloudflare announced that it had become the first internet infrastructure provider to block AI crawlers that access content without permission or compensation by default, and that website owners could choose whether AI crawlers could reach their content. Under the new arrangement AI companies could state whether a crawler is used for training, inference, or search, so that owners could decide which crawlers to allow. The press release quoted Matthew Prince, Cloudflare's chief executive, and executives from Condé Nast, Dotdash Meredith, and Gannett.16
Cloudflare Announces Default Blocking of AI Crawlers

The same day, Cloudflare described pay per crawl, a private beta that uses the HTTP 402 Payment Required status code. A site owner can set a flat per-request price, and the crawler is then offered three choices: allow free access, pay the configured price, or be blocked. Cloudflare acts as merchant of record, and it uses the Web Bot Auth proposals to prevent anyone from impersonating a paying crawler.17
July 21, 2025: Semrush Compares AI Citations With Google's Rankings
On July 21, 2025, Semrush published a comparison of citations on four platforms, Google Search with its AI Overviews, Google AI Mode, ChatGPT, and Perplexity, for 5,000 randomly selected keywords split evenly among informational, transactional, commercial, and navigational intent. For each platform it counted how many of the cited domains and URLs also appeared in Google's top 10 organic results for the same query. Perplexity overlapped the most, at more than 91 percent of domains and 82 percent of URLs. AI Overviews overlapped on about 86 percent of domains and 67 percent of URLs, AI Mode on about 54 percent and 35 percent, and ChatGPT showed the lowest overlap. Semrush also reported that AI Overviews linked to an average of three unique domains per query and AI Mode to seven.18
December 2025: Ahrefs Studies Best Lists and Brand Visibility
On December 4, 2025, Glen Allsopp of Ahrefs published a study of 750 top-of-the-funnel prompts in three categories, software, products, and agency recommendations, run through ChatGPT. Across 26,283 source URLs, "best X" blog lists accounted for 43.8 percent of all page types. The study counted both first-party mentions, where a recommended brand's own site is cited, and third-party mentions. In a separate check of 250 "best X software" search results pages, Allsopp found that 169, or 67.6 percent, featured a list in which the company that wrote the article ranks itself first.19
On December 12, 2025, Rand Fishkin posted about the study, writing that it showed how to "effectively spam your way into AI mentions" with self-promotional lists.20 The same day, Ahrefs published a second study of 75,000 brands, this time covering ChatGPT, AI Mode, and AI Overviews. It found that YouTube mentions correlated most strongly with AI visibility, at about 0.737, and that ChatGPT showed weaker correlations with classic authority measures, with branded search volume at 0.352 and Domain Rating at 0.266. The three assistants largely mentioned the same brands, with an output overlap correlation of 0.779.21
January to June 2026: Guidance and Reporting From the Platforms
Microsoft Advertising's January guide to GEO, The Wall Street Journal's January 30 feature, and Google's May guide are covered in The Origins of GEO. In June, Google began rolling out Search Console insights on which pages appear in AI responses and in which countries, as Google Answers Back describes.22
Key Organizations
Key Organizations in The Citation Economy, 2023 to 2026
| Organization | Role in this period | Key dates |
|---|---|---|
| Ahrefs | Published studies of which factors and page types correlate with mentions in AI answers.13,19,21 | May 26, December 4, and December 12, 2025 |
| Answer.AI | Published the llms.txt proposal.8 | September 3, 2024 |
| The Associated Press | Licensed part of its text archive to OpenAI.1 | July 13, 2023 |
| Axel Springer | Partnered with OpenAI on summaries with attribution and links in ChatGPT.2 | December 13, 2023 |
| Cloudflare | Blocked AI crawlers by default and began a pay per crawl beta.16,17 | July 1, 2025 |
| News Corp | Gave OpenAI permission to display content from its mastheads.6 | May 22, 2024 |
| OpenAI | Signed licensing agreements and launched ChatGPT search with citations.2,6,10 | July 2023 to October 31, 2024 |
| Perplexity AI | Launched a publisher program with revenue sharing.7 | July 30, 2024 |
| Licensed data access to Google and OpenAI.3,5 | February 22 and May 16, 2024 | |
| Semrush | Compared citations on four platforms with Google rankings; its June 2025 analysis of cited domains was charted by Visual Capitalist.18,14 | June and July 21, 2025 |
| Tow Center for Digital Journalism | Tested the citation accuracy of eight generative search tools.12 | March 6, 2025 |
Further reading
- AI Search Has a Citation Problem: Klaudia Jaźwińska and Aisvarya Chandrasekar, Tow Center for Digital Journalism, March 6, 2025.
- Introducing pay per crawl: Will Allen and Simon Newton, Cloudflare, July 1, 2025.
- Do Self-Promotional "Best" Lists Boost ChatGPT Visibility?: Glen Allsopp, Ahrefs, December 4, 2025.
- /llms.txt: a proposal to provide information to help LLMs use websites: Jeremy Howard, Answer.AI, September 3, 2024.
Notes
- "AP, Open AI agree to share select news content and technology in new collaboration," The Associated Press, July 13, 2023. https://www.ap.org/media-center/press-releases/2023/ap-open-ai-agree-to-share-select-news-content-and-technology-in-new-collaboration/
- "Partnership with Axel Springer to deepen beneficial use of AI in journalism," OpenAI, December 13, 2023. https://openai.com/index/axel-springer-partnership/
- "Expanding our Partnership with Google," Reddit, February 22, 2024. https://redditinc.com/news/reddit-and-google-expand-partnership
- "Reddit strikes $60M deal allowing Google to train AI models on its posts, unveils IPO plans," NBC News (Associated Press), February 23, 2024. https://www.nbcnews.com/tech/tech-news/reddit-strikes-60m-deal-allowing-google-train-ai-models-posts-unveils-rcna140168
- "OpenAI and Reddit Partnership," OpenAI, May 16, 2024. https://openai.com/index/openai-and-reddit-partnership/
- "A landmark multi-year global partnership with News Corp," OpenAI, May 22, 2024. https://openai.com/index/news-corp-and-openai-sign-landmark-multi-year-global-partnership/
- "Introducing the Perplexity Publishers' Program," Perplexity, July 30, 2024. https://www.perplexity.ai/hub/blog/introducing-the-perplexity-publishers-program
- Jeremy Howard, "/llms.txt: a proposal to provide information to help LLMs use websites," Answer.AI, September 3, 2024. https://www.answer.ai/posts/2024-09-03-llmstxt
- Google Search Central, "AI features and your website," documentation, read October 30, 2026. https://developers.google.com/search/docs/appearance/ai-features
- Kylie Robison, "OpenAI's search engine is now live in ChatGPT," The Verge, October 31, 2024. https://www.theverge.com/2024/10/31/24283906/openai-chatgpt-live-web-search-searchgpt
- Kyle Wiggers, "OpenAI launches its Google challenger, ChatGPT Search," TechCrunch, October 31, 2024. https://techcrunch.com/2024/10/31/openai-launches-its-google-challenger-chatgpt-search/
- Klaudia Jaźwińska and Aisvarya Chandrasekar, "AI Search Has a Citation Problem," Columbia Journalism Review, Tow Center for Digital Journalism, March 6, 2025. https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php
- Louise Linehan, "An Analysis of AI Overview Brand Visibility Factors (75K Brands Studied)," Ahrefs, May 26, 2025. https://ahrefs.com/blog/ai-overview-brand-correlation/
- "Ranked: The Most Cited Websites by AI Models," Visual Capitalist, charting a Semrush analysis of more than 150,000 citations from June 2025. https://www.visualcapitalist.com/ranked-the-most-cited-websites-by-ai-models/
- Pew Research Center, "Google users are less likely to click on links when an AI summary appears in the results," July 22, 2025. https://www.pewresearch.org/short-reads/2025/07/22/google-users-are-less-likely-to-click-on-links-when-an-ai-summary-appears-in-the-results/
- "Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large; Permission-Based Approach Makes Way for A New Business Model," Cloudflare press release, July 1, 2025. https://www.cloudflare.com/press/press-releases/2025/cloudflare-just-changed-how-ai-crawlers-scrape-the-internet-at-large/
- Will Allen and Simon Newton, "Introducing pay per crawl: Enabling content owners to charge AI crawlers for access," Cloudflare Blog, July 1, 2025. https://blog.cloudflare.com/introducing-pay-per-crawl/
- "How Google's AI Mode Compares to Traditional Search and Other LLMs [AI Mode Study]," Semrush, July 21, 2025. https://www.semrush.com/blog/ai-mode-comparison-study/
- Glen Allsopp, "Do Self-Promotional \"Best\" Lists Boost ChatGPT Visibility? Study of 26,283 Source URLs," Ahrefs, December 4, 2025. https://ahrefs.com/blog/best-lists-research/
- "AI Assistants Keep Turning to \"Best\" Lists, New Study Shows," Stan Ventures, December 12, 2025, quoting a post by Rand Fishkin. https://www.stanventures.com/news/ai-assistants-keep-turning-to-best-lists-new-study-shows-6261/
- Louise Linehan, "Top Brand Visibility Factors in ChatGPT, AI Mode, and AI Overviews (75k Brands Studied)," Ahrefs, December 12, 2025. https://ahrefs.com/blog/ai-brand-visibility-correlations/
- Mrinalini Loew, "New opportunities, control and insights for website owners," Google, June 3, 2026, updated August 31, 2026. https://blog.google/products-and-platforms/products/search/new-controls-website-owners/