Skip to content

The ChatGPT Citations Paper, Explained

Published November 19, 2026

When someone asks ChatGPT a question that needs current information, ChatGPT searches the web, looks through the pages it finds, and shows a few of them as citations next to its answer. For a brand, being one of those cited pages is what puts its website in front of the person asking. Most studies of AI search look only at the finished answer and the links it shows. The steps in between, where ChatGPT decides what to search for and which of the pages it found deserve a citation, have stayed mostly out of view, so a team of researchers set out to record them directly.

Our researchers read Understanding ChatGPT Citations: Evidence from Production Web Search Across Model Generations, a paper by Yunxuan Jiang, Shiqi Zhang, Anqi Xing, Xiangyu Chang, and Xiaokui Xiao of Xi'an Jiaotong University and the National University of Singapore, and wrote this report to explain its findings in simple terms and what they mean for brands. The paper was posted to arXiv on October 11, 2026. Between March and July 2026, the authors asked the ChatGPT website 80 real user questions many times over, using four versions of ChatGPT, and recorded the searches ChatGPT wrote, every page it found, and the pages it chose to cite. They then tested how well two outside sources that anyone can query, Google search results and OpenAI's developer API, could predict which pages ChatGPT cites.

In the following sections, we explain how the experiment worked, what the authors concluded, how each version of ChatGPT searched, how many of the pages it found it went on to cite, which pages were most likely to be cited, how well outside tools can stand in for ChatGPT, and the limits to keep in mind.

The Main Experiment

The authors wanted to see the whole path from a user's question to the citations in ChatGPT's answer, and to learn whether that path changes from one version of ChatGPT to the next.

The questions

The authors took their questions from Search Arena, a public collection of about 24,000 real conversations between people and AI chatbots that search the web. They picked 80 questions across four kinds of request: looking up a fact, pulling information together from several places, analysis, and asking for recommendations. Each kind had 20 questions, half in English and half in Chinese.

The four versions of ChatGPT

When ChatGPT answers in a web browser, its servers send the answer back in small pieces, and that stream of data also carries the searches ChatGPT runs and the pages it finds. The authors recorded this stream in their own logged-in ChatGPT sessions for four versions. GPT-5.3 Instant and GPT-5.5 Instant answered all 80 questions 20 times each. GPT-5.2 Instant and GPT-5.4 Thinking were retired from ChatGPT before the authors finished, so their results cover fewer questions.

The Four ChatGPT Versions in the Study
The Four ChatGPT Versions in the Study
ChatGPT versionQuestionsTimes each question was askedAnswers recorded
GPT-5.2 Instant327 to 28547
GPT-5.3 Instant80201,600
GPT-5.4 Thinking304 to 20491
GPT-5.5 Instant80201,600

Key Findings

  • The authors counted a page as cited when ChatGPT chose it as a reference for its answer. That count includes supporting pages ChatGPT attached to a chosen reference, some of which the user never sees as a separate link.
  • They studied the results at three levels: the exact page (one web address), the website (with blog.example.com counted separately from example.com), and the brands, companies, products, and other entities mentioned in a page's title and summary.
  • For the outside comparison, they asked OpenAI's developer API each question 20 times with web search turned on, using the comparable models gpt-5.3-chat-latest and gpt-5.5, for 1,600 calls per model. They also collected the first five pages of Google results for each question, which held 43.2 results on average. Both were gathered within three days of the ChatGPT answers to the same question.

What the Paper Concluded

The authors conclude that being found by ChatGPT and being cited by ChatGPT are two separate steps. ChatGPT finds many more pages than it cites, it finds a different mix of pages each time the same question is asked, and the pages it finds again and again are the ones most likely to be cited. Google results and OpenAI's developer API reproduce only part of this process. They match the websites ChatGPT cites far better than the exact pages, and even combined they contain only about a quarter of the exact pages ChatGPT cites.

The way ChatGPT searched also changed from one version to the next. For anyone trying to measure or improve their visibility in ChatGPT, the authors recommend tracking the searches ChatGPT writes for itself, asking each question many times, watching websites and brands alongside individual pages, and testing strategies again whenever OpenAI changes the model.

How Each Version of ChatGPT Searched

Before it searches, ChatGPT rewrites the user's question into one or more searches of its own. The authors counted how many searches each version ran per answer and how many separate rounds of searching it used.

How Each ChatGPT Version Searched Before Answering
How Each ChatGPT Version Searched Before Answering
ChatGPT versionSearches per answerAnswers with more than one searchRounds of searching per answerAnswers with more than one roundAnswers that searched the user's exact words
GPT-5.2 Instant2.36100.0%1.1817.4%5.5%
GPT-5.3 Instant1.066.2%1.065.5%0.0%
GPT-5.4 Thinking11.8299.4%3.2071.3%0.0%
GPT-5.5 Instant1.4531.3%1.043.8%0.0%

Key Findings

  • GPT-5.2 Instant ran more than one search in every answer, while GPT-5.3 Instant did so in 6.2% of answers and GPT-5.5 Instant in 31.3%. All three Instant versions usually searched in a single round.
  • ChatGPT almost never searched for the user's question word for word. GPT-5.2 Instant did so in 5.5% of answers, and the two later Instant versions never did.
  • The search language changed between versions. For questions asked in Chinese, GPT-5.2 Instant searched in both Chinese and English in 96.8% of answers, while GPT-5.3 Instant and GPT-5.5 Instant searched in English alone in 67.0% and 69.6% of answers.

GPT-5.4 Thinking searched far more, with 11.82 searches over 3.20 rounds per answer. It used several rounds in 87.6% of its answers to questions that pulled information together, 85.2% of analysis questions, and 74.4% of recommendation questions, compared with 10.8% of simple fact lookups. Its later searches grew more targeted. The share that limited results to one website with the site: command rose from 25.1% in the first round to 54.7% in later rounds, and the share with a phrase in quotation marks rose from 1.0% to 22.7%. Of its 4,077 searches after the first round, only 13 repeated an earlier search.

How Many of the Pages It Found ChatGPT Cited

The authors then compared the number of pages each version found while searching with the number it cited in the answer.

Pages ChatGPT Found and Pages It Cited per Answer, by Version
Horizontal bar chart of the average number of web pages ChatGPT found and cited per answer. GPT-5.2 Instant found 39.4 and cited 5.9. GPT-5.3 Instant found 18.7 and cited 5.6. GPT-5.4 Thinking found 73.5 and cited 7.8. GPT-5.5 Instant found 23.5 and cited 6.5.

Key Findings

  • GPT-5.4 Thinking found 73.5 pages per answer and cited 7.8 of them. The three Instant versions found between 18.7 and 39.4 pages and cited between 5.6 and 6.5.
  • Most of the pages ChatGPT finds while it searches never appear among its citations. For a brand, turning up in ChatGPT's search results is the first step, and being cited is a separate decision that comes after it.

Asking the same question again brought back many different pages. Across 20 runs of one question, GPT-5.5 Instant found 132.2 different exact pages on average and GPT-5.3 Instant found 96.4. The authors sorted every page, website, and entity by the number of those 20 runs it appeared in.

How Often a Source Reappeared Across 20 Runs of the Same Question
Stacked bar chart of how many of 20 runs of the same question each source appeared in. Exact pages: 40.8% appeared in only one run for GPT-5.3 Instant and 44.5% for GPT-5.5 Instant, while 9.4% and 7.7% appeared in 14 or more runs. Websites: 31.3% and 35.2% in one run, 17.0% and 15.1% in 14 or more. Brands or entities: 33.4% and 35.1% in one run, 17.4% and 15.0% in 14 or more. The rest appeared in 2 to 13 runs.

Key Findings

  • Of the exact pages GPT-5.3 Instant found, 40.8% showed up in only one of the 20 runs, and 44.5% for GPT-5.5 Instant. Only 9.4% and 7.7% showed up in 14 or more runs.
  • Websites and entities came back more reliably. For GPT-5.3 Instant, 17.0% of websites and 17.4% of entities appeared in 14 or more runs, and for GPT-5.5 Instant the shares were 15.1% and 15.0%.
  • The authors also scored how much the lists from two runs of the same question overlapped, from 0 for nothing shared to 1 for identical. The overlap was 0.29 to 0.30 for exact pages and 0.40 to 0.42 for websites and entities, so ChatGPT often found different pages from the same websites, about the same brands, on each run.

Which Pages Were Most Likely to Be Cited

To find signs of which pages get cited, the authors set each run aside in turn and checked whether a page's appearances in the other 19 runs predicted whether it was cited in the run set aside. They scored each signal with a measure called AUC, where 0.5 means the signal does no better than a coin flip and 1.0 means it separates cited pages from uncited pages perfectly.

Pages found in more of the other runs were more likely to be cited. With GPT-5.3 Instant, this signal scored 0.619 for exact pages, 0.666 for websites, and 0.688 for entities. With GPT-5.5 Instant, it scored 0.651, 0.709, and 0.711. The second signal was the snippet, the short summary of a page that ChatGPT sees in its search results.

Share of Search Snippets With 500 or More Characters, Cited and Uncited Pages
Bar chart of the share of search snippets with at least 500 characters. For GPT-5.3 Instant, 16.5% of cited pages and 1.5% of pages not cited. For GPT-5.5 Instant, 16.9% of cited pages and 1.9% of pages not cited.

Key Findings

  • Among cited pages, 16.5% had a snippet of at least 500 characters with GPT-5.3 Instant and 16.9% with GPT-5.5 Instant, compared with 1.5% and 1.9% of pages that were not cited. Used alone, snippet length scored 0.612 and 0.625, a weaker signal than appearing across many runs.
  • The authors treat snippet length as a pattern they can observe and stop short of calling it a cause. A long snippet may give ChatGPT a complete piece of evidence to quote, and length can also reflect the type of page or where it sits in the results.
  • Position in the search results helped somewhat, scoring 0.561 for GPT-5.3 Instant and 0.623 for GPT-5.5 Instant. Even so, 24.0% and 19.0% of pages ranked below 15th place were still cited.
  • How recently a page was published made no clear difference, with scores of 0.482 and 0.535 that sat within the range of chance.

The type of page made some difference as well. Among the three kinds of reference both versions used, which were general search results, news, and academic sources, news pages had the highest share cited at least once with GPT-5.3 Instant, at 52.0%, and academic sources had the highest with GPT-5.5 Instant, at 49.2%.

How Well Outside Tools Stand In for ChatGPT

Brands cannot look inside ChatGPT the way the authors did, so the authors tested two sources that anyone can query: Google results and OpenAI's developer API, which reports both the pages its model found and the pages it cited. For each source they measured two things: the share of the source's pages that ChatGPT also cited, and the share of ChatGPT's cited pages that the source contained.

How Closely Each Outside Source Matched GPT-5.5 Instant's Citations
How Closely Each Outside Source Matched GPT-5.5 Instant's Citations
Outside sourceExact pages: share ChatGPT also citedExact pages: share of ChatGPT's citations foundWebsites: share ChatGPT also citedWebsites: share of ChatGPT's citations found
Google results4.1%6.4%12.5%22.8%
OpenAI API, all pages found7.1%22.5%17.1%38.1%
OpenAI API, pages cited23.3%13.0%40.5%23.9%
Google and API pages found, combined5.7%26.2%12.8%48.7%

Key Findings

  • The API's citations were the closest match. GPT-5.5 Instant also cited 23.3% of the exact pages and 40.5% of the websites the API cited, up from 16.4% and 27.7% for GPT-5.3 Instant.
  • The API's full list of found pages covered more of ChatGPT's citations, at 22.5% of exact pages and 38.1% of websites. Google alone was the weakest match, at 6.4% and 22.8%.
  • For anyone tracking AI visibility with these tools, the API's citations are a closer guide to what ChatGPT cites than Google rankings, although none of the sources comes near a full match.

The authors also combined Google results with the API's found pages to see how much of ChatGPT's citations the two could cover together.

Share of ChatGPT's Cited Sources Found by Google and the OpenAI API Combined
Bar chart of the share of ChatGPT's cited sources that Google results and the OpenAI API's found pages contained when combined. Exact pages: 28.3% for GPT-5.3 Instant and 26.2% for GPT-5.5 Instant. Websites: 50.3% and 48.7%. Brands or entities: 50.7% and 45.1%.

Key Findings

  • Together, Google and the API contained 28.3% of the exact pages GPT-5.3 Instant cited and 26.2% of those GPT-5.5 Instant cited. They contained 50.3% and 48.7% of the cited websites, and 50.7% and 45.1% of the cited entities.
  • Because exact pages are so hard to match from the outside, the authors suggest watching which websites and brands ChatGPT cites alongside the individual pages.

Predicting Which Pages ChatGPT Will Cite

Finally, the authors trained a machine-learning model to predict ChatGPT's citations from the outside signals alone: a page's rank in Google and in the API, how often it appeared across the 20 API runs, how often the API cited it when it appeared, how many pages the API found on the same website or about the same entity, snippet length, and the type and language of the question. The model could rank only pages that Google or the API had found. The table shows its AUC scores for GPT-5.5 Instant beside simpler methods.

How Well Each Method Picked Out GPT-5.5 Instant's Cited Sources (AUC)
How Well Each Method Picked Out GPT-5.5 Instant's Cited Sources (AUC)
MethodExact pageWebsiteBrand or entity
Trained model using all outside signals0.7790.7630.742
Combined ranking of all outside signals0.7720.7560.742
Combined Google and API ranks0.6990.7160.708
OpenAI API citations alone0.6990.6930.674
Random guess0.5000.5010.499

Key Findings

  • The trained model scored highest in five of the six combinations of version and level and tied the combined ranking for GPT-5.5 Instant entities. For GPT-5.3 Instant its scores were 0.782 for exact pages, 0.763 for websites, and 0.743 for entities.
  • Rank was the strongest signal for exact pages and websites. At the exact-page level, its share of the model's predictive weight fell from 49.7% for GPT-5.3 Instant to 34.9% for GPT-5.5 Instant, while the weight of how often the API found and cited a page rose from 19.0% to 28.3%.
  • For entities, the largest signal was the number of different pages the API found about the same entity, at 23.9% and 24.6% of the weight.
  • These scores apply only to pages that Google or the API found, so even perfect predictions would recover only 26.2% to 28.3% of the exact pages ChatGPT cited.

Limits to Keep in Mind

  • The study covers one AI engine, the ChatGPT website, in four versions recorded between March and July 2026. Its 80 questions were in English and Chinese only.
  • Supporting pages attached to a chosen reference counted as cited even when ChatGPT did not show them as separate links, so the citation counts are somewhat broader than what a user sees.
  • All of the signals are associations. The authors stress that publishing more pages, or making a page appear more often in search results, will not necessarily get it cited.
  • Two of the authors' sessions were switched to a fallback model without notice after 14 and 16 successful runs, so anyone measuring ChatGPT repeatedly should check which model answered each time.
  • GPT-5.2 Instant and GPT-5.4 Thinking were retired before collection finished, so their results rest on fewer questions that differ from the 80 used for the other two versions.

What This Means for Brands in 2027 and Beyond

For a brand, the study separates two jobs: being found when ChatGPT searches and being picked as one of the few pages it cites. ChatGPT found up to 73.5 pages per answer and cited fewer than 8 of them, and the pages that turned up across many runs of the same question were the ones most likely to be cited. Pages with long summaries in the search results were cited more often too. The authors treat these patterns as associations and make no claim that changing a page would change the result, but they do suggest that the brands ChatGPT cites most are the ones that turn up across the many different searches it writes for a question.

The study also changes how AI visibility should be measured. ChatGPT rewrites each question before searching, finds a different set of pages on every run, and changed how it searched between versions, so a single check of a single question shows a small slice of what it can cite. Google rankings matched only a small share of its citations. Tracking the websites and brands ChatGPT cites, across many repeats of each question, gives a steadier picture than tracking single pages, and every result needs a fresh check after OpenAI releases a new model. An earlier University of Toronto study of what drives AI citations found that ChatGPT's own retrieval explained 32% of the differences in which pages on a website got cited, against 2.8% for page features, which fits the link found here between the pages ChatGPT finds across its searches and the pages it cites. Brands that want help tracking their AI visibility can compare specialist firms in our directory of GEO and AEO agencies.

So if you track how ChatGPT cites your brand, ask each question many times, watch your websites and brand mentions as well as your individual pages, and check your results again whenever ChatGPT's model changes.

Data Collection and Methodology

We read the arXiv version of the paper (2610.15382v1, posted October 11, 2026) and took every figure in this article from the paper's text, tables, and figures.

The number of questions, repeats, and answers for each version comes from the paper's Table 1, the pages found and cited per answer from its Table 2, and the search counts from its Table 3, where GPT-5.3 Instant's searches were visible in 1,339 of its 1,600 answers. The shares of sources appearing in one run and in 14 or more runs come from the paper's Figure 2 and its results text. The share appearing in 2 to 13 runs is what remains after subtracting those two shares from 100%. The overlap scores, the AUC scores for repeat appearances, snippet length, position, and recency, the snippet shares, and the page-type shares come from the paper's Section 3.3. The outside-source figures come from its Tables 4 and 5 and the combined coverage figures from its Sections 4 and 5. The prediction scores come from its Table 7 and the predictive weights from its Figure 5 and Section 5.3. The paper publishes 95% confidence intervals for several figures, and we left them out of the charts for readability.

Sources