The C-SEO Bench Paper, Explained

C-SEO Bench: Does Conversational SEO Work? is a 2025 paper by Haritz Puerto, Martin Gubri, Tommaso Green, Seong Joon Oh, and Sangdoo Yun, researchers at Parameter Lab, the Technical University of Darmstadt, the University of Mannheim, the University of Tübingen, and NAVER AI Lab. It was published at the 39th Conference on Neural Information Processing Systems (NeurIPS 2025). The paper retests the content edits proposed in the original GEO paper on six kinds of searches and four AI models, and it adds a test where many competing websites use the same edits at once (pages 1 to 3, Sections 1 and 2).

Our researchers read through the paper and created this report to break down its findings in simple terms and talk about what they mean for brands and marketers. In the following sections, we explain how the benchmark was built, what the authors concluded, how each content edit performed, why a page's position in search results mattered more than its wording, what happened when competitors copied each other, and the limits the authors point out themselves.

The Main Experiment

The authors call AI tools that search the web and then write an answer with citations "conversational search engines," and they call the practice of editing pages to appear earlier in those answers conversational SEO, or C-SEO (page 1, Section 1). Their question was whether today's C-SEO methods move a page up in AI answers across many kinds of searches.

The setup

An AI model receives a user's question along with a list of documents found by a regular search engine, and it writes an answer with inline citations to those documents (page 4, Section 3). For product questions, the model sees 10 product descriptions in random order and must recommend the best five, using only those descriptions (page 5, Section 4). The researchers ran this setup with four answer models: OpenAI's gpt-4o-mini, o3, and o4-mini, and Anthropic's Claude 3.5 Haiku. All rewrites were made with gpt-4o-mini (page 8, Section 6.1).

The benchmark

The benchmark covers two tasks with three subject areas each. The table below lists them, using Table 2 of the paper (page 5).

• The product data came from the Amazon Shopping Queries dataset, Steam video game pages, and the Google Books API (pages 5 and 6, Section 4.1).
• The question data came from real Google search questions with pages retrieved through the Brave Search API, from sets of news articles about the same event, and from a debate dataset with sources on both sides of a question (page 6, Section 4.1).
• In total, the benchmark holds more than 1,900 questions and 16,000 documents (page 6, Section 4.1).

What they tested

The researchers tested the eight rewriting methods from the original GEO paper: a more authoritative tone, added statistics, added citations, smoother writing, unusual words, technical terms, simpler language, and added quotes. They also tested two new methods. Content Improvement combines all eight edits into one rewrite, and LLM Guidance adds a short summary in the style of an llms.txt file to the top of the page (page 7, Section 5). In the main test, one document per question was picked at random and rewritten, so its position in the search results would not affect the outcome (page 8, Section 6.1).

How they measured success

• The main score is the change in citation order. If a page was cited fourth before the edit and second after it, it gained two positions (page 6, Section 4.2).
• Each result was checked with a statistical test, adjusted for the number of comparisons, to see whether the gain was larger than chance (page 6, Section 4.2).
• For the competition test, the share of documents using the same method was raised from 10% to 100% of up to ten documents per question (page 8, Section 6.1).

What the Paper Concluded

The authors concluded that most current C-SEO methods do little to move a page up in AI answers and often move it down. In their words, the methods "are not only largely ineffective but also frequently have a negative impact on document ranking" (page 1, Abstract). Out of 54 cases, they found only three where a rewrite produced a statistically significant gain, all in product recommendations (page 9, Section 6.2). Where a document sat in the list the AI read had a much larger effect than any rewrite, and the gains from rewriting shrank as more competitors adopted the same method (pages 9 and 10, Sections 6.3 and 6.4). The authors' view is that "C-SEO will not replace SEO, but will complement it" (page 2, Section 1).

How Each Content Edit Performed

The heatmap below shows the average number of positions each method gained or lost in each subject area when gpt-4o-mini wrote the answers, using Table 3 of the paper (page 8). The bottom row shows what happened when the same document was simply moved to first place in the list the AI read.

Re-charted from Puerto et al. Table 3.

• LLM Guidance produced significant gains for retail products and video games, and Content Improvement did so for retail only. No method worked for the question-answering tasks, and no method worked when Claude 3.5 Haiku wrote the answers (page 9, Section 6.2).
• Most edits left the ranking where it was. With LLM Guidance on retail products, 61.0% of ranks were unchanged, 26.2% improved, and 12.8% got worse, so the average effect stayed small (pages 8 and 9, Section 6.2).
• Adding statistics was the most harmful edit. It lowered rankings in 19 out of 24 settings, and it cost a debate source 0.80 positions on average with gpt-4o-mini (page 9, Section 6.2; page 8, Table 3).
• The losses were larger on some models. With Claude 3.5 Haiku, 26 of 30 product recommendation cases showed significant drops, and with o4-mini, 19 of 30 question-answering cases did (page 9, Section 6.2).

Why this differs from the original GEO paper

The original GEO paper reported clear gains from edits like adding quotes and statistics. The authors explain that the gap comes from what is measured. The original paper's main score counts how many words of the answer are tied to a page, and a page can be discussed at length without being cited first. The authors add that the original paper's position-weighted results "show a general decrease in the scores," so on citation order the two papers agree (page 9, Section 6.2).

Why Search Position Mattered More

To measure the effect of traditional SEO, the authors took a random document and placed it at each position in the list the AI read, from first to tenth, then measured how its citation rank changed (page 9, Section 6.3). The first chart compares the best rewrite in each subject area with the effect of moving the document to first place.

Re-charted from Puerto et al. Table 3.

Re-charted from Puerto et al. Table 5.

• The top three positions gave the largest gains, and the last few positions pushed documents down, since most documents moved to a worse slot than they started in (page 9, Section 6.3).
• Moving a document to first place gave a significant gain in every subject area for all four answer models (pages 15 and 16, Tables 7, 8, 10, and 12).
• The authors conclude that "the position of a document in the input context has a dominant effect on how the LLM ranks or selects content," and that traditional SEO remains critical for visibility in AI answers (page 9, Section 6.3).

What Happened When Competitors Copied Each Other

The authors also simulated a market where more and more websites use the same method. They tested the two strongest methods, LLM Guidance and Content Improvement, on retail products and video games, the two areas where gains showed up (page 9, Section 6.4).

• Early adopters saw meaningful gains, but the average gain per adopter fell steadily as more documents used the same method and approached zero when every document used it (page 10, Section 6.4).
• Measured across all adoption levels, LLM Guidance on retail products held up best, with an area-under-the-curve score of 1.88, compared with 1.05 for Content Improvement on the same products (page 10, Figure 5).
• The authors describe C-SEO as "both a congested and a zero-sum game," because one page's move up the citation list pushes another page down (page 10, Section 6.4).

Limits the Authors Note

The authors list several limits in Section 7 (page 10).

• The study covers only "white-hat" methods, meaning honest edits a site owner makes to its own pages. Manipulative techniques such as hidden instructions to the AI were left out.
• They did not test how SEO and C-SEO methods might work together or against each other.
• All answer models were commercial proprietary models.
• Every document fit inside the AI's reading window, so the study does not cover cases where pages have to be shortened or summarized first.
• The data is English only, which may limit how well the findings carry over to other languages.

The position test also simulated search rank by reordering the documents the AI read, and all rewrites were produced by one model, gpt-4o-mini (page 8, Section 6.1; page 9, Section 6.3).

What This Means for Brands in 2027 and Beyond

AI answers start from a search. The pages an AI system reads first are the ones it tends to cite first, so the work that lifts a page into the top few results of the search engine behind an AI tool still carries most of the weight in whether that page gets mentioned.

Rewriting copy to sound more authoritative or to pack in numbers carries a real risk of doing harm, and the small wins in this study were limited to product listings. Content changes aimed at AI tools are worth testing on a few pages at a time, with a plan to undo the ones that do not help.

The competition test also suggests that any rewriting trick that works will fade as rivals copy it. That pushes lasting advantage toward things that are harder to copy, such as strong search rankings and the reputation that earns them. Keep investing in the search rankings of the pages you want AI tools to cite, and treat AI-specific rewrites as experiments you measure.