Aleksandra Urman
Postdoctoral Researcher, Social Computing Group
University of Zurich
Aleksandra Urman is a computational social scientist and postdoctoral researcher in the Social Computing Group at the University of Zurich. She audits search engines and AI chatbots to measure what information they show, and has studied how chatbots answer political questions and how reliable Google's AI Overviews are.
Early Warnings on Chatbot Search
"It's completely untransparent how [AI-powered search] is going to work, which might have major implications if the language model misfires, hallucinates or spreads misinformation," Urman told Nature in February 2023, as Microsoft and Google added chatbots to search. She argued that people may trust a conversational answer more than a list of links, and that enough errors could instead erode the view of search engines as impartial. Her own survey work found that almost 80% of respondents considered Google's featured snippets and knowledge panels accurate, and about 70% thought they were objective.
Chatbots and Political Questions
With Mykola Makhortykh of the University of Bern, Urman tested how Google Bard and other chatbots answered questions about political leaders in Russian, Ukrainian and English. Asked in Russian, Bard declined to discuss Vladimir Putin in 90% of cases, compared with 30% to 40% for Joe Biden, Volodymyr Zelensky and Alexei Navalny. Before the 2024 European elections she concluded in Correctiv that chatbots should not yet be used for political information or any other factual search by people who want reliable answers. Fixes, she added, depend on the companies: even a perfect technical solution would leave it to them "to decide, if and how they would want to implement it."
Auditing Search Results at Scale
Much of Urman's work uses virtual agents that run the same searches across engines, locations and dates. A 2022 audit she led compared conspiracy-related results on Google, Bing, DuckDuckGo, Yahoo and Yandex, and found Google returned the smallest share of conspiracy-promoting pages. More recently she co-authored an audit of Google's AI Overviews and featured snippets on 1,508 baby care and pregnancy queries with Anikó Hannák and Christo Wilson, which found the two features contradicted each other on a third of the pages where both appeared.
Urman completed her doctorate at the University of Bern, where she was a research assistant at the Institute of Communication and Media Studies and wrote her dissertation on political polarization on social media. She has published more than 50 peer-reviewed papers and has commented on Telegram for the Financial Times, the BBC and CNBC.