Technology
AI chatbots outperform search engines in debunking foreign propaganda, study finds
An experiment by NPR and NewsGuard found AI chatbots countered state-backed falsehoods about 75% of the time, while search engine AI summaries performed worse.
The short version
- An NPR and NewsGuard study testing 30 false narratives from China, Iran, and Russia found popular AI chatbots successfully debunked propaganda roughly three-quarters of the time.
- Search engine AI summaries underperformed compared to chatbots and failed to challenge false claims at a higher rate than traditional search engine results.
- Major tech companies, including Google and DuckDuckGo, defended their search products and challenged the experiment's methodology.
- Experts emphasized that AI performance relies on the availability of factual online sources and the language used in prompts.
Key facts
- NPR and NewsGuard tested 30 queries based on false narratives spread by China, Iran, and Russia between December 2025 and July 2026.[NPR]
- AI chatbots like ChatGPT and Gemini correctly debunked state-sponsored false narratives about three-quarters of the time, outperforming traditional web search results.[NPR]
- AI summaries at the top of search engine results failed to challenge false narratives at a higher rate than standard search links.[NPR]
- Google's AI Overview debunked false claims most of the time, whereas Microsoft Bing's AI summaries failed to debunk false narratives most of the time.[NPR]
- A study from Washington University in St. Louis found that approximately 1 in 9 factual claims in Google AI overviews lacked support from their cited sources.[NPR]
What remains uncertain
- Google and DuckDuckGo disputed the study's methodology, arguing the test queries were rare and that some flagged responses provided context and links for users to evaluate independently.[NPR]
- The extent to which language impacts AI accuracy remains variable, as a study in Nature indicated that asking questions in Chinese returned more favorable responses about China's government than asking in English.[NPR]