Who Chose Your Sources? A Journalist’s Guide to AI Research
AI agents can help reporters find documents, follow leads and compare accounts. The harder question is whether their source choices are visible enough to challenge. This is a guide to auditing the research, not ranking chatbots by political personality.
THE QUESTION. The notebook and the search window belong to the same reporting process. Delegating a search does not settle which evidence a story needs. Illustrative photograph; not a participant in the cited research. Photograph: MJ Duford / Unsplash.
Imagine a reporter asking an AI assistant why a government changed a policy. The reply arrives quickly, complete with a chronology and five credible-looking links. Every link opens. Every quotation checks out. But all five accounts ultimately depend on the same government announcement.
Nothing in that hypothetical answer has to be fabricated for the reporting to remain incomplete. A court challenge, a local investigation or the testimony of someone affected could alter the story. The most consequential choice may be the source the assistant never brought into view.
That is the problem behind a deceptively simple question: who chose your sources? A web-enabled assistant can do more than retrieve material. It can reformulate the question, select evidence and organize it into a narrative before the reporter has inspected the underlying record. OpenAI and Google describe versions of that search-and-synthesis process in their documentation. [4] [5]
For journalism, the opportunity is real: delegate preliminary discovery, not the final judgment about what the evidence supports. An agent is most useful when it expands the reporter’s options and exposes its limitations, not when its fluency makes an unfinished investigation feel complete.
What the 31 percent finding actually says
In its 2025 News Integrity in AI Assistants study, the BBC and European Broadcasting Union reported significant sourcing issues in 31% of the responses in their core-question analysis. The problems included claims unsupported by the cited material, absent sources and incorrect or unverifiable sourcing assertions. Forty-five percent had at least one significant issue across the assessed categories. [1]
The wider project involved 22 public-service media organizations across 18 countries and 14 languages, assessing more than 3,000 responses. Its assistants were ChatGPT, Microsoft Copilot, Google Gemini and Perplexity. Claude and Grok were not part of that comparison. [2]
The denominator matters. The report’s high-level core-question chart lists 2,709 responses from the free or consumer versions; the broader dataset included custom questions. Those findings are a dated evaluation, not a current error rate for every 2026 model, subscription or research agent. Nor is a sourcing error automatically evidence of political bias. [1]
What the audit does establish is a failure mode reporters can check: a citation can be attached to a sentence without supporting it. The presence of a source list is the beginning of verification, not its conclusion.
Bias is not one setting inside a chatbot
NIST distinguishes systemic, computational/statistical and human-cognitive sources of bias. Its framework recognizes that harmful bias can arise without discriminatory intent. That is a useful starting point for an investigation that might otherwise assign each chatbot a simplistic political identity. [3]
For a reporter, it is more useful to examine the stages separately. Did the relevant record enter the searchable material? Did the search find it? Did the assistant select it? Did the summary preserve its meaning? And did the journalist’s own question already assume the desired answer? These are different problems, requiring different checks.
A missed document might reflect limited access, an unsuccessful query, a language gap, a date error or a choice to omit it. A single omission does not establish which explanation is correct. Repeated patterns can justify further investigation; motive still requires evidence.
Accuracy, completeness and fairness are also different tests. A correct statement can lack material context. More sources can repeat one origin. A neutral tone can conceal weak evidence. Fair reporting does not require equal weight for a well-supported finding and an unsupported counterclaim.
THE RECORD. Museums Victoria identifies this scene as the newspaper section of Emily McPherson College Library, circa the 1960s. An archive is a reminder to ask which records a digital search can actually reach. Historical photograph used for illustration. Photograph: Museums Victoria / Unsplash.
What the research tools actually disclose
The following is a documentation-based snapshot checked on October 18, 2026, not a head-to-head performance test. A feature documented for a developer API does not automatically exist in the consumer app. Record the interface, enabled tools and account conditions actually used.
ChatGPT: OpenAI says search sometimes uses partner providers and typically rewrites a question into one or more targeted queries when it does. Relevant saved memories can influence that rewriting. Its Sources view can contain cited material and other relevant links; it is not necessarily a complete record of everything considered. [4]
Gemini: Google documents a workflow in which its search-grounding tool generates queries, processes results and returns citation information. The current API documentation exposes search-call queries and citation annotations in supported responses. That is valuable audit material, but it is not a guarantee that a consumer interface exposes an equivalent log. [5]
Claude: Anthropic’s web-search documentation describes search with citations and controls such as domain restrictions and a maximum-use setting. Supported versions can use dynamic filtering to narrow results before they enter the model’s context. The documentation examined here does not provide a complete account of the underlying ranking system. [6]
Grok: xAI documents X Search and Web Search as separate tools. The first can search posts, profiles and threads on X; the second reaches web content. Whether a given answer drew on social posts, web pages or both should be established from that session’s available evidence, not inferred from the brand. [7] [8]
Microsoft Copilot Chat and agents: Microsoft’s documented experience generates short queries for Bing. Its Sources control can display the exact query and used sources. Access varies with the account, license and organizational settings. This description should not be generalized to every product carrying the Copilot name. [9]
Perplexity Pro Search: Its documentation describes multiple web searches, source links and a choice of underlying models. That last detail matters when comparing products: a different interface does not necessarily mean a different model. Log the selected model where the service exposes it. [10]
These disclosures help identify what to inspect. They do not establish which assistant is least biased. The stronger unit of evaluation is a particular research task, performed with a recorded configuration and checked against the relevant evidence.
A partnership is a lead, not proof of preferential treatment
OpenAI’s May 2024 Reddit partnership announcement documents access to Reddit’s Data API. It does not establish that ChatGPT prioritizes Reddit in every answer, that other assistants cannot access public Reddit material, or that a specific story’s sources were selected because of the agreement. [11]
A commercial relationship deserves scrutiny. To connect it to a sourcing claim, a reporter would need to examine actual results, define a meaningful comparison, consider other explanations and seek the provider’s response. Replacing that work with a suspicion would reproduce the very problem being investigated.
Public visibility remains incomplete. Stanford’s December 2025 Foundation Model Transparency Index identifies continuing opacity around training-data acquisition and composition. The index evaluates disclosure, not truthfulness or political neutrality; greater transparency is not itself proof of more accurate reporting. [12]
There is a further distinction inside vendors’ own evaluations. OpenAI’s 2025 political-bias assessment explicitly excluded web-search behavior because retrieval and source selection involved separate systems. Whatever that evaluation says about model responses, it cannot settle the fairness of the entire search-enabled research process. [13]
Asking a chatbot to describe its own bias may produce a helpful explanation, but it is still generated text. Preserve the tool records that actually exist. If a query log is unavailable, write “not exposed”; do not invite the model to invent a plausible account of searches it supposedly ran.
Three agreeing chatbots are not three independent witnesses
A second assistant can help challenge a first draft, but agreement is not a vote that establishes truth. A peer-reviewed ICML 2025 paper by Elliot Myunghoon Kim and colleagues evaluated more than 350 language models across leaderboard datasets and a résumé-screening task. It found substantial error correlation, including among models from different providers. That was not a newsroom field trial, but it cautions against assuming that different brands produce independent judgments. [14]
Source independence requires another check. Three assistants citing different republications of one wire report still depend on that report. Three assistants citing the same press release do not independently confirm the release’s underlying claim.
A useful second assignment is therefore adversarial rather than repetitive: identify unsupported sentences, trace claims to their earliest available origin, find consequential counterevidence and separate observation from inference. Then check the challenge too. A second system can introduce a new error as readily as it can expose an old one.
The number of links is not the number of independent sources.
A sports rumor and a foreign policy demand different access
Consider two hypothetical assignments. Neither is a test result from this article.
For a sports story asking why a player missed a game, establish the exact player, fixture and date first. Seek the relevant official availability report, a timestamped team statement, attributable beat reporting and an opportunity for the appropriate representative to respond. A fan post can identify a rumor; it does not establish an injury diagnosis. A team statement proves what the team publicly said, not necessarily the full reason behind a decision.
For a story about a foreign government’s new policy, obtain the original text and distinguish announcement, enactment and implementation. Search in the relevant language, consult independent local reporting, and seek knowledgeable and affected people. Separate predictions of impact from measured outcomes. An English summary should not substitute for the operative wording of a consequential provision.
THE LOCAL VIEW. Newspapers for sale in Dakar, Senegal, in an image published by the photographer in January 2022. Local-language reporting should be sought and evaluated on its merits, not assumed to be represented by an English-language summary. Illustrative photograph, not documentation of the hypothetical policy example. Photograph: Ewien van Bergeijk - Kwant / Unsplash.
The appropriate access may involve a library subscription, a specialist database, a public-records request or a conversation. Search access is not permission to bypass a paywall, and an inaccessible document must remain an acknowledged gap. Translation assistance can help locate material; consequential quotations still need competent review in context.
X and Reddit are not automatically worthless because they are social platforms. An authenticated post can establish that its author made a statement. Community discussion can reveal a question worth pursuing. But repetition, a screenshot or an account’s claimed proximity to events does not independently establish the allegation.
The practical objective is evidence diversity, not a decorative mix of domains: original records, independent scrutiny, relevant expertise and affected voices, evaluated for what each can actually know.
Two tests a newsroom can run before trusting the workflow
The supplied research brief proposes both open-web and same-evidence testing. The following is a refined protocol, not an experiment already conducted for this article.
First, test discovery. Choose a tightly defined question and establish a reference packet through separate research, while recognizing that the packet may itself be incomplete. Give the selected assistants the same neutral assignment within a narrow time window. Save their full answers, citations and exposed tool logs, including refusals and access failures. Identify important sources they missed and new sources they genuinely contributed.
Second, test synthesis. Supply each assistant the same documents and request a claim-by-claim evidence memo, with web access disabled where possible. Ask it to distinguish supported findings, attributed claims, contradictions and unanswered questions. Differences from the open-web results can help locate weaknesses in discovery or interpretation, without perfectly isolating their internal causes.
Record model and tool versions where exposed, interface, date, time, language, location context, account tier, document limits and personalization settings. Do not assume that opening a new conversation resets all context. OpenAI’s current temporary-chat documentation, for example, distinguishes personalized and unpersonalized sessions; the name of the mode alone is not enough. [15]
Repeat the assignments to see whether important results change, and retain failures rather than selecting the best-looking answer. Use a predefined rubric: factual and quotation accuracy, citation support, important omissions, source independence, chronology, language coverage and treatment of uncertainty. Report counts with denominators and keep discovery value separate from error rate.
A small pilot can reveal a broken citation or recurring omission. It cannot establish a permanent league table of the least biased AI. Nor should a setting such as zero temperature be treated as a guarantee that a changing web service will reproduce an identical result.
The evidence memo should come before the story
For routine work, a newsroom does not need to repeat a full comparison every time. It needs a record that lets another editor challenge the chain from question to claim. The workflow below is a proposed editorial practice, not a validated scoring instrument.
1. Define the question without embedding the conclusion. Specify the period, place, relevant actors and what would count as evidence against the working explanation. Ask what the story would need even if no chatbot existed.
2. Request leads before prose. Ask for original documents, responsible institutions, independent reporting, local-language sources and unresolved access needs. Require the assistant to distinguish material it opened from material known only through a snippet or another author’s description.
3. Inspect each consequential citation. Open the original record. Check author, date, exact passage, quotation, units, denominator and whether the statement is a finding, allegation or prediction. A later update may change the meaning of a once-correct summary.
4. Search outside the assistant’s shortlist. Follow document references, query the relevant institution directly, consult archives, and seek evidence that could defeat the provisional explanation. Another general search engine can help, but it is not automatically an independent editorial authority.
5. Record what each source proves. Use a claim-evidence log containing the draft claim, original URL, page or passage, source role, date, independent corroboration, unresolved conflict and publication decision. Mark a paywall, broken page or unavailable interview as an access gap, not as evidence of absence.
6. Attribute the evidence, not the assistant. Once a claim is verified against a record, cite that record. “According to a chatbot” normally describes the tool’s output, not independent knowledge of the event. Disclose material AI assistance separately and correct the publication when an error is established.
THE FOLLOW-UP. An agent may find a published statement or a public contact route. It cannot supply an interview that never happened. Illustrative photograph; no interview was conducted for this article. Photograph: Ed Rojas / Unsplash.
The Associated Press provides a useful professional benchmark: its published guidance treats generative-AI output as unvetted source material requiring editorial judgment and warns against entering confidential or sensitive information into AI tools. That is AP’s stated standard, not a guarantee about every newsroom’s practice. [16]
For an agent with connected accounts, extend that caution to permissions. Use only the access necessary for the assignment; require human authorization before contacting sources, submitting forms or publishing. A research question can itself reveal a confidential lead. Removing a name is not a substitute for assessing whether the remaining details identify the person. Use newsroom-approved security arrangements, not a consumer convenience setting, for protected material.
A useful research brief might read: “Build an evidence memo, not an article. Distinguish documents you opened from search snippets and secondhand references. Trace major claims to their origins, identify contrary evidence and local-language reporting, and list access gaps. Preserve exact quotations with their locations. Do not invent a search log, infer a diagnosis or contact anyone. Separate what is established from what still requires reporting.”
That instruction is not a cure for bias. It makes the desired work inspectable.
The point is not to stop using agents
The promise of an agent is not that it has no perspective. It is that it can help a reporter discover more useful material than the reporter could efficiently locate alone. Whether it achieves that should be judged by the evidence it contributes and the work required to verify it, not by the length of its answer.
The reporter still decides which question matters, whose account deserves scrutiny, which contradiction requires a call and when the record supports publication. Those responsibilities cannot be inferred from a tidy bibliography.
AI can widen a journalist’s reach. It should not quietly decide where the reporting ends.
Sources and reporting notes
This article develops the supplied research brief through checks against original studies and current product documentation. It is document-based analysis, prepared with AI assistance in research, drafting and website production. No original interviews or cross-platform performance experiment were conducted for this piece. The sports and foreign-policy assignments and newsroom protocol are explicitly proposed examples.
Verification corrected outdated product descriptions and removed unsupported claims that a platform partnership proves preferential sourcing. The brief’s illustrative response tables, overlap percentages and chart placeholders are not presented as measured results. Tool descriptions were checked October 18, 2026; interfaces and availability can change.
All four photographs are illustrative and individually credited. The historical library image retains the collection’s dating; none depicts a named study or an interview conducted for this article. Photo source pages identify the images as available under the Unsplash License.
Comments