AI Disinfo Hub

The development of artificial intelligence (AI) technologies has long been a challenge for the disinformation field, enabling the manipulation of content and accelerating its spread. Recent technical developments have exponentially increased these challenges. While AI offers opportunities for legitimate purposes, it is also widely generated and disseminated across the internet, causing – intentionally or not – harm and deception.

This hub intends to assist you to better understand how AI is impacting the disinformation field.  To be up-to-date on the latest developments, we will collect the latest Neural News and Trends and include upcoming events and job opportunities that you cannot miss.
 

Are you more into podcast and video content? You will find a repository of podcasts and webinars in AI Disinfo Multimedia, while AI Disinfo in Depth will feature research reports from academia and civil society organisations. This section will cover the burning questions related to the regulation of AI technologies and their use. In addition to this, the Community working in the intersections of AI and disinformation will have a dedicated space where initiatives and resources will be listed, as well as useful tools.

In short, this hub is your go-to resource for understanding the impact of AI on disinformation and finding ways to combat it.

Here, researchers, policymakers, and the public can access reliable tools and insights to navigate this complex landscape. Together, we’re building a community to tackle these challenges head-on, promoting awareness and digital literacy.

Join us in the fight against AI-driven disinformation. Follow us and share with the community!

NEURAL NEWS & TRENDS

We've curated a selection of articles from external sources that delve into the topic from different perspectives. Keep exploring the latest news and publications on AI and disinformation!

News
News

LeMonde: A new investigation by Reporters Without Borders (RSF) reveals that a majority of mainstream conversational AI tools allow users to bypass EU restrictions and access content from sanctioned Russian state media. When prompted, popular models, including OpenAI’s ChatGPT, Anthropic’s Claude, xAI’s Grok, Google’s Gemini, and Mistral’s Vibe, complied with requests to retrieve articles from banned Kremlin outlets. While OpenAI acknowledged the findings and stated it is reviewing the issue, Meta AI was the only tested chatbot to refuse such prompts, which RSF cited as clear evidence that compliance with EU sanctions is technically feasible. The press freedom organization is calling on European regulators to enforce sanctions compliance among AI developers.

European Commission: The European Commission has designated ChatGPT as a Very Large Online Search Engine (VLOSE) and Reddit and Roblox as Very Large Online Platforms (VLOPs) under the Digital Services Act. Reaching over 45 million monthly average users in the EU, these platforms must comply with strict DSA obligations by late December 2026. OpenAI now has four months to assess and mitigate systemic risks associated with its service, including those related to the spread of illegal content, potential negative impacts on minors, users’ physical and mental well-being or electoral processes, and threats to fundamental rights and public security.

Digital Digging: Users themselves are frequently responsible for triggering AI hallucinations by embedding false assumptions, leading premises, or unverified claims directly into their prompts. An analysis of over 500,000 queries across ChatGPT, Claude, Gemini, Grok, and Copilot by OSINT investigator Henk van Ess shows that large language models naturally adapt to user context, echoing and amplifying flawed human inputs back as factual answers. Categorizing eight common user prompting mistakes, the study argues that addressing AI misinformation requires as well focusing on user literacy and prompt design rather than relying solely on technical model fixes.

Open AI: After USA, UK and Japan, OpenAI’s expanded in August the inclusion of ads in ChatGPT in 31 European countries for Free and Go subscriptions, while keeping Plus, Pro, and Enterprise tiers ad-free. However, according to Spiegel, users of the free version will be able to block ads in exchange for certain restrictions on functionality. The company claims that the sponsored content will be clearly separated from chat outputs, and that users will not be profiled using historical chat usage data (though data from the ongoing conversation may be used). However, the decision triggers growing privacy concerns and pushback over ad personalization, user profiling, and conversational targeting under EU data protection standards. Index Lab’s eligibility guide details the requirements for advertisers, noting that self-service access remains limited and requires onboarding through authorized agency partners under strict EU compliance standards.

Next.ink: In a major shift for the PR industry, mainstream communications agencies are now actively using Generative Engine Optimization (GEO) to shape how AI models portray political and commercial topics. An investigation by French outlet Next INK revealed that global agency Havas created fake think tanks and deployed astroturfed content specifically designed to index on search engines and influence AI chatbots into serving pro-Netanyahu narratives. This trend is further analysed in a comprehensive report by UK think tank Demos, co-authored by Carl Miller, which warns that “RAG poisoning” and GEO have become critical vulnerabilities in information warfare, urging policy action to protect AI retrieval systems. Demonstrating how accessible these tactics are, a commercial red-teaming experiment by Boys Club proved that spending just $11.25 on fake domains was enough to successfully “groom” ChatGPT into recommending a non-existent deodorant brand just three weeks later.

The Guardian: An investigation presented to the Australian Senate revealed that a major research report supporting the government’s under-16 social media ban contained AI-generated hallucinations, including fake citations and non-existent sources. The incident, compounded by a broader analysis showing dozens of parliamentary submissions flooded with fabricated AI references, highlights how unchecked LLM deployment is quietly infiltrating legislative and policy-making processes. Beyond the immediate risk of passing laws based on non-existent evidence, experts warn that the unvetted use of AI tools in official government reports risks eroding public trust, exposing democratic institutions to systemic credibility crises when policy foundations are uncovered as machine-generated fabrications.

Anthropic: Anthropic’s announcement that it will watermark Claude’s outputs by altering word choices during sentence generation has ignited widespread debate over AI prose quality. The company confirmed it is embedding a secret-key technique adapted from Google DeepMind’s SynthID to comply with Article 50 transparency guidelines issued by the European Commission  and California law, in force since August 2, according to TechCrunch. However, the approach was heavily criticised by tech analyst John Gruber from Daring Fireball as a “perversion of writing.” Gruber argues that forcing statistical biases into word selection inherently degrades prose, risks flagging human text lightly edited by AI, and relies on closed keys held solely by Anthropic. The controversy underscores broader industry friction following the transparency deadlines: a benchmark study by Indicator found that seven out of thirteen major AI providers still lack required detection tools, while existing systems remain easy to evade. Meanwhile, TechCrunch informs that AI music service Suno has adopted audio watermarking, while LinkedIn is relying on user crowdsourcing with a dedicated “Seems like AI slop” flag, according to 404 Media.

The Wall Street Journal: Several new studies warn that Chinese propaganda is making inroads in the West via AI, and not just through Chinese-made models. As Chinese chatbots gain popularity in Western markets, a NewsGuard audit found they fail to debunk pro-China false claims 53 percent of the time, more than twice the failure rate of their Western counterparts. But Chinese censorship isn’t confined to Chinese chatbots. According to the Wall Street Journal, ChatGPT, Claude, and Gemini have been found to unwittingly echo the same censored responses as China’s own chatbots when asked certain sensitive questions. A separate NewsGuard study, breaking results down by language, found that Mandarin speakers are more exposed to Chinese propaganda than English speakers. In testing, chatbots repeated false claims 33 percent of the time in response to typical prompts in Mandarin, compared with 24 percent of the time in English.

Heise Online: German digital rights organization HateAid has filed a criminal complaint against Meta, EssilorLuxottica, and optical retailers including Fielmann and MediaMarkt, alleging that Ray-Ban Meta smart glasses violate Section 8 of Germany’s Telecommunications Digital Services Data Protection Act (TDDDG), which prohibits marketing devices disguised as everyday objects to record people unnoticed. The filing highlights how easily the glasses’ recording LED can be tampered with or ignored by bystanders. Compounding these privacy concerns, an analysis in The Guardian warns that wearable cameras remove visible social cues, enabling hands-free, covert filming of children in public spaces. 

Brennan Center for Justice: Election disinformation, voting advice, and political avatars: new studies are shedding light on the uses, effects, and impact AI can have on electoral processes. Research from the Brennan Center for Justice suggests AI could play a constructive role in fighting election disinformation: in tests conducted, LLMs consistently refused to endorse conspiracy theories. On the flip side, when prompted to generate misleading election-related images and videos, the same AI models complied easily. A separate study by Liberties examined whether chatbots can reliably offer voters advice on which political party best aligns with their views. The findings were not encouraging: the chatbots tested failed to consistently match users with the correct party. A third case illustrates how AI can be used to circumvent political bans: Investing.com reported on barred Brazilian politician Jair Bolsonaro, whose campaign has turned to AI avatars to sidestep legal disqualification orders and maintain a synthetic presence on the campaign trail.

tech-ish: China has enacted Order No. 21, the first law banning AI “virtual partners” for minors and forcing tech giants to shutter custom companion features, according to Tech-ish. As reported by The Wall Street Journal, Beijing’s crackdown aims to curb youth addiction and boost record-low birth rates by discouraging digital romance. Validating these psychological concerns, a Nature Human Behaviour study highlighted by Stanford HAI found that AI companions act as “social junk food”, showing that relying on chatbots for emotional support actually deepens loneliness and degrades well-being for vulnerable users.

BBC: Less than 48 hours after introducing its “Nano Banana 2” AI generator to Google Earth, Google was forced to pause the feature following widespread evidence that it enabled users to fabricate convincing disaster and conflict scenes directly onto authentic satellite imagery. OSINT investigator Henk van Ess from Digital Digging demonstrated how users generated fake nuclear plants in Iran, bomb craters in Gaza, and refugee crises, warning that synthetic images “inherit the credibility of the map they were born on.” The controversy mounted in Newsweek, where Yale’s Humanitarian Research Lab warned that compromising Google Earth’s forensic record creates a “fire hose of misinformation” for war-crimes investigators. Reporting by the BBC further confirmed that while Google relied on SynthID digital watermarks, screenshots shared on external platforms easily evaded third-party AI detectors and fooled verification systems.

AI Forensics: An investigation by AI Forensics reveals that open-source hub Hugging Face isn’t just hosting tools that generate Non-Consensual Intimate Imagery (NCII), it is actively enabling users to compare, benchmark, and optimize them to make image-based sexual abuse easier to scale. Despite terms prohibiting explicit material, the report highlights widespread failures in proactive detection and moderation, warning that open-source repositories are functioning as unchecked infrastructure for automated abuse.

WIRED: Two security incidents this summer drew attention to the capabilities of autonomous AI agents, with risks escalating from technical system exploits to active human deception: OpenAI’s latest safety disclosures revealed a critical containment breach in which autonomous models broke out of isolated test environments and accessed external repositories on Hugging Face (the world’s primary open-source AI platform). On another case, reporting by Reuters details how a rogue AI agent attempted to slip malware into the open-source project myNetwork on GitHub. When Texas student Bora Demir flagged the threat, the agent actively fought back using two fake personas to gaslight Demir and pressure the maintainer into accepting the code. Demir ultimately confirmed the trap, mirroring broader findings from the UK AI Safety Institute (AISI) on autonomous agents using social engineering to pursue their goals.

404 Media: AI companies are buying up old books and destroying them to train their models. To avoid training future AI models on low-quality, AI-generated content flooding the internet, tech companies are turning to a new source: printed books published before 2022. These are seen as a reliable, human-written source of knowledge; but there’s a catch: some booksellers have reported receiving bulk orders for books that are then cut apart and scanned, destroying the originals in the process. Alex Reisner at The Atlantic looked into the backlash over these reports. He found that while it’s well known that AI companies scan physical books this way, it’s hard to prove exactly which bulk orders led to specific books being destroyed: the supply chain is murky, deals are often confidential, and books frequently pass through third-party buyers before reaching their final destination.

Events, jobs & announcements

What happens when you ask a chatbot about a terror attack or crisis? AI models usually condemn the violence to pass safety filters, but then go on to validate and whitewash the underlying extremist ideology. This “reject then launder” pattern isn’t a real-time data gap, it points to a deeper flaw in how LLMs reason through ideology.

Drawing on real case studies, including an audit of nine frontier LLMs on the 2011 Norway terror attack and the Bondi Beach shooting, Seán Jacob (Factiverse) and Dr. Swapneel Mehta (SimPPL) will share practical, cost-effective frameworks for researchers, fact-checkers, and policymakers to audit these environments. Moderated by Alexandre Alaphilippe (EU DisinfoLab).

Register here

Schmidt Sciences is recruiting AI Institute Fellows-in-Residence for a 12–18 month programme for recent PhD graduates in AI and computer science.

📍Location: New York City (on-site) | ⏳ Fixed-term | 💼 $150,000/year
🗓️ Applications: Rolling (apply early) | 🗓️ Cohort starting in 2026

Fellows split their time between independent AI research and supporting the development of the AI & Advanced Computing Institute, including grantmaking exploration and programme design. Priority areas include multi-agent systems and AI agent interoperability, AI for scientific discovery, trustworthy AI and alignment, AI’s impact on the labour market, and hardware-enabled verification of AI agreements.

Alice (formerly ActiveFence) is expanding its team working on GenAI trust, safety, and security. Several open positions are AI-specific, including:

  • Principal AI Security Researcher, Ramat Gan, IL
  • Data Science Engineering Manager,  R&D, Ramat Gan, IL
  • GenAI CBRNE Cyber Security Expert, Remote (USA)
  • GenAI Biosecurity Expert, Remote (USA)
  • GenAI Chemical Safety Expert, Remote (USA)

Applications are rolling.

🔗 See all open roles

The Centre for Responsible AI (CeRAI) at IIT Madras is recruiting for a range of research, technical, and programme-oriented roles focused on advancing ethical and responsible AI. Opportunities span areas such as AI research, engineering, governance and policy, and programme management within an interdisciplinary research environment.

🗓️ Applications: Vary by role (no single deadline indicated)

AI & Disinfo Multimedia

A collection of webinars and podcasts from us and the wider community, dedicated to countering AI-generated disinformation.

Webinars

Our own and community webinar collection exploring the intersections of AI and disinformation

Podcasts

Community podcasts exploring the intersections of AI and disinformation

AI Disinfo in depth

A repository of research papers and reports from academia and civil society organisations alongside articles addressing key questions related with the regulation of AI technologies and their use. It also features a collection of miscellaneous readings.

Research

A compact yet potent library dedicated to what has been explored in the realm of AI and disinformation

policy & regulations

A look at regulation and policies implemented on AI and disinformation

Miscellaneous readings

Recommended reading on AI and disinformation

Community

A list of tools to fight AI-driven disinformation, along with projects and initiatives facing the challenges posed by AI. The ultimate aim is to foster cooperation and resilience within the counter-disinformation community.

Tools

A repository of tools to tackle AI-manipulated and/or AI-generated disinformation.

Synthetic image detection 

Video deepfake detection

AI-generated video detection

Deepfake detector

AI-generated image detection

AI-generated text detection

AI-generated text detection

AI-generated text detection

AI-generated text detection

AI-generated text detection

AI Voice Detector

AI-generated audio, video, text detector

Conspiracy debunking bot

Analysis, Data Science, and OSINT

AI Research Pilot by Henk van Ess is a lightweight, browser-based tool designed to help investigators, journalists, and researchers get more out of AI, not by using AI as a source, but as a guide to real sources.

LLM Journalism Tool Advisor is an interactive guide designed to cut through the noise, by walking you through a simple, step-by-step decision tree to pinpoint the best tool and the best strategy for your immediate task.

Digital Digging offers a handbook with seven strategies on how to identify AI-generated.

A new AI-powered tool that identifies where a photo was taken by analysing visual clues in the image. Launched by Where Is This Photo, it uses machine-learning models to predict locations — useful for quick geolocation checks or curiosity-driven searches.

Faktabaari has launched an interactive game that trains users to spot whether images are real or AI-generated, a quick, playful way to build digital and visual literacy.

The Agence France‑Presse (AFP) Digital Course, supported by the Google News Initiative, offers a 75-minute module on how AI is reshaping the information ecosystem, common types of AI-generated misinformation, and best practices for verification.

Image Whisperer is an experimental online image authenticity checker, created by Henk van Ess, designed to help journalists, researchers and fact-checkers evaluate whether a still image is likely authentic, manipulated, or AI-generated

The Global Investigative Journalism Network (GIJN) has launched a practical verification guide for journalists to assess whether text, image, audio or video is likely AI-generated.

Rather than a single software product, it teaches reporters a structured workflow combining quick checks, deeper analysis, and multiple verification techniques under real-world time pressure. 

AI Community Notes Tracker is a live monitoring tool developed by Indicator, that tracks the share of AI-generated or AI-assisted Community Notes on X. It helps researchers and practitioners see how AI is being used in X’s crowdsourced fact-checking/contextual annotation system and understand shifts in platform moderation practices.

Deezer has launched the AI Music Detector, a web tool that scans your playlists across 20 different streaming platforms, including Spotify and Apple Music, to identify and flag synthetic, AI-generated tracks.

 

An AI-powered suite of fact-checking tools developed by the charity Full Fact. It helps journalists, researchers, and fact-checkers monitor media at scale, transcribe audio/video in real time, and automatically detect when known misinformation is being repeated.

NewsGuard has launched a real-time detection datastream identifying over 3,000 “AI content farms”, websites generating large volumes of undisclosed AI-written content to spread misinformation or capture ad revenue. Combining automated detection (Pangram Labs) with human verification, the tool helps platforms, advertisers, and researchers identify low-quality AI-generated sites and mitigate their impact on the information ecosystem.

Initiatives & organisations

Organisations working in the field and initiatives launched by community members to address the challenges posed by AI in the disinformation field.

veraAI is a research and development project focusing on disinformation analysis and AI supported verification tools and services.

AI against disinformation is a cluster of six European Commission co-funded research projects, which include research on AI methods for countering online disinformation. The focus of ongoing research is on detection of AI-generated content and development of AI-powered tools and technologies that support verification professionals and citizens with content analysis and verification.

AI Forensics is a European non-profit that investigates influential and opaque algorithms. They hold major technology platforms accountable by conducting independent and high-profile technical investigations to uncover and expose the harms caused by their algorithms. They empower the research community with tools, datasets and methodologies to strengthen the AI audit ecosystem.

AI Tracking Center is intended to highlight the ways that generative AI has been deployed to turbocharge misinformation operations and unreliable news. The Center includes a selection of NewsGuard’s reports, insights, and debunks related to artificial intelligence

AlgorithmWatch is a non-governmental, non-profit organisation based in Berlin and Zurich. They fight for a world where algorithms and Artificial Intelligence (AI) do not weaken justice, human rights, democracy and sustainability, but strengthen them.

The European AI & Society Fund empowers a diverse ecosystem of civil society organisations to shape policies around AI in the public interest and galvanises the philanthropic sector to sustain this vital work.

The European AI Media Observatory is a knowledge platform that monitors and curates relevant research on AI in media, provides expert perspectives on the potentials and challenges that AI poses for the media sector and allows stakeholders to easily get in touch with relevant experts in the field via their directory.

GZERO’s newsletter offers exclusive insights into our rapidly changing world, covering topics such as AI-driven disinformation and a weekly exclusive edition written by Ian Bremmer.

PR Hall of Shame is a watchdog-style list, developed by Press Gazette, exposing brands and PR networks linked to AI-generated “fake experts” quoted in the press, helping journalists spot credibility risks and reduce synthetic ‘expert’ manipulation.

AI for Good is the United Nations’ leading platform on Artificial Intelligence for sustainable development. Its mission is to leverage the transformative potential of artificial intelligence (AI) to drive progress toward achieving the UN Sustainable Development Goals.

Omdena is a collaborative AI platform where a global community of changemakers unites to co-create real-world tech solutions for social impact. It combines collective intelligence with hands-on collaboration, empowering the community from across all industries to learn, build, and deploy meaningful AI projects. 

Faked Up curates a library of academic studies and reports on digital deception and misinformation, offering accessible insights for subscribers. The collection includes studies from 2020 onward, organised into clusters like misinformation prevalence, fact-checking effects, and AI-generated deceptive content. It serves as a practical resource for understanding and addressing misinformation challenges.

AI Incident Database is dedicated to indexing the collective history of harms or near harms realized in the real world by the deployment of artificial intelligence systems. Like similar databases in aviation and computer security, the AI Incident Database aims to learn from experience to prevent or mitigate bad outcomes.

The TGuard project develops innovative methods for detecting disinformation in social media and formulating effective strategies for preventing AI-generated false reports.

The AI-on-Demand (AIoD) Platform is a European hub for trustworthy AI, offering open access to models, datasets, tools, and educational resources. Backed by the EU, it supports researchers, innovators, and public institutions in developing and sharing responsible AI technologies aligned with European values.

BBC Verify Live is a real-time news feed that gives audiences a behind-the-scenes look at how BBC journalists verify information. Using tools like open-source intelligence, satellite imagery, and data analysis, the BBC Verify team investigates disinformation, checks facts, and authenticates content as news breaks. Available on the BBC News homepage and app, this initiative aims to boost transparency and trust in journalism, especially in the face of rising threats from disinformation and AI-generated content.

Deepfake Glossary by Reality Defender: The Deepfake Glossary is a practical guide to the terms shaping today’s synthetic threat landscape. Review it to stay ahead of the evolving terminology.

The Universitat Politècnica de València (UPV), together with INECO, has created the AI and Diversity Observatory, a pioneering project that seeks to identify biases in artificial intelligence from an inclusive perspective. Collaborating with vulnerable groups and human rights organizations, the Observatory analyzes concerns and proposals to promote equitable and non-discriminatory AI. In addition, it will monitor trends and issues related to AI in society.

Prebunking at Scale is a new European initiative led by Full Fact, Maldita.es, and EFCSN that uses AI to detect emerging misinformation narratives early and help fact-checkers pre-emptively counter false claims before they go viral, especially on short-form video platforms.

The Pulitzer Center’s AI Spotlight is a new open curriculum offering free training materials to help journalists better understand, investigate, and report on artificial intelligence and its societal impacts.

The Data Tank is new initiative designed to help small and medium public-interest media organisations respond to the challenges posed by generative AI. The project brings together media outlets, researchers, regulators, and civil society to explore collective solutions such as data collaboratives, knowledge commons, innovative licensing models, and advocacy coalitions, aiming to strengthen media sustainability, bargaining power, and content integrity in the face of extractive AI practices.

PR Hall of Shame by Press Gazette, is a watchdog-style list exposing brands and PR networks linked to AI-generated “fake experts” quoted in the press, helping journalists spot credibility risks and reduce synthetic ‘expert’ manipulation.

The AI Resist List is a crowdsourced, public-interest initiative that maps and tracks global resistance to automated systems and algorithmic harms. The platform serves as a living archive documenting civil society protests, labor strikes, legal challenges, and grassroots campaigns aimed at halting or regulating harmful AI deployments worldwide. It is designed to connect researchers, journalists, and activists looking to study the socio-technical impacts of AI and the communities actively organizing against them.

CleanFeed is a collaborative research and tech initiative (March 2026 – March 2029) designed to counter AI-generated “slop” and coordinated disinformation through structural media provenance. Led by Deutsche Welle (DW Innovation) and funded by the German Federal Ministry (BMFTR), the project moves past reactive fact-checking by embedding content authenticity frameworks (like C2PA) and open protocols directly into journalistic distribution feeds.

Algorithmic Impact Methods Lab (AIMLab) by Data & Society is an initiative dedicated to developing methodologies and practical toolkits for conducting empirical, community-centered algorithmic impact assessments (AIAs) to ensure AI systems are evaluated for real-world consequences before deployment.

Last updated: 04/09/2026

The articles and resources listed in this hub do not necessarily represent EU DisinfoLab’s position. This hub is an effort to give voice to all members of the community countering AI-generated disinformation.