Author: Raquel Miguel, EU DisinfoLab
Reviewer: Noémie Krack, KU Leuven Centre for IT & IP Law – imec
Last updated: 6 December 2023
Executive Summary
The development of artificial intelligence (AI) technologies has long been a challenge for the disinformation field, allowing content to be easily manipulated and contributing to accelerate its distribution. Focusing on content, recent technical developments, and the growing use of generative AI systems by end-users have exponentially increased these challenges, making it easier not just to modify but also to create fake texts, images, and audio pieces that can look real. Despite offering opportunities for legitimate purposes (e.g., art or satire), AI content is also widely generated and disseminated across the internet, causing -intentionally or not- harm and deception.
In view of these rapid changes, it is crucial to understand how platforms face the challenge of moderating AI-manipulated and AI-generated content that may end up circulating as mis- or disinformation. Are they able to distinguish legitimate uses from malign uses of such content? Do they see the risks embedded in AI as an accessory to disinformation strategies or copyright infringements, or consider it a matter on its own that deserves specific policies? Do they even mention AI in their moderation policies, and have updated these policies since the emergence of generative AI to address this evolution?
Answers to these questions are crucial as the Digital Services Act (DSA) will provide new complaint mechanisms for users on the lack of enforcement of terms and conditions. The DSA will also require platforms to assess their mitigation measures (and results) against systemic risks.
The present factsheet delves into how some of the main platforms – Facebook, Instagram, TikTok, X (formerly Twitter), and YouTube – approach AI-manipulated or AI-generated content in their terms of use, exploring how they address its potential risk of becoming mis- and disinformation.
The analysis concluded that definitions are divergent, leaving users and regulators with diverse mitigation and resolution measures. First, only Facebook and TikTok mention “artificial intelligence” (including deepfakes in the case of Facebook) directly in their policies aiming to tackle disinformation. TikTok and X include “synthetic media” in their policies about manipulated and misleading media. In a blog post released on November 2023, YouTube announced new measures targeting “synthetic content”, explicitly referencing “AI”. Concurrently, Meta addressed “digitally created or altered content” in the context of political ads.
While the distinction between general misinformation policies and AI-specific considerations isn’t always evident, there’s a growing trend among platforms to incorporate specific guidelines for content altered or generated by AI, such as recently YouTube or Meta. However, the platforms often overlook mentioning AI-generated text and refer mainly to images and videos in their policies.
In cases, like TikTok, where platforms explicitly address synthetic or manipulated media with AI, they try to distinguish between allowed and banned uses. Little variations in the rationale behind content moderation exist: the driving force is either the misleading and harmful potential or a more compliance-oriented approach in terms of copyright and quality standards of the content.
On a different note, all the studied platforms qualify as Very Large Online Platforms (VLOPs) according to the DSA). The DSA is technically neutral, i.e., it applies regardless of the technology used to produce the content. Meanwhile, the strengthened Code of Practice on Disinformation has been reinforced by the co-regulatory mechanism and additional obligations to combat disinformation (including through systemic risks assessment and mitigation, crisis protocols, users empowerment measures and increased transparency requirements) set up by the DSA. In its 15th commitment, relevant signatories of the Code (all of the studied platforms except X) are specifically called to “establish or confirm their policies in place for countering prohibited manipulative practices for AI systems that generate or manipulate content, such as warning users and proactively detect such content”.
Consequently, even though X has withdrawn from the Code, it still has to abide by the DSA. Therefore, all the five studied platforms must comply with the DSA due diligence obligations and justify the means they deploy to combat disinformation on their services. This could require that they adopt new measures: among other required actions, platforms should update their policies to meet new needs in the face of rapidly evolving technologies, enhance cooperation with experts, and clarify the burden of responsibility on this complex topic.
Since the initial release of this document in September 2023, YouTube and Meta have announced, in November 2023, additional measures regarding AI-manipulated or generated content, that are incorporated into this updated version. The implementation of these new measures is expected in 2024 in the case of Meta, and in a non-specific moment “over the coming months” in the case of YouTube.
Platforms’ policies on AI-manipulated and generated misinformative content
EU DisinfoLab has developed an analytical framework to analyse and compare the policies of five platforms on different misinformative topics. Factsheets on electoral, health, and climate change misinformation have already been published following this framework. The same methodology (focusing on definitions and actions, and types of actions) is applied to AI-generated and manipulated misinformation. As far as applicable, the notes included in the table are verbatim mentions of the platforms’ policies. In other cases, for the sake of simplification, the notes are a summary or analysis by the author.
This page has been updated on 6 December 2023 with the version 2 of the factsheet that includes the last policy updates announced by Meta and YouTube in November 2023. The Executive Summary on this page has been also adjusted accordingly. You can find the version 1, published on 28 September 2023, here (pdf).
