By Ana Romero-Vicente, EU DisinfoLab Researcher
Introduction
Facebook, Instagram, Twitter, and YouTube have all taken additional steps to crack down on Covid-19 falsehoods, although it is proven that they are not always effective, and their AI detection system fails to remove most of the problematic content.
More worryingly, malicious actors are a step ahead and have already been developing strategies to escape platforms’ content moderation systems. One of these strategies consists of using twisted keywords to escape platforms’ COVID-19 disinformation policies. Therefore, misleading and fearmongering coronavirus-related content can continue to spread quickly on traditional social media platforms in subtle disguises.
This case study briefly explores how misinformers camouflage virus-related Spanish keywords to spread bogus content. This represents a challenge for the platforms that seem unable to detect it and, therefore, to label or remove it in many cases.
Although limited in scope, the present contribution should serve as a case study and a call to action for platforms to improve their systems and keep implementing additional methods to improve their content moderation and mitigate disinformation related to the pandemic.
Methodology
The research focuses on three Spanish words related to Covid-19 that have been subtly modified in order not to be detected by Facebook, Instagram, YouTube, or Twitter. These pieces of unverified disinformation have been collected through various social media tracking tools, with the goal of collecting concrete and significant cases, rather than providing a comprehensive evaluation of the phenomenon.
The twisted Spanish words that have been analysed are the following: “vacunas” (vaccines), “pandemia” (pandemic) and “Covid-19”.
Even though our study focuses on the pandemic, camouflaged words are also used to spread other narratives. To illustrate this, we recall a pro-QAnon Facebook video claiming that “there is an operation against p3d0fili14 (pedophilia) and 4dr3n0cr0m0 (adrenochrome)”, which collected 48k views and over 2k likes.
In addition to our main blog post, we offer in an attached document with additional material that covers other modified keywords linked to the pandemic, namely “virus”, “grafeno” (graphene), “mascarillas” (masks), or “Ivermectina” (Ivermectin). These twisted keywords were also used to escape content moderation and spread disinformation: (Annex 1)
The word camouflage technique, or how to keep out of the sight of content moderation systems
Misinformers tend to subtly tweak keywords so that they are understandable for users while remaining undetected to social networks’ content moderation systems. Although they do not pursue the same goal, their techniques are similar to those used by cybercriminals while performing the practice called cybersquatting, playing on the spelling of domain names. To a certain extent, it also reminded us of our investigation “Indian Chronicles”, where some Indian actors regularly inverted letters in names, very likely in an attempt to confuse people and hide online tracks linked to their foreign influence operation.
In particular, we noticed a number of recurring strategies:
- Vowels are replaced by numbers due to their pictographic similarity:
- Vowel “A” replaced by the number 4: e.g. VACUNA (vaccine) 🡪 V4CUN4.
- Vowel “E” replaced by the number 3: e.g. GRAFENO (graphene) 🡪 GR4F3N0.
- Vowel “O” replaced by the number 0: e.g. COVID 🡪 C0V1D.
- Vowel “I” replaced by the number 1: e.g. PANDEMIA (pandemic) 🡪 P4ND3M14.
- Consonants are replaced by other consonants that, in Spanish, sound the same or very similar:
- “V” replaced by “B”: e.g. COVID 🡪 COBID; VANUNA 🡪B4CUN4.
- “D” replaced by “T”: e.g. COVID🡪 COVIT.
Or a combination of both: e.g. COVID🡪 COBIT.
- “C” replaced by “K” or “Q”: e.g. VACUNA (vaccine) 🡪 V4KUN4 🡪 V4QUN4.
- The order of the syllables is reversed:
- NACUVA instead of VACUNA.
- Punctuation marks and symbols are inserted between the letters (., …,!, $,@, π, €, ∆, etc.).
- Words are replaced altogether by other words with similar meanings: Bicho 19 (bug 19) instead of Covid-19.
From the above, there are many possible combinations: e.g.
- V4.kuna, b4cun4, V.A.C.U.N.A.S, va-cuna, va…cuna, v4-kuna, b4qVn4z, V@(u¬a, K4KVNA, v@kN4s instead of “vacuna/s” (vaccine/s).
- V1RU5, V1ru$ instead of “virus”.
- @#plan#demia, pl@πd€m1∆ instead of “pandemia” (pandemic).
- K0 B1T, k0b1d, C0 V ¡D, c0*vid, C0%VID-19, C(o(v(i(d-19, C0 •VID•19, CV I9 instead of “Covid-19”, etc.
In addition, we observed other camouflage methods in videos and images:
- Some of the analysed posts contain images (photos and screenshots), including altered keywords. Social networks seem, in some cases, unable to recognise Optical Character Recognition (OCR), i.e. the text encoded in visuals.
- In some videos, the speaker silences a keyword when pronouncing it and, instead, that keyword is superimposed on the video screen duly camouflaged:
It is important to note that the Covid-19 words camouflaged by some social media users do not always have misinformative intentions. However, the majority are malicious actors who deliberately use this method to spread disinformative messages to escape being deplatformed or labelled.
In this regard, content moderation by Facebook, Instagram, Twitter, and YouTube sometimes seems haphazard since the implementation of platforms’ policies has flaws, leaving out considerable amounts of disinformation. From our observation, many of the posts containing camouflaged words within a misinformation message go entirely unnoticed. However, there are times when they are detected and either moderated or addressed by applying standard information panels – these panels are designed to give more context about a topic regardless of the presence or absence of disinformation in a social media post. An example:
A June 2021 post wrote: “Maddie de Garay, 12 years volunteer for the study of b4.cu.n4s (vaccines) k0vbld (Covid), she is now in a wheelchair and eats through a tube”.
In this case, Facebook detected that the post contained a message linked to Covid-19 vaccines. The platform added an information panel saying that “Covid-19 vaccines go through many tests for safety and effectiveness and are monitored closely”. It is unclear if the information panel was added by Facebook because of the platform detecting the twisted keywords or due to some other elements in the post.
In this case, we can notice that the generic information panel doesn’t address the misleading information here, which consists in using the very specific case of a young American woman to suggest unproven conclusions about the overall dangerousness of COVID-19 vaccines.
Moderation gaps
The items below are a brief selection of several posts containing hidden keywords that have not been detected, labelled by independent fact-checkers, or removed from traditional social media platforms, despite explicitly breaking their Covid-19 policies (i.e. Facebook, Instagram, Twitter, and YouTube).
Concerning the narratives, word camouflage techniques are mainly spreading anti-vaccine stances, conspiracy messages, and denialist content.
Keyword 1: Vacuna (vaccine) vs. V4CUN4, B.4.C.U.N.4, NACUVAS, V.A.C.U.N.A.S, V4 CUN 4
The screenshots in Figure 3 include a post with 84k views and 5k shares, claiming that “the V4CUN4 (vaccine) has four potential dangers and that they can lead to the death of foetuses, female infertility and serious side effects”. The other posts state that there are safer treatments “than the B.4.C.U.N.4 (vaccine)” or that “Covid-19 has a 99,96% recovery rate. We do not need quarantines, nor protocols, we do not need V.A.C.U.N.A.S (vaccines)”.
The third group of figures shows a tweeted video that reached 13k views, claiming that “this is not a V4CUN4 (vaccine),it is a horror experiment on humans. It’s a war”. Another tweet maintains that “the so-called NACUVAS (vaccines) are not FDA-approved”. Another item declares that “Margarita’s 38-year-old brother died after being V.A.C.U.N.A.R.S.E (vaccinated), a victim of experimental V.A.C.U.N.A.S. C.O.V.I.D.-1.9 (Covid-19 vaccines)”.
The selected examples show how Instagram’s OCR detection system failed to tackle disinformation.
The first post obtained over 2k likes by claiming that “being anti-cov1t (Covid) V4CUN4s (vaccines) is not being anti-vax (…) V4CUN4s (vaccines) are experiments and f4rm4ceuticas (pharmaceutical companies) make us sign a contract that says they are not responsible for any adverse effects”.
The other photo, which received almost 3k likes, people hold a banner that reads: “The manufacturer is not responsible; the government is not responsible; doctors are not responsible. But if you do not get V.A.C.U.N.A.S (vaccinated), you are irresponsible. They are cheating you. Wake up now”.
YouTube
The caption of the video below reads: “The most dangerous V4 CUN 4 (vaccine)”, as the video proceeds to proclaim the vaccine a mass experiment to turn every patient into “Bill Gates’ mutant pet”. Interestingly, the title of the video also suggests that YouTube might have already taken some actions against this video when it was published on another YouTube channel.
Keyword 2: Covid-19 vs. K0 B1T, K0b1d 1984, KOBiD, Bicho 1’9, ¢ 0V-! D, C.O.V.I.D
The first post states that “Italy follows France and announces the green pass as a requirement to enter restaurants. The Italian people have taken to the streets to claim their right to free movement and to protest against the tyranny of the K0b1d 1984 (Covid) dictatorship”. The second post is from a disinformative outlet according to which “38.700 deaths and 3.968.000 adverse reactions have been registered due to the K0 B1T (Covid-19) injection”.
The tweet below claims that “forcing more than 99%, which is the healthy population, to get vaccinated, while less than 1% are sick with the bicho (bug – Covid) is like asking healthy people to undergo chemotherapy”.
The above post received almost 2k views and claims that “the circus is over. On May 6, 2021, Texas physicians testified before the Senate to oppose the mandatory v @ q-n4s (vaccines) for ¢ 0V-! D (Covid)…”
YouTube
The first video, with over 11k views, has the following caption: “Doctors from around the world explain how they Fals3an (falsify) deaths by the bicho (bug)”. In the other video, extracted from a TV interview, which has more than 60k views and 3k likes, former Miss Spain talks about “Data of KOBiD_B4Kun4s (Covid-19 vaccines)” and expresses several Covid-19 denialist messages on the allegedly low Covid-19 mortality rate as opposed to the danger of vaccines.
Keyword 3: pandemia (pandemic) vs. P4ND3M14, Pandemi4, P4ndemi4, P4nd3m1A
The post (with more than 1k views) spread the conspiracy that Tanzanian President John Magufuli was disappeared because “He was the only world leader to fully expose the entire p4ndemi4 (pandemic)” and “to go against the v4cun4s (vaccines) plot” aimed at poisoning citizens, but “now the mainstream fake news media say he died of c0v1d (Covid). This example is particularly relevant to highlight the widespread use of word camouflage beyond the virus-related lingo, as terms like “T4NZ4NI4NO” (from Tanzania), ASES1N4D0 (killed), or “desapar3ció” (disappeared) employ the same technique.
The first item (left) is relevant as it shows that disinformation travels across platforms. In fact, the tweet asks: “What do people die of if the P4nd3m14 (pandemic) is not real?” and attaches a video from a smaller, fringe platform (Odysee) where the video received more than 5k views. Similarly, the other tweet (right) asks: “Doctor Luis de Benito, where is the P4nd3m1A (pandemic)?” and contains a link to a Covid-19 denialist video on YouTube, which has not been removed by the platform, with 30k views.
This post claims to be “an excerpt from an interview by James Corbett (The Corbett Report) in which the p4nd3m1a (pandemic) is mentioned as a smokescreen to what could happen”, projecting a ‘deep state’ conspiracy.
Conclusions and recommendations
- The skill of misinformers in developing new techniques to continue spreading their messages cannot be ignored. Although there are other platforms where false and deceptive content spreads without any restriction, mainstream social media platforms have the widest audiences, drawing the interest of malicious actors. Besides escaping moderation, the use of word camouflage also reinforces in-group identity, the idea of being part of a community sharing the same secret code and opposing the censorship power of big tech companies.
- Tackling the phenomenon of camouflaging keywords to spread false and harmful messages is a complex challenge. An excessively limited list of disinformation-related keywords can wllow misinformation content to still find its way onto the platforms. However, the opposite can lead to the removal of content that does not violate the rules, like Twitter labelling as COVID-19 disinformation all tweets that mentioned 5G and oxygen together, irrespective of their non-conspiratorial nature. Blocking lists and techniques must be carefully and continually re-evaluated and optimised to achieve this delicate balance.
- Overall, there are actions that platforms should embrace, such as making better use of Optical Character Recognition (OCR) to tackle content that violates their terms and conditions, or improving other detection techniques including their automated moderation systems, to deal with current difficulties.
