Threat Actors looking to manipulate us can exploit vulnerabilities to increase their probability of success. Sometimes a vulnerability is in a platform, sometimes the vulnerability exists in our very own BRAINS!!!! This one is a brain one.
What is the Illusory Truth Effect?
If you see something repeated enough times, it seems more true. Multiple studies show that it works on 85% of people. Worse, it still happens even if the information isn’t plausible & even if you know better.
Ethan Mollick (@emollick) on 30 Oct 2022
Ethan also linked to other studies providing more insight into the effect;
(1): It works after just five repetitions for obviously false statements (e.g. “George Washington was born in China”).
(2): Having knowledge about a topic before being repeatedly exposed to related false information does not reduce the effect’s impact.
(3): Repetition of false information strengthens belief regardless of when repetition occurs, (but the effect is stronger the sooner the repetition occurs; researches tested exposure immediately, one day later, one week later, and one month later).
Which Techniques benefit from the Illusory Truth Effect?
The repeated publication of identical information was a feature in A pro-government disinformation campaign on Indonesian Papua, which Flooded the Information Space (T0049) using bots (T0049.003):
These synchronous tweets on [government policy] share an additional characteristic: most are duplicates or near duplicates of each other. At face value, 22,479 of the 23,853 tweets […] are original tweets about special autonomy, posted by 2,668 different authors. In reality, just 760 of these tweets are unique (3.4%) […]. The most duplicated tweet appears 286 times, posted by 246 distinct authors, and declares that [the policy] demonstrates the seriousness of the Indonesian government in improving welfare in Papua.
Evidence of Technique Flooding the Information Space (T0049) taken from the article.
A heavily disproportionate number of tweets in the dataset were posted in the minute starting 6:55 am, and then from mid-April 2021 onwards, in the minute starting 8:00 am Jakarta time. If we consider that nearly all these tweets mention special autonomy, and most received little or no engagement from other users, coordinated automated posting is the only plausible explanation.
Evidence of Technique Bots Amplify via Automation (T0049.003) taken from the article.
There may yet be more techniques which take advantage of this vulnerability to increase their success rate. As more associated techniques are identified (and if spotlights of vulnerabilities prove useful – this is the first one!), they will be added here.