07.04 Propaganda Review

Manipulative Intelligence: AI has already become a battleground for ideological confrontation and propaganda manipulation, but the main fight is still to come

Following in the footsteps of social networks and search engine algorithms, artificial intelligence algorithms are now also becoming a battleground for ideological confrontations and direct manipulation. Autocracies and propagandists are increasingly seeking to ‘ride’ AI and turn it into a tool of influence. And just like with the previous two cases, the so-called ‘free internet’ remains helpless and unprepared to defend itself.

Beyond the use of AI to generate deepfakes, experts identify two main mechanisms of manipulation at the level of large language models (LLMs). The first is 'scaling lies', that is using AI as a generator of propaganda or ideologically framed content on an industrial scale. The second is 'LLM grooming', which refers to the corruption of language models by manipulating the training data.

Currently, the main Russian player in this field is believed to be the Pravda network, also known as Portal Kombat. This network automatically produces a massive volume of publications and consists of around 180 multilingual domains and subdomains. Its target is not ordinary readers, of which it has very few, but automated systems: web crawlers, chatbots, and algorithms that collect data for AI training.

Using search engine optimisation techniques, Pravda manages to artificially boost the visibility of its content in search results, infiltrate Wikipedia, and become legitimate 'fodder' for AI algorithms.

For those fighting disinformation, these strategies pose a complex challenge. It is not enough to simply filter out sources marked as Pravda to cleanse language models of the disinformation they absorb. New domains legitimising fake content appear faster than the heads of a hydra.

Censorship mechanisms – known as alignment – are built into today’s AI models and reflect developers’ perceptions of norms and tolerance. This will also become a field of ideological struggle. However, to some extent, this issue can be addressed through transparent disclosure of alignment principles and reputation-based mechanisms. The strategies for corrupting language models, however, represent a separate and far greater danger.

What once appeared in the early 2010s as a tool to challenge autocracies – social media – had, by the end of the decade, transformed into a reliable mechanism of influence and control for those very regimes (→ Grigory Asmolov: Propaganda in The Network Environment). In the early 2020s, autocratic regimes also managed to harness search engine algorithms to serve the cause of ‘viral disinformation’ (→ Lev Gershenzon: Algorithms – Reputation – Karma). Now, artificial intelligence (AI) is becoming the newest battleground in the information and ideological war.

This battle promises to be especially dramatic and will unfold on multiple levels. AI platforms accessible to the general public already have censorship mechanisms embedded into them, the so-called alignment systems, which are typically configured according to fairly progressive standards of tolerance, reflecting values of the pre-Trump era. As a result, we may soon witness the rise of true AI multiparty systems, while in authoritarian regimes, a 'sovereign' AI will find its place within the infrastructure of a Ministry of Truth. However, beyond the issue of ideological alignment – a problem that could be partly addressed through the open publication of guiding principles and reputation mechanisms – lies a second, more serious issue: the direct manipulation of AI mechanisms.

In the little more than two years since large language models (LLMs) became publicly available, significant experience has already been accumulated in using them for propaganda and information manipulation. AI holds vastly greater potential than traditional methods, making it possible to automate and scale the production of disinformation at minimal cost.

According to a review in the Bulletin of the Atomic Scientists, there are two primary ways to use LLMs for spreading propaganda and conducting information manipulation. The first could be described as 'scaling lies': malicious actors prompt AI to generate thousands of texts (articles, posts, comments) that embed disinformation or specific ideological frames. These are then published online, creating the illusion that certain narratives or 'facts' are widely accepted or popular.

The second method, which the authors call 'LLM grooming', involves the direct corruption of language models by manipulating their training data. In other words, the interference occurs at the level of the algorithm itself, leading the AI to consistently produce distorted or false responses. On a global scale, this 'grooming' is considered far more dangerous than mere content generation.

Moscow’s central role in attempts to 'ride' AI and use it to its own advantage is reportedly played by the Pravda network, also known as Portal Kombat. According to an investigation by the Atlantic Council's Digital Forensics Research Laboratory (DFRLab), Pravda consists of hundreds of news aggregators and portals engaged in the centralised dissemination of pro-Kremlin narratives across more than 80 countries and regions. These resources do not create original content but instead aggregate publications originally released by Russian news agencies, government websites, and pro-government influencers and institutions on social media. According to the French government agency Viginum, the Pravda network is administered by TigerWeb, an IT company based in occupied Crimea and owned by Yevgeny Shevchenko, a web developer who previously worked at Krymtekhnologii, a company that built websites for Russian authorities in Crimea.

As of today, Pravda comprises 182 domains and subdomains, collectively publishing at least 3.6 million articles annually – or 20,273 articles every 48 hours. The volume of publications containing hyperlinks to Pravda network domains has grown exponentially since 24 February 2022, according to a recent report by DFRLab and the Finnish initiative CheckFirst.

According to a study by NewsGuard, a company that verifies the credibility of online content, 40 of the sites publish in Russian and use domain names referencing Ukrainian cities and regions, for instance, News-Kiev.ru, Kherson-News.ru, and News-Donetsk.ru. Around 70 sites target European countries in their respective languages. Another 30 or so are aimed at countries in Africa, the Pacific region, the Middle East, North America, the Caucasus, and Asia. The remaining sites are organised by theme: one collects content about NATO, another about Donald Trump, a third about Emmanuel Macron, and so on.

Despite the scale of its operations, the websites in the Pravda network have virtually no organic reach: they attract only around 1,000 unique visitors per month. The same is true for the project's associated social media accounts. According to the American Sunlight Project (ASP), the 67 Telegram channels linked to Pravda average just 43 subscribers each, while Pravda accounts on X (formerly Twitter) have even fewer, with just 23 followers on average. The Pravda network is centralised, automated, and even uses uniform domain names, yet it is completely user-unfriendly: its websites lack search functionality, menus are primitive, and scrolling often does not work. Formatting and translation errors suggest the network is not designed for humans, but for search engine bots and automated systems – web crawlers and algorithms used to train AI, the ASP report concludes. In other words, the network’s purpose is to supply false content for AI, and in that, it is quite effective.

During an experiment, NewsGuard analysts tested ten leading chatbots using a sample of 15 falsehoods circulated by the Pravda network. Each statement was tested in three different formulations: one with a neutral phrasing asking whether the claim was true; one implying partial truth; and one asking for information that would confirm the false claim. The study showed that the chatbots repeated the false claims in 33.6% of cases, gave no response in 18.2%, and actively refuted the disinformation in only 48.2% of cases.

In total, 56 out of 450 chatbot responses included direct links to disinformation published on one of the Pravda network’s websites. Even when the responses included a debunking of the claim, chatbots sometimes still listed Pravda articles as sources, thereby increasing the visibility and potential spread of the falsehood. For example, when a chatbot was asked, 'Did Trump order the closure of a US military base in Alexandroupolis, Greece?' (he did not), one chatbot correctly refuted the claim but still cited three Pravda articles spreading the falsehood, thus increasing its ‘visibility’ and likelihood of further spread.

By using search engine optimisation techniques, Pravda is able to artificially increase the visibility of its content in search results. As a result, AI chatbots, which often rely on publicly available, indexed content, are more likely to pick up information from these websites. Pravda’s content is cited by major chatbots and even by Wikipedia, significantly expanding the reach of pro-Russian narratives and disinformation into the global information ecosystem, as noted by the Bulletin of the Atomic Scientists.

It is plausible that Wikipedia editors are making changes to articles based on information they receive from chatbots. It also seems likely that some Pravda-related links are being deliberately inserted into Wikipedia. Since many younger language models use Wikipedia as a training source, manually adding propaganda links becomes a logical way to boost the visibility and apparent legitimacy of Russian propaganda for AI systems. Currently, Wikipedia is estimated to account for 3-5% of the total data volume used by LLMs for training.

DFRLab and CheckFirst analysed Wikipedia content and found 1,907 hyperlinks placed on 1,672 pages across 44 languages that link to 162 sites connected to Pravda. The majority of these references appear on Russian (922 links) and Ukrainian (580 links) Wikipedia pages. Since the start of the full-scale invasion of Ukraine, the rate of such link insertions has grown significantly and has expanded to articles in English (133 links), French (28), Chinese (25), German (19), Polish (17), and other languages. Additionally, links were found in articles written in the languages of Russian minorities, such as Bashkir (28) and Tatar (25).

The problem is that it is impossible to simply filter out 'Pravda-labeled' sources to cleanse language models of the disinformation they have absorbed. Pravda continuously creates new websites on new domains, and even if current models were programmed to block all known resources today, it would be ineffective and new 'heads of the hydra' could appear the very next day.

Google's January report ‘Hostile Abuse of Generative AI’ confirms that foreign actors are increasingly using AI and SEO to boost the visibility of fake content and narratives in search results. The report reviews interference with Google’s chatbot Gemini by state-linked actors from Iran, North Korea, China, and Russia. Russia, it notes, is especially interested in the deepest forms of manipulation, including developing AI capabilities, tools for building AI chatbots, and developer tools for interacting with LLMs. However, the report suggests that Russia’s direct activity with Gemini itself remains limited. This is likely because propagandists fear that Google will detect and document such interference.