Loading ad...

OpenAI Stops Russia-Linked Campaign That Used ChatGPT for Fake Social Media Content

OpenAI Stops Russia-Linked Campaign That Used ChatGPT for Fake Social Media Content

OpenAI says it disrupted a new covert influence operation linked to Russia that used ChatGPT to produce social media content promoting a supposedly independent “expert community.”

The company recently banned a cluster of ChatGPT accounts believed to originate in Russia after an investigation uncovered activity connected to the International Burke Institute (IBI). This organization presented itself as an expert community based in Israel. The incident also highlights the growing importance of understanding the broader AI systems and infrastructure behind generative AI, beyond the chatbot itself.

What initially appeared to be a collection of AI-generated social media posts turned out to be part of a wider operation. According to OpenAI, the campaign involved a website filled with copied and misattributed academic material, a “sovereignty index” that portrayed Russia favorably, and attempts to disguise the real origins of the people operating the network.

OpenAI said the operation appeared to have had a relatively limited reach, but its structure and efforts to appear legitimate made it notable.

How ChatGPT Was Used in the Campaign

The operators reportedly used ChatGPT mainly to create social media posts promoting articles from the IBI website.

OpenAI identified AI-generated content connected to the campaign on X, LinkedIn, Facebook, Substack, and Telegram. Some posts were published through accounts using the IBI name and logo, while others appeared to come from ordinary users whose primary activity involved sharing IBI-related material.

The operators reportedly prompted ChatGPT in Russian but instructed the system to produce most of the content in English. They also attempted to remove linguistic clues that could reveal their Russian background.

According to OpenAI, the users accessed ChatGPT through VPNs because the service is not available in Russia.

The campaign also involved generating replies to posts made by genuine Substack users. These replies often promoted IBI’s channels, allowing the operation to place its messaging within existing online conversations rather than relying only on its own accounts.

The case demonstrates how generative AI can be used to produce content quickly across several platforms. At the same time, AI adoption is growing for legitimate purposes around the world. OpenAI has been expanding its activities in different markets, including Brazil. You can read more about this development in our report on OpenAI’s expansion in Brazil as AI adoption surges.

The International Burke Institute

At the center of the operation was the International Burke Institute, or IBI.

The organization presented itself as an expert community based in Israel and maintained a website containing articles and research covering a range of international issues.

However, OpenAI’s investigation found serious problems with the material published on the website.

The company reviewed a sample of 36 articles that were linked to experts on the IBI website. Of those, 34 appeared to have been copied from other sources online.

Some of the articles were also attributed to people who did not originally write them. OpenAI gave examples involving academic work that appeared to have been taken from legitimate publications but presented under different names.

The apparent purpose was to make the website look more credible by filling it with authentic-looking research while making it difficult for readers to identify the original sources.

OpenAI also found evidence that some content appeared to have been translated from Slavic languages into English, producing unusual wording that could reveal the background of those responsible.

Promoting a Pro-Russia Narrative

Another major part of the website was a so-called “sovereignty index.”

The index was used to discuss the level of sovereignty of different countries, but OpenAI said the reports frequently portrayed Russia favorably while criticizing Western countries and institutions.

Reports focused on countries including France, Germany, the United States, and members of the European Union.

The campaign also produced German-language content for a Telegram channel called “Lahme Ente,” which criticized Ukraine, the EU, and the German government while promoting improved relations with Russia.

Another operator reportedly used ChatGPT to create logos for several Telegram channels focused on countries such as Germany, the United States, France, Poland, and Türkiye.

The activity shows that the campaign was not simply about generating individual posts. Instead, AI was incorporated into a broader effort involving websites, social media accounts, branding, translations, summaries, and promotional content.

AI and the Changing Digital Landscape

The incident highlights a growing challenge surrounding generative AI: the same technology that can help organizations work faster can also be misused to create and distribute deceptive information at scale.

Businesses are increasingly using AI tools to speed up research, software development, communications, and other everyday tasks. One example is Stampli’s use of ChatGPT Work and Codex, which helped the company reduce product launch time by 68%.

The contrast is important. AI itself is not inherently harmful or beneficial; its impact depends heavily on how people use it. While businesses can use AI to improve productivity, bad actors can use similar capabilities to create convincing content, operate multiple accounts, and spread narratives more efficiently.

For platforms and AI companies, identifying these patterns may require looking beyond individual pieces of content to examine how websites and online activity connect.

What Was the Impact?

Despite the sophistication of the operation, OpenAI said its immediate impact appeared to be limited.

Many of the social media posts received relatively low numbers of views, and the official IBI accounts had relatively few subscribers.

However, some of the Telegram channels connected to the operation reportedly attracted between 10,000 and 20,000 followers.

OpenAI assessed the campaign as being at the lower end of Category Three on the Brookings Breakout Scale, meaning it operated across multiple platforms and showed some signs of reaching authentic audiences.

The relatively small audience does not necessarily make the activity insignificant. The operation’s structure demonstrated how AI-generated content could be combined with other online tactics to create an appearance of legitimacy.

OpenAI’s Response

OpenAI said it banned the ChatGPT accounts associated with the operation and investigated how the technology had been used.

The company emphasized that detecting and disrupting covert influence operations is part of its broader effort to prevent misuse of its AI systems.

The investigation also illustrates why transparency and verification remain important in the digital environment. Websites that appear professional, social media accounts that look authentic, and articles that seem academic may not always accurately reflect who is behind them or where their information originated.

As generative AI becomes more widely available, distinguishing genuine communication from coordinated manipulation may become increasingly important for platforms, researchers, journalists, and everyday internet users.

Final Thoughts

OpenAI’s latest investigation provides another example of how AI technology is becoming part of modern influence operations.

In this case, ChatGPT generated social media content and supported activity across several online platforms. At the same time, the wider operation relied on a website, copied research, misleading attribution, and pro-Russia messaging.

Although the campaign’s overall reach appeared limited, its organization demonstrates how AI can be incorporated into a larger influence network.

The challenge for technology companies will be to continue supporting legitimate uses of AI while identifying and disrupting attempts to use these tools for deception, manipulation, or covert influence.

Leave a Reply

Your email address will not be published. Required fields are marked *