Deepfakes

• Through various spoofing techniques, individuals can impersonate another person or company to gain their victims’ trust and deceive, manipulate, or defraud them. Deepfake-enhanced scams such as CEO fraud (or “president fraud”), in which criminals impersonate company executives to trick employees into revealing sensitive data or transferring funds, have risen sharply. For instance, in early 2024, an employee in Hong Kong transferred €23 million to scammers who simulated an entire videoconference using AI, convincing him he was speaking with his manager and the company’s board of directors.
• Even biometric verification systems (such as facial or voice recognition) can be bypassed by deepfakes, enabling perpetrators to gain access to smartphones, computers, bank accounts, or other sensitive data.
• Classic fraud schemes, such as the “grandparent scam”, “shock calls”, “romance scams”, and other online frauds, have become far more effective through the use of deepfakes. Personal data from potential victims can easily be collected from public profiles and analyzed to create detailed psychological profiles or generate personalized conversation scripts. Messages can be flawlessly translated, mass-sent, and even automated. With the increasing quality of deepfakes and the personalization of approaches, such scams have become more credible than ever. Many victims feel reassured when they believe they are speaking live to a familiar person whose voice or face they recognize on a call, unaware that it is a synthetic imitation. Once trust is established, they often share more sensitive information. Fraudsters can then build emotional connections and exploit these relationships to commit further crimes, such as grooming (approaching minors online for sexual exploitation), cyberstalking, or sextortion (blackmail using intimate images).
• In this context, social engineering techniques exploit cognitive biases and emotions to make deceptive narratives seem credible. Applied to deepfakes, they reinforce and accelerate all forms of scams by:
  • creating a sense of urgency or fear that short-circuits critical thinking;
  • impersonating the appearance, voice, or authority of trusted individuals to elicit obedience;
  • exploiting victims’ goodwill or empathy to extract information or money.

Deepfakes can serve dual purposes: on one hand, to discredit or defame someone by attributing false statements or actions to them; on the other, to exploit the credibility and public image of well-known figures to lend weight and legitimacy to misleading messages.

Political figures are often prime targets of manipulation campaigns, but other public personalities, such as actors, presenters, or well-known media brands, are also used because of their visibility and the trust they inspire. Since a large amount of public audiovisual material exists about them, it is easy to feed AI systems with such data to produce highly convincing deepfakes.

Deepfakes are also used to produce and disseminate coordinated disinformation campaigns that aim to:

• manipulate public opinion;
• sow doubt and undermine trust in public institutions, the media, and democracy as a whole;
• compromise the integrity of elections and democratic processes;
• divide society.

AI-generated content can serve multiple manipulative purposes, particularly in the run-up to elections, posing a serious threat to democratic integrity.

Two days before the parliamentary elections in Slovakia in September 2023, for example, an AI-generated audio recording circulated in which opposition leader Michal Šimečka appeared to admit to manipulating the vote in his party’s favor. Shortly before the New Hampshire primary in the United States in January 2024, an automated robocall mimicking Joe Biden’s voice was sent out, apparently to discourage voters from going to the polls. As early as April 2023, the Republican Party had released, immediately after Biden announced his bid for re-election, an AI-generated campaign video depicting a dystopian scenario of his second term: scenes of chaos, economic collapse, war, and uncontrolled migration, all designed to dissuade voters. The case of Rumeen Farhana, an opposition politician in Bangladesh, whose AI-generated images showing her in a bikini circulated in late 2023, illustrates that significant harm can occur even when the manipulation is obvious. Although the fake was quickly exposed, the images provoked public outrage in a predominantly Muslim country. And then there is Donald Trump, who regularly shares deepfakes featuring himself, including a recent one showing him dropping excrement from a fighter jet onto protesters from the “No Kings” movement, mocking opponents of his government who were demonstrating against his authoritarian drift and growing concentration of power. Malicious actors use deepfakes to influence electoral choices in various ways: by damaging the integrity of political opponents, deterring certain voters from participating in the election, or deliberately spreading false electoral information. These technologies are also used to stage fear-inducing or threatening scenarios, transforming abstract ideas into vivid images that make the conveyed messages more credible, striking, and persuasive. More broadly, they are employed to steer public opinion, often by mobilizing outrage around sensitive topics such as migration or refugee reception, to reinforce specific political agendas. At the same time, they help consolidate and amplify pro-government narratives, distract attention from critical or oppositional issues, and saturate the information space with misleading or irrelevant content.

The proliferation of disinformation and the ease with which AI-generated content can mimic reality are undermining a fundamental human capacity: to understand the world through their senses, and to rely on what they see and hear to make sense of reality. This blurring of the boundary between true and false leads to a loss of trust; the feeling that nothing can be verified, and weakens collective confidence. This situation is further aggravated by what researchers call the “liar’s dividend”: the tendency for political actors to exploit the very existence of manipulated content to deny genuine facts that may be unfavorable to them. Thus, deepfakes become tools of strategic denial, contributing to the erosion of trust in the media, institutions, and even the concept of truth itself.

In January 2023, Luxembourg’s main political parties (DP, LSAP, déi gréng, ADR, Déi Lénk, Piraten, Fokus, and Volt) signed an electoral agreement committing to conduct fair and factual campaigns for the 2024 European elections. In this agreement, they expressed their intention to use social media responsibly and to refrain from personal attacks, the spread of false information, defamatory campaigns, and the tampering with or manipulation of other parties’ campaign materials. Although the text does not explicitly mention a ban on deepfakes, it does include a pledge not to use bots or automated propaganda or manipulation campaigns. It can therefore reasonably be inferred that such AI-based technologies should not be employed for disinformation purposes.

This category notably includes deepnudes: pornographic content generated without the consent of the people depicted. The production, possession, or distribution of child-pornographic material, whether AI-generated or not, that depicts minors is strictly prohibited and punishable by law.

Deepfakes are rarely disseminated in isolation; they are often embedded in hybrid campaigns that combine various AI-based tactics to maximize their reach and impact.

In Luxembourg, deepfakes have circulated featuring, among others, Prime Minister Luc Frieden, the Mayor of Luxembourg City Lydie Polfer, Grand Duke Henri, and former Prime Minister Jean-Claude Juncker.

In these manipulated or AI-generated videos and articles, the public figures appeared to give financial advice and encourage the public to invest through cryptocurrency platforms. Journalists from RTL, Mariette Zenners, Caroline Mart, and Lynn Cruchten, as well as actress Désirée Nosbusch, were also victims of deepfakes used to make these scams more convincing.

The fraudsters combined several methods:

• reproducing the design of reputable news websites (such as RTL Luxembourg or Tagesschau), copying logos, colors, and layouts;
• creating fake articles with sensational headlines promising “exclusive revelations”;
• promoting these fakes via fake social-media profiles and paid advertising services on Facebook and Instagram.