Deepfakes

• Through various spoofing techniques, individuals can impersonate another person or, for example, a company in order to gain their victims’ trust and deceive, manipulate, or defraud them. Deepfake-enhanced scams such as CEO fraud (or “president fraud”), in which criminals impersonate company executives to trick employees into revealing sensitive data or transferring funds, have risen sharply. For instance, in early 2024, an employee in Hong Kong transferred €23 million to scammers who simulated an entire videoconference using AI, convincing him he was speaking with his manager and the company’s board of directors.
• By bypassing biometric verification systems (such as facial or voice recognition) using deepfakes, enabling fraudsters to gain access to smartphones, computers, applications, bank accounts and, consequently, sensitive data.
• Through the use of deepfakes to enhance traditional fraud schemes, such as the so-called “grandparent scam,” “shock calls”, “romance or love scams”, or other forms of online fraud. Data on potential victims can be easily compiled from publicly available profiles and analysed to create detailed personal profiles or customised conversation scripts. Messages can be translated without error and disseminated on a large scale. In addition, the ability to automatically generate personalised responses significantly increases the scalability of fraudulent activities, enabling perpetrators to conduct numerous conversations in parallel and thereby reach a much larger number of people. Improved deepfake quality and increased message personalisation enhance the credibility of these fraud attempts. Many victims feel reassured when they believe they are speaking in real time with a familiar person whose voice or face they recognise during a phone or video call, without realising that it is an imitation. Once trust has been established, victims often disclose further sensitive information. Fraudsters can then build emotional ties and exploit these relationships to commit additional crimes, such as grooming (the online approach of a minor for the purpose of sexual exploitation), cyberbullying, or “sextortion” (blackmail involving intimate images).
• In this context, social engineering techniques exploit cognitive biases and emotions to make deceptive narratives seem credible. Applied to deepfakes, they reinforce and accelerate all forms of scams by:
  • creating a sense of urgency or fear that short-circuits critical thinking;
  • impersonating the appearance, voice, or authority of trusted individuals to elicit obedience;
  • exploiting victims’ goodwill or empathy to extract information or money.

Deepfakes can serve dual purposes: on one hand, to discredit or defame someone by attributing false statements or actions to them; on the other, to exploit the credibility and public image of well-known figures to lend weight and legitimacy to misleading messages.

Political figures are often the primary targets of manipulation campaigns. However, other public figures, such as well-known actors, prominent presenters or popular media brands, are also affected due to their high visibility and the trust they inspire. The large amount of publicly available audiovisual material about them further facilitates the use of this data to train AI systems, enabling the creation of highly realistic deepfakes.

Deepfakes are increasingly used as part of coordinated disinformation campaigns designed to:

• manipulate public opinion;
• sow doubt and undermine trust in public institutions, the media, and democracy as a whole;
• compromise the integrity of elections and democratic processes;
• divide society.

AI-generated content can serve multiple manipulative purposes, particularly in the run-up to elections, posing a serious threat to democratic integrity.

Two days before the parliamentary elections in Slovakia in September 2023, for example, an AI-generated audio recording circulated in which opposition leader Michal Šimečka appeared to admit to manipulating the vote in his party’s favor. Shortly before the New Hampshire primary in the United States in January 2024, an automated robocall mimicking Joe Biden’s voice was sent out, apparently to discourage voters from going to the polls. As early as April 2023, the Republican Party had released, immediately after Biden announced his bid for re-election, an AI-generated campaign video depicting a dystopian scenario of his second term: scenes of chaos, economic collapse, war, and uncontrolled migration, all designed to dissuade voters. The case of Rumeen Farhana, an opposition politician in Bangladesh, whose AI-generated images showing her in a bikini circulated in late 2023, illustrates that significant harm can occur even when the manipulation is obvious. Although the fake was quickly exposed, the images provoked public outrage in a predominantly Muslim country. And then there is Donald Trump, who regularly shares deepfakes featuring himself, including a recent one showing him dropping excrement from a fighter jet onto protesters from the “No Kings” movement, mocking opponents of his government who were demonstrating against his authoritarian drift and his tightening grip on power.

Malicious actors use deepfakes to influence electoral choices in various ways: by damaging the integrity of political opponents, deterring certain voters from participating in the election, or deliberately spreading false electoral information. These technologies are also used to stage fear-inducing or threatening scenarios, transforming abstract ideas into vivid images that make the conveyed messages more credible, striking, and persuasive. More broadly, they are employed to steer public opinion, often by mobilizing outrage around sensitive topics such as migration or refugee reception, to support specific political agendas. At the same time, they serve to reinforce and amplify pro-government narratives, while diverting attention from issues critical of those in power and saturating the information space with irrelevant, misleading or false information.

The proliferation of disinformation and the ease with which AI-generated content can mimic reality are undermining a fundamental human capacity: to understand the world through their senses, and to rely on what they see and hear to make sense of reality. This blurring of the boundary between truth and falsehood fosters the feeling that nothing is true anymore and can be verified, thereby undermining collective trust. This situation is likely to be further exacerbated by what researchers refer to as the “liar’s dividend”, which enables political actors to challenge, or even deny the authenticity of facts that prove inconvenient or unfavorable to them by invoking the possibility of deepfake manipulation. Thus, deepfakes become tools of strategic denial, contributing to the erosion of trust in the media, institutions, and even the concept of truth itself.

In January 2023, Luxembourg’s main political parties (DP, LSAP, déi gréng, ADR, Déi Lénk, Piraten, Fokus, and Volt) signed an electoral agreement committing to conduct fair and factual campaigns for the 2024 European elections. In this agreement, they expressed their intention to use social media responsibly and to refrain from personal attacks, the spread of false information, defamatory campaigns, and the tampering with or manipulation of other parties’ campaign materials. Although the text does not explicitly mention a ban on deepfakes, it does include a pledge not to use bots or automated propaganda or manipulation campaigns. It can therefore reasonably be inferred that such AI-based technologies should not be employed for disinformation purposes.

This category notably includes deepnudes: pornographic content generated without the consent of the people depicted. The production, possession, or distribution of child-pornographic material, whether AI-generated or not, that depicts minors is strictly prohibited and punishable by law.

Deepfakes are rarely disseminated in isolation; they are often embedded in hybrid campaigns that combine various AI-based tactics to maximize their reach and impact.

In Luxembourg, deepfakes have circulated featuring, among others, Prime Minister Luc Frieden, the Mayor of Luxembourg City Lydie Polfer, the abdicating Grand Duke Henri, as well as former Prime Minister Jean-Claude Juncker. In these manipulated or AI-generated videos and articles, the public figures appeared to give financial advice and encourage the public to invest through cryptocurrency platforms. Journalists from RTL, Mariette Zenners, Caroline Mart, and Lynn Cruchten, as well as actress Désirée Nosbusch, were also victims of deepfakes used to make these scams more convincing.

The fraudsters combined several methods:

• reproducing the design of reputable news websites (such as RTL Luxembourg or Tagesschau), copying logos, colors, and layouts;
• creating fake articles with sensational headlines promising “exclusive revelations”;
• promoting these fakes via fake social-media profiles and paid advertising services on Facebook and Instagram.