Navigating the realms of unreality: How to protect yourself from deepfake fraud

Coined from "deep learning" algorithms that fuel its magic and the undeniable "fakeness" it brings to the table, deepfakes are the ultimate illusionists of the internet age, capable of creating videos, images, or sounds that mimic reality so closely, they blur the lines between what's real and what's AI-generated. Suddenly, seeing isn't believing anymore, especially when celebrities and public figures can be made to appear as if they're saying or doing things that are purely fictional. As we navigate through a sea of digital content, from unlocking our phones with a glance to binge-watching video feeds, deepfakes loom as a trendy yet troubling challenge. How do we keep it real when fakes feel just as authentic?

We invite Nation readers to dive into the heart of this digital dilemma with us, where we explore the cool, the quirky, and the downright creepy world of deepfakes.

Deepfake is an image, a video or sound that is manipulated or created anew to make it look and sound authentic like a real thing or real person using algorithms based on a type of AI called Deep Learning (hence, the name “deep”). As it is created without the actual person, often a celebrity, ever saying or doing the thing we saw, it is “fake”. With so many of these media being digitised and used online in many ways, from authentication by face or voice, to video feed consumption, deepfake has become a new frontier in the problem of how to distinguish between authentic and fake digital media.

Deep Learning is an AI which mimics (or more accurately, is inspired by) the human neural system that learns and deduces. This underlying technology is used to create a deepfake by having two Deep Learning AI’s competing against each other. One is trained to create the most convincing deepfake whereas the other one is trained to detect the fakeness of what the first AI created. The second AI then gives feedback to the first AI so that it can adjust its system to create something that is more convincing. The process is repeated until the second AI cannot detect fakeness in the resulting deepfake, which is what we ultimately get to see or hear. Since the breakthrough of this technique in creating deepfakes, people have used it to bring deceased artists back to life performing on stage for the fans, create John Lennon’s voice in The Beatles’ latest hit “Now and Then” released in late 2023, and render things that are not possible in physical world generating the wow factor in the cinematography of many films. On the other hand, the deepfake technique has also been used to generate a voice of a president saying things he did not say, and a video and images of a drunk politician or celebrity in a compromising position that never happened. As in most cases, technology can be used for good or evil purposes, depending on the user of technology. Deepfakes fall into the same category.

In a sophisticated case of fraud reported by The Wall Street Journal, the CEO of a UK-based energy firm was deceived into transferring €220,000 (slightly more than 10 million baht) by a fraudster who used AI voice technology to imitate the voice of his boss, the CEO of the firm's German parent company. The fraudster's convincing use of the German executive's subtle accent and speech patterns prompted the UK CEO to make the transfer to a supposed Hungarian supplier. The scam was executed through three phone calls, with the fraudster initially requesting the transfer, then falsely claiming reimbursement had been made, and finally seeking a follow-up payment. Suspicion arose when the reimbursement failed to materialise and the call's origin, an Austrian phone number, was noted.

The transferred funds were quickly moved from Hungary to Mexico before being dispersed further. This incident marks a notable use of AI in committing financial fraud, underscoring the potential for machine learning technologies to be weaponised. Despite the transfer of the initial amount, no subsequent payments were made upon once the scam was unveiled. This case sheds light on the emerging threats posed by AI technology in the realm of financial security, with the fraudsters' methodology and the specific software used remaining largely unknown.

Another case study, the Taylor Swift incident during the Super Bowl, involved viral images of a digitally manipulated doppelgänger of pop star Taylor Swift, created using deepfake technology. Initially crafted by a digital artist and spread rapidly on social media, these lifelike images sparked debates on privacy, ethics, and legal issues surrounding digital impersonation and deepfake technology.

After reading all this, one question must have popped up in the reader's mind: how do we spot deepfakes from the real thing. Here are some guidelines:

1. Use your common sense and don’t believe something that is too unbelievable: Bruce Lee playing table tennis with his nunchuks is impossible, maybe apart from one ball hit (fake!). A celebrity with a decent track record suddenly behaving in an unruly way? Probably not real (likely fake!).

2. When trying to verify a person, use some form of two-factor authentication. If you receive a LINE message asking to borrow some money, call the person to verify and vice versa, if you receive an unusual call with the convincingly authentic voice of the person, before acting, verify by texting back or calling back.

3. Finally, do not totally believe anything you see or hear online. Keep in mind that anything digital could have been modified. We already know about Photoshop where people can manually manipulate images at will very convincingly. These days, with deepfake technology, fakes can be made faster than ever before, sometimes maybe on the fly at the time of use (imagine voice authentication being fake saying the passphrase on the spot). It is always good to be a little sceptical about things. Just a simple pause to think can help avoid disaster,

Many responsible AI software developers provide the tools that can identify deepfake content. For example Adobe Photoshop now provides a tool to let the user know if the content has been manipulated by its software. ZeroGPT is an AI tool which can identify content written by ChatGPT.

But we still require much more regulation and rules on how AI can make use of content to learn and generate fake content, both in terms of copyright and how to protect the public at large from fake content. Just like cyber security problems where the good has to keep up with the evil given the ever-changing advancement in technology, deepfake is another battleground of the two camps in the years to come.

Nattida Sanguansin, Managing Director and Co-founder of BUZZEBEES Co., Ltd. and

Pamornpol Jinachitra, CEO and Board of Director Member of AI GEN, Co., Ltd.