Undetected AI: A DeepFake scam!

When Mr. Nair (name changed) received a video call from an unknown number, he picked it up to find a former colleague on the other end. After a brief conversation, where the colleague mentioned the names and details of common acquaintances, he requested a sum of Rs. 40,000 for hospitalization expenses for his child.

Mr. Nair, transferred the money to his colleague’s account. Within a short time, when the colleague asked for Rs. 35,000 more, he grew suspicious and called his acquaintance on the number he had saved on his phone. It was then that he realized that he has been scammed. Mr. Nair had become one of the first victims of DeepFake fraud in India.

DeepFakes have surpassed even the strongest authentication platforms put into practice, they are computer-generated images or videos that replace or alter a person's voice or other distinguishing characteristics. Their algorithms have the ability to replace one person's face with another in a video or make the person lip-sync to the audio that has been replaced.

Moreover, DeepFakes can be used to mimic the sound of another person's speech, tricking even those who know them well. Deep learning is a class of AI techniques employed by these algorithms; it is the same class of techniques used to automatically recognize things in films and tag friends in photographs posted on social media.

India, home to 800 million internet users, is emerging as a hub for the deepfake epidemic. From 2022 to 2023, there was a notable 10x increase in the number of deepfakes found worldwide in all industries.According to the 2023 State of Deepfakes, the introduction of inexpensive AI technologies has caused a dramatic rise in DeepFake videos, which have increased more than five-fold (550 percent) since 2019.

The earliest DeepFakes were easy to detect. Researchers pointed out that DeepFake characters did not blink normally. Soon after this research was published, DeepFake videos with realistic blinking were created. DeepFake videos also show some obvious signs such as bad lip sync, pixelation, and discoloration around the edges of the transposed faces in videos. As algorithms got better and increased the sophistication shown by financial scammers, the quality consistently improved. This proved a threat to detection systems to a level that solving them got even more difficult.

Paradoxically, the best weapon against Deepfakes might be artificial intelligence, the very technology used to create them. One of the earliest attempts was Facebook’s DeepFake Detection challenge, which attempted to “accelerate development of new ways to detect DeepFake videos”. Though the results of the contest were encouraging, Deepfake creation has made good progress.

With increasingly powerful computing available on demand and plenty of freely available code, it has become simple for a person with basic programming knowledge to create realistic facsimiles that pretend to be another person.

Today, we are seeing an arms race between scamsters who come up with better Deepfakes and technology companies that try to catch them.

A combination of vision, segmentation and object tracking algorithms to detect DeepFakes is used in this current generation by top players in the field.

The rise of DeepFakes presents a formidable threat, not just as a technological manipulation but as a potential tool for scams and deception. The malicious use of DeepFake technology poses significant risks to individuals, businesses, and society as a whole. Preventing the proliferation of deepfakes is not merely a technological challenge; it is a societal imperative to preserve trust, security, and the integrity of our digital landscape.

-- Ashok Hariharan, CEO, IDfy.