Deepfake videos and images grow ahead of Indian polls, stirring unease

The video that regenerated the former Tamil Nadu chief minister through artificial intelligence (AI) got much applause at the gathering in Salem, especially when the AI-generated Karunanidhi scolded the national government under Prime Minister Narendra Modi for “robbing the state” of its due share of funds. 

A month later, Mr Karunanidhi’s AI avatar appeared again in the leader’s trademark yellow shawl and dark glasses at the autobiography launch of party veteran T.R. Baalu, congratulating his dear friend. 

This video, broadcast live by several Tamil news channels, was playing on the television in a Chennai mall’s break room as housekeeper Frieda Devadasan, 56, sipped her tea. She watched the broadcast, but only realised days later that the video of the leader she had grown up seeing had been “created by computers”. 

“I was fooled! I thought the whole time that the channel was replaying one of his old videos!” she said, more thrilled than embarrassed. 

AI-generated videos and voices have become a handy tool for political parties in India, with many launching trials before fine-tuning them for extensive use in election campaigning in the national polls due in April. 

“Since AI videos and voices have a novelty factor, we want to use them as entertainment to draw crowds. There will be more use of AI-generated content towards the national election,” said DMK spokesman Dharanidharan Selvam. 

Not to be left behind, DMK’s rival, the All India Anna Dravida Munnetra Kazhagam (AIADMK), also released in late February an AI-generated audio clip of its leader J. Jayalalithaa, who died in 2016.

The AI-generated voice of Tamil Nadu’s only female chief minister repeated her popular catchphrase: “I am because of the people, I am for the people” (Makkalaal naan, makkalukkaagave naan), before asking people to stand with Mr Edappadi Palaniswami, the current party president.

Mr Senthil Nayagam, the founder of Muonium AI, which made the AI-generated Karunanidhi video for Mr Baalu, said: “Earlier, making such videos would require a visual effects person to spend three days. But AI technology has cut the cost and time taken to a fraction. We can generate any voice, any face, remove people in an existing video, add people, make them say anything.”

His first attempts were to market his AI-generated content to the Tamil film industry, but Mr Senthil said he is in “talks with many political parties, who all want AI-generated content for the upcoming (national) election”. 

Drawing in voters with AI-generated content that is acknowledged to be fake is one thing. But its use by political parties, which escalated in India in 2023 during elections in five states, can often be insidious.

In the southern state of Telangana, whose capital is the tech city of Hyderabad, the Bharat Rashtra Samithi (BRS) released AI-generated photos last October of its top leaders meeting the elderly, waving to enormous crowds and visiting the ill in hospitals. These occasions never happened.

Mr Archis Chowdhury from the Indian fact-checking website Boom Live also noted that while some AI-generated content was linked to a political party, the real danger came from AI-generated videos and images used for “shadow campaigning”, whose origins were untraceable.

For instance, on the day of polling for the Telangana state assembly on Nov 30, a seven-second video of BRS leader K.T. Rama Rao asking people to vote for rival Congress went viral, even ending up on the state Congress’ official social media page. The video turned out to be a deepfake. 

Deepfakes are manipulated media that create a person’s likeness in an image or a video. They are created via AI using deep learning algorithms.

Mr Rao’s party BRS complained to the Election Commission, asking it to “prevent any further harm to the fair conduct of the election” and accusing the Congress, but as election results rolled in, no action was taken. 

Hyderabad’s Assistant Commissioner of Police, Cybercrime, Mr Shiva Maruti, said the difficulty in differentiating genuine and manipulated content led to public confusion, and was “a potential threat to the integrity of the electoral process in the run-up to the elections”.

Ahead of the Madhya Pradesh state elections in November 2023, a video of Congress leader Kamal Nath saying he would cancel a subsidy for girls, and another of Bharatiya Janata Party leader Shivraj Chauhan expressing doubt about his party’s return to power, turned out to be fabricated. Boom Live found they were made using voice cloning technology.

“After the use of AI-generated content in elections in Pakistan, Bangladesh and Indonesia, companies like OpenAI (which made ChatGPT) and (video, image generator) Midjourney banned use of their publicly available, free tools for political purposes by putting some filters on prompts, but people are bypassing it quite easily in India,” Boom Live fact checker Archis Chowdhury told The Straits Times. 

He anticipated that parties with the most funds, like the ruling BJP, opposition Congress, Tamil Nadu’s DMK and AIADMK, and West Bengal’s Trinamool Congress, would invest in AI content, especially in constituencies with close contests.

In the southern state of Andhra Pradesh, the Telugu Desam Party’s national spokesman Pattabhi Ram Kommareddy said his party was already using AI “in positive ways” to do quick research on constituencies to make speeches more relevant. But after deepfake images of its leader Chandrababu Naidu in prison for alleged corruption emerged online some months ago, the party is now also wary about the misuse of AI. 

The founder of the technology policy analysis website Medianama, Mr Nikhil Pahwa, is fairly certain that “2024 will be the year of Deepfake Elections”.

He wrote in The Times of India that “just-in-time, targeted, malicious audio and deepfake videos” released a few days before polls would be the trickiest, given how little time there would be to combat them.

But on top of those, India could also expect a barrage of “videos mixing fact with fiction, AI-generated false testimonials, manipulation of historical records, fake polling data and analysis, synthetic news articles, and counterfeit documents as if they are from election authorities”. 

Regulators are barely able to keep up.

The capacity of fact-checkers, media, politicians and the Election Commission to address disinformation is limited, compared with teams that make and disseminate the content, said Mr Pahwa.

Minister of Electronics and IT Rajeev Chandrasekhar has said that online platforms are “mandated to remove such content within 36 hours upon receiving a report from either a user or government authority”. The Election Commission also puts the onus on social media platforms to detect and remove deepfake narratives, but this approach has not worked.

Political campaign consultant Shivam Shankar Singh said at Medianama’s seminar on deepfakes on Jan 24 that in the 2019 national election, the Election Commission had the same mechanism to take down misinformation but was so flooded with complaints that it could not forward them to the platforms fast enough.

Many experts believe that the only way out is to ban the use of deepfakes for election campaigns, while other experts suggest labelling them clearly as AI-generated content.

Taking on the trend, Congress parliamentarian Shashi Tharoor released a video on Feb 10 in which he took questions on his work and hobbies from his deepfake version. “It’s interesting definitely, but also slightly troubling,” the kurta-clad leader says in the video to his suit-wearing AI avatar.

After many on social media said they could easily identify the real Mr Tharoor by his “buttery English accent” while the AI-Tharoor had an American twang, the leader conceded the point, but said: “The scary thing is that technology will only get better!” 

Rohini Mohan

The Straits Times

Asia News Network