Scams and deepfake technology remain key challenges in India: MeitY report

As artificial intelligence (AI) advances in the age of emerging technologies, instances of deepfake and AI-generated content (ART) misuse are also increasing.

The Delhi High Court has received a report from India’s Ministry of Electronics and Information Technology (MeitY).

The report draws attention to the proliferation and abuse of AI scams and deepfake technology in India, particularly during state elections.

It also emphasizes that rather than enacting a new law, significant and stringent measures should be taken.

In November 2024, MeitY established a nine-member committee to investigate the growing prevalence of deepfakes in India.

During a meeting with technology and policy stakeholders on January 21, the stakeholders expressed concerns about an “over-reliance on intermediary liability frameworks for AI-generated content regulation.”

They highlighted the need to "improve the capacity of investigative and enforcement agencies rather than introducing new regulations" and emphasized that the focus should be on combating malicious actors rather than regulating the creative or benign uses of deepfake technology.

Additionally, the stakeholders emphasized that “there should be regulation around mandatory AI content disclosure, labeling standards, and grievance redressal mechanisms, while focusing on malicious actors rather than creative or benign uses of deepfake technology.”

According to the report, many groups agree that existing laws, including the Information Technology Act, 2000 (IT Act), the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021 (IT Rules, 2021), and the Bharatiya Nyaya Sanhita, 2023 (BNS), are sufficient to prevent the misuse of harmful deepfakes.

However, these laws need to be enforced more rigorously, with improved forensic tools.

Two concerning trends regarding the misuse of deepfake technology have been identified by the Deepfakes Analysis Unit (DAU), an initiative under the Meta-supported Misinformation Combat Alliance (MCA).

The DAU has flagged that deepfakes are specifically targeting women during state elections.

The report also highlights AI-driven scams that occur after elections and notes that detecting deepfake audio remains a significant challenge due to the difficulty in distinguishing real from manipulated voice recordings.

Furthermore, the Ministry of Electronics and Information Technology is working with the Ministry of Information and Broadcasting to collect testimonies from individuals affected by deepfake content.

The committee has requested a three-month extension from the Delhi High Court to finalize these discussions, and the court has granted the request.