The National Institute of Standards and Technology (NIST) has unveiled a groundbreaking initiative, GenAI, aimed at tackling the escalating threat of deepfakes and AI-generated misinformation. GenAI, a comprehensive program developed by NIST, is set to assess the capabilities and limitations of generative AI technologies, particularly in text and image generation.

In a press release issued on Monday, NIST detailed the objectives of GenAI, which include releasing benchmarks, developing content authenticity detection systems, and fostering the creation of software to identify the source of fake or misleading AI-generated content. The program seeks to promote information integrity and guide the responsible use of digital content in an era where AI-generated misinformation proliferates.

One of GenAI's primary focuses is a pilot study to distinguish between human-created and AI-generated media, starting with text. This project invites participation from academia, industry, and research labs to submit both content-generating AI systems and systems designed to identify AI-generated content. The study sets strict parameters for generators and discriminators, aiming to ensure fairness and compliance with applicable laws and regulations.

Registration for the pilot study begins on May 1, with the first round scheduled to close on August 2. Final results from the study are anticipated to be published in February 2025.

The launch of GenAI coincides with a concerning trend highlighted by Clarity, a deepfake detection firm, reporting a staggering 900% increase in the creation and dissemination of deepfakes compared to the previous year. This surge in AI-generated misinformation has prompted widespread alarm, with an overwhelming 85% of Americans expressing concerns about misleading deepfakes spreading online, according to a recent YouGov poll.

NIST's initiative aligns with President Joe Biden's executive order on AI, which emphasizes transparency from AI companies and sets forth new standards, including labeling content generated by AI. GenAI marks the first major AI-related announcement from NIST following the appointment of Paul Christiano, a former OpenAI researcher, to the agency's AI Safety Institute.

However, Christiano's appointment has sparked controversy due to his stark warnings about the potential risks of AI development. Critics, including scientists within NIST, fear that Christiano's perspectives may steer the AI Safety Institute towards focusing on hypothetical scenarios rather than addressing immediate AI-related risks. NIST maintains that GenAI will play a pivotal role in informing the work of the AI Safety Institute.

As the proliferation of AI-generated misinformation continues unabated, initiatives like GenAI underscore the urgent need for collaborative efforts to safeguard information integrity and mitigate the risks posed by rapidly evolving AI technologies.