Think tank calls for AI incident reporting system

The Centre for Long-Term Resilience (CLTR) has called for a comprehensive incident reporting system to urgently address a critical gap in AI regulation plans.

According to the CLTR, AI has a history of failing in unexpected ways, with over 10,000 safety incidents recorded by news outlets in deployed AI systems since 2014. As AI becomes more integrated into society, the frequency and impact of these incidents are likely to increase.

The think tank argues that a well-functioning incident reporting regime is essential for effective AI regulation, drawing parallels with safety-critical industries such as aviation and medicine. This view is supported by a broad consensus of experts, as well as the US and Chinese governments and the European Union.

The report outlines three key benefits of implementing an incident reporting system:

Monitoring real-world AI safety risks to inform regulatory adjustments

Coordinating rapid responses to major incidents and investigating root causes

Identifying early warnings of potential large-scale future harms

Currently, the UK’s AI regulation lacks an effective incident reporting framework. This gap leaves the Department for Science, Innovation & Technology (DSIT) without visibility on various critical incidents, including:

Issues with highly capable foundation models

Incidents from the UK Government’s own AI use in public services

Misuse of AI systems for malicious purposes

Harms caused by AI companions, tutors, and therapists

The CLTR warns that without a proper incident reporting system, DSIT may learn about novel harms through news outlets rather than through established reporting processes.

To address this gap, the think tank recommends three immediate steps for the UK Government:

Government incident reporting system: Establish a system for reporting incidents from AI used in public services. This can be a straightforward extension of the Algorithmic Transparency Recording Standard (ATRS) to include public sector AI incidents, feeding into a government body and potentially shared with the public for transparency.

Engage regulators and experts: Commission regulators and consult with experts to identify the most concerning gaps, ensuring effective coverage of priority incidents and understanding stakeholder needs for a functional regime.

Build DSIT capacity: Develop DSIT’s capability to monitor, investigate, and respond to incidents, potentially through a pilot AI incident database. This would form part of DSIT’s central function, initially focusing on the most urgent gaps but eventually expanding to include all reports from UK regulators.

These recommendations aim to enhance the government’s ability to responsibly improve public services, ensure effective coverage of priority incidents, and develop the necessary infrastructure for collecting and responding to AI incident reports.

Veera Siivonen, CCO and Partner at Saidot, commented:

“This report by the Centre for Long-Term Resilience comes at the opportune moment. As the UK hurtles towards a General Election, the next government’s AI policy will be the cornerstone for economic growth. However, this requires precision in navigating the balance between regulation and innovation, providing guardrails without narrowing the industry’s potential for experimentation. While implementing a centralised incident reporting system for AI misuse and malfunctions would be a laudable first step, there are many more steps to climb.

The incoming UK government should provide certainty and understanding for enterprises with clear governance requirements, while monitoring and mitigating the most likely risks. By integrating a variety of AI governance strategies with centralised incident reporting, the UK can harness the economic potential of AI, ensuring that it benefits society while protecting democratic processes and public trust.”

As AI continues to advance and permeate various aspects of society, the implementation of a robust incident reporting system could prove crucial in mitigating risks and ensuring the safe development and deployment of AI technologies.

See also: SoftBank chief: Forget AGI, ASI will be here within 10 years

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Tags: ai, artificial intelligence, Centre for Long-Term Resilience, cltr, ethics, government, legal, regulation, safety, Society