The Need for Post-Market Monitoring and Incident Reporting
Why reform is needed
The nature of advanced AI technology means that it is not possible to fully predict or prepare for its possible harms. Instead, harms and issues will emerge unpredictably from AI being used in untested contexts, specific user interactions, and AI being integrated with other tools. The use of a single foundational AI model across multiple sectors or use cases (such as in finance, health, logistics and defence settings) also creates systemic risk. Potential failure modes or vulnerabilities can rapidly propagate across sectors, each of which might answer to a different regulator.
However, at present, it is largely only developers who have access to early warning signs of issues and thus the ability to accurately assess risk from their products. Buyers, regulators, and insurers must construct their risk assessments based on disjointed reporting, expert analysis, and voluntary disclosures by the major AI labs. This results in risk being mis-priced, uncertainty limiting trust, and uptake being restrained.
Without economy-wide monitoring and reporting of AI incidents, Australia will remain blind to emerging threats and forfeit its potential AI dividend.
