The Policy Challenge
The Australian Government has recognised the need to regulate AI development and deployment in Australia. However, the nature of this technology means that it is not possible to fully predict its real-world consequences nor prevent all of its possible harms, especially for General Purpose AI (GPAI). These harms will emerge unpredictably from use of AI in untested contexts, specific user interactions, and integration with other tools (including other AI systems).
Modern AI systems have repeatedly exhibited unpredicted behaviors. This includes reward hacking (also known as specification gaming), unexpected biases in language and image generation, and manipulation of users. Until and unless the technical science of understanding machine learning models advances considerably, unexpected harms remain a real risk. We must therefore anticipate and plan for unexpected, delayed harms that will only arise after deployment of each AI product.
The Proposed Solution
Given the unpredictable nature of this technology, in addition to strong ex-ante guardrails, Australia’s approach to AI regulation must include robust post-deployment monitoring and reporting mechanisms.
Australia already has rigorous post-market monitoring and reporting requirements for a range of products, including automobiles, airplanes, medical devices and consumer goods (see Box 1). These monitoring, evaluation, reporting, and response mechanisms have been key to keeping Australians safe and building public trust. Similar mechanisms must also be applied to AI systems to ensure they are safe and can be modified or recalled if they give rise to unacceptable adverse incidents.
A rigorous approach to post-deployment AI monitoring mustinclude:
1. Active post-deployment evaluation and surveillance of AI in the Australian market;
2. An adverse incident reporting system (with both mandatory and voluntary components) including mechanisms for sharing information publicly and privately to other AI developers while protecting business-sensitive information;
3. Documentation and dissemination of adverse incidents, as appropriate;
4. Robustfeedback loops to inform future ex-ante testing and guardrails; and
5. The power to take corrective action where necessary, such as to recall faulty, malfunctioning or harmful AI deployments.
Box 1. Practices in other regulated industries in Australia
- General consumer product safety: The Australian Consumer Law empowers the Australian Competition and Consumer Commission (ACCC) to enforce safety standards, ban unsafe products, and oversee and initiate product recalls. For example, in 2018 over four million defective airbags were compulsorily recalled due to their connection with injuries and fatalities as part of the world’s largest automotive recall. Where a separate agency is responsible for safety regulation of specific products they are also able to lead their recall when appropriate (including in relation food, agricultural products, therapeutic goods, and vehicles).
- Vehicle safety: Part 3 of the Road Vehicle Standards Act 2018 allows for recalls where a safety issue arises with a vehicle or component that may cause injury or where the vehicle or component does not comply with the applicable standards. As the Department of Infrastructure, Transport, Regional Development, Communications, Sports and the Arts has noted “Vehicle and component recalls are a critical part of keeping you and the vehicles on our roads safe.”
- Air and transport safety: The Australian Transport Safety Bureau (ATSB) receives mandatory notifications of accidents and serious incidents (“Immediately Reportable Matters”) that affect aircraft safety. This information allows ATSB to enhance transport safety and fulfil its regulatory role under the Transport Safety Investigation Act 2003. In 2024, an ATSB investigation into an Airbus EC130 helicopter accident led to the operator enhancing its safety protocols and practices and modifying and upgrading certain equipment. ATSB also made recommendations to the Civil Aviation Safety Authority to improve helicopter safety, which CASA accepted.
- Medical devices and therapeutic goods: The Therapeutic Goods Administration (TGA) has a robust post-market monitoring and surveillance system for therapeutic goods under the Therapeutic GoodsAct 1989. This involves actively monitoring adverse events and product problems reported by health departments, healthcare professionals, consumers, pharmaceutical companies, and medical device suppliers. The TGA uses this information to identify safety concerns and take regulatory action such as issuing safety alerts, product recalls, or updating product information. During the COVID-19 vaccine rollout, the TGA was a valuable source of information and assurance on the safety of the vaccine and to identify potential side effects, interactions and risk factors for certain users.
