This Global Shield Australia primer details AI-related harms already occurring in Australia and overseas, and the far greater dangers coming in the near future. These demonstrate the clear need for government action to safeguard Australians and build the trust necessary to realise the benefits of AI.
AI is already causing harm, current AI models have…
Accelerated sexual bullying and abuse in schools – sharing of explicit deepfake images of underage Australians has doubled in the past 18 months.
Exposed children to harmful material online – the eSafety Commissioner has warned that AI companions risk exposing children and young people to drug-taking advice, self-harm and suicidal ideas, eating disorders, and highly sexualised content.
Been used for coercive control and domestic abuse – abusive partners have used AI models to legitimise their claims and reinforce their control over victim-survivors.
Violated privacy – Clearview AI has improperly scraped Australian’s biometric information for use in its facial recognition tool.
Driven suicides – a 30 year old Belgian father in 2023 and a Florida teenager in late 2024 took their lives after extended exchanges with chatbots.
Enabled domestic terrorism – a bomber of a Palm Springs fertility clinic used an AI assistant to obtain information on how to make explosives.
Proliferated child sexual abuse material (CSAM) – the FBI has warned that AI models are being used to create CSAM, including from images of actual minors.
Enabled improper biometric surveillance of consumers – the Office of the Australian Information Commissioner found that Bunnings breached Australians’ privacy by collecting sensitive information through a facial recognition system.
Amplified foreign disinformation – Russia ran AI-enhanced disinformation campaigns during recent elections in Australia and Germany and to prop-up friendly regimes in West Africa.
Spoofed senior governmental officials – the FBI has warned of deepfaked voice messages of senior US officials during election campaigns.
Fired workers without allowing review – Amazon used an automated AI evaluation system to dismiss drivers, without allowing for challenge or appeal of the decision.
These are just a few examples of the existing harm that has resulted from inadequately regulated and poorly governed deployments of AI technologies. AI is not just a hypothetical threat in the future. It requires effective policy action now.
Tomorrow’s AI models bring even greater threats, including of…
Rapid bioweapon design and proliferation
Researchers used an AI model to generate 40,000 potential bioweapons in just six hours. OpenAI says that its models can be “red-teamed” into sharing guidance for creating deadly pathogens.
AI-enabled chemical, biological, radiological, and nuclear (CBRN) weapons proliferation and use by rogue actors
Anthropic has warned that its Claude Opus 4 may have the capability to support the development or acquisition of CBRN weapons. Jailbreakers have forced AI models to output instructions for producing chemical weapons.
AI models blackmailing users and seeking to resist shutdown attempts
OpenAI’s o1 model has the capability to seek to avoid shut down in certain contexts. Frontier models have shown deceptive behaviours, including hiding intentions from users and attempting to copy themselves.
Supercharging cyberattack capabilities of non-State actors and State-backed groups
Anthropic has said that its AI models advanced from high school to undergraduate competency in cybersecurity and cyberattack capabilities in one year. Google found threat actors using its Gemini model to support cyberattacks.
Producing and distributing health dis- and mis-information at scale
The University of South Australia identified tools in OpenAI’s public GPT database actively producing health disinformation. Researchers said: “This is not a future risk. It is already possible, and it is already happening.”
Many ofthese ‘future threats’ are only being prevented by voluntary safeguards. These offer no protection against open sourced models, rogue developers, or accidental releases.
Government action is needed to prevent potentially catastrophic outcomes.
What can the government do?
Australia needs an AI Act,with:
1. Appropriate coverage, at least of high-risk AI systems and general purpose AI models;
2. Robust safety testing and standards,aligned with international best practice; and
3. Monitoring and surveillanceto identify harms and enable appropriate responses.
