The public sector in Australia leverages generative artificial intelligence on a large scale. However, it faces challenges due to insufficient infrastructure and security measures to manage this technology effectively. A report released in June by Nutanix and Vanson Bourne revealed that 94% of public sector organisations currently employ GenAI workflows.
However, 92% acknowledge they are not taking sufficient measures to secure their models and applications. The identified gap will manifest throughout Canberra and all state and territory agencies by mid-2025. Public systems encounter challenges with fragmented data infrastructures and outdated platforms that hinder secure data collection, sharing, and cloud storage. Nutanix notes that 76% of organisations acknowledge the need for improvements to their current IT infrastructure to effectively support native cloud applications on a large scale.
Security remains a primary focus. Greg O’Connell, Vice President for Federal Sales at Nutanix, states, “Generative AI is no longer a future concept it’s already transforming how we work. 94 percent of public sector organisations are already putting AI to work and expect returns in as little as one year.” He emphasises that “now is the time to invest in AI‑ready infrastructure data security privacy and training to ensure long‑term success.”
Generative AI risks
The Australian Cyber Security Centre’s (ACSC) latest guidance identifies several structural security challenges in adopting generative AI across the public sector:
1. Provenance and integrity of AI-generated content
The Australian Cyber Security Centre (ACSC) warns about the potential misuse of AI-generated multimedia in official communications. It emphasises the importance of verifying its origin to prevent public deception. A recent paper discusses challenges posed by deepfakes and synthetic text. It highlights their potential to impersonate government entities and spread disinformation. Without embedded content credentials, agencies cannot ensure the authenticity of their published materials. This erodes confidence in public information and diminishes the credibility of digital government services.
2. Insecure data collection and data sharing practices
Many agencies still rely on unreliable or unrecorded methods to gather and share datasets used in AI models. The ACSC emphasises that poor governance practices for data collection and sharing increase the risk of exposing sensitive or personally identifiable information. Using datasets that lack verified lineage or suitable privacy measures can expose sensitive citizen information in model outputs. Departments without uniform data governance frameworks increase the likelihood of compliance violations under the Privacy Act.
3. Cloud storage misconfigurations
The transition of AI operations to cloud environments has led to a notable increase in vulnerabilities from misconfigured storage buckets and inadequate identity access controls. The ACSC highlights that many incidents occur because public-facing cloud instances retain sensitive training data without proper encryption or role-based access controls. Internal agency evaluations highlighted an instance where a model developed using internal documents became accessible through an unsecured cloud API. Misconfigurations leave agencies vulnerable to external threats and compromise their data protection responsibilities.
4. Data silos inhibiting collaboration
AI initiatives depend on strong cross-agency data to develop reliable models. However, many departments still operate independently with outdated systems that obstruct modern data interoperability. These silos hinder collaborative model development. They lead to duplicated efforts, inconsistent outputs, and gaps in the delivery of public services. For example, health, education, and social services might develop distinct models using incomplete datasets and overlook the chance to recognise trends or at-risk groups. Insufficiently secure and efficient data collaboration pipelines undermine service accuracy and hinder effective strategic planning.
5. Low organisational preparedness and skills shortage
The ongoing lack of internal expertise poses a significant barrier to the safe implementation of generative AI technologies. The 2025 ACSC guidance highlights a significant gap within government teams. It notes a deficiency in personnel who can evaluate AI model risks, track data drift, or perform security audits. This indicates that agencies might launch AI pilots without fully understanding the consequences of prompt injection attacks, data poisoning, or model hallucinations. The lack of expertise heightens operational risk and hinders the implementation of cybersecurity best practices throughout public infrastructure.
Check out: “Protecting little lives with big AI”
Government enacts AI policy
The Australian government has implemented the Policy for Responsible AI Use in Government in response to security concerns as of September 1, 2024. This policy mandates agencies to designate responsible officials, release statements on AI transparency, and participate in pilot assurance as outlined in the National Framework for AI Assurance endorsed by Data and Digital Ministers on June 21, 2024.
On January 30, 2025, the Australian Cyber Security Centre partnered with the NSA UK NCSC and CCCS to announce a set of recommendations that aim to incorporate cryptographically verified media provenance into government digital outputs. The January 2025 Strengthening Multimedia Integrity report highlights the importance of awareness about the provenance of digital content.
In 2024, the government tested the Microsoft 365 Copilot with 5,000 employees from 60 agencies to assess data governance, cloud storage security, and potential threats to digital collaboration. Generative AI evolves in digital government, tackling critical aspects like data infrastructure, governance, privacy, and sharing, as well as regulatory challenges surrounding cybersecurity.
AI threatens trust
Generative artificial intelligence serves a significant role in Australia’s digital government, but security flaws threaten public confidence and service integrity. A Mastercard study found that deepfake frauds involving public figures cost Victorians over AU $8 million in 2023 and damaged government agency confidence. The Australian Cybersecurity Centre warns that AI-generated content without credentials may harm government communications. In 2024, agencies testing Microsoft 365 Copilot discovered cloud storage and data sharing issues that could lead to sensitive breaches or misconfigurations.
Research suggests AI may boost efficiency but reduce individuals’ sense of influence over government choices if governance doesn’t change. Declining public trust, rising privacy concerns, and inadequate technological protections threaten digital governance. Australia must enhance its data infrastructure, governance, and cybersecurity to sustain public service confidence as AI becomes more integrated into government processes.
The public sector in Australia faces a pivotal moment in digital governance as generative AI emerges as a crucial element in enhancing service delivery. Many agencies have developed strategies for GenAI, but significant gaps still exist. A recent poll conducted by Nutanix in June 2025 reveals that 83% of organisations plan for GenAI and 54% actively implement it. Seventy-six percent of respondents face challenges with IT infrastructure. Ninety-two percent report inadequate security measures.
The Australian Cybersecurity Center’s May 2025 guideline emphasises the importance of data encryption, content provenance, and secure cloud storage to ensure the integrity of AI models and bolster public confidence. Upgrade cloud-native infrastructure, improve data governance frameworks, and integrate content credentials in all AI outputs as viable solutions. Authorities must prioritise cybersecurity training and close data silos in order to advance sustainable artificial intelligence initiatives. By improving its digital government infrastructure, Australia can maintain public trust in an AI-driven future.
Justin Lavadia is a content producer and editor at Public Spectrum with a diverse writing background spanning various niches and formats. With a wealth of experience, he brings clarity and concise communication to digital content. His expertise lies in crafting engaging content and delivering impactful narratives that resonate with readers.
Latest
Popular