The Australian Digital Transformation Agency (DTA) has completed a comprehensive evaluation of a government-wide trial of generative AI services, specifically Microsoft 365 Copilot, within the Australian Public Service (APS). Conducted from January to June 2024, this trial represents a major advancement in the safe and effective integration of AI tools in public sector operations.
It involved over 7,600 employees across more than 60 government agencies. The insights and recommendations from this evaluation will shape future AI adoption strategies in the government, with a focus on boosting productivity, ensuring data security, and maintaining ethical standards.
Boosting productivity with AI
The trial centred on Microsoft 365 Copilot, equipping APS staff with an integrated AI tool to streamline daily operations. The DTA’s report highlighted that 77% of participants were satisfied with the AI tool, and 86% expressed a desire to continue using it. The trial demonstrated significant productivity improvements, with 69% of users completing tasks faster and 61% reporting enhanced work quality.
Lucy Poole, the General Manager of Strategy, Planning, and Performance at the DTA, underscored the agency’s dedication to fostering innovative solutions in government. She stated, “The DTA has always been clear in its goal to not shy away from innovation. We identified a tool that would quickly integrate into most workplaces with minimal disruption.” Poole noted that participants saved an average of one hour per day on administrative tasks, with 40% of that time being redirected towards skill-building and team engagement activities.
The trial highlighted AI’s ability to improve routine tasks such as note-taking, document preparation, and agenda setting. Advanced features like data summarisation and programming assistance showed potential in transforming job roles, allowing APS staff to explore new applications within their positions. Confidence in using AI was linked to the frequency of training, with 75% of participants who received three or more training sessions feeling confident, compared to 47% of those with minimal training.
Improving data accuracy
Despite the positive outcomes, the evaluation highlighted areas needing improvement, particularly in content accuracy and context sensitivity. Some participants noted the necessity of closely reviewing AI-generated content, which affected perceived efficiency gains. Poole acknowledged these limitations, stating, “as we are testing these tools at such an early stage, there are clear opportunities for tailored solutions to be developed that can handle highly technical material.”
To tackle these challenges, the DTA has advised agencies to adopt custom AI solutions tailored to their specific operational needs and use cases. This strategy aligns with the DTA’s broader vision of providing the APS with tools that enhance adaptive and efficient digital service delivery while minimising disruptions.
Ensuring ethical AI adoption
Alongside the AI tool trial, the DTA piloted an AI Assurance Framework to ensure responsible and ethical AI adoption. Developed under Australia’s AI Ethics Principles, this draft framework provides guidelines for evaluating AI solutions based on fairness, reliability, safety, and data privacy. It aims to help government entities identify and mitigate AI-related risks at all stages of AI deployment.
Poole highlighted the critical need for transparency and accountability in AI usage, stating, “bias can emerge in data when it’s incomplete, unrepresentative, or mirrors societal prejudices. AI models might replicate these biases from their training data, resulting in misleading or unfair outputs.” To address these risks, the framework advocates for a “human-in-the-loop” approach, emphasising the importance of human oversight in decision-making to ensure AI outputs are consistent with public sector values.
Enhancing workplace inclusivity
The trial’s findings highlight the potential of generative AI to enhance workplace accessibility and inclusivity within the APS. Several agencies have shown interest in leveraging AI to support diverse needs, advocating for tools that promote fairness and reduce disparate impacts on specific groups. The DTA’s ongoing AI initiatives include resources for responsible AI use in government, such as training guidelines and transparency policies, aimed at strengthening these capabilities across agencies.
The trial underscores the potential of AI tools to attract and retain talent within the APS, especially in roles that benefit from automating repetitive tasks. Poole concluded, “Keeping with new innovations ensures the APS remains a competitive workplace.” By integrating AI tools that improve work quality and efficiency, the APS aims to create an environment that encourages strategic thinking and cross-functional collaboration.
The DTA’s comprehensive trial of generative AI across the government has provided valuable insights into the advantages and challenges of AI adoption within Australia’s public sector. By balancing innovation with ethical responsibility, the DTA aims to leverage AI’s potential to improve service delivery, streamline operations, and protect data integrity. As the government continues to refine its AI framework and expand digital capabilities, these initiatives position Australia’s public sector to effectively achieve future digital transformation goals, ensuring a secure, transparent, and inclusive digital environment for all Australians.