AI has quickly outgrown the data science team.

The capability that once owned machine learning models and advanced analytics simply can’t own all of AI anymore. Nor should they. The majority of internal AI use cases built today centre on business process automation powered by agents and LLMs, territory that requires domain expertise data science teams don’t have and doesn’t fit centralised ownership. 

With AI now embedded in everyday tools (e.g. CRMs and SaaS products), and low-code agent platforms, less technically skilled users have the opportunity to build their own AI-enabled solutions. For CIOs and CDOs, this is both an opportunity and a risk – the barrier to building AI solutions has dropped, but so has the barrier to building flawed ones. 

The expanding governance gap 

So who owns AI when it’s everywhere? The logical answer, to enable business users to build their own solutions via low/no-code platforms, breaks down at scale. Ungoverned AI solutions, data leakage, erroneous logic, and untraceable outputs become real risks. And because AI agents sit in an awkward middle ground between traditional machine learning, software, and Robotic Process Automation (RPA), they fall through existing governance structures that weren’t designed with autonomous agents in mind. The result is a widening gap where organisations are taking on more AI risk while the existing challenges of AI literacy, skills, and data access remain unsolved. 

For government, this is where it gets serious. If an agent is automating decisions that affect grants, benefits, or penalties, the accountability chain must be correct, explicit, and auditable. The last thing anyone needs is Robodebt proliferated across dozens of use cases. 

Three operating models, three failure modes 

How do we close this expanding gap? Organisations typically reach for one of three operating model approaches based on pre-Gen/Agentic AI world thinking, and each breaks in its own way. 

1. Fully centralised: an AI team or Centre of Excellence owns all AI. In a landscape where AI is everywhere, this creates bottlenecks and scalability issues, and rarely brings the domain expertise to design agents that map to business processes. 
2. Fully decentralised: business units build what they need. Scalable on paper, but this is where shadow AI thrives: no visibility, duplicated solutions, inconsistent practices, wasted spend, and compliance risk. It also assumes data and AI talent embedded in every business unit, which doesn’t exist at scale across large organisations. 
3. A hybrid ‘Hub and Spoke’ model: most organisations land in this ‘middle ground’ by default, not by design – and this is the most insidious. There’s some central oversight, some local autonomy, but no clear lines on who reviews agent logic before it goes live or who’s accountable when something goes wrong. Worse, it can create the illusion of AI governance without the substance of it. 

What a deliberate model looks like and how to implement it 

Not all AI capability requires the same governance response. Data science and traditional machine learning may reasonably remain in a centralised function or hub-and-spoke model. GenAI and agentic AI present a different challenge: they’re being adopted across every function by people without technical backgrounds and applied to decisions that directly affect operations and outcomes. The operating model needs to match the capability. 

A deliberate approach starts with three questions: 

1. What must remain central: risk appetite, approval thresholds, assurance requirements, and the guardrails that are non-negotiable regardless of where AI is built. 
2. What can be safely delegated to domains: workflow design, agent configuration within guardrails, and the application of AI to specific business problems. This is where speed and domain expertise live; centralising it kills both. 
3. How do you maintain line of sight across distributed deployments: which agents exist, what decisions they’re influencing, whether outputs are being monitored, and who is accountable when something goes wrong. 

Most organisations have frameworks they can adapt for the first two. The third, maintaining genuine line of sight, is where almost everyone struggles. 

For government, two additional layers apply. Many departments have multiple agencies underneath them, each with distinct mandates and risk profiles. The governance model needs to be clear on what is set at the enterprise level and what agencies can determine for themselves. Machinery of Government changes are a recurring reality too. Restructures and transfers of function happen regularly, and a governance model needs to be built to survive them. 

Read also: Nothing Artificial About Australian AI Adoption: Business and Government Trends

Central standards are only as effective as the mechanisms that enforce them. A risk framework sitting in a SharePoint folder does nothing on its own. The real implementation challenge is building the feedback loops and monitoring infrastructure that give a central function genuine visibility across domains. That means knowing which agents exist, what decisions they’re influencing, and whether anyone is watching the outputs. 

Without that, a federated model looks deliberate on paper but behaves like an ungoverned decentralised one in practice. This provides a false sense of governance which is the worst position to be in. 

Solving this also enables AI to scale. Organisations that solve the visibility and feedback loop problem reduce risk and create the conditions for a repeatable path from use case ideation through to production – the whole point of hub and spoke, centralised governance, and distributed accountability. A centralised governance function with genuine line of sight across domains becomes the hub that standardises how AI is identified, evaluated, delivered, and operated. Business units become the spokes. They move faster because of the guardrails in place. Reactive governance is a ceiling. Deliberate industrialisation is a growth engine. 

Why this should matter to you: 

This risk is already playing out. 

Consider a service delivery team that has built a workflow to pre-screen grant applications. A Policy Manager used an AI coding assistant to build the decision logic. It ran for six months unreviewed and unmonitored with no unit testing, or observability. You find out when a budget shortfall reveals the agent was approving ineligible applicants due to a missing condition. 

AI agents are almost certainly being built across your department already. What matters is whether you have line of sight into where they are, what they’re doing, and who is accountable when one gets it wrong. 

If you don’t know the answer, the operating model conversation is overdue. 

Nick Koleits

Lead Data & AI Consultant at Mantel |  + postsBio

Nick Koleits is a Lead Data & AI Consultant at Mantel and holds a Master of Data Science from Monash University. He works at the intersection of AI strategy and technical delivery, helping organisations move data and AI use cases from concept to production and modernise the data platforms behind them. His recent work spans agentic systems and enterprise data architecture across Retail, Energy, Education, and Financial Services.