Your AI Agents are already working. Do you know what they're doing?

The pace of AI adoption inside organisations has surprised almost everyone.

Two years ago, conversations centred on whether AI tools were worth trialling. Today, the question has shifted dramatically. Employees are building AI agents independently, attaching them to live data systems, and deploying them across their teams, often before any formal governance or risk assessment has taken place.

That shift has created a problem that is urgent, and largely invisible.

"Agents are being created across the business without the controls in place to manage them. Organisations don't know how many agents are in use, who's using them, and don’t have the processes in place to assess their risk." Adrian Buxton, Co-founder & CTO, Increment 

If that statement resonates with you, you're not alone. It's the most common conversation we're having with customers right now.

The scale of the challenge

The numbers paint a clear picture. According to IDC, there will be 1.3 billion AI agents in use by 2028. Microsoft's own data shows the number of active agents in the Microsoft 365 ecosystem has grown 15x year-over-year, and 18x in large enterprises. Meanwhile, research from Microsoft's Work Trend Index found that while 81% of business leaders plan to integrate agents into their AI strategy, only 24% have deployed them organisation-wide in a governed way.

That gap between intent and governance is where risk lives.

The problem isn't that employees are building agents, that's a good thing, and it reflects genuine enthusiasm for productivity gains. The problem is that agents, unlike traditional software tools, can take actions, access sensitive data, and operate continuously, without the guardrails we'd apply to any other system or person with similar levels of access.

When good intentions create real risk

Consider two scenarios applicable to the Australian market.

A coordinator at a school builds a student-facing study and feedback agent, grounding it on the school's learning management system and student records platform. The agent is helpful, popular, and, unknowingly, has access to individual learning plans, counselling notes, and child protection restricted contact flags. Under the teacher's full staff account, the agent can expose that data in any interaction, with no escalation pathway for sensitive disclosures.

In local government, a digital services team builds a resident query agent that quickly becomes popular. Other departments clone and customise it. But the original agent had broad read access to case records containing hardship details and domestic violence safe address flags. With no mechanism to track how many clones exist or what data each can access, the risk surface has quietly multiplied, and nobody knows where it ends.

These aren't hypothetical scenarios. They're the kinds of situations that surface when well-meaning people use powerful tools without the right framework in place.

The most alarming real-world example we've seen involved an AI agent with a connector to an ERP system that hadn't been fully authorised through proper channels. During an email drafting task, the agent queried the ERP and surfaced non-public financial information about upcoming business events. The immediate consequence wasn't a minor data leak; the customer initiated a full systems lockdown because they believed they'd been hacked.

"It's not just the direct implications, you might be leaking data, but what are the additional impacts? Businesses thinking they've been hacked. Having to notify customers. It creates a whole degree of risk in business continuity and operations." Adrian Buxton, Increment 

The four risk vectors to understand

When we assess agent risk with customers, we consistently see exposure across four dimensions:

Visibility: Agents proliferate across departments without a central inventory. IT has no reliable picture of what agents exist, who owns them, or what data they access.

Accountability: Agents are built and deployed without clear ownership. When something goes wrong, there is no defined chain of accountability, making it difficult to remediate compliance issues quickly.

Lifecycle: Agents get copied, cloned, and forgotten. Access persists long after the original purpose of an agent has changed or ceased. There's no equivalent of the offboarding process we apply to employees.

Controls: Agents can exfiltrate data at speed and scale. Without controls on what data they can access, what connectors they can use, and what actions they can take, the blast radius of a single misconfigured agent can be enormous.

Agent Governance starts with Identity Governance

The good news is that we already know how to solve this problem. The frameworks exist. The tools exist. What's required is applying to agents the same disciplined approach we apply to human identities to a new class of digital actor: the AI agent.

When you onboard an employee, you assign them a manager, define their access rights based on the principle of least privilege, conduct periodic access reviews, and offboard them when they leave or change roles.

Agent governance follows the same pattern.

Every agent needs an owner. Every agent needs a defined purpose. Every agent needs a permission boundary. Every agent needs to be reviewed, monitored, and eventually retired when it is no longer required.

The difference is that agents are not passive identities. They can reason across data, invoke tools, trigger workflows, and take actions in business systems. That means Agent Governance cannot stop at access control. It must extend identity governance with controls for autonomy, tool use, human approval, observability, auditability, and safe shutdown.

"When we talk about agents like users, agents need a manager or an owner. Managing the agent life cycle works in a very similar way to the human identity life cycle." Ryan Pool, Head of Information Security & Governance, Increment

This is exactly the model Microsoft has embedded into Agent 365, and it's why organisations with mature identity governance practices are best placed to extend those practices to their agent fleet.

What Agent 365 actually does

Agent 365 is Microsoft's control plane for AI agents. It brings together the security and governance capabilities across Microsoft Entra, Microsoft Purview, and Microsoft Defender, and extends them specifically to AI agents, whether those agents were built by Microsoft, created in Copilot Studio, or deployed from third-party platforms.

At its core, Agent 365 provides five things:

Registry: A single, searchable inventory of every agent in your environment, including who owns it, what data it accesses, and any high-risk signals associated with it.

Access Control: Policy-based management of what agents can do, including conditional access rules, access packages, and the ability to automatically reassign ownerless agents to managers.

Visualisation: A dynamic, real-time view of how agents interact with each other and with your data, making it possible to spot unusual usage patterns and respond quickly.

Interoperability: The ability to govern agents regardless of where they were built, including partner-built agents from vendors like Adobe, ServiceNow, SAP, and Databricks.

Security: Integration with Microsoft Purview for DLP and sensitivity labelling, Defender for threat detection and endpoint control, and Entra for identity protection and zero trust for AI.

The result is a single place to observe, govern, and secure your entire agent estate, without having to navigate across multiple portals or tools.

Getting started

Agent 365 is a powerful capability, but it requires the right foundation to deliver its full value. That means having:

• A clear AI policy covering how agents may be deployed and used

• Defined ownership and accountability roles for agents across the business

• An AI governance framework that includes risk assessment, model selection criteria, and board-level reporting

• The prerequisite Microsoft licences, including M365 E5 (or equivalent) and, for full identity governance, Microsoft Entra ID Governance or the M365 E7 bundle

The organisations that will navigate this transition most effectively are those that treat agent governance as a business discipline, not just a technical configuration.

If you'd like to understand what that looks like for your organisation, Increment's Getting Agent Ready engagement is designed to help you assess your current maturity, see Agent 365 in action, and leave you with a prioritised roadmap tailored to your environment.

Contact Us