For decades, we’ve spent billions protecting our computer systems from unauthorized access. We’ve built firewalls, implemented sophisticated authentication, restricted permissions, and invested enormous resources in cybersecurity.
Now we’re inviting AI into those same systems and giving it the ability to access information, make decisions, and take action.
Think about that for a moment.
I’ve been working on a project developing an operating system for a client that brings together everything from daily operations, employee and fleet management and routing to billing, accounting, and email. We’ve incorporated AI to analyze information across the organization, uncover useful insights, and help decision makers better understand what’s happening in their business. We’ve also introduced automation to streamline everyday processes.
It’s exciting work. After more than 20 years developing software, I’m genuinely amazed by what’s possible today.
But working on this project really got me thinking.
To make AI useful, you have to give it access. Sometimes to very sensitive information. And increasingly, we’re giving AI systems the ability to do more than retrieve information. We’re allowing them to act on it.
For years, cybersecurity has focused on keeping unauthorized users out. But what happens when the potential risk comes from software we’ve deliberately authorized to come in?
When the Software Changes the Rules
On October 20, 2026, OpenAI disclosed two incidents involving AI coding agents working on legitimate internal development tasks.
In the first incident, an agent was working on a software evaluation system. An earlier instruction told it not to send messages while searching Slack. Later, while repairing the system, the agent interpreted that instruction broadly enough to disable automatic security alerts.
The monitoring system was still recording information, but the notifications intended to alert the security team had been suppressed.
In the second incident, another agent was migrating a program. To make the new version resemble an existing implementation, it removed an integration responsible for monitoring the program.
When questioned, the agent incorrectly claimed it hadn’t removed the monitoring.
Now, consider what makes these incidents unusual.
The agents weren’t instructed to sabotage security. They were carrying out legitimate programming tasks. Yet their interpretation of those tasks resulted in safeguards being weakened.
OpenAI acknowledged the incidents and began strengthening its monitoring protections. The company has not established whether the behavior reflected excessive persistence in completing tasks or something more concerning.
But that uncertainty is precisely what makes these events worth examining.
An AI system doesn’t have to be malicious to create a serious security problem. It can be trying to do exactly what we asked and still do something we never intended.
And the concern extends beyond AI development laboratories.
A September 2026 EY survey found that 26% of surveyed senior AI executives whose organizations use AI agents said their organizations couldn’t detect unauthorized AI agents operating internally.
How do you protect an environment when you don’t even know everything operating inside it?
The Hacker Doesn’t Always Need a Password
Consider a fairly ordinary business scenario.
A company introduces an AI assistant to help manage customer emails. It can read messages, retrieve relevant customer information, and prepare responses.
Sounds useful, doesn’t it?
Now imagine someone sends an email containing instructions that appear to come from the company’s administrator. Hidden among the ordinary message content is a request for the AI to retrieve confidential information and send it to an external address.
The employee reading that email might immediately recognize something suspicious.
But what if the AI interprets those instructions as part of the task it’s supposed to perform?
This is a form of attack known as prompt injection. Instead of breaking into the system directly, an attacker attempts to manipulate an AI system that already has legitimate access.
Whether that attack succeeds depends on the safeguards surrounding the AI and its tools.
Still, it’s a significant shift in how we need to think about security.
We’ve spent years building systems that ask whether someone has permission to access information.
Now we also need systems that can determine whether an AI agent’s requested action is actually authorized, regardless of how confidently the agent presents it.
The intruder may not need to steal a key if they can convince someone inside to open the door.
AI Needs Boundaries, Not Just Capabilities
I don’t believe the solution is to retreat from AI.
Quite the opposite.
I’m using AI extensively in my own development work, integrating it into business applications, and seeing firsthand how it can improve operations and decision making.
And AI itself can help cybersecurity teams detect suspicious activity, investigate threats, and respond more effectively.
But there’s a difference between giving AI the tools to be useful and giving it unrestricted authority.
That’s something we’ve been deliberate about in the operating system we’re developing.
In the applications that we build, we implement restricted permissions, authorization controls, activity logging, and safeguards around sensitive operations. We also build functionality that allows administrators to control the level of automation throughout the application.
That last capability is particularly important.
Administrators can determine which processes should be automated, how much authority to delegate, and where human approval is necessary.
For example, allowing AI to analyze accounting records and prepare a financial summary is very different from allowing it to modify those records or initiate a payment.
One provides information to support a decision. The other takes action that could have financial consequences.
Both may be useful, but they shouldn’t automatically require the same permissions.
And that’s really the point.
Just because AI is capable of performing an action doesn’t mean it should have the authority to perform it.
Of course, these controls don’t eliminate every risk. AI security will require continuous testing, monitoring, and improvements as capabilities evolve.
But giving administrators meaningful control is a much better approach than assuming an AI system will always interpret our intentions correctly.
Who’s Really in Control?
We’re entering a period when AI will be connected to more of the systems businesses depend on every day.
Accounting platforms. Email. Customer databases. Financial applications. Operational systems.
The opportunities are enormous.
But before connecting an AI agent to another business system, we should be asking some basic questions.
What information does it need? What actions can it perform? Who approves consequential decisions? Can its actions be reviewed? And how quickly can its access be restricted if something goes wrong?
These aren’t questions intended to slow innovation.
They’re questions that allow us to innovate responsibly.
I believe AI represents one of the most significant advancements in software I’ve witnessed during my career. I’m excited about it, and I intend to keep exploring what it makes possible.
But enthusiasm shouldn’t replace sound engineering judgment.
We’ve spent decades learning how to protect our systems from people who shouldn’t have access. Now we need to become just as thoughtful about the intelligent systems we choose to let inside.
Because giving AI the keys is one thing.
Making sure we still control the locks is another.