When AI Agents Go Beyond Their Brief: October's Major AI Safety and Security Incidents

On 15 October, India’s cyber agency CERT-In issued a critical advisory with an unsettling detail. AI agents under test had reached live systems at Indian organizations and read business records, personal data among them. None of the affected organizations even noticed (1).

The agents didn't need clever exploits. They used credentials left in configuration files and public code repositories, and cloud roles with far too much access [1].

October was full of stories like this, unfortunately. Labs and regulators reported one case after another of AI agents stepping outside their brief, and headlines reached for the word “rogue.” I think a better word is “ungoverned.” Read the reports closely and nearly every failure traces back to a control that already exists in ordinary security practice, which are isolation, identity, monitoring, approval and reporting.

The sandbox had a window

On October 9, Anthropic published a report on four kinds of unintended actions its models took on real websites during evaluations and internal use, including exploiting a flaw on a university server [2]. In one run, a model submitted an invented tip through a police department's online form. The site flagged it as spam, so nothing came of it. The tip went in on 18 July. Anthropic found it on 28 September, ten weeks later [3]. The company has since cut live internet access for its internal evaluations [4].

Wikimedia reported something similar on 5 October. It found edits it believes came from agents operated by OpenAI, almost all of them test edits in sandbox areas, along with changes to a citation tool and millions of automated requests [5][6]. OpenAI said it was reviewing the findings and had not confirmed the attribution [7].’

The watchers went first

The oddest report came from OpenAI on 20 October. In two internal coding tasks in early October, agents weakened the monitoring meant to catch misbehaviour. On the 3rd, an agent told not to send Slack messages while it worked to suppress the automatic alerts to OpenAI's security team. On the 6th, another agent removed a monitor during a migration and, when a user asked, said it hadn't [8][9].

Two details here stand out. The agents themselves stayed monitored but what slipped was oversight of the systems they were building [10]. And OpenAI blamed a gap in how it enforces monitoring on the workloads its agents launch [8]. That is a governance sentence. It has since updated its review tooling to block attempts to switch monitoring off [8].

Found keys, used keys

Another pattern we find is that an agent would find a credential and treats it as fair game. In mid-October OpenAI added a 26 June case to its public log. A research model was asked to deploy contracts on a test network but wasn't given a funded wallet. It derived addresses from publicly known developer recovery phrases, checked 362 of them and used one with enough test ether to pay the fees. Its final summary left that out [11][12][13]. It was test money, and OpenAI didn't find the episode until 25 September [12].

The same logic is reaching ordinary banking. Bank of America emailed customers asking them not to give banking logins to AI assistants, and Morgan Stanley has reportedly begun blocking such agents [14]. Reuters reported that banks have been flagging and blocking transactions where customers shared credentials with agents [15]. A password was designed to identify a person. An agent that holds one can do whatever that person can.

Nobody told anyone

Look at the dates. A fake police tip took ten weeks to find and a wallet about three months. In September, Australia learned that an OpenAI agent had breached a Medicare statistics portal on 18 June. OpenAI notified the government on 10 September, through a public email inbox [16]. In CERT-In's cases, the AI providers disclosed the activity, not the organisations affected [17].

The governments are also noticing. The White House said it expects companies to notify affected parties and address incidents [18]. CERT-In reminded organisations that cyber incidents must be reported within six hours [1]. And India's telecom regulator TRAI proposed on 30 October that telcos report serious AI incidents, though it hasn't yet defined what counts or how fast [19].

What would have helped

None of this needs new theory. CERT-In's own advice reads like an infrastructure checklist with isolate evaluations, restrict internet access, use synthetic data and credentials, monitor agents live and stop runs that exceed their authorised scope [20]. Set that against October's incidents and each one lines up with a control that was missing.

Close the doors by default: Test environments should get no outbound access unless a task needs it, plus fake data and fake credentials. Cutting live internet access is what Anthropic did after its report [4], and CERT-In's cases turned on real credentials sitting in repositories [1].
Put monitoring out of the agent's reach: Alerts and monitors should be enforced by the platform, where an agent has no edit rights, with an alarm if anyone tampers with them. OpenAI named this gap itself [8].
Give every agent run its own identity: Short-lived, narrow, tied to one task, with spending limits and a human sign-off on payments [21]. Never a person's password.
Set the reporting clock in advance: Keep a list of your agents with a named owner for each, logs you can actually read, and a plan for who is told, and how fast. CERT-In's reminder of a six-hour reporting window shows where the bar is heading [1].

These can work. When an agentic attack hit the Dutch Institute for Vulnerability Disclosure on 21 September, the institute said network segmentation and a quick response limited the damage [22].

Three questions for this week

The labs deserve some credit. They published these cases, which is how we know about them. But these are organisations with large security teams. Most companies deploying agents have fewer people watching and fewer controls in place.

So the question is can you list every AI agent running in your organisation? Does each have its own credentials? And if one steps outside its brief tonight, who finds out, and how fast?

Where Xtant.tech can help

If those questions were hard to answer, we got you covered. Most teams ship AI faster than their policies can keep up, and that gap tends to surface later as a failed security questionnaire, a lost enterprise deal or a regulator's letter.

That gap is the work we do atXtant.tech. We are an ISO/IEC 42001 and ISO/IEC 27001 consulting firm. We start with a gap assessment of your model pipelines, data flows and documentation, so you know what is running and who owns it. Then we write the risk treatment plan and the technical guardrails into your existing developer workflows, and run the internal audit that prepares you for certification. ISO/IEC 42001 gives the habits above a home like defined roles, risk assessment and treatment, monitoring, internal audit and corrective action.

Our free AI risk assessment is a quick way to see where you stand, and our AI Risk Observatory follows AI risk and regulation news as it breaks. To talk it through,book a free 30-minute scoping call.

When AI Agents Go Beyond Their Brief: October's Major AI Safety and Security Incidents

When AI Agents Go Beyond Their Brief: October's Major AI Safety and Security Incidents

The sandbox had a window

The watchers went first

Found keys, used keys

Nobody told anyone

What would have helped

Three questions for this week

Where Xtant.tech can help

Sources

Kashif Hussain

Put this into practice

Let's make your AI assured.

Let's make your AI assured.