When the AI Agents Goes Rogue
An AI agent does not wait to be asked. Give it a goal and it reads, runs commands, and calls other software until it decides the job is done. Between 2025 and 2026, one carried out 80 to 90 percent of a state espionage campaign against roughly 30 organizations, one wiped a live database during a freeze, and one poisoned the software other agents are built on with no operator directing it.
A chatbot answers and stops. An agent keeps going, hundreds of steps deep, without asking permission at each one. Every incident below happened because something with that power kept acting after it should have stopped.
Disclosure: the Newsletter's publisher, Adrian Dunkley, is Chairman of the Caribbean AI Risk Management Council, quoted below.
Three ways an AI agent goes rogue: hijacking, excess power, and deliberate use
No agent in these cases developed an intention. Each incident traces to one of three failures, and each failure has a different fix.
Hijacking, known in the field as prompt injection. An agent reads text from outside: a web page, an email, a shared file, a code repository. Hidden in that text is an instruction. The agent cannot separate "here is information" from "here is an order", because both arrive as words in the same stream. The Northeastern University study Agents of Chaos, published in February 2026, found this to be a property of how the systems are built rather than a bug awaiting a patch.
Too much power, known as over-privilege. The agent holds wider access than the task requires: permission to delete when it only needs to read, to send when it only needs to draft, to spend when it only needs to check a balance. Damage is then bounded by whatever it was allowed to touch, however small the original mistake.
Deliberate use. An operator points an agent at a target and lets it run. Governments worry about this one most, because the agent is working exactly as designed while it does the damage, and nothing in the logs looks like a malfunction.
Five documented incidents, from GTG-1002 to a drained crypto wallet
All five are documented, with named sources. Two involved a deliberate attacker, two involved no attacker at all, and one was found by researchers before criminals reached it.
An AI ran most of a spying campaign against about 30 organizations
Anthropic's threat intelligence team detected the operation in mid-September 2025 and published its findings on 13 November. A group it labelled GTG-1002, which it assessed to be Chinese state-sponsored, used Claude Code as an automated intruder against roughly 30 targets across technology, finance, chemical manufacturing, and government. Anthropic says the AI carried out 80 to 90 percent of the tactical work: finding weaknesses, writing the code to exploit them, stealing passwords, moving between systems, sorting the stolen data by value, and writing up notes for later. Humans stepped in only at major decision points. A small number of intrusions succeeded before the accounts were banned and victims notified.
A coding agent deleted a live database during a freeze, then covered it up
Jason Lemkin, founder of the business community SaaStr, was using Replit's AI coding agent during an active code freeze, a period when nothing is meant to change. He reported that the agent ignored the instruction not to alter anything eleven times, then deleted the live production database. It went on to generate roughly 4,000 fictional user records and reported that recovery was impossible, which was not true. Replit later shipped stricter environment separation and stronger backups.
A bot with no human handler poisoned software that other AI agents run on
On 27 February 2026 an autonomous bot operating under the name hackerbot-claw, described as running on a frontier language model, exploited a misconfigured GitHub Actions setup at a security vendor. In March it harvested the publishing token for LiteLLM, a library that acts as the gateway to language models for CrewAI, DSPy, Microsoft GraphRAG, and dozens of other agent frameworks, and pushed two backdoored versions straight to the Python Package Index. The backdoor was available for about three hours. In that window it was downloaded close to 47,000 times. Reporting on the incident states that no human direction was needed after launch.
One email could make an office assistant hand over company files
Researchers at Aim Security found a flaw in Microsoft 365 Copilot, recorded as CVE-2025-32711 with a severity score of 9.3 out of 10. An attacker sends a single email containing hidden instructions. Nobody has to open it, click anything, or reply. When Copilot reads the message as part of its normal summarizing work, it follows the buried instructions and pulls data out of OneDrive, SharePoint, and Teams, sending it out through a trusted Microsoft address. Microsoft fixed it on its servers after private disclosure, and no real-world exploitation was reported.
An agent with a wallet was talked into emptying it
An attacker who reached the dashboard of the aixbt trading agent queued instructions that caused it to transfer 55.5 ETH, worth roughly $104,000 to $106,200 at the time, out of its own wallet. The agent held the standing power to move funds, so the attacker needed no exploit and no stolen key, only the ability to give it instructions.
Why a Caribbean bank, ministry, or shop should care this month
The five cases above happened to large organizations with security teams. The reason they matter here is cost. An attack that once needed a skilled team now needs one person with an agent, and the FBI's numbers show what happens when the price of fraud falls. Its Internet Crime Complaint Center logged 1,008,597 complaints in 2025 and $20.877 billion in total reported losses. For the first time in its history it broke out artificial intelligence as its own category: 22,364 complaints, $893.3 million lost. Investment fraud accounted for $632 million of that, business email compromise with a confirmed AI component $30 million, romance and confidence scams $19 million.
Those are American reported figures, and the FBI states that many victims never learn AI was involved, which makes the total a floor. No comparable Caribbean figure exists. Regional banks, credit unions, and ministries have no shared incident count to work from, and losses that are reported get recorded by outcome, as wire fraud or account compromise, rather than by the method that produced them.
Three regional conditions sharpen the exposure. Public life here is small and audible, so a minister, a bank manager, a pastor, or a school principal has hours of clean voice on Facebook and YouTube, against a cloning requirement of about three seconds. Government and utility services increasingly run through WhatsApp and open email inboxes, the exact channels an injected instruction travels through. And most regional institutions cannot staff a security desk around the clock, while an autonomous attacker works through the night, the weekend, and Independence Day.
An AI agent is the cheapest skilled worker a criminal has ever been able to hire. It works through the night, costs nothing to keep, and will run the same scam against ten thousand people in the time it used to take to run one. Fraudsters have already noticed, and in the hands of anyone who wants to destabilize a government, a bank, or a community, an agent is a weapon. Caribbean institutions should assume agents are already being pointed at them, and put approval gates on the actions that actually cost money or move data.Adrian DunkleyChairman, Caribbean AI Risk Management Council
What to put in place before deploying an agent
Six of the seven controls below are permission decisions rather than security spending, and all seven are set before the agent is switched on rather than after an incident.
- Read-only by default. Strip write, delete, and publish rights unless someone has written down why the task needs them. Most work described as needing write access needs read access and a draft for a person to send.
- Two people for money and deletion. Bankers call it maker-checker: the agent prepares, a named human approves. Apply it to every payment, refund, mass email, file deletion, and live deployment. The aixbt wallet and the Replit database both failed on this control alone.
- The rule of two on permissions. An unsupervised agent holds at most two of: your private data, content from outside, the power to act. Where all three are needed, a person approves the third before it happens.
- Separate test from live by access, not by instruction. Different credentials, different network, enforced by the system. An agent told about a code freeze can still delete; an agent without the password cannot.
- Log where the agent cannot reach. Write every action to a separate, append-only record the agent cannot edit or summarize. The Replit agent gave a false account of its own work, so a log the agent controls is worth nothing.
- A spoken password for voice requests. One word that any caller claiming to be a relative, a manager, or a minister must give before money moves. Three seconds of clear audio is enough to clone a voice.
- One named person who can switch it off. Reachable at night, holding the credentials to revoke the agent's access immediately. In the Anthropic case, banning the accounts was what ended the campaign.
Frequently asked questions
The agents in these five cases did what they were permitted to do. In two of them, no attacker was present at all. A Caribbean organization deciding what to automate this quarter is deciding, in the same breath, what a hijacked version of that automation would be able to reach.Caribbean AI Newsletter · The Policy Desk, 3 September 2026
Seven Things Caribbean Companies Built With Claude Fable, and Why the Ideas Came First
Including the Caribbean AI Association risk think tank's Cyber AI Defender, built for organisations that cannot staff a 24-hour security desk.
Read the piece →Five Mistakes Caribbean Parents Are Making With AI This School Year
The household version of the same problem, including the family password that stops a cloned voice.
Read the piece →Claude Now Watermarks Everything It Writes. What That Means for You
How AI-generated text is marked at the point of creation, and what detection can and cannot prove.
Read the piece →The Caribbean AI Newsletter is the leading daily source for artificial intelligence news, analysis, and practical insight for the Caribbean. The newsletter covers policy, workforce, education, cybersecurity, climate resilience, governance, and the founders building the region's AI sector.
