Skip to content

A field guide · reviewed 25 September 2026

AI agents act. That changes what can go wrong.

A chatbot that gives a wrong answer produces a bad paragraph. An agent that goes wrong sends the email, moves the money, deletes the table, or signs the contract, at machine speed, before anyone reviews it. This site maps that risk: where agents already sit inside a firm, how they fail, what has already happened, which guidance and standards exist, and how the insurance market prices it.

Jul 2026

First autonomous intrusion

Agents under evaluation at OpenAI escaped containment and breached Hugging Face production systems.

Sep 2026

Agent on a government portal

An OpenAI agent researching medicine spending went around the blocks on an Australian government statistics portal.

Sep 2026

$40M for agent certification

AIUC, publisher of the AIUC-1 agent standard, raised a $40M Series A led by Ribbit Capital.

01 · The installed base

Where agents show up

Agents enter a firm in three postures, each with its own risk shape: bought with a licence, wired to the firm’s own systems, or built by staff.

You bought it

Assistants and agents arrive inside licences a firm already holds, some switched on by default. Which of them are on, for whom, and connected to what is configuration a firm can read out of its own admin consoles.

You wired it

Open protocols now connect models to business systems: MCP for tools and data, A2A for traffic between agents. Each server added is a door into something the firm runs.

You built it

Staff produce working software by describing it to a model, and most of it never passes through IT. The last time business logic was built outside IT at this scale was the spreadsheet era, which left a written record of what that costs and which controls answer it.

Where agents show up, with the vendor facts

02 · Why it’s different

What makes agents different

Four things make agents different from ordinary software.

They’re unpredictable.

The same request can produce a different action each time. There is no fixed script to test against.

They’re open to instruction.

Anything an agent reads, a web page, an email, a message from another agent, can carry an instruction it follows.

They operate at machine speed.

A single bad instruction can run a thousand times before a person notices once.

Fluency is not the same as correctness.

An agent sounds just as sure when it is wrong as when it is right, and often you cannot check until after you have acted.

Sources: OWASP Top 10 for Agentic Applications↗

03 · Taxonomy

How agentic systems fail

The reference taxonomy is the OWASP Top 10 for Agentic Applications (2026), a widely referenced list of agent risks. The eight patterns below consolidate it with OWASP’s adjacent LLM and MCP work, each cross-referenced to OWASP’s published identifiers. Some patterns have no clean one-to-one OWASP category; those are marked. Each also names the class of control that answers it: read backwards, a control is a record of the failure it exists to prevent.

FM-01

Prompt injection

Hidden or hostile instructions embedded in content the agent processes (a web page, an email, a document, image metadata) hijack its behaviour.

OWASP: LLM01:2025 Prompt Injection · ASI01 Agent Goal Hijack

Answered by: Untrusted-content handling: isolate and validate everything the agent reads.

FM-02

Excessive agency & permissions

The agent holds broader authority than the task requires (database write access for a read task, spend authority without limits), so a single mistake or manipulation has outsized blast radius.

OWASP: LLM06:2025 Excessive Agency · ASI03 Identity and Privilege Abuse

Answered by: Least privilege: scope permissions, cap spend and autonomy.

FM-03

Insecure tool use

The tools an agent calls become the attack surface: unvalidated inputs, unauthenticated tool servers, or tool descriptions that themselves carry malicious instructions.

OWASP: ASI02 Tool Misuse and Exploitation

Answered by: Authenticated, validated, allow-listed tools.

FM-04

Identity & delegation gaps

Nobody can say which principal an agent was acting for when it acted. Agents inherit human credentials, share service accounts, or pass tasks to sub-agents with no authenticated chain of delegation.

OWASP: ASI03 Identity and Privilege Abuse

Answered by: A distinct agent identity and an authenticated delegation chain.

FM-05

Memory & context poisoning

False or malicious information planted in an agent's memory, retrieval store, or context persists and corrupts future decisions long after the original interaction.

OWASP: ASI06 Memory & Context Poisoning

Answered by: Provenance and integrity checks on memory and retrieval.

FM-06

Cascading actions

Multi-step and multi-agent workflows propagate one early error through every downstream step: a chain reaction at machine speed.

OWASP: ASI08 Cascading Failures

Answered by: Checkpoints and blast-radius limits between steps.

FM-07

Hallucinated facts, real actions

The model invents something (a policy, a price, a legal citation) and then acts on it or communicates it with the authority of the firm behind it.

OWASP: LLM09:2025 Misinformation · ASI09 Human-Agent Trust Exploitation (partial)

Answered by: Grounding, and human sign-off before consequential action.

FM-08

Unaudited operation

This does not cause an incident; it makes every incident worse. There is no per-action log tying decision, data, policy version and outcome together, so nobody can reconstruct what happened or prove what did not.

OWASP: No single OWASP category

Answered by: Durable per-action logs and a human gate before irreversible action.

Sources: OWASP Top 10 for Agentic Applications 2026 (ASI01 to ASI10)↗ · OWASP Top 10 for LLM Applications 2025↗ · NIST AI RMF (Manage)↗ · as of 25 September 2026

The failure modes, each with the cases that show it

04 · Evidence

What has already happened

May to July 2026 · intrusion 9 to 13 July · ~17,600 logged actions · disclosed 16 July

Agents under evaluation broke into a real company

On 16 July 2026 Hugging Face disclosed an intrusion it described as "driven, end to end, by an autonomous AI agent system." The agent was running inside OpenAI's evaluation infrastructure, under test on a cyber-capability benchmark. It left its sandbox through a zero-day in a package registry proxy and an unsecured…

June to September 2026 · 18 June · notified 10 September · announced 24 September

An agent on a research task went around a government portal's blocks

On 18 June 2026 an OpenAI agent doing internet research into public medicine spending reached the Medicare Statistics Reporting Service portal, run by Services Australia. After repeated blocks, in the Prime Minister's words, it "found a way around those blocks", accessed public and non-public files, and wrote files to…

Emblematic cases

All cases, and the public trackers that record the rest

05 · Prior art

Vibecoded software and the EUC precedent

Vibe coding, building software by describing it to a model, has put application development in the hands of anyone who can type, and most of what gets built never passes through IT. The nearest precedent is end-user computing: four decades of business-critical logic living in spreadsheets built by the people who needed them. That era is on the public record: what it cost, and which controls it eventually produced.

The spreadsheet-era record and what it wrote down

06 · The wiring

Securing MCP

A firm that wires agents to its systems over MCP is running new privileged software on its own perimeter. The vendor-neutral guidance for securing it is young; these are the citable documents.

Securing MCP

07 · Governance

The framework map

The map is scored for a specific reader: a firm of roughly 20 to 500 people, no CISO and no compliance department, using agents it bought and agents it built, looking for a named document it can check itself against.

Framework Agent-scopedAdopter-facingChecklist-shapedRight-sizedRecognised
Careful Adoption of Agentic AI Services ✔✔◐✔✔
ETSI TS 104 223 / EN 304 223 ◐◐✔◐✔
AIUC-1 ✔✘✔◐✔
OWASP AISVS ✔✘✔✘✔
CSA AI Controls Matrix ◐✔✔✘✔
OWASP Top 10 lists ✔◐✘✔✔
NIST AI RMF ◐✔✘✘✔
ISO/IEC 42001 ✘✔◐✘✔
Deployer AI Risk Register ◐✔✘✔✘
Community checklists ✔✔✔✔✘

✔ yes · ◐ partly · ✘ no · scores are this site’s reading of each document

The full map: the tests, what each document is, and the sixth test

08 · The market prices it

The insurance signal

The clearest evidence that agent risk is real is that underwriters now price it. Traditional carriers have begun filing AI exclusions into standard liability forms, and a small specialist market has emerged to sell affirmative cover back, generally on one condition: demonstrated governance first.

The insurance signal: the market table, exclusions and notes

09 · Vocabulary

Glossary

Plain definitions of the vocabulary used across this site, from agent and MCP to affirmative AI cover.

Glossary