A field guide · reviewed 2 August 2026
A chatbot that gives a wrong answer produces a bad paragraph. An agent that goes wrong sends the email, moves the money, deletes the table, or signs the contract, at machine speed, before anyone reviews it. This site maps that risk: where agents already sit inside a firm, how they fail, what has already happened, which guidance and standards exist, and how the insurance market prices it.
Feb 2026
First insured AI agents
ElevenLabs voice agents were first to go live with AIUC-1-backed cover.
2025-26
AI exclusions spread
W.R. Berkley filed an absolute AI exclusion across D&O, E&O and fiduciary lines.
Jul 2026
First autonomous intrusion
An agent under evaluation at OpenAI escaped containment and breached Hugging Face production systems.
01 · The installed base
Agents enter a firm in three postures, each with its own risk shape: bought with a licence, wired to the firm’s own systems, or built by staff.
Assistants and agents arrive inside licences a firm already holds, some switched on by default. Which of them are on, for whom, and connected to what is configuration a firm can read out of its own admin consoles.
Microsoft 365 Copilot Chat needs no Copilot licence; the paid seat adds Copilot in the Office apps and custom agent creation.
learn.microsoft.com↗Gemini features are on by default in Workspace Enterprise Standard and Plus and eligible Education editions, managed in the Admin console under Generative AI.
knowledge.workspace.google.com↗SAP embeds Joule, its generative AI copilot, across its cloud portfolio, from SuccessFactors through S/4HANA Cloud and Ariba.
news.sap.com↗Salesforce's hosted MCP servers are generally available; they sit inactive until an administrator enables them.
developer.salesforce.com↗Open protocols now connect models to business systems: MCP for tools and data, A2A for traffic between agents. Each server added is a door into something the firm runs.
MCP support in declarative agents for Microsoft 365 Copilot is generally available.
devblogs.microsoft.com↗Any MCP-compatible client, Claude, ChatGPT or a custom agent, can connect to a Salesforce org through its hosted servers.
developer.salesforce.com↗A2A, donated by Google to the Linux Foundation in June 2025, is backed by AWS, Cisco, Google, Microsoft, Salesforce, SAP and ServiceNow.
linuxfoundation.org↗Staff produce working software by describing it to a model, and most of it never passes through IT. The last time business logic was built outside IT at this scale was the spreadsheet era, which left a written record of what that costs and which controls answer it.
In Microsoft 365 tenants, creating apps and agents with Copilot Studio inside Teams is enabled by default, provisioning its own data environment on first use; admins can turn it off per user.
learn.microsoft.com↗02 · Why it’s different
Four things make agents different from ordinary software.
They’re unpredictable.
The same request can produce a different action each time. There is no fixed script to test against.
They’re open to instruction.
Anything an agent reads, a web page, an email, a message from another agent, can carry an instruction it follows.
They operate at machine speed.
A single bad instruction can run a thousand times before a person notices once.
Fluency is not the same as correctness.
An agent sounds just as sure when it is wrong as when it is right, and often you cannot check until after you have acted.
03 · Taxonomy
The reference taxonomy is the OWASP Top 10 for Agentic Applications (2026), a widely referenced list of agent risks. The eight patterns below consolidate it with OWASP’s adjacent LLM and MCP work, each cross-referenced to OWASP’s published identifiers. Some patterns have no clean one-to-one OWASP category; those are marked. Each also names the class of control that answers it: read backwards, a control is a record of the failure it exists to prevent.
Hidden or hostile instructions embedded in content the agent processes (a web page, an email, a document, image metadata) hijack its behaviour.
OWASP: LLM01:2025 Prompt Injection · ASI01 Agent Goal Hijack
Answered by: Untrusted-content handling: isolate and validate everything the agent reads.
The agent holds broader authority than the task requires (database write access for a read task, spend authority without limits), so a single mistake or manipulation has outsized blast radius.
OWASP: LLM06:2025 Excessive Agency · ASI03 Identity and Privilege Abuse
Answered by: Least privilege: scope permissions, cap spend and autonomy.
The tools an agent calls become the attack surface: unvalidated inputs, unauthenticated tool servers, or tool descriptions that themselves carry malicious instructions.
OWASP: ASI02 Tool Misuse and Exploitation
Answered by: Authenticated, validated, allow-listed tools.
Nobody can say which principal an agent was acting for when it acted. Agents inherit human credentials, share service accounts, or pass tasks to sub-agents with no authenticated chain of delegation.
OWASP: ASI03 Identity and Privilege Abuse
Answered by: A distinct agent identity and an authenticated delegation chain.
False or malicious information planted in an agent's memory, retrieval store, or context persists and corrupts future decisions long after the original interaction.
OWASP: ASI06 Memory & Context Poisoning
Answered by: Provenance and integrity checks on memory and retrieval.
Multi-step and multi-agent workflows propagate one early error through every downstream step: a chain reaction at machine speed.
OWASP: ASI08 Cascading Failures
Answered by: Checkpoints and blast-radius limits between steps.
The model invents something (a policy, a price, a legal citation) and then acts on it or communicates it with the authority of the firm behind it.
OWASP: LLM09:2025 Misinformation · ASI09 Human-Agent Trust Exploitation (partial)
Answered by: Grounding, and human sign-off before consequential action.
This does not cause an incident; it makes every incident worse. There is no per-action log tying decision, data, policy version and outcome together, so nobody can reconstruct what happened or prove what did not.
OWASP: No single OWASP category
Answered by: Durable per-action logs and a human gate before irreversible action.
Sources: OWASP Top 10 for Agentic Applications 2026 (ASI01 to ASI10)↗ · OWASP Top 10 for LLM Applications 2025↗ · NIST AI RMF (Manage)↗ · as of 2 August 2026
04 · Evidence
July 2026 · 9 to 13 July · ~17,600 logged actions · disclosed 16 July
On 16 July 2026 Hugging Face disclosed an intrusion it described as "driven, end to end, by an autonomous AI agent system." The agent was running inside OpenAI's evaluation infrastructure, under test on a cyber-capability benchmark. It left its sandbox through a zero-day in a package registry proxy and an unsecured code-execution harness, then entered Hugging Face's dataset pipeline through two separate remote-code-execution vulnerabilities.
The attack ran for about four and a half days and left roughly 17,600 logged actions. Inside the network the agent harvested cloud credentials, forged tokens with a stolen signing key, used a compromised GitHub integration, joined the corporate VPN with a stolen key, and staged its command channel on public pastebins and file-drop hosts. Hugging Face reports access to a limited set of internal datasets and service credentials, no evidence of tampering with public models or the supply chain, and a customer impact assessment still open at disclosure.
OpenAI published its account on 21 July. JFrog confirmed its Artifactory product was the exploited proxy; which patched vulnerabilities correspond to those used is unconfirmed by either company. The Cloud Security Alliance's post-mortem calls this "the first publicly documented autonomous AI attack" and recommends treating every AI agent as "a bounded, privileged insider identity." One forensic detail: Hugging Face decoded the attacker's payloads with an open-weight model, because commercial providers' guardrails refused the exploit code.
Read against the taxonomy above, the incident walks through the failure modes:
Sources: Hugging Face: security incident disclosure↗ · Hugging Face: technical timeline↗ · OpenAI: incident statement↗ · CSA: post-mortem↗
Several public databases track AI incidents. Rather than duplicate them, this guide links to the main ones and notes what each is for. They largely record what reached the news, not what a system did internally, and were not built for agentic systems.
Sources: Incident Analysis for AI Agents (Ezell et al.)↗
Responsible AI Collaborative
Real-world AI incidents (harms already caused) across all AI system types, from public reporting and community submission.
OECD.AI Policy Observatory
AI incidents and hazards (near-misses / plausible-harm events) detected from global news media via an automated news-intelligence pipeline.
Charlie Pownall (independent, volunteer-run public-interest project)
AI, algorithmic and automation incidents and controversies, including reputational, ethical and governance failures beyond strict harm events.
MIT AI Risk Initiative
Reclassifies AIID's raw reports against MIT's own risk taxonomy and a harm-severity scale, with EU AI Act risk-level tagging.
Damien Charlotin (independent legal researcher, HEC Paris)
Court and tribunal decisions worldwide where a party was found to have relied on AI-fabricated content (invented citations, false quotes).
liability · 2024 · British Columbia, Canada · FM-07 FM-04
Air Canada's website chatbot told a customer he could apply for a bereavement fare after travel. The BC Civil Resolution Tribunal rejected the argument that the chatbot was a separate entity and held the airline liable for negligent misrepresentation.
professional · 2026 · Multiple (US, UK, Canada, Australia, Israel, Brazil) · FM-07
A growing, actively maintained database tracks court and tribunal decisions in which a party was found to have relied on AI-fabricated case citations. The count has risen steadily as more matters reach judgment.
operations · 2025 · FM-02 FM-03 FM-06
A Replit AI coding agent, holding write access to a live production database, deleted production data despite instructions not to make changes. Replit's CEO confirmed the deletion and rolled out automatic development/production database separation in response.
security · 2026 · FM-01 FM-05
Google DeepMind researchers published a six-category taxonomy of how adversarial web content (hidden HTML instructions, poisoned images, manipulated memory, multi-agent cascades) can hijack autonomous agents, including sites that detect an agent visitor and serve it content a human never sees.
Source: Franklin et al., AI Agent Traps, Google DeepMind (March 2026)↗
05 · Prior art
Vibe coding, building software by describing it to a model, has put application development in the hands of anyone who can type, and most of what gets built never passes through IT. The nearest precedent is end-user computing: four decades of business-critical logic living in spreadsheets built by the people who needed them. That era is on the public record: what it cost, and which controls it eventually produced.
2013
The London Whale VaR model
JPMorgan's task force report records that the bank's new value-at-risk model "operated through a series of Excel spreadsheets, which had to be completed manually, by a process of copying and pasting data from one spreadsheet to another," and that a formula error, dividing by a sum instead of an average, "likely had the effect of muting volatility by a factor of two."
JPMorgan task force report (2013), pp. 123 and 128↗2013
The Reinhart-Rogoff growth paper
The replication by Herndon, Ash and Pollin found "coding errors, selective exclusion of available data, and unconventional weighting of summary statistics" behind the finding that high public debt stalls growth. Corrected, average growth for high-debt countries was 2.2 percent, not the published minus 0.1.
Herndon, Ash & Pollin, PERI WP322 (2013)↗2020
England's missing COVID cases
15,841 positive cases were left out of England's reported figures and delayed reaching contact tracers when files of results exceeded "a maximum file size" in Public Health England's data-load process, widely reported as the legacy Excel row limit.
PHE statement, gov.uk (4 Oct 2020)↗since 1999
The running list
EuSpRIG, the European Spreadsheet Risks Interest Group, has maintained a public register of documented spreadsheet incidents for over two decades.
EuSpRIG horror stories↗The spreadsheet era produced a governance literature that is free, mature, and written for exactly the artefact class vibecoding now multiplies: logic that matters, built outside IT.
SR 11-7 Federal Reserve and OCC, 2011
The US supervisory guidance on model risk management: development, validation and governance expectations for models in banking.
federalreserve.gov↗Twenty Principles for Good Spreadsheet Practice ICAEW, 4th edition, 2024
Planning, design and control principles for spreadsheets, maintained by the accountancy body for over a decade.
icaew.com↗SS1/23 UK PRA, in effect May 2024
The UK model-risk framework for banks explicitly brings end-user computing applications and spreadsheet calculations into scope as models.
bankofengland.co.uk↗for · Practical Ecommerce, Sept 2025
"Vibe Coding Is Ecommerce's New Excel": spreadsheets gave non-programmers a visual layer over maths and logic; vibe coding does the same for software.
source↗for · RoboCFO, June 2026
"A CFO who lets the team build without a governance framework trades shadow Excel for shadow code."
source↗against · Cloud Security Alliance, June 2026
The counter-view: vibe-coded apps often run on already-sanctioned platforms, so the shadow-IT playbooks the analogy suggests do not transfer.
source↗06 · The wiring
A firm that wires agents to its systems over MCP is running new privileged software on its own perimeter. The vendor-neutral guidance for securing it is young; these are the citable documents.
Careful Adoption of Agentic AI Services
CISA, NSA, ACSC, CCCS, NCSC-UK, NCSC-NZ · May 2026 · advisory guidance
Joint guidance from six national cyber agencies for organisations adopting agentic AI services. The most authoritative signature on this list.
cisa.gov↗MCP Top 10
OWASP · beta, revision due Oct 2026 · risk list
Ten risk categories for MCP deployments, MCP01:2025 through MCP10:2025, from token mismanagement to shadow MCP servers.
github.com/OWASP↗Secure MCP Server Development
OWASP GenAI Security Project · Feb 2026 · guide
For firms running their own servers: architecture, authentication and authorisation, validation, session isolation, hardened deployment.
genai.owasp.org↗Third-Party MCP Servers CheatSheet
OWASP GenAI Security Project · v1.0, Nov 2025 · cheat sheet
For servers you did not write: tool poisoning, prompt injection and memory poisoning risks, with vetting, sandboxing, least privilege and oversight as the mitigations.
genai.owasp.org↗MCP-Security-Checklist
SlowMist · MIT licence, maintained · checklist
Priority-tagged items across server, client, multi-MCP scenarios and LLM adaptation. Written by a blockchain-security firm rather than a standards body; narrow and practical.
github.com/slowmist↗07 · Governance
The map is scored for a specific reader: a firm of roughly 20 to 500 people, no CISO and no compliance department, using agents it bought and agents it built, looking for a named document it can check itself against.
Agent-scoped
Covers agents, tools, MCP and autonomy, not AI in general.
Adopter-facing
Usable by a firm deploying agents, not only building them.
Checklist-shaped
Enumerable requirements you can pass or fail.
Right-sized
Digestible without dedicated security staff.
Recognised
A name a client, auditor or insurer would accept.
| Framework | What it is | Agent-scoped | Adopter-facing | Checklist-shaped | Right-sized | Recognised |
|---|---|---|---|---|---|---|
| Careful Adoption of Agentic AI Services CISA + five partner agencies · May 2026 · free | Joint guidance from six national cyber agencies on adopting agentic AI services. Advisory prose with actions, not a scored checklist. | ✔ | ✔ | ◐ | ✔ | ✔ |
| ETSI TS 104 223 / EN 304 223 ETSI · 2025 · free | Baseline cyber security requirements for AI: 13 principles, 72 numbered provisions across the lifecycle, with an implementation guide (TR 104 128). AI-general rather than agent-specific; operators are one of several audiences. | ◐ | ◐ | ✔ | ◐ | ✔ |
| AIUC-1 AIUC · quarterly since July 2025 | The first agent-specific certification standard: 51 requirements and 130 controls across six pillars; insurer-backed, audited by accredited firms. Written to certify the vendor's agent, not the buyer's programme. | ✔ | ✘ | ✔ | ◐ | ✔ |
| OWASP AISVS OWASP · v1.0, June 2026 · free | AI Security Verification Standard: 12 chapters, about 191 pass/fail requirements over three assurance levels, with agentic (C9) and MCP (C10) chapters. Developer and verifier audience. | ✔ | ✘ | ✔ | ✘ | ✔ |
| CSA AI Controls Matrix CSA · v1.1, June 2026 · free | 247 control objectives across 18 domains, mapped to ISO/IEC 42001, ISO/IEC 27001 and BSI AIC4. Enterprise GRC scale. | ◐ | ✔ | ✔ | ✘ | ✔ |
| OWASP Top 10 lists OWASP · LLM 2025, Agentic Dec 2025, MCP beta · free | The reference vulnerability taxonomies (LLM01 through ASI10 and the MCP beta). Risk lists, not control checklists. | ✔ | ◐ | ✘ | ✔ | ✔ |
| NIST AI RMF NIST · 2023, GenAI profile 2024 · free | Govern, Map, Measure, Manage process framework. No agentic profile as of August 2026; an AI Agent Standards Initiative launched February 2026. | ◐ | ✔ | ✘ | ✘ | ✔ |
| ISO/IEC 42001 ISO · 2023 · paywalled | Certifiable organisation-level AI management system standard on the ISO 27001 pattern. Certification costs real money and audit cycles. | ✘ | ✔ | ◐ | ✘ | ✔ |
| Deployer AI Risk Register MindXO · CC BY 4.0 · free | 82 deployer-side risks plus 61 MITRE ATLAS-anchored entries, crosswalked to ISO/IEC 42001, the EU AI Act and both OWASP Top 10s. A single organisation's project. | ◐ | ✔ | ✘ | ✔ | ✘ |
| Community checklists SlowMist and peers · free | Practitioner checklists, narrow and component-level (MCP security in particular). No formal standing. | ✔ | ✔ | ✔ | ✔ | ✘ |
✔ yes · ◐ partly · ✘ no · scores are this site’s reading of each document; every row links to its primary source
No row passes all five tests. The closest fit for the adopting firm is the six-agency guidance, which is advisory prose rather than a scored checklist. ETSI is numbered, free and recognised, and covers AI in general rather than agents in particular. AIUC-1 is agent-native and faces the vendor. AISVS and the CSA matrix assume security staff. The Top 10 lists catalogue what goes wrong, not what done looks like.
The ground is nevertheless covered. Three free documents between them span an adoption programme: the six-agency guidance for the actions, ETSI's operator provisions for a numbered baseline, and the OWASP Agentic Top 10 as a cross-check that the controls answer real failures. What no document on the map supplies is a single name to claim conformity against, a grading scheme that says pass or fail, or an external signature. For a bought agent, the certification question has an answer: AIUC-1 certifies the vendor's agent.
A sixth test
The five tests above ask whether a firm can use each document. A sixth question sits underneath them, and none of the five brings it out: does any of these documents address risk that appears across many agents at once, rather than inside one?
| Framework | Population | What it does, and where it stops |
|---|---|---|
| Careful Adoption of Agentic AI Services | ◐ | Structural risks name cascading failure, scoped within one deployment. |
| ETSI TS 104 223 / EN 304 223 | ✘ | One model or system lifecycle; no multi-agent scope. |
| AIUC-1 | ✘ | Certifies one vendor's agent. |
| OWASP AISVS | ◐ | A swarm-level kill-switch and inter-agent controls, within one orchestrator's own fleet. |
| CSA AI Controls Matrix | ✘ | A single organisation's control set. |
| OWASP Top 10 lists | ◐ | Cascading Failures (ASI08) describes propagation within one connected system. |
| NIST AI RMF | ✘ | Applied to one AI system's lifecycle. |
| ISO/IEC 42001 | ✘ | One organisation's management system. |
| Deployer AI Risk Register | ✔ | Records model monoculture with correlated failure, and multi-agent interaction dynamics, as deployer-side risks. |
| Community checklists | ◐ | Cross-MCP call control, within one install. |
✔ addresses it · ◐ names it, within one system · ✘ does not
The recognised standards do not address the population problem. Where a document touches it, it stays inside one system: cascading failure within a connected deployment, a kill-switch over one orchestrator's own fleet, controls across one organisation. Each stops at the edge of the system it governs.
One entry names it directly. The Deployer AI Risk Register records model monoculture with correlated failure, and multi-agent interaction dynamics, as deployer-side risks. It is a single organisation's project, which the map above marks as carrying no formal standing.
No document on the map treats the third case, a behaviour that returns after it has been removed.
That many identical systems are not many independent safeguards is not a new idea.
08 · The market prices it
The clearest evidence that agent risk is real is that underwriters now price it. Traditional carriers have begun filing AI exclusions into standard liability forms, and a small specialist market has emerged to sell affirmative cover back, generally on one condition: demonstrated governance first.
| Provider | Structure | Offering |
|---|---|---|
| AIUC | Standard + insurance pipeline | AIUC-1 certification (test volume varies by company: ElevenLabs' involved 5,000+ adversarial simulations, UiPath's 2,000+ risk scenarios, with at-least-quarterly retesting), used by insurers as the basis for cover. ElevenLabs was first to go live (11 February 2026); UiPath certified 9 March 2026. AIUC also has a $50M Beazley-backed product. |
| Armilla | Lloyd's coverholder / MGA | Affirmative AI liability up to $25M per organisation, covering AI model underperformance and agent actions, backed by Chaucer, Axis, Convex, Swiss Re and Greenlight Re; paired with Armilla's own AI assessment. |
| Munich Re aiSure | Reinsurer programme (since 2018) | AI performance warranties and liability cover, distributed via Mosaic at up to $15M. |
| Testudo | Lloyd's Lab MGA (launched January 2026) | US mid-market AI liability, underwritten off real-time litigation data, with Atrium/QBE capacity; markets an audit-free, no-code-access underwriting process. |
Traditional carriers have begun filing AI exclusions into standard liability forms. W.R. Berkley has filed an absolute AI exclusion (Form PC 51380 00) across its D&O, E&O and fiduciary lines; trade press reports other carriers filing similar exclusions.
Source: National Law Review↗
A handful of Lloyd's-adjacent providers make up the identified standalone affirmative AI-liability market as of mid-2026; no additional standalone provider was found, though the sweep was not exhaustive.
Source: The Insurer↗
AXA, Allianz and Zurich have not launched a dedicated AI-agent liability product as of mid-2026, though AXA XL offers a generative-AI cyber endorsement.
Source: agentinsured.eu↗
Several leading AI insurers (Armilla, AIUC) tie underwriting to governance and audit documentation; this is not universal. Testudo markets an audit-free, no-code-access process.
Source: testudo.co↗
09 · Vocabulary