A field guide · maintained in Johannesburg · reviewed 11 July 2026
A chatbot that gives a wrong answer produces a bad paragraph. An agent that goes wrong sends the email, moves the money, deletes the table, or signs the contract, at machine speed, before anyone reviews it. This site maps that risk: how agentic systems fail, what has already happened, which standards and insurers now govern the space, and what it means for firms in regulated markets.
Feb 2026
First insured AI agents
ElevenLabs voice agents were first to go live with AIUC-1-backed cover.
2025-26
AI exclusions spread
W.R. Berkley filed an absolute AI exclusion across D&O, E&O and fiduciary lines.
2027-28
EU AI Act deadlines moved
High-risk obligations were postponed from Aug 2026 to Dec 2027 and Aug 2028.
01 · Definition
An AI agent is an AI system that does not just generate text: it takes actions. It calls tools, queries databases, browses the web, writes and runs code, sends messages, and increasingly delegates work to other agents. The connecting layer is often an open protocol such as MCP (the Model Context Protocol), introduced by Anthropic in November 2024 and now widely adopted, which gives a language model structured access to real systems.
Agent risk is the family of risks that arises specifically because the system acts: a wrong output becomes a wrong action, executed with real permissions, often in chains where one mistaken step feeds the next. Security researchers identify threat vectors that do not exist in passive AI: prompt injection through content the agent reads, privilege escalation through over-broad permissions, unintended action chains, data exfiltration through tool use, and supply-chain compromise.
Three properties set this apart from classic software risk. Agents are probabilistic: the same input can produce different actions. They are open to instruction: anything an agent reads, a web page, a PDF, an email, another agent’s message, is potentially an instruction, which is why prompt injection has no clean software analogue. And they operate at machine speed and scale: a flawed policy executes a thousand times before a human notices once.
Those three properties describe how an agent acts. A fourth difficulty is about how its work is checked. Agent output behaves as a credence good: fluent text is not the same as correct, secure or well-governed work, and the person relying on it often cannot tell the difference before acting on it, after acting on it, or in some cases ever. That distance between fluent and verified is why the assurance of agents is treated as a separate problem from testing ordinary software, and it is the subject the companion book takes up.
Sources: Anthropic on MCP↗ · OWASP Top 10 for Agentic Applications↗ · Incident Analysis for AI Agents (Ezell et al.)↗ · Credence goods (Darby & Karni, 1973)↗ · Agent Assurance↗
02 · Taxonomy
The reference taxonomy is the OWASP Top 10 for Agentic Applications (2026), a widely referenced list of agent risks. The eight patterns below consolidate it with OWASP’s adjacent LLM and MCP work, each cross-referenced to OWASP’s published identifiers. Some patterns have no clean one-to-one OWASP category; those are marked. Each also names the class of control that answers it: read backwards, a control is a record of the failure it exists to prevent, and the companion book traces that mapping in full.
Hidden or hostile instructions embedded in content the agent processes (a web page, an email, a document, image metadata) hijack its behaviour.
OWASP: LLM01:2025 Prompt Injection · ASI01 Agent Goal Hijack
Answered by: Untrusted-content handling: isolate and validate everything the agent reads.
The agent holds broader authority than the task requires (database write access for a read task, spend authority without limits), so a single mistake or manipulation has outsized blast radius.
OWASP: LLM06:2025 Excessive Agency · ASI03 Identity and Privilege Abuse
Answered by: Least privilege: scope permissions, cap spend and autonomy.
The tools an agent calls become the attack surface: unvalidated inputs, unauthenticated tool servers, or tool descriptions that themselves carry malicious instructions.
OWASP: ASI02 Tool Misuse and Exploitation
Answered by: Authenticated, validated, allow-listed tools.
Nobody can say which principal an agent was acting for when it acted. Agents inherit human credentials, share service accounts, or pass tasks to sub-agents with no authenticated chain of delegation.
OWASP: ASI03 Identity and Privilege Abuse
Answered by: A distinct agent identity and an authenticated delegation chain.
False or malicious information planted in an agent's memory, retrieval store, or context persists and corrupts future decisions long after the original interaction.
OWASP: ASI06 Memory & Context Poisoning
Answered by: Provenance and integrity checks on memory and retrieval.
Multi-step and multi-agent workflows propagate one early error through every downstream step: a chain reaction at machine speed.
OWASP: ASI08 Cascading Failures
Answered by: Checkpoints and blast-radius limits between steps.
The model invents something (a policy, a price, a legal citation) and then acts on it or communicates it with the authority of the firm behind it.
OWASP: LLM09:2025 Misinformation · ASI09 Human-Agent Trust Exploitation (partial)
Answered by: Grounding, and human sign-off before consequential action.
This does not cause an incident; it makes every incident worse. There is no per-action log tying decision, data, policy version and outcome together, so nobody can reconstruct what happened or prove what did not.
OWASP: No single OWASP category
Answered by: Durable per-action logs and a human gate before irreversible action.
Sources: OWASP Top 10 for Agentic Applications 2026 (ASI01 to ASI10)↗ · OWASP Top 10 for LLM Applications 2025↗ · NIST AI RMF (Manage)↗ · Agent Assurance (the controls in full)↗ · as of 11 July 2026
03 · Evidence
Several public databases already track AI incidents. Rather than duplicate them, this guide links to the main ones and notes what each is for. They largely record what reached the news, not what a system did internally, and were not built for agentic systems.
Sources: Incident Analysis for AI Agents (Ezell et al.)↗
Responsible AI Collaborative
Real-world AI incidents (harms already caused) across all AI system types, from public reporting and community submission.
OECD.AI Policy Observatory
AI incidents and hazards (near-misses / plausible-harm events) detected from global news media via an automated news-intelligence pipeline.
Charlie Pownall (independent, volunteer-run public-interest project)
AI, algorithmic and automation incidents and controversies, including reputational, ethical and governance failures beyond strict harm events.
MIT AI Risk Initiative
Reclassifies AIID's raw reports against MIT's own risk taxonomy and a harm-severity scale, with EU AI Act risk-level tagging.
Damien Charlotin (independent legal researcher, HEC Paris)
Court and tribunal decisions worldwide where a party was found to have relied on AI-fabricated content (invented citations, false quotes).
liability · 2024 · British Columbia, Canada · FM-07 FM-04
Air Canada's website chatbot told a customer he could apply for a bereavement fare after travel. The BC Civil Resolution Tribunal rejected the argument that the chatbot was a separate entity and held the airline liable for negligent misrepresentation.
professional · 2026 · Multiple (US, UK, Canada, Australia, Israel, Brazil) · FM-07
A growing, actively maintained database tracks court and tribunal decisions in which a party was found to have relied on AI-fabricated case citations. The count has risen steadily as more matters reach judgment.
operations · 2025 · FM-02 FM-03 FM-06
A Replit AI coding agent, holding write access to a live production database, deleted production data despite instructions not to make changes. Replit's CEO confirmed the deletion and rolled out automatic development/production database separation in response.
security · 2026 · FM-01 FM-05
Google DeepMind researchers published a six-category taxonomy of how adversarial web content (hidden HTML instructions, poisoned images, manipulated memory, multi-agent cascades) can hijack autonomous agents, including sites that detect an agent visitor and serve it content a human never sees.
Source: Franklin et al., AI Agent Traps, Google DeepMind (March 2026)↗
04 · Governance
No single framework covers agent risk end to end; mature programmes layer several. NIST AI RMF for governance process, ISO/IEC 42001 for a certifiable management system, the OWASP lists for engineering-level vulnerabilities, MITRE ATLAS for adversary techniques, and, newest, AIUC-1, the first certification standard written specifically for AI agents.
| Framework | What it is | Certifiable? | Agent-specific? |
|---|---|---|---|
| AIUC-1 | 50+ safeguards across six pillars (data & privacy, security, safety, reliability, accountability, society), updated quarterly (latest release 15 April 2026). Built with a consortium including MITRE, Stanford, Orrick and the Cloud Security Alliance; certification can qualify an agent for insurance. | Yes, via accredited auditors (Schellman was first, 3 Feb 2026) | Yes, the first |
| ISO/IEC 42001 | The first international AI management system standard (December 2023), built on the ISO 27001 pattern. It is not a harmonised standard under the EU AI Act, and a certificate does not by itself confer a presumption of conformity. | Yes, via accredited bodies (BSI, Schellman, A-LIGN, KPMG) | No, organisation-level |
| NIST AI RMF | A voluntary US framework (2023) structured Govern / Map / Measure / Manage. It does not mandate specific runtime technical controls, leaving implementation to adopting organisations. | No | Extensible, not specific |
| OWASP lists | Three vulnerability taxonomies: the LLM Top 10 (2025), the Top 10 for Agentic Applications (2026, identifiers ASI01 to ASI10), and the MCP Top 10 (in beta as of mid-2026). | No, a taxonomy | Yes (agentic & MCP lists) |
| MITRE ATLAS | A living knowledge base of adversary tactics and techniques against AI systems; a technical contributor to AIUC-1. | No, threat intelligence | Partially |
| EU AI Act | Binding, risk-tiered law with extraterritorial reach (Art. 2). Its high-risk obligations, originally set for 2 August 2026, were postponed by the 2026 Digital Omnibus to 2 December 2027 (stand-alone systems) and 2 August 2028 (AI embedded in regulated products); the Article 50 transparency obligations remain on track for 2 August 2026. | Conformity assessment | Indirectly |
05 · The market prices it
The clearest evidence that agent risk is real is that underwriters now price it. Traditional carriers have begun filing AI exclusions into standard liability forms, and a small specialist market has emerged to sell affirmative cover back, generally on one condition: demonstrated governance first.
| Provider | Structure | Offering |
|---|---|---|
| AIUC | Standard + insurance pipeline | AIUC-1 certification (test volume varies by company: ElevenLabs' involved 5,000+ adversarial simulations, UiPath's 2,000+ risk scenarios, with at-least-quarterly retesting), used by insurers as the basis for cover. ElevenLabs was first to go live (11 February 2026); UiPath certified 9 March 2026. AIUC also has a $50M Beazley-backed product. |
| Armilla | Lloyd's coverholder / MGA | Affirmative AI liability up to $25M per organisation, covering AI model underperformance and agent actions, backed by Chaucer, Axis, Convex, Swiss Re and Greenlight Re; paired with Armilla's own AI assessment. |
| Munich Re aiSure | Reinsurer programme (since 2018) | AI performance warranties and liability cover, distributed via Mosaic at up to $15M. |
| Testudo | Lloyd's Lab MGA (launched January 2026) | US mid-market AI liability, underwritten off real-time litigation data, with Atrium/QBE capacity; markets an audit-free, no-code-access underwriting process. |
Traditional carriers have begun filing AI exclusions into standard liability forms. W.R. Berkley has filed an absolute AI exclusion (Form PC 51380 00) across its D&O, E&O and fiduciary lines; trade press reports other carriers filing similar exclusions.
Source: National Law Review↗
A handful of Lloyd's-adjacent providers make up the identified standalone affirmative AI-liability market as of mid-2026; no additional standalone provider was found, though the sweep was not exhaustive.
Source: The Insurer↗
AXA, Allianz and Zurich have not launched a dedicated AI-agent liability product as of mid-2026, though AXA XL offers a generative-AI cyber endorsement.
Source: agentinsured.eu↗
Several leading AI insurers (Armilla, AIUC) tie underwriting to governance and audit documentation; this is not universal. Testudo markets an audit-free, no-code-access process.
Source: testudo.co↗
06 · Jurisdiction
South Africa has no AI-specific statute. A Draft National AI Policy was gazetted on 10 April 2026 but withdrawn (gazetted withdrawal 12 June 2026) after fabricated, AI-generated citations were found in it; a revised draft is not expected before late 2026 or early 2027. Until then, agent risk falls inside obligations that already exist.
POPIA addresses automated decision-making at section 71 and security safeguards at section 19. King IV assigns the governance of technology and information to the governing body under Principle 12. For financial institutions, relevant FSCA and Prudential Authority instruments include the Joint Standard 1 of 2023 (IT Governance and Risk Management, commenced 15 November 2024), the Joint Standard 2 of 2024 (Cybersecurity and Cyber Resilience, commenced 1 June 2025), and the Joint Standard 1 of 2024 (Outsourcing by Insurers, effective 1 December 2024).
No Information Regulator guidance specifically addressing automated decision-making or AI has been identified as of 11 July 2026.
07 · Vocabulary