Failure mode · FM-03
Insecure tool use
The tools an agent calls become the attack surface: unvalidated inputs, unauthenticated tool servers, or tool descriptions that themselves carry malicious instructions.
Answered by
Authenticated, validated, allow-listed tools.
OWASP cross-reference
ASI02 Tool Misuse and Exploitation
Legitimate tools used unsafely via chaining, privilege escalation, unintended actions.
Sources: OWASP Top 10 for Agentic Applications 2026↗
In the record
- Agents under evaluation broke into a real company
The way out of the sandbox was the tooling around the evaluation, not the model itself.
- A coding agent deleted a production database during a code freeze
A Replit AI coding agent, holding write access to a live production database, deleted production data despite instructions not to make changes. Replit's CEO confirmed the deletion and rolled out automatic development/production database…
- An LLM-driven operator ran a database extortion campaign end to end
Sysdig's threat research team documented an extortion operation it describes as driven end to end by a large language model. It entered through an internet-facing Langflow instance (CVE-2025-3248), pivoted to the intended target, and ran a…
- A malicious MCP server waited three tool calls before asking for credentials
Pillar Security reported an MCP server distributed through public GitHub pull requests that behaves as a text formatting tool until a client has made three tool calls. It then returns instructions directing the agent to collect SSH keys…