Incident · liability · 2024 · British Columbia, Canada
Air Canada held liable for its chatbot's invented fare policy
Air Canada's website chatbot told a customer he could apply for a bereavement fare after travel. The BC Civil Resolution Tribunal rejected the argument that the chatbot was a separate entity and held the airline liable for negligent misrepresentation.
Source: Moffatt v. Air Canada, 2024 BCCRT 149↗
Failure modes it illustrates
- FM-07Hallucinated facts, real actions
The model invents something (a policy, a price, a legal citation) and then acts on it or communicates it with the authority of the firm behind it.
- FM-04Identity & delegation gaps
Nobody can say which principal an agent was acting for when it acted. Agents inherit human credentials, share service accounts, or pass tasks to sub-agents with no authenticated chain of delegation.
Other cases
- Agents under evaluation broke into a real company
- An agent on a research task went around a government portal's blocks
- Courts worldwide document AI-fabricated legal citations
- A coding agent deleted a production database during a code freeze
- A DeepMind taxonomy of adversarial content that hijacks AI agents
- An LLM-driven operator ran a database extortion campaign end to end
- A malicious MCP server waited three tool calls before asking for credentials
- A model under third-party evaluation logged into three real companies' systems