Key points
- Large language models (LLMs) can be attacked through the text they read, the data they were trained on, the tools they can use and the software around them.
- Prompt injection is the leading risk. The UK NCSC has warned it may never be fully mitigated, because LLMs do not separate instructions from data.
- EchoLeak (CVE-2025-32711), a zero-click flaw in Microsoft 365 Copilot, showed how one crafted email could make an AI assistant leak internal data. Microsoft fixed it in June 2025.
- The best defence is design: give AI systems as little privilege as possible and keep untrusted content away from sensitive actions.
Organisations are connecting AI assistants and large language models (LLMs) to email, documents, code and business systems. That makes the models themselves a target. An attacker no longer needs to break into a server if they can persuade the AI that already has access to do the work for them.
This article sets out the main ways attackers target AI systems, using the two most widely used reference frameworks: the US National Institute of Standards and Technology’s taxonomy of adversarial machine learning (NIST AI 100-2 E2025) and the OWASP Top 10 for LLM Applications (OWASP). MITRE also maintains ATLAS, a knowledge base of real attack techniques against AI systems (MITRE ATLAS).
Why AI systems are a new attack surface
Traditional software follows code. An LLM follows text, and it cannot reliably tell the difference between an instruction from its owner and an instruction hidden in a web page, email or document it has been asked to read. Once a model can send messages, browse, run code or query databases, a successful manipulation can have real effects.
Prompt injection
In a direct prompt injection, a user types instructions that override the system’s rules. In an indirect prompt injection, the attacker plants instructions in content the AI will later process, such as a web page, a shared file or an email. OWASP ranks prompt injection as the number one risk for LLM applications (OWASP).
The clearest real example is EchoLeak (CVE-2025-32711, CVSS 9.3), found by Aim Security in Microsoft 365 Copilot. An attacker sent an ordinary-looking email containing hidden instructions. When the user later asked Copilot a routine question, Copilot pulled the email into its context together with internal data, followed the hidden instructions and leaked the data through links. The user did not need to click anything. Microsoft fixed the flaw in its June 2025 updates, and there was no evidence it had been exploited (The Hacker News).
In December 2025 the UK’s National Cyber Security Centre warned that prompt injection should not be treated like SQL injection, because LLMs have no built-in boundary between instructions and data. It said such attacks “may never be totally mitigated” and that organisations should design systems to limit the damage instead (NCSC).
Data and model poisoning
Poisoning attacks corrupt the data a model learns from, or the model itself, so it behaves wrongly in specific situations. An attacker might plant content in public sources likely to be scraped for training, tamper with a fine-tuning dataset, or insert a hidden “backdoor” trigger that changes the model’s output when a particular phrase appears. NIST classifies poisoning as one of the core attack types against machine learning (NIST). Retrieval-augmented systems, which look up documents at answer time, can be poisoned simply by adding misleading documents to the knowledge base.
Supply chain attacks on AI
Most organisations build on models, libraries and datasets made by others. Model files downloaded from public hubs can contain malicious code, because some common file formats, such as Python’s pickle format, can run code when they are loaded. OWASP lists supply chain weaknesses among the top risks for LLM applications (OWASP).
Privacy attacks and model theft
Attackers can query a model to extract information it should not reveal: fragments of its training data, personal information, or its hidden system prompt. With enough queries, they may also copy a model’s behaviour to build a cheaper imitation, which NIST describes as model extraction (NIST). OWASP lists sensitive information disclosure and system prompt leakage as separate risks (OWASP).
Jailbreaks and misuse
A jailbreak is a prompt designed to make a model ignore its safety rules, for example by role-play, by splitting a harmful request into harmless-looking parts, or by encoding it. Jailbreaks matter because they turn a general-purpose assistant into a tool for writing malware, phishing or disinformation. How attackers use AI tools in their operations is covered in our companion article on how AI agents are used in cyber attacks.
Excessive agency
The more an AI system is allowed to do, the worse a successful manipulation becomes. OWASP calls this “excessive agency”: an assistant with permission to send email, change records or make payments can be steered into doing so by an attacker’s planted text (OWASP). This is the risk that grows fastest as companies deploy AI agents.
The OWASP Top 10 for LLM applications (2025)
- Prompt injection
- Sensitive information disclosure
- Supply chain
- Data and model poisoning
- Improper output handling
- Excessive agency
- System prompt leakage
- Vector and embedding weaknesses
- Misinformation
- Unbounded consumption
Source: (OWASP Top 10 for LLM Applications 2025).
How to defend AI systems
- Assume the model can be manipulated. Give it no more access than the least trusted person whose content it reads (NCSC).
- Use deterministic controls. Require human approval or rule-based checks for sensitive actions such as sending data outside the organisation or making payments.
- Separate trust zones. Keep external content, such as inbound email and web pages, apart from confidential data in the model’s context where possible.
- Check the supply chain. Use trusted model sources, safe file formats and signed artefacts, and scan models before loading them.
- Log and monitor. Record prompts, outputs and tool calls so suspicious behaviour can be spotted and investigated.
- Test adversarially. Red-team AI applications before and after deployment, using frameworks such as MITRE ATLAS.
Related research
For how attackers use AI as a weapon rather than a target, see how AI agents are used in cyber attacks. For attacks on physical systems where lives are at stake, see the series Cyberattacks That Endanger Human Lives.
Frequently asked questions
What is the most common cyber attack on large language models?
Prompt injection, where an attacker gets instructions into the text an LLM processes so that it ignores its rules. OWASP ranks it as the top risk for LLM applications.
What was EchoLeak?
EchoLeak (CVE-2025-32711) was a zero-click vulnerability in Microsoft 365 Copilot found by Aim Security. A crafted email could make Copilot leak internal data without the user clicking anything. Microsoft fixed it in June 2025 and reported no evidence of exploitation.
Can prompt injection be fixed?
Not completely, according to the UK NCSC. Because LLMs do not separate instructions from data, organisations should reduce the likelihood and impact of attacks through system design rather than expect a single fix.
What is data poisoning in AI?
Corrupting the data a model learns from, or the documents it retrieves, so that it gives wrong or attacker-chosen answers in specific situations.
Last reviewed: 22 October 2026.
