If you are deploying LLMs, it helps to hold one idea in mind before you get into the specifics. Almost every attack below is a variation on a small number of failures: untrusted input treated as instruction, sensitive data flowing where it should not, unverified components pulled in from outside, and a model trusted to act or answer without anyone checking. Fix those at the boundaries and most of this list gets much smaller.
The stakes are highest in regulated industries, where a single leak or a single bad action can mean exposed customer records, a compliance breach, or patient harm, so the examples below are drawn mostly from banking, financial services, and healthcare.
1.Prompt Injection
A model cannot reliably tell the difference between its own instructions and the input a user provides. So an attacker who is told ‘no’ can often get to ‘yes’ by reframing the request, or by hiding an instruction where the model will read it.
Picture a retail bank’s support assistant instructed never to reveal another customer’s account details. A direct request is refused, but the attacker reframes it as a fraud review and the assistant hands the details over. That is the direct form. In the indirect form, a claims team asks the assistant to summarize a document a customer uploaded, say a scanned loan application, and hidden inside that file is an instruction telling the model to ignore its rules and forward account data elsewhere.
The defense is layered. Tighten the system prompt with your own rules, but do not rely on it alone, because you will never anticipate every case. Put an AI gateway between the user and the model to inspect what goes in and what comes out, blocking unwanted requests and redacting sensitive responses. Then test it yourself with a batch of injection attempts and see where it breaks.
2. Sensitive Information Disclosure
Models get trained or grounded on real business data: customer records, health information, financials, and proprietary material. With the wrong controls, an attacker can coax that information back out with a well-phrased prompt. A hospital that grounds a clinical assistant on patient histories can find it surfacing one patient’s diagnosis inside another patient’s session, and a bank that fine-tunes on transaction data risks the same with account numbers and balances. A patient attacker can go further, querying the model over and over and recording each answer until they have reconstructed large parts of it, an approach known as a model inversion or extraction attack. Defend it by sanitizing the data so only what you need reaches the model, scanning outbound responses for sensitive patterns like account or policy numbers, and enforcing strong access controls on the model, the data, and the users. And close the basics: outdated software, weak authentication, and unencrypted data are all leaks waiting to happen.
3. Supply Chain Vulnerabilities
A model does not appear from nowhere. It begins with data, gets trained and tuned, and sits under an application, and almost no organization builds its own from scratch. Instead, teams pull pre-built models, datasets, and libraries from public sources that are far too large to inspect by hand, which means unverified material entering your environment. A payments firm or a hospital network that adopts an open model to accelerate a project inherits every weakness in that model and the code around it, often without a full inventory of what it has taken on. The infrastructure underneath counts too. Vet every component and the parties behind it, trace the provenance of what you bring in, scan and run adversary-style testing against your systems, and keep everything patched. A supply chain is only as strong as its least examined link.
4. Data and Model Poisoning
If data is the lifeblood of a model, corrupted data is a slow toxin. An attacker who mixes a little error into training data, or tampers with a document the model treats as ground truth, can degrade accuracy, introduce bias that compounds over time, or plant behavior that acts like hidden malicious code.
In banking, poisoned data behind a fraud or credit model can quietly tilt decisions, letting through what should be flagged or skewing who gets approved for a loan. In healthcare, a tampered reference behind a clinical assistant can bend its guidance in ways nobody notices until there is harm. These attacks are often subtle and hard to detect, which is exactly what makes them dangerous as you come to rely on the model. The recurring theme applies here more than anywhere: know your sources. Control who can touch the model, the training data, and any grounding sources, and put change control around all three so nothing gets altered quietly.
5. Improper Output Handling
The risk does not end when the model produces an answer. If that output flows into another system, into a web page, a database, or a command line, it can carry a payload with it, including cross-site scripting, SQL injection, or remote code execution. A bank that lets an assistant generate queries against account systems, or that renders model output directly in a customer portal, can pass an injection straight into production if that output is never checked. The failure is treating model output as trustworthy simply because it came from your own system. Treat it instead the way you would treat input from a stranger. Validate and sanitize anything the model returns before it becomes code, a query, or markup that something downstream will execute.
6. Excessive Agency
The more you connect a model to, the more damage it can do when something goes wrong. A model with access to tools, APIs, plug-ins, and systems that act in the real world becomes a serious liability the moment it is hijacked through an injection, or simply gets an action wrong. The stakes rise fast once the model can act on its own: an assistant able to initiate a payment, adjust a credit limit, or update a patient’s medical record can turn a single hijacked prompt into real financial or clinical damage.
The fix is least privilege applied to AI: give the model only the reach it genuinely needs, and disconnect it from anything it does not. For any action with real consequences, keep a person in the approval loop, and never let the model make a high-impact change to money, operations, or safety on its own.
7. System Prompt Leakage
The system prompt sets the model’s context, and sometimes teams put things in it that do not belong there, such as the credentials the model needs to reach another application. Asked the right way, the model can be persuaded to reveal them. If that prompt holds the login the assistant uses for a payment gateway or an electronic health record system, a single leak hands an attacker the keys to far more than the chatbot.
The answer is straightforward: keep secrets out of the prompt entirely, store them securely, and let the model reach them through a controlled path. A guard on the output that catches sensitive values gives you a second line of defense, but the real win is that a secret which was never in the prompt has nothing to leak.
8. Vector and Embedding Weaknesses
Many deployments give the model documents from a store to help it answer, and that store is a target. A tampered document can bleed bad information into responses, and if you are not careful, that information settles in and starts to function as knowledge the model treats as its own. A bank’s assistant that answers from a store of policy and product terms, or a hospital assistant grounded in clinical guidelines, becomes unreliable the moment a doctored document slips into that store. Control what is allowed in, validating every document before it lands and locking down who can add to it. And keep retrieved content temporary, so it supports a single answer and then washes over the system rather than becoming permanent.
9. Misinformation
At some point the question is simply whether you can trust what the model tells you. These systems can be manipulated, and they can also produce confident answers that are wrong. Decisions built on either one rest on shaky ground. A confidently wrong answer about a lending rule, a fee, or a customer’s eligibility can create real compliance exposure for a bank, and a fabricated but plausible clinical detail can be dangerous in a care setting. There is no purely technical fix for this one.
The defense is discipline: keep critical thinking in the loop, ask whether an answer is reliable and whether it makes sense, and cross-check against trusted sources before acting. Verification has to become a habit, not an afterthought.
10. Unbounded Consumption
Left without limits, an AI system can be overwhelmed the same way any service can. Flood it with too many requests, unusually long ones, or ones that force heavy computation, and it becomes unavailable to everyone else, a denial of service in familiar clothing. There is a financial version too, often called denial of wallet, where an attacker keeps the system busy and runs up your bill. A customer-facing banking assistant knocked offline at month-end, or a metered clinical tool driven into a runaway cost, is both an availability problem and a budget problem. Put bounds in place: rate limit requests per user, set timeouts so a single request cannot tie the system up, and monitor usage with alerts and spending caps.
The Pattern Underneath
Read the list again and the same handful of controls keep reappearing. Know where your models, data, and documents come from. Put inspection at every trust boundary, on the way in, on the way out, and around who has access. Validate rather than trust, especially anything the model produces or ingests. And test your own systems the way an attacker would. None of this is exotic. It is the disciplined application of security fundamentals to a new kind of system, and in regulated industries it is also what regulators and auditors will expect to see. Build it in from the start rather than bolting it on after an incident. The attackers already know this list. Your team should know it better.
