Glossary

Short definitions of the recurring terms in the guide. Not comprehensive — reach for a textbook when you need real depth on a concept.

A

A2A (Agent-to-Agent) protocol. Standard for how agents discover, describe, and delegate to peer agents. Covered in Chapter 05.

Adapter. A small trainable module attached to a base model that specializes it for a domain (LoRA is the dominant format).

Adversarial example. An input crafted to be misclassified by an ML model despite being similar to a correctly-classified input. Chapter 10.

Adversarial suffix. A short token sequence appended to a prompt that reliably steers an LLM toward a target output. Reference: GCG.

Adversarial training. Training augmented with adversarial examples to raise robustness.

Agent. An LLM wrapped in a control loop that observes, plans, and acts. Chapter 05.

Agent card (A2A). A published descriptor an agent exposes so peer agents can discover its identity, capabilities, and endpoint under the A2A protocol. Chapter 05.

ALGEN. Model-agnostic embedding-inversion technique using canary injection and linear alignment. Chapter 08.

Alignment matrix. Linear map between two embedding spaces used in ALGEN.

ATLAS. MITRE's adversarial threat matrix for AI systems. Analog to ATT&CK. Reference throughout the guide.

Attention. Transformer's mechanism for weighting relevant parts of the input sequence.

B

Backdoor. A hidden behavior in an ML model activated by a specific trigger. Same as trojan.

BadNets. Reference paper for data-level trojan attacks on image classifiers.

Beam search. Search algorithm keeping the top-k candidate sequences at each step. Used in Zero2Text embedding inversion.

BM25. Classical keyword-based retrieval scoring. Common component of hybrid RAG retrievers.

BUILD opcode. Pickle opcode used with __setstate__ — sometimes bypasses pickle scanners.

Byteplot. Visualization of a binary file as an image; used as input to CNN-based malware classifiers.

C

Canary injection. Inserting known texts into a target vector store to obtain alignment pairs for embedding inversion.

Clean-label attack. Data poisoning where labels remain correct; the feature vectors are the poisoned content.

Chunker. RAG component that splits documents into retrievable pieces. Chapter 06.

Confused deputy. Attack pattern where a low-privilege caller induces a high-privilege service or agent to perform an action on their behalf, because authorization is checked at the wrong hop rather than at every step of a delegation chain. Seen in both classical OAuth flows and multi-agent tool-calling chains. Chapter 07, 15.

CVE-2025-23266. NVIDIA container-toolkit escape via a misconfigured OCI hook, dubbed NVIDIAScape. Chapter 12.

CVE-2025-53109 / CVE-2025-53110. MCP filesystem reference-server flaws: CVE-2025-53109 allowed symlink-based access outside configured directories; CVE-2025-53110 allowed an outside path to pass when it shared the allowed-directory prefix. Chapter 07.

CVE-2025-6514. OS command injection in mcp-remote, triggered by an unsanitized authorization_endpoint URL returned during the MCP OAuth handshake; a confused-deputy-flavored supply-chain flaw in a widely used MCP proxy. Chapter 07, 12.

D

DAN. "Do Anything Now" — reference jailbreak persona from LLM safety folklore.

Deceptive Delight. Jailbreak pattern that embeds an unsafe request inside a sequence of otherwise benign narrative or creative tasks, relying on the model's tendency to complete a story coherently. Chapter 04.

DeepFool. Adversarial attack computing minimum-norm perturbation to cross the decision boundary. Chapter 10.

Divergence attack. Technique that induces an LLM to regurgitate verbatim training data by exploiting repetition or degenerate decoding modes. Chapter 08, 13.

DP-SGD. Differentially-private stochastic gradient descent. Chapter 13.

Diffusion model. Iterative denoising generative model, common for image generation.

E

ε (epsilon). Perturbation budget for adversarial attacks (typically L∞ or L2) OR privacy budget for DP.

Echo Chamber. Jailbreak technique that steers a model toward an unsafe completion by building up context from the model's own earlier, individually benign responses until the accumulated context biases the next output. Chapter 04.

EAD (ElasticNet Attack). L1-regularized adversarial attack via FISTA. Chapter 10.

ECR. Elastic Container Registry. AWS container image storage frequently containing credentials in image ENVs. Chapter 12.

Embedding. Fixed-dimensional numeric representation of an input's semantic content. Chapter 01.

Embedding collision. Document crafted to have high similarity to many query embeddings. Chapter 06.

Embedding inversion. Recovering text from a stored embedding vector, e.g. via Vec2Text-style supervised decoders. Chapter 08.

Extraction (model theft). Building a functional copy of a model via query-based training. Chapter 13.

F

Feature. Numeric representation of an input the model consumes.

Feature store. Managed system for storing and serving precomputed features. Attack surface for training-time poisoning.

FGSM. Fast Gradient Sign Method — single-step L∞ adversarial attack. Chapter 10.

FISTA. Fast Iterative Shrinkage-Thresholding Algorithm — proximal-gradient optimization used in EAD.

Foundation model. Large pretrained model that serves as a base for many downstream applications.

Function calling. LLM API pattern where the model emits structured tool invocations.

G

GCG (Greedy Coordinate Gradient). Method for generating adversarial suffixes for LLM jailbreaks.

GGUF. Quantized model file format used by llama.cpp and compatible runtimes; a common distribution format for locally-run open-weight models. Chapter 09, 11.

GoodWords attack. Adding class-representative words to shift a text classifier's decision. Chapter 10.

GPU passthrough. Container configuration granting direct GPU hardware access. Weakens container isolation. Chapter 12.

GraphRAG. RAG variant that retrieves over a knowledge graph rather than (or in addition to) a flat vector index, using entity and relationship structure to guide retrieval. Chapter 06.

Guardrail. Input or output filter that blocks unsafe LLM interactions. Chapter 14.

H

Hallucination. LLM output that is fabricated but presented as fact. Chapter 04.

Homoglyph. Character that visually resembles another (Cyrillic а ~= Latin a). Used for filter bypass.

Human-in-the-loop. Design pattern requiring explicit human approval before an agent or automated pipeline executes a sensitive action. A common mitigation for tool abuse and confused-deputy chains, and a common target for attacks that try to make the approval step meaningless (e.g. rug pulls that change tool behavior after approval). Chapter 05, 07, 14.

Hybrid retrieval. RAG retrieval combining dense (vector) and sparse (keyword) methods.

I

I-FGSM (BIM). Iterative FGSM — multi-step L∞ adversarial attack. Chapter 10.

IAM role chaining. Sequence of sts:AssumeRole calls that escalates privilege across cloud identities. Chapter 12.

IMDS. Instance Metadata Service. Cloud endpoint returning workload credentials; SSRF target. Chapter 12.

Indirect prompt injection. Prompt injection delivered via content the LLM retrieves. Chapter 04.

Ingestion poisoning. Inserting malicious content into a RAG corpus at ingestion time. Chapter 06.

Inference probing. Fingerprinting technique that identifies an embedding model by comparing similarities against candidate models. Chapter 03.

J

Jailbreak. Prompt attack that bypasses an LLM's policy refusals. Chapter 04.

Jinja. Python template engine; common SSTI sink in tool chains. Chapter 05.

JSMA. Jacobian-based Saliency Map Attack — L0-bounded adversarial attack. Chapter 10.

K

Kubelet. Kubernetes node agent. Root on a node typically yields kubelet credentials, path to cluster-admin. Chapter 12.

L

L0, L1, L2, L∞ norms. Bounds on adversarial perturbation size. Chapter 10.

Label flipping. Data poisoning by changing labels. Chapter 09.

LangChain / LangGraph. Python frameworks for building LLM applications.

LD_PRELOAD. Environment variable specifying a shared library to load before others. Used in CVE-2025-23266 container escape.

Line jumping (MCP). Attack pattern where an MCP client or malicious server invokes a tool capability before the normal discovery and user-approval flow has completed, skipping the step meant to give the user visibility into what is about to run. Chapter 07.

LLM (Large Language Model). Autoregressive transformer trained on large text corpora.

LoRA. Low-Rank Adaptation — parameter-efficient fine-tuning method producing small adapter files. Chapter 11.

LoRA adapter poisoning. Embedding a backdoor or sleeper-agent trigger inside a small LoRA adapter rather than the base model, so the poisoned artifact is small, easy to distribute through an adapter registry, and hard to distinguish from a legitimate fine-tune under casual inspection. Chapter 09, 11, 15.

LSB flipping. Modifying least-significant bits of floating-point weights for tensor steganography. Chapter 09.

M

MCP (Model Context Protocol). Standardized protocol for LLM-tool integration. Chapter 07.

Membership inference. Determining if a specific input was in a model's training set, including refinements such as Min-K%++ (thresholding on minimum-token-probability statistics) and SPV-MIA (self-calibrated probabilistic variation). Chapter 13.

MLflow. Open-source model registry and experiment tracker. Common target of registry attacks. Chapter 12.

MITRE ATLAS. See ATLAS above.

Model jacking. Hijacking a hosted model endpoint, registry entry, or serving alias so that requests intended for a trusted model are served by an attacker-controlled one instead. Chapter 11, 12.

Model signing. Cryptographically signing a model or adapter artifact at publish time so downstream consumers can verify it has not been altered before loading it. Emerging supply-chain defense against registry poisoning and model jacking. Chapter 11, 12.

Multi-agent. Systems with several cooperating agents, often coordinated by an orchestrator. Chapter 05.

Multi-turn jailbreak. Jailbreak that succeeds only across a sequence of prompts rather than a single one, gradually shifting context so a request that would be refused outright becomes acceptable in the accumulated frame. Crescendo is the reference technique. Chapter 04, 15.

N

Nemo Guardrails. NVIDIA's open-source guardrail framework for LLM applications.

Norm. Function measuring the size of a vector. Adversarial perturbations are bounded by norms; Chapter 10.

O

OpenAI-compatible API. REST convention for LLM inference (/v1/completions, /v1/chat/completions) adopted by many self-hosted servers.

Orchestrator. Central component in a multi-agent system that routes tasks between specialists. Chapter 05.

OWASP LLM Top 10. Reference list of most common LLM application vulnerabilities.

P

PATE. Private Aggregation of Teacher Ensembles — DP training via teacher-student aggregation. Chapter 13.

PGD. Projected Gradient Descent — reference iterative adversarial attack with restarts. Chapter 10.

Pickle. Python serialization format that executes bytecode on load. Chapter 09, 11.

Pickle deserialization. The act of loading a pickle file, which — unlike loading a plain data format — executes embedded bytecode and is therefore a code-execution primitive if the file is attacker-controlled. Scanners that look for known-dangerous opcodes can be bypassed by techniques such as nullifAI. Chapter 09, 11.

Policy Puppetry. Jailbreak framing that wraps a disallowed request inside a fake system-configuration or policy document (e.g. XML/JSON that looks like an override the model should obey as configuration rather than as a user instruction). Chapter 04, 15.

Prompt guard. A classifier or filter positioned in front of an LLM specifically to detect and block prompt-injection or jailbreak attempts before they reach the model. A subtype of guardrail. Chapter 14.

Prompt injection. Attack that shifts LLM input from data to instruction channel. Chapter 04.

Provenance. Signed record of how a model or dataset was built. Emerging supply-chain defense.

Q

Qdrant. Popular open-source vector database.

Quantization. Reducing weight precision (32-bit → 8-bit → 4-bit) to shrink model size. May reduce inversion vulnerability at cost of accuracy.

R

RAG (Retrieval-Augmented Generation). Architecture pairing an LLM with a retrieval store. Chapter 06.

RBAC. Role-Based Access Control. Kubernetes and cloud IAM permission model.

ReAct. Agent loop pattern alternating Reason and Act. Chapter 05.

Ridge regression. Linear regression with L2 regularization. Used in ALGEN alignment.

RLHF. Reinforcement Learning from Human Feedback. Standard LLM alignment technique.

RSA (Rules of Engagement / ROE — engagement doc). Constraints and permissions for a security engagement.

Rug pull (MCP). Attack pattern in which a tool or resource that was benign at the time a user or client approved it is later changed — silently, server-side — to carry malicious instructions or behavior, exploiting clients that do not re-verify approved tools on every use. Chapter 07, 15.

S

SageMaker. AWS ML platform. SageMakerExecutionRole commonly over-privileged. Chapter 12.

Safetensors. Non-executable format for storing tensors, designed specifically so that loading a model file cannot trigger arbitrary code execution the way an unpickled file can. Chapter 09.

Saliency map. Per-feature attribution of a model's prediction. Basis of JSMA.

SBOM. Software Bill of Materials. Emerging for ML models (model provenance metadata).

Shadow AI. Unsanctioned or unmanaged use of AI tools, models, or agents inside an organization, outside the visibility of security and IT — e.g. a team standing up its own LLM integration or MCP server without going through review. Expands the real attack surface beyond what the security team's inventory shows. Chapter 03, 14.

Shadow model. Auxiliary model trained to mimic a target, used in privacy attacks. Chapter 13.

Slopsquatting. Supply-chain attack that publishes malicious packages under names that LLM coding assistants are observed to hallucinate, betting that a developer will pip install or npm install a plausible-but-nonexistent dependency the model suggested. Chapter 11.

Slow drip poisoning. RAG poisoning that spreads malicious changes across many small ingestion events. Chapter 06.

Sleeper agent. A model or adapter that behaves normally on all ordinary inputs and only activates a hidden, malicious behavior when a specific trigger is present — the general pattern behind trojans, backdoors, and poisoned LoRA adapters. Chapter 09, 11, 15.

Sparsity attack. L0-bounded adversarial attack. Chapter 10.

SSMA (SSM Parameter Store). AWS Systems Manager parameter store; frequently contains plaintext secrets. Chapter 12.

SSTI. Server-Side Template Injection. Common tool-chaining sink. Chapter 05, 07.

Sympify. SymPy function that internally uses eval; common pickle-scanner bypass gadget. Chapter 09, 11.

T

T5 / T5-small. Encoder-decoder transformer model used as inversion decoder in ALGEN and Vec2Text.

Template poisoning. Poisoning tool descriptions rather than tool code. Chapter 07.

Token. Atomic unit of text an LLM processes. Chapter 01.

Tokenizer. Component mapping text to token IDs. Chapter 01.

Tool. Action primitive an agent can invoke. Chapter 05.

Tool poisoning. Hiding malicious instructions inside a tool's metadata (most often its free-text description field) rather than its executable code, so that the instructions are read and obeyed by the calling LLM without being visible in a typical human review of the tool's function. Chapter 07, 15.

Trojan. Hidden model behavior activated by a trigger. Same as backdoor. Chapter 09.

Trust boundary. Point where one component's authority transitions to another's. Chapter 02.

U

Universal adversarial perturbation. Single perturbation that fools many inputs of a target model. Chapter 10.

V

Vault. HashiCorp Vault — dominant secrets manager for ML deployments. Chapter 12.

Vec2Text. Supervised embedding-inversion technique. Chapter 08.

Vector database. Store for embedding vectors with similarity search, queried by nearest-neighbor lookup to power RAG retrieval. The primary data store attacked in embedding-inversion and retrieval-poisoning techniques. Chapter 06, 08.

vLLM. Open-source LLM inference server. Common target in Chapter 12.

W

Weaviate. Popular open-source vector database.

weights_only. PyTorch load option that refuses pickle bytecode execution. Chapter 09.

X

xp_cmdshell. MSSQL stored procedure executing OS commands. Common escalation primitive. Chapter 15 case B.

XSRF/CSRF. Cross-site request forgery. Applies to any cookie-authenticated LLM interface.

Z

Zero-shot inversion. Embedding inversion without prior training on the target's model. Chapter 08.

Zero-width character. Unicode characters that render as no visible glyph (U+200B, U+200C, U+200D, U+2060, U+FEFF). Used for filter bypass; a subtype of the broader character-injection family alongside homoglyphs and Unicode tag blocks.

Zero2Text. Beam-search embedding inversion technique. Chapter 08.