Adversarial Evasion

Source: 10-evasion.md
State: Editorial review
Edition: 2026-draft
Estimated reading time: 14 min

Draft chapter under editorial review

This material is available for early reading, but it has not reached the reviewed 1.0 release. Technical references, examples, and wording may change.

Adversarial evasion is the family of attacks that produce inputs designed to make a model misclassify. The distinctive property is that the perturbation is small — often imperceptible to a human — but reliably fools a specific classifier. The mechanism is that neural networks fit high-dimensional decision boundaries that are locally continuous but globally not aligned with human perception; small directed nudges can cross the boundary without changing perceptual features.

This chapter is technical. If you have not seen gradient-based optimization before, the ideas are: for any differentiable model, we can compute how the model's output changes with respect to each input pixel/token/feature; taking small steps in the direction that maximizes loss (rather than minimizes it) reliably moves the input toward misclassification. The chapter closes with two operationally important extensions of the same idea beyond image classifiers: evasion of ML-based malware/EDR classifiers, and evasion of the lightweight guardrail classifiers that sit in front of LLMs.

10.1 Norm-bounded perturbations

An adversarial perturbation is bounded by a norm that measures its "size." The four norms that matter:

L-infinity (max) — the largest change to any single feature. Bounded by a budget epsilon, no feature may change by more than epsilon. Attacks tend to change many features by small amounts.

L2 (Euclidean) — the geometric length of the perturbation vector. Bounded by epsilon, the total change measured Euclideanly is at most epsilon. Attacks concentrate change smoothly across many features.

L1 (Manhattan / taxicab) — the sum of absolute changes across all features. Bounded by epsilon, the total absolute change is at most epsilon. Attacks concentrate change in a moderate number of features.

L0 (sparsity) — the count of changed features. Bounded by k, at most k features may be modified. Attacks concentrate all change into as few features as possible — the sparsity-attack family (Section 10.5).

The choice of norm reflects the attack's imperceptibility model. For images, L-infinity perturbations spread noise across every pixel; L0 attacks concentrate change into a few pixels. For text, L0 corresponds to changing a small number of characters; L1 corresponds to synonym substitution over the whole sentence.

10.2 FGSM (Fast Gradient Sign Method)

The foundational adversarial attack (Goodfellow et al., 2014). Given a differentiable model, an input x, and a target output y:

adversarial_x = x + ε * sign(∇_x L(model(x), y))

Read: compute the gradient of the loss with respect to the input; take its sign (which gives ±1 per feature, an L-infinity-bounded direction); scale by ε; add to the input. The result is the input pushed by ε in the direction that maximizes the loss for the label y — the direction of decreasing model confidence in the correct class.

Untargeted FGSM. Uses the loss with respect to the true label. Moves the input away from being classified correctly; the resulting class is whatever is nearest.

Targeted FGSM. Uses the negative loss with respect to a chosen wrong label. Moves the input toward being classified as the chosen wrong label.

Single-step vs iterative. FGSM is a single-step attack — take one gradient step, done. Simple, fast, low quality. Iterative variants (Section 10.3) take many small steps and are much more effective.

Preprocessing gotchas. Real models rarely feed raw pixels/tokens directly. They normalize (subtract mean, divide by std), tokenize, or embed. Gradients must be computed with respect to the input at the layer where the attacker can actually change values. If the target normalizes images before the model, the attacker must apply the inverse normalization to convert the FGSM perturbation from model-input space to raw-pixel space.

Success rate of raw FGSM is modest (30-60% on non-defended models); iterative variants push into 90%+.

10.3 I-FGSM (Iterative FGSM) and PGD

Iterative FGSM (also known as BIM, Basic Iterative Method, Kurakin et al., 2016):

for i in range(steps):
    x = x + α * sign(∇_x L(model(x), y))
    x = clip(x, x_original - ε, x_original + ε)   # stay within L-infinity ε-ball

Small step α (often ε/steps), repeated. Clipping after each step keeps the total perturbation within the L-infinity budget. Success rate against undefended models: near 100%.

Projected Gradient Descent (PGD). A generalization: multiple random restarts within the ε-ball, iterative attack from each, best result taken. Considered the standard adversarial attack for benchmark evaluation. PGD-40 (40 iterations) is a strong baseline; PGD-100 is thorough. Papers report robust accuracy under PGD-N.

Targeted variants. Same substitution as targeted FGSM — use loss with respect to the target label with a negative sign. Success rate for targeted attacks is 5-15% lower than untargeted, because the model must not only move away from the correct class but toward a specific wrong one.

10.4 DeepFool

DeepFool (Moosavi-Dezfooli et al., 2015) computes the minimum perturbation needed to cross the nearest decision boundary. Rather than take fixed-size steps, it linearizes the model at the current input, computes the exact minimal step to cross the boundary, and takes that step. Iterates until the input is misclassified.

Contrast with FGSM: FGSM applies a full ε-budget every step; DeepFool applies the minimum step needed. Result is smaller total perturbations, closer to the true decision boundary. Metric of interest is the "minimal perturbation magnitude" — how close is the input to the boundary? Smaller = more robust; larger = more robust to noise, but the model has more room around each input.

DeepFool is expensive per input (repeated linearization) but produces the tightest possible adversarial example. Useful for:

Batch generation — generate adversarial examples across a dataset, measure model's robustness distribution
Perturbation analysis — study which classes are close to which; understand the model's decision geometry

The published implementation targets L2 by default; L-infinity variant exists. Universal Adversarial Perturbations (Moosavi-Dezfooli et al., 2016) generalize DeepFool to find a single perturbation that fools many inputs — an idea that Section 10.6 extends to modern data-free and vision-language settings.

10.5 Sparsity attacks

L0-bounded attacks change only a few features while keeping the change per feature unrestricted. The perceptual model: a single-pixel change to an image is invisible to a human; a whole-image ε-noise is a blurry version of the image. Sparsity attacks are especially relevant for structured inputs (text, network data, tabular records) where "changing a few features" corresponds to a natural attack action.

10.5.1 JSMA (Jacobian-based Saliency Map Attack)

Papernot et al., 2015. Computes the Jacobian of the model with respect to the input — the matrix of partial derivatives of each output with respect to each input feature. Constructs a saliency map: for each feature, how much would changing it help toward the target misclassification. Modifies the top-saliency features iteratively.

Two subvariants:

Single-pixel JSMA — change the single highest-saliency feature at each step
Pairwise JSMA — change pairs of features together (some features are only useful jointly)

Success at L0 budgets as small as 3-5 features on MNIST; harder on more complex datasets.

Search-space growth. The Jacobian is O(features × classes). For MNIST (784 features × 10 classes), tractable; for ImageNet (150,528 features × 1,000 classes), the Jacobian is 150M entries per input. Pruning heuristics (top-K feature filtering, gradient-based candidate selection) make it practical.

Iterative cycle. Compute Jacobian → compute saliency → pick top feature → apply change → check misclassification → repeat. Terminate on misclassification or when the L0 budget is exhausted.

10.5.2 ElasticNet Attack (EAD)

Chen et al., 2018. Instead of pure L0 or pure L1, EAD optimizes a mixture: L2 for smoothness plus L1 for sparsity. Uses FISTA (Fast Iterative Shrinkage-Thresholding Algorithm) with proximal operators to efficiently optimize the mixed objective.

The FISTA setup:

Adversarial loss — Carlini-Wagner-style formulation (f(x') = target-class logit minus max other logit, or similar)
Distortion regularization — β * L1 + λ * L2^2
Proximal operator — soft-thresholding for L1, standard for L2

The optimization iterates:

z = x' - α * ∇(f + λ*L2)   # gradient step
x' = shrink(z, α*β)         # L1 proximal (soft-thresholding)

Binary search over c (a coupling constant between adversarial loss and distortion) finds the smallest distortion that achieves misclassification.

Sparsity outcomes: 10-20% of features changed on typical images; comparable to JSMA in L0 but often better in perceptual quality.

10.5.3 Perturbation analysis

Sparsity attacks give a discriminating view of model robustness. A model that requires 3 pixel changes to be fooled is less robust than one requiring 30. Practical use:

Comparing two candidate models for deployment
Identifying which classes are close in decision space (the "perturbation distance" from A to B is a similarity measure)
Localizing which features are most sensitive (which pixels/tokens carry model decisions)

10.6 Universal adversarial perturbations, 2025 update

Universal Adversarial Perturbations (Section 10.4) generate a single, instance-agnostic noise pattern that fools a model across many different inputs. Two 2025 lines of work extend this considerably:

Data-free UAPs. A 2025 method generates universal perturbations without requiring any training data — the perturbation is built starting from pure noise, recursively extracting "pseudo-semantic priors" from the UAP during its own training process. This achieves state-of-the-art average fooling rate and improved cross-architecture transferability among CNNs, outperforming even prior UAP methods that do depend on training data. Operationally, this removes "the attacker needs a representative sample of the training distribution" as a precondition for building a universal, reusable perturbation against a target.

UAPs against vision-language models. Work presented at ICCV 2025 extends universal adversarial perturbations to vision-language pretraining (VLP) models, demonstrating that these multimodal models are equally susceptible to instance-agnostic perturbations as single-modality vision classifiers. A single crafted perturbation pattern, once found, degrades a wide range of downstream vision-language tasks without per-input optimization — a meaningful escalation given how much production tooling now routes images through multimodal LLMs rather than dedicated vision classifiers.

10.7 Text-specific evasion

For text inputs, "small perturbation" is discrete — replacing a word, adding a character, changing case, inserting invisible unicode. Adversarial NLP techniques:

GoodWords attack. For spam classifiers and other binary text classifiers, adding "good words" (tokens strongly associated with the non-spam class) to a spam message shifts the classifier's decision toward non-spam. The attack does not remove spam content — it dilutes it. Success rate high against Naive Bayes and logistic regression classifiers; moderate against modern transformers.

Black-box GoodWords. When the classifier is not accessible for gradient computation, discovery uses query-based methods:

Adaptive discovery. Query the classifier with candidate words; measure impact on classification confidence; retain high-impact words as "good."
Genetic-algorithm search. Population of candidate augmentations; fitness = misclassification rate; standard mutation and crossover.
Substitute-model attack. Train a local surrogate on labeled examples elicited via query; run white-box GoodWords on the surrogate; transfer to target.

Character-level attacks. Insert typos, homoglyphs, or unicode variants that preserve meaning to a human but confuse tokenization or embedding lookup. DeepWordBug (Gao et al., 2018) systematizes character perturbation to maximize model impact per edit.

Character injection against LLM guardrails. A 2025 study (arXiv 2504.11168) demonstrates that inserting invisible Unicode characters, zero-width joiners, and homoglyphs — text that a guardrail classifier does not recognize as meaningfully different but that the target LLM's tokenizer still resolves to the original semantic content — achieves up to 100% evasion against some tested guardrails, including commercial systems such as Azure Prompt Shield and Meta Prompt Guard. This is the modern, LLM-guardrail-specific descendant of DeepWordBug-style character perturbation: instead of fooling a spam filter's bag-of-words features, it fools a moderation classifier that is itself just another neural text classifier subject to the same norm-bounded evasion logic as everything else in this chapter. A companion technique in the same study — algorithmic, imperceptible perturbations computed via word-importance ranking against a white-box offline surrogate and then transferred to a black-box target — achieves high attack success against BERT/DeBERTa-class detection systems, confirming that classical white-box-to-black-box transfer (Section 10.8) applies directly to guardrail classifiers, not only to image models.

Adversarial paraphrasing. Rewrite the input to preserve meaning while shifting classification. Effective when the target uses simple bag-of-words features; less effective against embedding-based classifiers that generalize across paraphrases.

Prompt-level evasion. For LLM classifiers that classify by prompting a base model, wrap the input in a fictional framing ("consider the following as a story about a person receiving a spam email") that shifts the target's classification without changing the underlying content.

10.8 Transferability

An adversarial example crafted against model A frequently succeeds against model B if B is trained on similar data or shares architecture. This is the transferability phenomenon and it is the reason black-box attacks work: an attacker without direct access to the target trains a surrogate model with similar architecture, generates adversarial examples against the surrogate, and applies them to the target.

Practical rules of thumb:

Same-architecture transfer is very high (>80% for FGSM/I-FGSM)
Cross-architecture transfer is moderate (30-70%)
Cross-task transfer is low (<20%)
Ensemble crafting (generate adversarials effective against several surrogates) increases transfer to the actual target

10.8.1 Is white-box robustness a good proxy for black-box robustness?

This question matters directly for scoping an engagement: if a target can only be assessed as a black box, is a public white-box robustness benchmark for a similar model still informative? RobustBlack (2024-2025 study) evaluated advanced black-box transfer attacks — LGV, BASES, TREMBA — and found they struggle to succeed even against models with only simple adversarial training, and that models optimized for white-box robustness against AutoAttack also exhibit improved resilience against black-box attacks. The practical conclusion is a qualified yes: white-box robustness (Section 10.8.2) is a reasonable proxy for black-box robustness, which is useful when a target's model card cites a RobustBench leaderboard position but you only have query access.

10.8.2 Standardized benchmarking: RobustBench and AutoAttack

RobustBench remains the reference benchmark for adversarial robustness in image classification, using AutoAttack — an ensemble of white-box and black-box attacks (multiple PGD variants with different loss functions plus a black-box Square Attack) — as the standardized evaluation method, with active leaderboards for CIFAR-10, CIFAR-100, and ImageNet under L-infinity and L2 norms. The leaderboard covers more than 120 models and offers unified access to more than 80 robust models through its Model Zoo. Current top-performing models reach approximately 60% robust accuracy under AutoAttack at L-infinity=8/255 on CIFAR-10 — a figure some researchers consider "unrealistically strong" compared to real-world threats, since it produces near-perfect detection rates for adversarial examples even with simple detection algorithms. When benchmarking a target's own robustness claims, treat a reported AutoAttack score as an upper bound on real-world resilience, not a floor — production inputs and attacker knowledge rarely match the benchmark's clean assumptions.

10.9 Evasion of ML-based malware and EDR classifiers

Everything in this chapter about norm-bounded perturbations and query-based black-box search applies directly outside image classification, to one of the highest-value targets in a red-team engagement: machine-learning-based malware and EDR detection.

Mechanics. Attackers modify PE headers, API call sequences, or PDF/document structures so the underlying malware executes unmodified while the ML classifier's verdict flips from malicious to benign. This is functionally identical to the FGSM/PGD/CW attacks earlier in this chapter, just applied to a malware feature space (for example, against the open EMBER feature representation) instead of pixels.

White-box. Full access to the classifier's architecture and weights allows precise gradient-based perturbation of the feature vector — the direct analogue of Sections 10.2-10.5 applied to PE-header and import-table features rather than image channels.

Black-box / query-based. Without model internals, an attacker submits samples, observes the classifier's verdict, and iterates — a query-based search analogous to the black-box GoodWords methods of Section 10.7, adapted to binary file features. This is the practically relevant case against commercial ML-augmented engines, where the attacker only has query access.

Defensive implication. When detection based purely on ML fails, there is no alert, no log entry, and no SOC ticket — the attacker proceeds to post-exploitation with zero visibility for the defender. This is a materially different failure mode from a signature-based engine missing a sample: an ML classifier that is evaded produces a confident, silent "benign" verdict rather than a near-miss. Effective mitigation requires layering behavioral detection, adversarial training of the classifier itself, input validation, and human review — no single layer is sufficient on its own, mirroring the defense-in-depth conclusion from image classifiers in Section 10.10.

10.10 Evasion of LLM guardrails treated as classifiers

Lightweight guardrails ("prompt guards") — small classifiers that filter malicious inputs before they reach the main LLM — present a structural attack surface: they are themselves classifiers, and classifiers are the subject of this entire chapter.

Controlled-release prompting (2025). This attack exploits the resource asymmetry between the guardrail (a lightweight model) and the main LLM (a much more capable one). The method encodes a known jailbreak in a way the lightweight guardrail cannot decode, but the more capable main model can — evading the input filter and activating the jailbreak by reference, similar in spirit to indirect prompt injection. It has been demonstrated successfully against Google Gemini 2.5 Flash/Pro, DeepSeek Chat (DeepThink), Grok 3, and Mistral Le Chat (Magistral), while preserving the quality of the resulting harmful response. The underlying principle generalizes beyond this specific technique: any two-tier defense where the front-line filter is meaningfully weaker than the model it protects has an exploitable capability gap, and an attacker's job is to find an encoding that survives that gap.

GCG resurgence, 2025 findings. A systematic 2025 re-appraisal of Greedy Coordinate Gradient (GCG) and its annealed variant (T-GCG) against open-source LLMs of varying scale produced three findings relevant to test planning:

Attack success rate decreases with model size, reflecting more complex, non-convex loss landscapes in larger models — meaning GCG-style suffix attacks should be budgeted as a smaller-model technique first, with escalating cost against frontier-scale targets.
Prefix-based heuristics substantially overestimate attack effectiveness relative to semantic evaluation using an LLM judge (for example, GPT-4o) — a reminder to score jailbreak success semantically, not by string-matching a refusal prefix.
Code-related prompts are significantly more vulnerable than general-safety adversarial prompts, suggesting that the model's own reasoning process over code can be exploited as an attack vector — worth prioritizing when the target is a coding assistant or has tool-calling capability tied to code generation.

10.11 Defense encounters

Common defenses and their bypass patterns:

Adversarial training. The model is trained on adversarial examples generated during training. Substantially raises robust accuracy against known attack types. Bypass with attacks the model was not trained on (different norm, iterative variants, higher ε).

Gradient masking. The model is designed to produce non-informative gradients (defensive distillation, quantization, non-differentiable preprocessing). Foils gradient-based attacks. Bypass with black-box attacks (surrogate + transfer) or with backward-pass differentiable approximation (Athalye et al., 2018).

Input transformations. JPEG compression, bit-depth reduction, spatial smoothing before classification. Removes small perturbations. Bypass by including the transformation in the attack's differentiable path (BPDA, EOT).

Randomized smoothing. Add Gaussian noise at inference; classify by majority. Provides certified robustness against L2 attacks below the noise scale. Bypass with larger perturbations exceeding the certificate.

Certified defenses (interval bound propagation, randomized smoothing). Provide provable guarantees within a specific budget. Bypass with attacks exceeding the certified budget or with attacks on different threat models.

10.12 Practice checklist

Identified whether the target is differentiable (white-box) or opaque (black-box)
For white-box: chose norm (L-infinity, L2, L1, L0) matching the attack's imperceptibility model
Started with FGSM as sanity check; escalated to I-FGSM/PGD for reliability
For minimum-perturbation analysis: applied DeepFool
For sparsity-critical inputs (text, tabular): applied JSMA or ElasticNet
Considered data-free UAPs and vision-language UAPs where a single reusable perturbation is valuable
For text targets: considered GoodWords, character-level/character-injection, and paraphrasing evasion of guardrail classifiers
For black-box: trained surrogate, generated adversarials, tested transfer; cited RobustBench/AutoAttack robustness as a proxy where full white-box access is unavailable
For malware/EDR-adjacent targets: assessed PE-header / feature-space evasion and confirmed whether an ML-only verdict flip produces zero SOC visibility
For guardrail-protected LLM targets: assessed controlled-release prompting and the resource asymmetry between guardrail and main model
If using GCG-style suffix search: budgeted for lower ASR against larger models and scored success semantically, not by prefix matching
Documented model's robust accuracy under multiple attacks (not just one)

MITRE ATLAS references

ID	Technique
AML.T0015	Evade ML Model
AML.T0043	Craft Adversarial Data
AML.T0043.000	Craft Adversarial Data: White-Box Attacks
AML.T0043.001	Craft Adversarial Data: Black-Box Attacks
AML.T0043.002	Craft Adversarial Data: Transferability