Don't Touch the Model. Just Add Words Until It Flips.
GoodWords Evasion Against a Naive Bayes Spam Classifier. White-Box and Black-Box.

Lab Setup

Both labs run locally in the conda ai environment. No SSH tunnel or target machine is required for this module.

conda activate ai
python3 -m pip install scikit-learn numpy requests

The Module

The earlier modules on the AI Red Teamer path covered attacks that operate before or during deployment: corrupting training data, extracting models, injecting into prompts, exploiting what LLMs output. This module focuses on a fundamentally different threat model. Evasion attacks require none of that access. They work against a deployed model that is functioning correctly, by manipulating the input at inference time to cause a misclassification the attacker wants.

The critical distinction from data attacks is persistence. A successful data poisoning attack corrupts the model permanently; every inference carries the attacker's influence until the training data is cleaned and the model retrained. An evasion attack has no persistence at all. It produces one misclassification on one input. The model is correct on every other input. The attacker must craft an evasion payload for each query they want to influence.

That constraint makes evasion attacks easier to deploy in some ways: no access to training infrastructure is required, and the attack leaves no trace in model weights or training data. It makes them harder in others: every payload must be crafted, and the model's behaviour on all other inputs remains unchanged and correct.

How Naive Bayes Spam Classifiers Work

Understanding the GoodWords attack requires understanding what Naive Bayes is actually computing. Naive Bayes is a probabilistic classifier. For each word in the vocabulary, it has learned a probability: how likely is it that a message containing this word is spam, and how likely is it that a message containing this word is ham. These are the class-conditional word probabilities, stored as log probabilities for numerical stability.

To classify a new message, the classifier multiplies the prior probability of each class by the product of the word probabilities for every word in the message. Whichever class produces the higher probability wins. Because word probabilities multiply together, every word in the message contributes to the final score. A word that is strongly associated with ham pulls the score toward ham. A word that is strongly associated with spam pulls the score toward spam.

from sklearn.naive_bayes import MultinomialNB

# after training, the classifier stores per-feature log probabilities
# feature_log_prob_ shape: (n_classes, n_features)
# class 0 = ham, class 1 = spam

# for each word in vocabulary:
# feature_log_prob_[0][word_idx] = log P(word | ham)
# feature_log_prob_[1][word_idx] = log P(word | spam)

# classification = argmax over classes of:
# log P(class) + sum(log P(word | class) for word in message)

The GoodWords evasion strategy follows directly from this: find words where log P(word | ham) is much higher than log P(word | spam). Adding those words to a spam message shifts the cumulative probability score toward ham without changing the spam content. If enough high-ham-probability words are added, the sum tips past the decision boundary and the message is classified as ham.

Lab 1: GoodWords Challenge

The Goal

Take a spam message that the classifier correctly identifies as spam and inject words that flip the classification to ham, without access to the model's internals. The classifier's API accepts a message and returns a class label and confidence score. The attack must use only that feedback.

Technique: Greedy Word Injection

Build a candidate vocabulary of words that plausibly shift classification toward ham. For each candidate word, measure the impact on the spam probability score by querying the API with and without the word added. Rank words by their probability delta. Add the highest-impact word to the message, measure the new score, then repeat until the classifier flips to ham.

import requests

TARGET = "http://127.0.0.1:5000/predict"
SPAM_MESSAGE = "WINNER!! Claim your FREE prize now. Call 0800 FREE."

candidate_words = [
    "meeting", "please", "attached", "regards", "project",
    "schedule", "team", "update", "report", "review",
    "invoice", "document", "agenda", "discussion", "follow"
]

def get_spam_prob(message: str) -> float:
    resp = requests.post(TARGET, json={"message": message})
    return resp.json()["spam_probability"]

def greedy_evasion(message: str, words: list, max_rounds: int = 20) -> str:
    current = message
    for _ in range(max_rounds):
        prob = get_spam_prob(current)
        if prob < 0.5:
            print(f"[+] Flipped to ham: {prob:.4f}")
            return current

        best_word, best_prob = None, prob
        for word in words:
            candidate = current + " " + word
            p = get_spam_prob(candidate)
            if p < best_prob:
                best_word, best_prob = word, p

        if best_word is None:
            break  # no improvement from remaining candidates

        current = current + " " + best_word
        print(f"    Added '{best_word}': spam prob now {best_prob:.4f}")

    return current

evaded = greedy_evasion(SPAM_MESSAGE, candidate_words)
print(evaded)

Why Greedy Works

Naive Bayes is additive in log space. Each word's contribution to the classification score is independent of every other word. This is the "naive" assumption: word probabilities are treated as conditionally independent given the class. The independence assumption means the impact of adding a word to a message is approximately the same regardless of what other words are already present. Greedy selection is near-optimal for this classifier because there is no interaction term to account for.

On classifiers that capture word interactions, such as n-gram models or neural classifiers, greedy selection is a heuristic rather than a near-optimal strategy. The optimal evasion payload requires searching a combinatorially larger space. Greedy still works as a baseline but will require more injected words to achieve the same flip.

Constraint: Semantic Preservation

The practical requirement in most evasion contexts is that the payload must not obviously corrupt the meaning of the original content. Injecting "meeting please regards schedule team" into a spam message produces a message that looks semantically incoherent to a human reader. In a real deployment this might trigger a different detection layer or a human review process. Effective evasion selects words that are contextually plausible, limiting the candidate vocabulary to words that could appear in a legitimate message of the same apparent type.

Skills Assessment: White-Box and Black-Box Evasion

The Scenario

The skills assessment requires evading the same spam classifier twice: once with full access to the trained model object (white-box), and once with only an API endpoint that returns classification probabilities (black-box). Both approaches must flip the same target spam message to ham. The white-box solution should be optimal or near-optimal. The black-box solution is necessarily approximate and query-intensive.

White-Box: Log Probability Ranking

With access to the trained MultinomialNB object, the feature log probabilities are directly available. For each word in the classifier's vocabulary, compute the difference between its ham log probability and its spam log probability. Words with the highest positive difference contribute most to a ham classification relative to spam. Sort and inject in rank order until the classification flips.

import numpy as np

# clf is the trained MultinomialNB, vectorizer is the fitted TfidfVectorizer
vocab = vectorizer.get_feature_names_out()

# log P(word | ham) - log P(word | spam)
# high positive value = word strongly favours ham over spam
ham_idx, spam_idx = 0, 1
log_diff = (clf.feature_log_prob_[ham_idx] -
            clf.feature_log_prob_[spam_idx])

# rank all vocabulary words by discriminative power for ham
ranked = sorted(zip(vocab, log_diff), key=lambda x: x[1], reverse=True)
good_words = [w for w, _ in ranked[:50]]  # top 50 ham-discriminating words

# inject greedily until classification flips
message = SPAM_MESSAGE
for word in good_words:
    message = message + " " + word
    pred = clf.predict(vectorizer.transform([message]))[0]
    if pred == 0:  # 0 = ham
        print(f"[+] Flipped after adding '{word}'")
        break

The white-box approach is efficient because it ranks the entire vocabulary in a single pass over the log probability arrays. No repeated queries are needed. The optimal injection word is always chosen first, minimising the number of words added to achieve the flip.

Black-Box: API-Based Word Impact Estimation

Without model access, the feature log probabilities are not available. The black-box approach approximates them through query feedback. For each candidate word, query the API twice: once with the current message and once with the word added. The difference in spam probability is the word's estimated impact. Greedy selection picks the highest-impact word at each round.

import requests

TARGET = "http://127.0.0.1:5000/predict"

def spam_prob(message: str) -> float:
    return requests.post(TARGET, json={"message": message}).json()["spam_probability"]

def black_box_evasion(message: str, candidates: list) -> str:
    current = message

    while spam_prob(current) >= 0.5:
        baseline = spam_prob(current)
        impacts = []

        for word in candidates:
            trial = current + " " + word
            delta = baseline - spam_prob(trial)  # positive = reduces spam prob
            impacts.append((word, delta))

        impacts.sort(key=lambda x: x[1], reverse=True)
        best_word = impacts[0][0]

        if impacts[0][1] <= 0:
            break  # no candidate improves the score

        current = current + " " + best_word
        print(f"    Added '{best_word}': spam prob now {spam_prob(current):.4f}")

    return current

Query Cost

The black-box approach queries the API once per candidate word per round. With a candidate vocabulary of 50 words and 10 rounds of injection, that is 500 queries plus the baseline checks, roughly 520 total. The white-box approach required zero API queries for ranking and one query per injected word to verify, roughly 10 total. The query cost ratio is the fundamental difference between white-box and black-box evasion.

In a real deployment, high query volumes against a classification API trigger rate limiting and anomaly detection. An attacker in a black-box setting must balance injection quality against query volume. Strategies to reduce queries include pruning the candidate vocabulary to a small set of high-confidence ham words before the attack starts, using binary search-style probing instead of exhaustive ranking, and caching previous round results to avoid re-probing words that showed no impact.

Key Takeaways

Repository

Notebooks and scripts from this module are in the htb-ai-red-teamer repository.