Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
Authors: Samuel J Paech
Allen G Roush
Judah Goldfeder
Ravid Shwartz-Ziv
Venue: ICLR 2026 Poster (2026)
Keywords: AI slop
slop
constrained generation
delve
patterns
sampleing
dpo
preference optimization
fine-tuning
fine tuning
creativity
AI writing
Creative AI
Primary area: foundation or frontier models, including LLMs
TL;DR: We show several techniques for removing characteristic patterns from LLM generated texts at both the sampler level and at the model weights level.

Abstract: Repetitive lexical patterns in LLM output, termed "slop," degrade writing quality through over-use and make AI-generated text immediately recognizable. We present Antislop, a comprehensive framework providing tools to both detect and eliminate these overused patterns. Our approach combines three innovations: (1) The Antislop Sampler, which uses backtracking to suppress unwanted strings at inference time without destroying vocabulary. (2) An automated pipeline that profiles model-specific slop against human baselines and generates training data. and, (3) Final Token Preference Optimization (FTPO), a novel fine-tuning method that operates in logit-space on individual tokens, surgically adjusting logits wherever a banned pattern has appeared in an inference trace. We demonstrate that some slop patterns appear over 1,000 times more frequently in LLM output than human text. The Antislop Sampler successfully suppresses 8,000+ patterns while maintaining quality, whereas token banning becomes unusable at just 2,000. Most importantly, FTPO achieves 90% slop reduction while maintaining or improving performance in cross-domain evals including GSM8K, MMLU, and creative writing tasks. In contrast, DPO suffers significant degradation in writing quality and lexical diversity despite achieving weaker suppression. We release all code and results datasets under MIT license.

Published as a conference paper at ICLR 2026
ANTISLOP: A COMPREHENSIVE FRAMEWORK FOR IDENTIFYING AND ELIMINATING REPETITIVE PAT- TERNS IN LANGUAGE MODELS
Samuel J Paech'
Allen G Roush²
Liquid AI
Thoughtworks
spaech@gma warcch@@@aal..coomm com
a allen. hollenn.cuuttthhcugtttorssss
Judah Goldfeder
Ravid Shwartz-Ziv*
Columbia University
New York University
jag2396 umbia.edi edu
ravidzivegmail.com
ABSTRACT
Repetitive lexical patterns in LLM output, termed "slop." degrade writing qual- ity through over-use and make Al-generated text immediately recognizable. We present Antislop, a comprehensive framework providing tools to both detect and eliminate these overused patterns. Our approach combines three innovations: (I) The Antislop Sampler, which uses backtracking to suppress unwanted strings inference time without destroying vocabulary; (2) An automated pipeline that profiles model-specific slop against human baselines and generates training data; (3) Final Token Preference Optimization (FTPO), a novel fine- sne-ttning method that operates individual tokens, surgically adjusting logits wherever banned pattern has appeared in an inference trace. We demonstrate that some slop pat- terns appear over 1,000 more frequently in LLM output than human text. The Antislop Sampler successfully suppresses 8,000+ patterns while maintaining qual- ity. whereas token banning becomes unusable at just 2,000. Most importantly. FTPO achieves 90% slop reduction while maintaining or improving performance in cross-domain evals including GSMSK, MMLU, and creative writing tasks. contrast, DPO suffers significant degradation in writing quality and lexical diver- sity despite achieving weaker suppression. We release all code and results datasets under MIT license]
INTRODUCTION
Language models have ushered in of slop: Repetitive words and phrases that instantly recognizable as Al generated text(Wu et al. 2025). In creative writing, the ubiquitous Elara always speaks with "voice barely above whisper". In functional writing, patterns appearing everywhere; far more often than they would in human writing. In our tests, we find that these patterns occur thousands of times more frequently LLM text than in human writing, leading to the perception of repetition and over-use i.e. slop.
"it's not just X, it's Y
Existing approaches suppress unwanted patterns are brittle or ineffective. Token banning creates collateral damage- for instance, if we wish to ban "catatonic" and it tokenizes to ["cat", "atonic"], we will have banned all words that tokenize firstly to "cat". Instructing the model to avoid set of banned vocabulary has limited efficacy and may induce backfire effect due to the "Pink elephant problem" (Castricato al. 2024).
We present the Antislop Sampler: it detects unwanted patterns during generation words, phrases, and regex patterns then backtracks to the pattern's 's first token, reduces its probability, and resam- ples. Our sampler can suppress 8,000 patterns with configurable strength (from soft discouragement to hard banning), without degrading output.
To train slop suppression into the model, present Final Token Preference Optimization (FTPO). a training algorithm designed to surgically suppress slop with minimal collateral damage to
https://github: com/ http:///sppccc/aaooaaiiiaaaallla

---

Published as a conference paper at ICLR 2026
the model. Teaching model to disprefer its most preferred tokens requires large logit adjustments, which can damage the model. Our FTPO trainer implements several "soft-touch" mechanisms to minimize deviations from the reference weights. We measure substantial improvements over DPO and token banning on banlist suppression rates, lexical diversity and impact on writing quality.
We release all code and results datasets under MIT license 2
2 RELATED WORK
Degeneration in text outputs was highlighted by Holtzman al. (2020), who showed that maximum- likelihood decoding (e.g. beam search) can lead to bland, looping text. Stochastic decoding strate- gies like top-k, top-p (nucleus sampling), and min-p (Nguyen et al. 2025) have since been adopted to increase diversity and reduce incoherent outputs. Nonctheless, these strategies do not directly address repetitive writing tendencies in otherwise coherent outputs. Studies have found that rein- forcement learning from human feedback (RLHF) can significantly reduce output diversity com- pared to a supervised baseline (Kirk et al.) 2024), and similar effects have been documented for other alignment fine-tuning methods (O Mahony et al. 2024, Murthy et al.) 2024). Even the use of rigid chat-format templates can suppress creativity, phenomenon dubbed diversity collapse in LLMs (Yun al. 2025).
Several recent samplers attempt to improve creativity and diversity while suppressing repetition. XTC (Exclude Top Choices) removes the current highest-probability tokens above a threshold (Wei- dmann 2024b), and DRY (Don't Repear Yourself) prevents repetition of sequences that have already occurred verbatim in the text multiple times [Weidmann] 2024a). These methods encourage selec- tion of lesser-used continuations and reliably prevent local looping, but in our experiments they leave the global over-representation profile of words and trigrams essentially unchanged (App. N ExL- lama implements a string banning feature similar to our backtracking mechanism which hard-bans a provided set of strings at inference time (Turboderp] 2024).
Beam-search methods exclude forbidden words or phrases by pruning any beam that would produce them. Efficient variants use tries and a fixed beam budget enforce both positive and negative constraints (Hu et al., 2019). A recent benchmark compares decoding-time and training -ime ap- proaches, and notes that models can still slip around bans with small spelling changes or closely related word forms; they also test simple fixes to reduce this (Jon et al. (023).
similar approach by Zhang et al. (2025) trains a model to deploy a special [RESET] token when unsafe content is detected in the inference trace, triggering backtracking and a retry of the current sentence. Work by Roush et al. (2022) further explored lexical filtering at inference time. Their plug-and-play method enforced constraints (such as omitting the letter in lipogram) without fine-tuning the model.
Welleck et al. 2020) introduced an unlikelihood training objective that penalizes undesirable tinuations (c.g., repeated tokens or n n-grams) by adding a negative log- pooo-oaabilltyy term for explicitly marked "negative" events, and Li et al. (2020) extended this idea to dialogue generation. In this formulation, the core mechanism is a generic loss on disfavored tokens, while two key ingredients left open: how construct the dataset of negative events (which contexts and spans should be treated as slop, repetition, or otherwise undesirable), and what complementary "positive" objective should be paired with the unlikelihood term (e.g., standard MLE, task-specific supervision, or pref- erence data). Our methods can be viewed as making concrete choices for both of these aspects in the context of over-represented stylistic patterns in LLMs.
Our work closely connects to
methods like Direct Preference Optimization (Rafailov al. 2023), which align the model on preference pairs without relying on reward models. However, DPO has known failure modes, including lowering the likelihood of preferred responses, inducing diversity collapse and reducing syntactic and n- n-gram variety in outputs (Razin al. [204]]
Lanchantin et al. 2025 Shypula et al.] (022). To counter this, FTPO uses multi-term regularization
similar to RLHI'S S KL penalty (Stiennon al. 2020).
2

---

Published as a conference paper at ICLR 2026
3 FORENSIC ANALYSIS OF OVER-REPRESENTED PATTERNS
3.1 QUANTIFYING SLOP
To identify over-used patterns in LLM outputs, we analyze the statistical overrepresentation of words, bigrams (n 2) and trigrams (n = 3) versus human text. We limit our analysis to n S 3 due to practical constraints: with n 2 4, patterns typically appear fewer than 5 times across our 2,000 generated samples.
For each model, we generate 2,000 outputs using creative writing prompts from Reddit (NirralAA] 2024 and compute frequency ratios:
p(p) fLLM
fhuman
(p)
where fLLM (p) and fhamen (p) represent the frequencies of pattern p LLM and human corpora respectively. Our human baseline combines wordfreq (Speer et al. (018) for individual words and a curated corpus Reddit creative writing and Project Gutenberg texts for n-grams. For n- n-gram processing, we remove stop-words. We treat pattern p as over-represented when its ratio exceeds 1.0, adding the most over-vepresented subset of these to the banlist.
By collating list of the highest over-represented words and n-grams, we produce "slop finger- print" of the model's unique tendencies.
3.2 EMPIRICAL FINDINGS
Table reveals the severity of the problem. With genma 1aruma----11 3-12b, certain patterns show extreme overrepresentation:
Word
Ratio
Trigram
Ratio
elara
85,513x
heart hammered ribs
1, 1,192x
unsettlingly
3,833x
voice trembling slightly
731x
shimmered
2,882x
said voice devoid
693x
stammered
2,043x
felt profound sense
550x
Table 1: Top overrepresented patterns in gemna-3 3-12bb outputs, and their frequency ratio relative to human baseline.
The name "Elara" appears 85,513 times more frequently in gemma-3-aa creative writing out- puts than in human text, while the trigram "heart hammered ribs" shows 1,192x overrepresentation. We find similar ratios of over-use other models tested (Mistral-small-a Slop fingerprints cluster strongly within model families, but differ substantially between model fam- ilies (Appendix K). warranting a model-specific approach to slop identification and suppression.
and Llama-3-3-70b)
Our analysis reveals several distinct categories of slop. Models fixate specific character names ("Elara", "Kael"), sensory clichés ("voice barely above a whisper"), intensifiers ("a profound sense") and go-to set of overused descriptives ("unsettlingly", "shimmered"), We also count sentence-level constructions of the form "It's not X, it's Y" to be 6.3x more prevalent than human writing some models (Figure 7)
THE ANTISLOP SAMPLER
The Antislop Sampler provides inference-time suppression of unwanted patterns. It can suppress individual words ("tapestry"). multi-word phrases ("voice barely above a whisper"), and complex patterns defined by regular expressions ("It's not X, it's Y"). Unlike token banning, which triggers on the first token of a banned sequence and is prone to false positives, our sampler waits until the entire sequence appears in the inference trace before triggering a ban.
3

---

Published as a conference paper at ICLR 2026
barned pattern is detected (string. n-gram triggerin backtracking
Elara called out, her voice barely above a whisper
Elara called out, her v Cheart ]warning]
Apply logprab penalty first takan in the banned string and resample from
[meeart iwarming [voice]
fittarig)
3
Elara called out, her heart...
Inference continues.
Figure 1:
The Antislop backtracking mechanism detects unwanted patterns in the inference trace, backtracks to the first token of the banned sequence, lowers its probability, then resamples.
4.1 BACKTRACKING MECHANISM
During generation, maintain trace of all tokens and their logit distributions. After each new token (or chunk of inference), we sean for banned patterns. When detected, we backtrack to the position where the pattern began and lower the initiating token's probability by: Pnew Potd 10-10s where 0 ك < 1.0 is the configurable baanstrees rength parameter. We then renormalize the modified distribution so that it again sums to one. P = pi/ Σ; Pj. and resample using min-p filtering with fixed threshold of 0.1 constrain the distribution coherent candidates. If, after its probability is reduced, the same token is sampled again, the sampler ignores this violation in future checks avoid infinite loops. This ability allow banned patterns through if they high enough probability is a key part of our implementation, which we term "soft- noft-banning".
Algorithm Antislop Backtracking
1: while generating tokens do
2: generate token
3: if bannedipattern detected then
backtrack to pattern start
5: reduce probability
6: resample with min-p
end if
8: end while
4.2 SOFT BANNING: CONFIGURABLE SUPPRESSION STRENGTH
Imposing a hard-ban on a word or pattern can cause problems with coherence when there no good alternatives. Our soft-banning mechanism provides incremental control through the ban-strength parameter s. When s 0, patterns are allowed freely. Values between 0 and provide incremental suppression of the banlist, while
enforces complete blocking.
For example, this approach allows us to generally suppress the word "tapestry" while still per- mitting its use when directly requested in the prompt: "Write an essay about tapestries". At ban-strength 1. O, banned patterns are still allowed through when their probability is high enough compared the next highest token. See Appendix A for a worked example.
4.3 IMPLEMENTATION AND LIMITATIONS
The sampler is implemented two ways: single-threaded HuggingFace Transformers with stream- ing support, and a higher- throughut multithreaded OpenAl-APP-compatiblee version for production inference platforms like vLLM wwon et al. 2023)
The sampler suppresses patterns without fine-tuning but reduces throughput. Each backtracking event restarts inference at prior position, and this may occur hundreds of times per generation with large banlists. In practice, this reduces throughput by 69% up to 96% in worst cases, depending on banlist size (detailed performance analysis in Appendix Bb. For applications requiring maximum

---

Published as a conference paper at ICLR 2026
inference speed, this overhead motivates complementary approach: using the sampler's outputs to train permanently improved models via FTPO.
5 FINAL TOKEN PREFERENCE OPTIMIZATION (FTPO)
We develop Final Token Preference Optimization (FTPO), training method that permanently sup- presses unwanted patterns with minimal degradation model output. Suppressing slop is nontrivial because it requires large updates to the model's ' most preferred patterns, reducing their probabil- ity until other continuations are preferred. These large shifts can easily damage the model, leading to degradation or model collapse. Our trainer approaches this delicate procedure by incorporating several strategies to constrain logits to the reference, while avoiding collateral damage.
FTPO trains just a single continuation token at the end of incomplete inference trace. A final-token preference pair consists of three parts:
(1) The prompt, including the chat template and the model's response up to the point a banned sequence appeared.
Prompt: "# User: Write me a story. # Assistant: Once upon a time, Princess"
(2) A single rejected continuation token, corresponding to the first token of the banned sequence. Rejected: "Elara"
(3) A set of chosen coherent alternative continuation tokens. Chosen:
["Madelyne", "Nadia", "Freya", "Isolde"]
In our implementation, each final-token prefer- ence pair is instantiated at a single backtrack- ing event of the Antislop sampler: we cache the top-k logits at that position (with k = 20), remove the rejected token from this candidate set, renormalize over the remaining tokens, and sample 4-8 distinct, high-probability alterna- tives to form the chosen set C.
Identify overused patterns in baseline model
"The sky was lapestry of color"
Rejected: "tapestry"
Generate preference dataset with Antislop Sampler
Chosen: ["cascade", "brilliant", "crimson"]
The sky was Ilapestry foascade
5.1 LIMITATIONS
FTPO trains model to suppress slop and prefer coherent alternative tokens with minimal collateral damage
OF DIRECT PREFERENCE OPTIMIZATION
tbrilliant
Direct Preference Optimization (DPO) (Rafailov et al.] 2023] can also train final-token pairs to suppress slop like FTPO. However, DPO is limited to updating single chosen token per training sample, unlike FTPO which update set of preferred tokens in one step.
tcrimson
=purple
=bagel
Figure 2: Pipeline for identifying and suppressing overused writing patterns in language model.
More importantly, DPO's primary hyperparameter for constraining updates (8) is coarse tool, impairing learning at high values and causing model degradation by allowing large logit movements low values (Wu al. 2024).
5.2 THE FTPO FORMULATION
FTPO implements several mechanisms constrain logits reference, with a two-part regulariza- tion allowing larger shifts for chosen and rejected logits, relative to the remaining vocab. The loss function is formulated such: At the final position in the inference trace, define token (rejected) and chosen alternatives C. We optimize three loss objectives:
Preference loss with margin. We enforce that chosen tokens exceed the rejected token's logit by margin m:
Ecec We softplus((m
(m--))//))) Ac)/T)
Eares
EeEo We
5

---

Published as a conference paper at ICLR 2026
where A. y/cl y/rl is the logit gap between chosen and rejected, and the weight we c( A.)/m, O, 1) deactivates the loss when the margin is achieved (Figure I4).
Target regularization. We tether chosen and rejected ("target") logits to reference values frref from the frozen base model at the same position, calculating MSE loss directly on logit deltas (not logprobs). A zero-penalty window
Learget
Σ JET
max( - yres [i]l Tharget: 0)²
[T]
where 7' = C' {r} cache Yref during the data getteaannration step and reuse these cached logits during FTPO training.
contains all target tokens. To avoid redundant forward passes, we compute and
Non-target regularization. We strongly anchor the remaining vocabulary to prevent distribution drift:
Luontarget
Σ
(ubi) yres Lil)
(3)
I
JEN
where N represents all non non-target tokens.
The total loss, incorporating weighting coefficients Aaagge and Anontargeti
LFTPO Cyref + MargetCtarge + gnontargethnutten
(4)
All FTPO results in this paper single default configuration: margin of m 2.0 logits in the preference term, target tether of Atorget = 0.05 with zero-penalty window Ttarget = 0.5, and
non-target tether of Amottarget 0.4 (see App. D
5.3 KEY DESIGN PRINCIPLES
Three design choices make FTPO effective for targeted suppression of unwanted patterns:
Logit-space operation. With large logit updates to chosen and rejected, probability mass gets redistributed substantially after softmax, which would impose compensatory pressure on unrelated (non-target) logits we were to use KI.-loss as our regularizer. By using MSE loss on logits instead, we avoid this collateral pressure, localizing updates to just the logits we care about, i.e. the chosen rejected.
Margin-based dcactivation. The weight w. automatically reduces to zero when chosen tokens win by margin m, preventing overtraining. This self-limiting behavior maintains model stability even with extended training high preference accuracy.
Two-part regularization. The two-part MSE loss allows target logits to move relatively freely, while constraining the remaining vocabulary to the reference. This allows training high preference accuracy while avoiding destructive logit divergences.
5.4 AUTOMATED TRAINING DATA GENERATION
The Antislop Sampler provides an effective mechanism for generating training data for FTPO. At each backtracking event, we capture preference pair at the exact position where banned sequence would begin: the rejected token that initiated the unwanted pattern versus chosen viable alternatives from min-p filtering (Figure 2 This allows to build an end-to-end automated pipeline identify a model's overused patterns, generate targeted preference training set, and train model with FTPO to suppress these patterns. We release this automated pipeline open-source.
EXPERIMENTAL EVALUATION
6.1 EXPERIMENTAL. SETUP
Models We evaluate on three model families to represent different architectures and scales: gemma-3-11 sistra--Sae 1-3. 3.3. and T.lama-3, 3.3-0B.

---

Published as a conference paper at ICLR 2026
Datasets
We utilize distinct data subsets for training and evaluation to ensure data leakage:
Slop Profiling Training: We use 2,000 prompts from the Reddit Writing Prompts dataset (Nitral-Al, 2024) to generate the model's "slop fingerprint" and synthesize the FTPO train-
ing data.
Evaluation: We evaluate on hold-out set of writing prompts. To out-of-distribution generalization, we also evaluate on the EQ-Bench Creative Writing prompts (Paech 2023
Human Baseline: To calculate veer-eepresentation ratios, we utilize wordfreq (Speer al, 2018) for single words and curated corpus of Project Gutenberg texts for n-gram statistics.
Our primary metrics measure suppression efficacy, impact on lexical diversity and writing quality. We include MMLU and GSM8K assess impact out-of-domain tasks.
Metries
Banlist Suppression Rate:
The percentage reduction in the frequency of banned patterns appearing in outputs relative to the baseline model.
Writing Quality Rubric:
A GPT-5-as-Judh evaluation grammar, style, and formatting artifacts (see Figure (O). An aggregate, length-controlled metrie combining MATTR-500, Root- TTR, HD-D, and Distinct-n scores, normalized to the baseline model (100).
0-100 scale, assessing coherence,
Lexical Diversity:
MMLU:
5-shot evaluation of STEM and cross-domain knowledge (Hendrycks) et 2021
GSM8K:
8-shot evaluation of grade-school math reasoning (Cobbe et al. 2021)
Longform Writing:
A 30k-token multi-turn story generation task judged by Claude-3.5-
Sonnet, specifically designed to detect repetitive loops and degradation in extended contexts (Paech 2025).
Methods Compared. We evaluate four approaches: (1) token banning with logit bias -100, (2) Antislop Sampler with configurable ban-strength 8, (3) FTPO fine-tuning, and (4) DPO fine-tuning on identical preference pairs. We test banlist sizes of 2k, 4k, and 8k patterns to assess scalability.
Training Details. Our primary experiments train gemma-3-122 32b with FTPO and DPO banlist sizes 2k, 4k and 8k. FTPO uses the hyperparameter configuration specified in Appendix P DPO uses 3 = 0.1. To minimize perturbation of the original weights, we freeze all layers except the last 5 and Im head. We train a high-rank LoRA (Hu et al.) (021] with between 128 and 512. We find these high ranks allow higher preference accuracy targets be reached with lower degradation. Both methods train for I epoch with early stopping at target suppression rates. For the preference accuracy ablation (6.4). learning rate is scaled such that both methods reached the early stopping targets at approximately the same number of training samples processed.
6.2
MAIN RESULTS: SUPPRESSION PERFORMANCE vs. WRITING QUALITY
Figure visualizes the performance in banlist suppression for each method, plotted against output degradation as measured by our writing rubric. The
BL
4k
Writing Quality Score (0- 100)
Antislop Sampler achieves perfect sup-
pression (100%) while actually im-
8k
proving writing quality above baseline.
50
FTPO maintains quality within 1% of
the baseline performance of gemma-3- 12b, while achieving 83-92% suppres- sion rates.
40
Token Banning
FTPO Finetuned
Antistop Sampler
30
8k
DPO Finetuned
In contrast, DPO and token banning show marked quality degradation. DPO drops 6-15 points in writing quality de- spite achieving only 80-82% suppres- sion. Token banning collapses even more severely, with quality falling
Gemma3-126 Baseline
80
85
90
100
Banlist Suppression (%)
Figure 3: Our methods (Antislop Sampler and FTPO) out- perform DPO and token banning for effective suppres- sion of gemma-3-12b's overused patterns, with negligible quality degradation. We test with range of banlist sizes from 2,000 to 8,000 banned items. Error bars are C195.

---

Published as a conference paper at ICLR 2026
28 (out of 100) 8k patterns. In prac- tice, this degradation manifests as se- vere repetition, spelling and grammar artifacting, and incoherence.
We evaluate non-overlapping sub- set of 1,000 prompts from the same Reddit Writing Prompts dataset (Nitral-Al 2024) and observe similar patterns out-of- distribution EQ-Bench Creative Writing set (Figure
We also train gemma-3-27b mixed-domain prompts (essays and creative writing), achieving 83% banlist suppression with no degradation in longform writing quality.
6.3 FTPO vs DPO: DETAILED COMPARISON
FTPO maintains strong suppression across models with minimal degradation (Table 2 The Key Findings: FTPO suppresses 90+% of slop in creative writing outputs for banlist sizes <= 4, 000 items, causing negligible impact on writing quality metrics, lexical diversity and math/STEM bench- marks.
Suppression effectiveness. FTPO achieves 8.5% stronger suppression than DPO at equivalent train- ing settings.
Capability preservation. FTPO maintains math reasoning on GSM8k and world- wardd-nnowledge capa- bilities MMLU within 1-3% of baseline. DPO degrades both metrics by 2-5%.
Long-form generation. The difference is most dramatic in the longform creative writing test, since repetition and other degradation modes exacerbated in extended multi-turn generation. Our FTPO-trained models cluster around the baseline gemma3 score for 2k, 4k and 8k banlist sizes; while DPO experiences large degradation in quality.
Lexical diversity. FTPO maintains or enhances diversity (95-102% of baseline), while DPO causes progressive collapse (74-92%). This confirms hypothesis: DPO has collateral effects proba- bility distributions, while FTPO's precise adjustments preserve vocabulary diversity.
This pattern holds from 12B 70B parameters, demonstrating that FTPO generalizes across chitectures. Some models are more sensitive to preference training and prone to repetition and artifacting: for Llama-3.3-70B we restrict LoRA updates Im_head avoid repetition, the cost of lower suppression rate of 66%.
To verify that FTPO is not simply trading one set of slop for another or drifting semantically, we profile each model's "slop fingerprint" (top over-represented words and n-grams relative to human baseline) and run a cosine-distance embedding analysis. After FTPO, average over-use of these patterns drops sharply instead of being replaced by and embedding distances to the baseline remain only modestly above natural sampling variability and far below simple style-prompt shift (App. L App. M).
set of equally extreme over-used patterns,
Table 2: FTPO & DPO evaluation results for models fine-tuned to suppress range of banlist sizes from of 1k to 8k items.
experiment
mmlu
gsm8k
longform
writing qual
diversity
ban %
gemma-3-12h baseline
0.590
0.888
51.3
67.80
100.00
0.00
gemma-3-12h FTPO 2k (Ours)
0.559
0.876
47.5
68.93
101.05
92.39
gemma-3-12b FTPO 4k (Ours)
0.565
0.880
49.4
67.31
97.68
90.15
gemma-3-12b FTPO 8k (Ours)
0.592
0.889
52.3
67.49
95.09
83.40
gemma-3-12b DPO
0.541
0.847
36.6
62.98
91.03
82.00
gemma-3-12b DPO 4k
0.549
0.861
34.8
58.24
81.92
80.64
gemma-3-122 DPO 8k
0.571
0.864
26.9
54.61
73.92
81.44
Mistral-Small baseline
0.812
0.900
56.03
72.93
100.00
0.00
Mistral-Small FTPO 1 (Ours)
0.811
0.895
58.38
74.60
102.10
89.46
Continued on next page
8

---

Published as a conference paper at ICLR 2026
experiment
mmlu
gsm8k
longform
writing qual
diversity
ban
1lama-3.3-70B baseline
0.801
0.929
36.77
64.34
100.00
0.00
Llama-3.3-70B FTPO 1K (Ours) 0.799
0.923
35.57
63.16
99.66
66.41
6.4 ROBUSTNESS TO OVERTRAINING
Compared with DPO, FTPO can train to a higher preference accuracy target on final-token prefer- pairs before degradation model collapse occurs. FTPO is designed precisely alter only the logits needed, switching off the training signal when chosen logits are winning by given margin over rejected. DPO lacks these "soft-touch" features, resulting chosen/rejected logits continuing to diverge as training progresses.
FTPO
DPO
Logit divergence from reference
Writing Quality Rubric (0-100)
70
7.5
60
5.0
2.5
0.0
-2.
30
FTPO
-5.0
20
DPO
-7.5
20
40
60
80
100
100
60
100
Proference accuracy
Preference accuracy
Preference accuracy
rejected
(a) FTPO maintains writing quality as training progresses to higher pref accuracies, while DPO degrades sharply after the 40% accuracy mark. This experiment trains gemmma-311 -aauma----12b on banlist of 1,000 items.
(b) With FTPO, logits stay close to reference due to (1) the MSE loss terms and (2) the carly switch-off feature which nulls the training signal for chosen tokens that already winning rejected. With DPO, logits diverge unconstrained as training continues. We posit this to be the main cause of FTPO's minimal degradation vs DPO.
Figure 4: (a) Impact on writing quality from training to high preference accuracy targets; (b) Logit divergence from reference as training progresses.
When training gemma-3-12b to increasing preference accuracy targets, we find FTPO can train
to nearly 100% preference accuracy with minimal degradation, while DPO only manages 40%,
after which substantial degradation occurs (Figure 4a). Increasing DPO's 3 hyperparameter to 1.0 mitigates this degradation, but impairs learnability, reducing ban suppression by 15.9% (Figure 8) We posit that FTPO's mechanisms for constraining logits the reference while allowing freedom of movement of target logits are the primary reasons outperforms DPO on this task.
6.5 REGEX BANS
The Antislop Sampler can also target sentence-level patterns using regex bans. The "It's not X, it's Y" construction recurring stylistic pattern that uses different words in each instance but follows some recognisable archetypes. We define these archetypes regex patterns in Appendix O In qwen3-4b, these patterns appear extremely frequently, at a rate of 1.10 occurrences per 1 1,000 characters (Appendix (). With the Antislop Sampler enforcing regex ban, the frequency drops to exactly zero. This demonstrates that the sampler can suppress structural templates-patteni where the specific vocabulary varies but the underlying syntactic form remains constant.
6.6 FTPO HYPERPARAMETER ABLATIONS
The FTPO trainer exposes hyperparameters to tune the strength of the MSE loss tether to the refer- ence, and also the margin specifying where gradients turn off for winning chosen logits. We train gemma-3-126 on hyperparameter ranges outside the defaults, observing poor preference accuracy

---

Published as a conference paper at ICLR 2026
and degradation at these sub-optimal values, and thus demonstrating the efficacy of these FTPO safeguards (Appendix D
DISCUSSION
Antislop Sampler achieves 100% suppression of over-used patterns without quality loss. FTPO outperforms DPO on our measured metrics, even for 30,000-token generations.
Detecting complex semantic patterns, such metaphor overuse narrative tropes is fundamen- tally harder than matching n-grams. These high-level repetitions evade simple lexical filters. Al- though methods like embedding-basse based clustering or syntactic parsing could identify them, they currently too expensive to run inside a training loop. We therefore focus on mechanical detection because it is tractable and captures the majority of artifacts, though we acknowledge that measuring abstract stylistic repetition remains a difficult challenge for the field.
There are limitations and tradeoff's our methods: Antislop Sampler reduces throughput by 69- 96% with vLLM at banlist sizes of 1,000 to 8,000 respectively. due to the frequency of backtracking events. performance-sensitie deployments, this a clear incentive to prefer a solution that trains suppression into the weights.
Anticipating these downstream needs, we develop pipeline that automatically profiles model's overused writing patterns, generates a training set, and trains the model to suppress these patterns. Our FTPO trainer is designed to make targeted adjustments to the model's over-used writing ten- dencies with minimal changes to its distribution otherwise. We hypothesize that FTPO's minimal degradation compared to DPO primarily due to its multi-part loss tethering to reference logits, and zeroing of gradient updates when chosen tokens winning versus rejected.
We encourage future work to explore Antislop's performance in domains other than creative writ- ing, human-rater replication of quality metrics, Al generated text detection, and suppression of toxic text.
Decoding-time diversity methods such as top-k, top-p/nucleus, min-p, temperature schedules, and more recent proposals like XTC, DRY, Mirostat, and top-no primarily manipulate the candidate set or its entropy without changing the local rank ordering of the few tokens that actually trigger stereo- typed phrasing. This explains their mixed success against "slop": they either (i) widen exploration and let more low-probability garbage through or (ii) prune more aggressively and entrench the same high-probability modes. By contrast, the AntiSlop sampler is sequence-ane aware and intervenes exactly the moment banned pattern would begin, backtracking to the initiating token and reshaping the immediate continuation by resampling from cached top-logprob candidates. FTPO then converts that intervention into durable, local preference change by enlarging the logit margin between the offending token and viable alternatives while keeping the rest of the vocabulary tightly tethered to reference logits. Empirically, this division of labor yields robust suppression with minimal collat- eral damage: the sampler guarantees enforcement at inference; FTPO makes the model want the alternative even when the sampler is disabled.
CONCLUSION
We introduced framework for eliminating overused stylistic patterns ("slop") in LLM outputs while preserving capabilities on our evaluated benchmarks. The Antislop sampler performs sequence- level enforcement with backtracking resample that preserves coherence, supports hard and soft bans, and can suppress string and regex patterns. Our automated pipeline extracts model-specific slop fingerprints by comparing the model's overused writing patterns against human baselines, then synthesizes a preference dataset without human intervention. Final Token Preference Optimization (FTPO) trains the model on these pairs, making suppression permanent. Across our tests, FTPO and the sampler achieved higher suppression than DPO and logit- token banning, with negligible measurable quality loss on our rubric. We release code and datasets under the MIT license
10

---

Published as a conference paper at ICLR 2026
AI USAGE DISCLOSURE
Language models were used to assist with early drafting of sections of this paper. All results were human designed and performed, and the citations were human-soureen and validated.
REPRODUCIBILITY STATEMENT
We provide all materials to reproduce our results. Algorithms are specified in Sections 4.2 5.2
including loss definitions and hyperparameters. The general configuration template FTPO/DPO training configuration, LoRA settings, carly-stopping criteria, and decoding parameters in App. P In addition, the data pipeline, prompts, judge rubric, and scoring template are included (Fig. T0P. For inference with Antislop, describe the implementation and throughput (App. B and include our antislop-vllm implementation supplementary materials. The supplemental materials contain necessary code and example configuration files run Antislop Sampler and the automated training pipeline with FTPO or DPO.
given
ETHICS STATEMENT
Our study operates on publicly available datasets and benchmarks: Reddit SFW Writing Prompts via Nitral-AI (Nitral-AT 2024p, EQ-Bench creative prompts (Paech 2023), Project Gutenberg texts (Project Gutenberg). and wordfreq statistics (Speer et al., 2018). We processed only public text and did not collect or annotate human subjects. No personally identifying information was collected, and no IRB was required.
We adhere the ICLR Code Ethics latpps::/.c.r..ccppbbiiiii
Potential harms include: (i) unintended suppression of legitimate dialects, or minority styles; (ii) attempts to evade Al-text detection. Mitigations: our code produces human-readable banlists which may be vetted by hand before deployment; we document and expose the ban-strength control (Sec. 4.2) and provide soft-ban defaults rather than hard blocking; implement whitelist to prevent terms from being automatically banned; we recommend human review any production banlist. Our methods do not target model safety filters and not intended to bypass them.
We transparently report throughput impacts (App. B to support energy-cost accounting. The au- thors declare conflicts of interest, external sponsorship that biases results, and disclose LLM assistance for drafting as stated in the paper's 's Al Usage Disclosure.
ACKNOWLEDCEMENTS
We thank Thoughtworks for generously providing compute for several of experiments.
REFERENCES
Louis Castricato, Nathan Lile, Suraj Anand, Hailey Schoelkopf, Siddharth Verma, and Stella Bi- derman. Suppressing pink elephants with direct principle feedback, 2024. URL https:
//axiiv.ox//ab//20000
Karl Cobbe, Vincet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jemry Tworek, Jacob Hilton, Reiihhro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 1rXiv:2110.14166 2021. URLBttts::/aaxiv..o
14168
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations (ICLR), 2021. URL htt.httts://// http://aavvv.v//..///a//......aa http:///aa.i.voorrr///a/////////
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case neural text degeneration. In International Conference Learning Representations (ICIR), 2020.
11

---

Published as a conference paper at ICLR 2026
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https: // aaaxiv.orq/aa.o. abs/ 2006. 09685
Edward Hu, Huda Khayrallah, Ryan Culkin, Patrick Xia, Tongfei Chen, Matt Post, and Benjamin Van Durme. Improved lexically constrained decoding for translation and monolingual rewrit- ing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 839-850, Minneapolis, Min- nesota, 2019. Association for Computational Linguistics. URL
Josef Jon, Dušan Variš, Michal Novák, João Paulo Aires, and Ondiej Bojar. Negative lexi- cal constraints in neural machine translation. arXiv preprint arXiv:2308. 036001 2023. URL
https://arxix: htt.v/.rggaa////00/// 03601
Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects rlhf on Ilm generalisation and diversity. arXiv preprint AXXv:23100006522 2024. 06452
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings the 29th ACM Symposium Operating Systems Principles (SOSP '23), pp. 61 611-626, New York, NY, USA, 2023. ACM. doi: 10.1 10.1145/3600006. 3613165.
Jack Lanchantin, Angelica Chen, Shehzaad Dhuliawala, Ping Yu, Jason Weston, Sainbayar Sukhbaatar, and Ilia Kulikov. Diverse preference optimization. arXiv preprint arXiv:2501.11001, 2025. URL
18101
Margaret Li, Stephen Roller, Ilia Kulikov, Sean Welleck, Y-Lan Boureau, Kyunghyun Cho, and Jason Weston. Don't 't say that! making inconsistent dialogue unlikely with unlikelihood training. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 4715-4728, 2020.
Sonia K. Murthy, Tomer Ullman, and Jennifer Hu. One fish, two fish, but not the whole Align- ment reduces language models' conceptual diversity. arXiv preprint arXiv:2411.04427, 2024.
URL
wwtpav/..rga////ooo/ooooooo /tt.a///4111.. 04427
Minh Nhat Nguyen, Andrew Baker, Clement Neo, Allen Roush, Andreas Kirsch, and Ravid Shwartz-Ziv. Turning up the heat: Min-p sampling for creative and coherent Ilm outputs, 2025.
abs/ 01082
Nitral-AI. Reddiissiw-wrrinngpooppteddeeeee
http://ggggg.ccccccccc/ctttettte Swweeddit-SS--Wiiiitt wwwwwriiing Prompts ShareGPT] 2024. Accessed: 2025-09-
16.
Laura O'Mahony, I'eo Grinsztajn, Hailey Schoelkopf, and Stella Biderman. Attributing mode collapse in the fine-tuning of large language models. In ICLR Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo), 2024.
Samuel J. Paech. Eq-bench: An emotional intelligence benchmark for large language models, 2023.
URL https://arxii: wwtp:o/g//a/aaaeo 06281
Samuel J. Paech. Longform creative writing benchmark.
Loagform-wwriingg--ee
2025. GitHub repository.
Project Gutenberg. Project gutenberg. URL http::://www.uuu.. http://wwww..uuebbbrr.rogrg
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model secretly a reward model. arXiv preprint 3005 /8290, 2023. htR.:ttps:://// / wwtp///wwww.w...aaaaaa/aaaaaaa/p 18290
12

---

Published as a conference paper at ICLR 2026
Noam Razin, Sadhika Malladi, Adithya Bhaskar, Danqi Chen, Sanjeev Arora, and Boris Hanin. Unintentional unalignment: Likelihood displacement in direct preference optimization. arXiv preprint TrXX::241000887,, 2024. Jttts::///aaza
wttpg//abb//1//// 08847
Allen Roush, Sanjay Basu, Akshay Moorthy, and Dmitry Dubovoy. Most language models can be poets too: An Al writing assistant and constrained text generation studio. In Proceedings of the Second Workshop on When Creative Al Meets Conversational Al (CAI). pp. 9-15, 2022. URL
httts:://aclanthol..o. wwtp://ww..ooog/.....ooo cai-1.2......
Alexander Shypula, Shuo Li, Botong Zhang, Vishakh Padmakumar, Kayo Yin, and Osbert Bastani. Does instruction tuning reduce diversity? a case study using code generation. ICLR 2025 Workshop on Deep Learning for Code (DL4C), 2025. URL https::/ net/
OpenReview.
Robyn Speer, Joshua Chin, Andrew Lin, Sara Jewett, and Lance Nathan. Luminosoinssghtwwordircy v2.2, October 2018. URL httpsstts:///...rr /ttp:///..../1///11////11/////
5281 /endddo 1443582
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Casey Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Turboderp. Exllamav2, 2024. wtt.t//ttg///ggttu....////eeeeea
Philipp Emanuel Weidmann. Dry: A modern repetition penalty that reliably prevents loop- ing. GitHub pull request #5677 aebaboogg/texxttgeeeeaaa... -n- May 2024a. URL https::/ //ggttttthb/////////// wtt http:////obbbbbtttttttteeeeeeeee gew.reettittgeeeeiiiiii.ii wwtpe/errrtttttaaaaorrrro/oaaaaa
Merged May 20, 2024.
Philipp Emanuel Weidmann. Exclude top choices (xte): sampler that boosts creativity,
breaks writing clichés, and inhibits non-verbatim repetition. GitHub pull request #6335 to
cobaaboogg/cxxtt-eeenrrtiioooo comm haan:/ooobbooog/eeeeggeeeeaaaaaa
September 2024b. URL https:///// /
Merged Sep 28, 2024.
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. Neural text generation with unlikelihood training. In International Conference on Learning Rep- resentations (ICLR), 2020.
Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. A survey Imm-generated text detection: Necessity, methods, and future directions. Computational Linguistics, 51(1):275-33 2025.
Junkang Wu, Yuexiang Xie, Zhengyi Yang. Jiancan Wu, Jinyang Gao, Bolin Ding, Xi- ang Wang, and Xiangnan He. B-dpo: Direct preference optimization with dynamic 8. In Advances Neural Information Processing Systems (NeurtPS), 2024. URL https://proceeste
neur wttp::pppppppppppppeeaeeeeeeee 2288881700000223222222225252-111
Longfei Yun, Chenyang An, Zilong Wang, Letian Peng. and Jingbo Shang. The price of format:
Diversity collapse 1lms. arXiv preprint arXiv:2505.18949, 2025. URL https: pectps://arxxr
org/abs/ 25055. 18949
Yiming Zhang, Jianfeng Chi, Hailey Nguyen, Kartikeya Upasani, Daniel Bikel, Jason Weston, and Eric Michael Smith. Backtracking improves generation safety, In Proceedings of the International Conference on Learning Representations (ICLR), 2025.
13

---

Published as a conference paper at ICLR 2026
APPENDICES
A SOFT BANNING
In real-world use cases, it is often not preferable ban word phrase outright. In these cases, a scalable "soft ban" is preferred, where there is general suppression effect, but the suppressed vocab may still be used there are no good alternatives.
An example of how soft-banning works when there are no good alternate candidates:
Step 1. We have the word "tapestry" in our banlist, and have ban-strength 0.2 and min-p 0.1.
Step 2. The user requests an essay on tapestry weaving.
Step 3. The model begins inference with, "The art of Tapestry-", triggering backtracking. In this example we will say "Tapestry" was the top token this position with 0.99 prob, with next highest token "Mural" at 0.0005.
Step 4. The "Tapestry" token reduced to prob = 0.99 x 10 10-0.2 0.0099.
Step 5. After probability rescaling, min-p still excludes "Mural" from consideration, since 0.0005 0.0099 R 0.05 < 0.1 (the min-p threshold), resulting in "Tapestry" remaining the only candidate for sampling.
Step 6. "Tapestry" is selected as the next token despite being on the banlist. This specific violation at this position is marked be ignored by Antislop in future checks, to avoid backtracking loop.
ban-strength value of 1.0 effectively hard ban, enforcing 100% suppression of the banlist.
To determine whether each method can still use the suppressed patterns when contextually necessary, construct an adversarial prompt:
Write short story (500 words) incorporating the target phrase exactly times in the story. The target phrase is: "(phrase)".
validates the soft-banning mechanism (Section 4.2), where ban-strength s controls suppres- sion intensity. The Antislop Sampler with 0.4 achieves optimal balance. suppressing patterns in 90% normal generation anverarvesariaa) while fully permitting them when explicitly requested.
Baseline + Antislop Sampler
FTPO/ DPO / Token banning
100
100
80
60
60
Optinal
Optinal
40
40
20
20
0
0.2
0.4
0.6
0.0
1.0
FTPO Rin tuned
DPo Rinettneed
Taken banning
Anti Sampler banistrength
Adversarial
Non-sdversarial
Figure 5: Our methods can suppress 90+ percent of banlist occurrences while allowing the banlist through when contextually necessary. Antislop Sampler, FTPO, DPO and token banning are com- pared on banlist suppression efficacy under normal writing conditions (non-adversarial prompts) and when the model is explicitly instructed use the banned vocab (adversarial prompts). We indi- cate optimal behavior for most real-world use cases to be maximal suppression in normal writing conditions, and minimal (preferably zero) suppression in adversarial conditions i.e. when the model has no coherent alternatives.
14

---

Published as a conference paper at ICLR 2026
B INFERENCE PERFORMANCE AND COST ANALYSIS
We release two implementations of the Antislop sampler: single-threaded version using Hug- gingface Transformers, and higher-throughput version that works with any OpenAl-compatible vl/completion endpoint that supports top logprobs. The sampler incurs significant throughput penalty, especially with larger banlist sizes, due to the backtracking events. There is additional performance lost with the API implementation, since it generates chunks, with banned pattern de- tection only occurring after chunk is generated. This could be optimized further by, for example, integrating the sampler into vLLM directly rather than generating chunkwise via the API.
The maximum token rate of our OpenAl API implementation is discovered with binary search on the number concurrent threads when generating with vLLM. Figures cited are using single Nvidia H100 gpu.
We measure a 69% reduction in throughput at a banlist size of 1,000, up to 96% reduction at banlist size 8,000. However, these should be considered worst-case values. A banlist of this size would be overkill for most real-world usage; we include it here as a stress-test.
Maximum token rate (parallel generation VILMM]
4000
1000
per
2000
1000
Baseline gpemya tib-ttttt baoned)
vive tumed bamee))
Fine fine-tneed banned)
nikmn harning benneu)
Antisuup Sampler 24 Dwnned
Artistup Sampler Dunned
Avtistup Sampler de burned
Antiscup Sampler Sk burneed
Figure 6: Rate of inference measured for each method when generating with optimal parallelism with vLLM.
B. PIPELINE COST ANALYSIS
We analyze the computational resources required to execute the complete Antislop pipeline, includ- ing the automated generation of the preference dataset via the Antislop Sampler and the subsequent FTPO fine-tuning. Costs are estimated based on standard on-demand cloud pricing for single NVIDIA H100 GPU (approx. $2.00-$2.5000
Table 3: Cost and resource analysis for complete slop removal pipeline (Dataset Generation + FTPO Training) on 1 1x H100 GPU.
Stage
Model Size
Time
Memory/Throughput
Est. Cost
Dataset Gen. (2,000 samples)
12B (gemma-3)
1.5 hours
1,506 tok/sec"
$3.00
(69% reduction vs baseline)
FTPO Training
12B (gemma-3)
2.3 hours
42 GB VRAM
$4.50
(10k samples)
27B
6.8 hours
68 GB VRAM
$ $13.30
Total Pipeline Cost:
The complete workflow for 12B model requires approximately hours and $7.50 in compute credits. This is comparable to standard supervised fine-tuning (SFT) workflows.
15

---

Published as a conference paper at ICLR 2026
Resource Requirements:
Training the 12B model requires 42 GB of VRAM (using LoRA 256 and b-bit quantization), fitting comfortably on a single A100/H100 (80GB). The 27B model requires 68 GB under similar settings. FTPO Training exhibits similar scalability properties to DPO. Because the method operates final-token preference pairs rather than full- sequence generation during training, it scales efficiently to larger mod- cls (e.g., 70B+) provided sufficient GPU memory available hold the base model weights and optimizer states.
Scalability:
Energy Consumption:
We estimate the average power draw of an H100 GPU at peak load to be approximately 700W. Given a total pipeline duration of 3.8 hours (1.5h generation + 2.3h training), the total energy demand to fine-tune the model is approximately 2,660 Wh (~2.6 kWh).
16

---

Published as a conference paper at ICLR 2026
C LONG-RANGE CONSTRAINT ENFORCEMENT VIA REGEX BANS
Some models exhibit stylistic slop such as the "not z; but y" family of constructions, which standard quality metrics rarely penalize and which are difficult unlearn post hoc. We prevent these forms at inference by compiling a small set of regular expressions into one alternation and scanning the full generated text each validation pass. On a match we locate the earliest offending span, map its first character to the corresponding generated generated-token index, and trigger backtracking at that position. Backtracking resamples from the cached top-logprob lists with the same decoding hyperparameters (temperature, top-p, top-k, min p). yielding coherent alternative continuation without another API call.
Figure 7 shows an example where the baseline gwen33-43 overuses the pattern, while Antislop with regex bans reduces its rate to zero.
"Not but patterns per 1k chars
(baseline)
gemma-3-4b-it
0.55
gemini-2.5--Be
4hatgpt-4n latest
0:48
Histral-Small-3-2225555
claude-opur
0.22
Human baseline
0.17
qwen3-4b (regex antislop)
0.00
0.0
0.2
0.4
0.6
0.8
1.0
per characters
Figure 7: Occurrences per 1k characters of the "not H. but y family across several models. The Antislop variant of qwen3- 4b enforces regex bans with backtracking and yields 0.00 hits.
17

---

Published as a conference paper at ICLR 2026
D HYPERPARAMETER ABLATIONS
The FTPO trainer exposes some tunable hyperparameters:
diipeepsioon logits: Clips the preference-loss component of the training signal for chosen logits that are already beating the rejected logit by this margin.
lambda mse target:
The strength of the tethering reference logits, specifically applied to the target (chosen & rejected) logits. Higher values prevent the target logits straying too far from refer- ence, but also make it harder for the trainer to achieve high preference accuracy. Lower values allow the model to learn more easily, but may lead to degradation or model collapse.
In this ablation, we train gemma-3-126 with FTPO on 10k samples with early stopping 95% pref- erence accuracy. We vary defaults, demonstrate the protective effect this feature of the trainer. We also ablate the lambda mse target parameter, setting it at O, 0.05 (default) and 0.4 while keeping other parameters defaults. We measure the impact on writing quality, average divergence of logits from reference, and the percent of training examples processed before the 95% preference accuracy early stopping condition triggered.
apsilon_logitt from 2 (default) to 16 while keeping other parameters
Table FTPO ablation results for clip clip.epsilon logits and lambda mas_target.
experiment
writing qual
ban %
early stop
Achosen
Arejected
Aother
gemma-3-122 baseline default params
67.80
0.00
N/A
N/A
N/A
N/A
67.89
84.51
66.00
1.23
-3.93
-0.26
no margin clipping
19.57
98.24
37.00
1.48
-7.02
-0.35
no target mse loss strong target mse loss
39.65
94.54
46.00
-2.91
-8.31
-3.17
69.68
55.86
100.00
1.18
-1.50
0.07
We find that setting the tiip.epsilon_logti parameter (the margin clip point that switches off prefer- ence loss for winning logits) to 16 effectively disabled results in model collapse. Logits diverge much further from reference, and output degrades to single-word repetitions. With this parameter set to 2 (the default), the model reaches the 95% preference accuracy stopping point with writing quality preserved.
With lambda Londda_ms_ttte target reduced to 0, disabling the reference tether for target logits, ob- serve faster training and logits diverging farther from reference. Writing quality degrades 71% from the baseline per our rubric, illustrating the protective effect of this loss component. When lambda_mse_tan starget is set to 0.4, logits diverged much less from reference, but the model was only able to achieve 74% preference accuracy by training completion. At the default value of 0.05, the model reached the 95% preference accuracy target without any substantial output degradation.
Hyperparameter Robustness. While FTPO operates in logit space, we find that our default hyper- parameters transfer well across diverse model families (gemma3, llama3, mustral-small, glm-4).
18

---

Published as a conference paper at ICLR 2026
E
DPO 3 HYPERPARAMETER ABLATION
50
Longform writing score
45
DPO $m0.01
40
DPO p=0.1
DPO ß=
DPO ß=1
35
FTPO
Baseline gemma-3-125 51.30
30
25
20
40
60
80
100
Banlist suppression (%)
Figure 8: examine the impact of DPO's 3 hyperparameter, training gemma-3-128 on our final- token preference set with several values of 3: 0.01, 0.1. 0.3 and 1.0. This training set suppresses banlist 1,000 items. With DPO, we observe an expected tradeoff learnability vs degradation (Wu et al. 2024). DPO manages 1% reduction in output quality at 1.0. but at the expense of significantly yy impaired banlist suppression (74.7%). At lower values of B3 output quality is markedly reduced for the DPO-trained models. In comparison, the FTPO model trained the same dataset achieves the highest suppression rate of 94.4% suppression, with neglibible (< 1%) degradation in longform writing score.
F SUPPRESSION PERFORMANCE vs WRITING QUALITY FOR EQ-BENCH DATASET
8k
Writing Quality (D-100)
8k
Token Banning Finetured
2k
Antisiop Sampler
Gemma3 12b Baseline
82.5
85.0
87.5
90.00
92.5
95.0
97.5
100.0
Banlist Suppression (%)
Figure 9: We replicate 6.2 with an out-of-distribution writing prompts dataset. While a smaller dataset size of 96 prompts (and correspondingly larger error bars), we observe similar pattern of banlist suppression rates and impact on writing quality for each method.
19

---

Published as a conference paper at ICLR 2026
G MOST COMMON OVER-REPRESENTED WORDS AND TRIGRAMS ACROSS
MODELS
pattern
percent models
flickered
98.5
flicker
94.0
flickering
92.5
leaned
82.1
muttered
82.1
gaze
80.6
grinned
80.6
containment
77.6
gestured
77.6
addendum
74.6
murmured
73.1
nodded
73.1
glint
68.7
hesitated
68.7
whispered
68.7
blinked
64.2
hummed
64.2
faintly
62.7
leans
62.7
unreadable
62.7
Table 5: Top overlapping words across 67 Al models. Each entry shows the % of models in which the token appears among their top 120 most over-represented words (relative a human baseline).
pattern
percent models
voice barely whisper
68.7
said voice low air thick scent took deep breath
61.2
49.3
44.8
smile playing lips
43.3
something else something
37.3
said voice barely
35.8
voice barely audible
35.8
take deep breath
32.8
could shake feeling
31.3
eyes never leaving
29.9
casting long shadows says voice low
28.4
26.9
something else entirely
26.9
heart pounding chest last time spreading across face air thick smell could help feel long shadows across
25.4
23.9
22.4
19.4
19.4
19.4
Table 6: Top overlapping trigrams across 67 Al models. Each entry shows the % of models in which the phrase appears among their top 40 most over-represented trigrams (relative to a human baseline).
20

---

Published as a conference paper at ICLR 2026
H WRITING QUALITY RUBRIC PROMPT
You expert in assessing creative writing. Your task is to score the test model response below, by several metrics, on 0-20 scale.
[PROMPT START]
(writing_prompt
[PROMPT END]
[TEST MODEL RESPONSE]
ITEST MODEL RESPONSE END
[Task]
You are expert in assessing creative writing. Your task is score the model's response below, by several metrics, on a 0-20 scale.
Scoring notes: In the output, write the metric names exactly as below so they can be parsed.
Use the designated output format exactly.
All criteria are "higher better"
You are critic, and your job is to be critical, especially of any failings amateurish elements.
Output format is:
[Scores]
Metric name: [Score 0-20]
Metric 2 name:
Now, rate the supplied model output on the following criteria:
Spelling/grammar
Formatting issues artifacts
Cohcrence
Consistency of tense, pronouns, perspective Repetition issues Overall quality of the piece
Figure 10: Writing quality rubric prompt: This prompt was used to assess the overall quality of
creative writing outputs in our experiments, with particular focus the common modes of degra- dation.
21

---

Published as a conference paper at ICLR 2026
1
IMPACT ON METRICS BY BANLIST SIZE
Bench Writing Prompts
Reddit Writing Prompts
20
20
ArtSlop Samplor
Sampler
DPO
DPO
Figure 11: Impact on writing quality per our LLM-judged rubric at several banlist sizes, for each suppression method (Token banning, FTPO, Antislop Sampler and DPO).
Bench Writing
Reddit Writing Prompts
69-
Taken Ban
60-
Tokan Ban
Sampler
DPD
DPO
netatrre
banfit size
banlist size
Figure 12: Impact on banlist suppression rates at several banlist sizes, for each suppression method (Token banning, FTPO, Antislop Sampler and DPO).
BQ-Bench Writing Prompts
Writtng Prompts
18
14
14
12
12
Raknn Ban
Taken Ban
FTPO
FTPO
AntiSlop Sampler
AntiSlop Sampler
2k
5k
is
2
burfist
banist
Figure 13: Impact lexical diversity at several banlist sizes, for each suppression method (Token banning, FTPO, Antislop Sampler and DPO).
22

---

Published as a conference paper at ICLR 2026
J FTPO Loss FUNCTION DEFINITION
Preference Loss Component:
For each chosen token index against rejected token index r, define the logit gap
A ylc y(r].
The margin requirement is m. A smooth penalty is applied if the gap is smaller than m. greel log( 1 + clm ml/))
with = 1 here. A taper weight
clamp nー 0,
shrinks the contribution as A approaches the margin. The preference loss is the weighted mean over chosen tokens:
Σ w goeef
Preet
Σ
w
Preference Loss Term
-6
-4
-2
A logilo seaiit_chheen-
& O: chosen lesing full softplus penalty
margin satisfied loss
winning tapering penalty
Figure 14: Preference loss component as a function of the logit gap A. When A < 0 (chosen losing), the penalty is large. As increases toward the margin m, the penalty smoothly tapers. Once A 2 the weight goes to zero and the preference loss no longer contributes.
MSE tether terms:
Let deviations be d, = ylol rf [j]. Define:
Target set (c) (r) (chosen and rejected indices).
Non-target set N (1,..., V} T.
Non-target MSE loss term:
Σjen d? [N
Cnontarget
Target MSE loss term with zero-penalty window
Σjer o²
e max |dyl Target 0),
Ctarget
[T]
23

---

Published as a conference paper at ICLR 2026
Per-logit MSE Loss
Zero-loss window for target logits
3
policy llgittlot---- ref logit
Non target logits: Aumse_nontinn nontarget (logit - refi
Target (chosen rejected): A_mse target maxi]logit refl target 01^2
Figure 15: MSE loss components as functions of logit deviation from the reference. The non- target term (blue) penalizes any deviation quadratically. The target term (orange) allows dead zone around zero, where no penalty applies, then grows quadratically once the deviation exceeds the zero-penalty window.
Here Taaget is zero-pente vero-penalty window: the chosen or rejected logits are within Target of the refer- ence, no penalty is applied.
Total objective:
With weighting coefficients Anonturget and Auaget the total FTPO loss is
Laret + Anontarget feventanget
Atarget Liarget-
This formulation allows the model learn clear preference signal while preventing uncontrolled drift of the logit distribution.
24

---

Published as a conference paper at ICLR 2026
K SLOP PROFILE CLUSTERING BETWEEN MODELS
Colloquially, slop may refer to over-used words, phrases, themes or writing styles. Here we focus over-used words and n-grams as they are relatively straightforward to ex- tract. For given model, generate outputs from a creative writing prompts dataset (Paech) 2023) and writing prompts dataset sourced from Reddit (Nitral-AI] 2024). We then com- pute list of the most over-tepresented words and bigrams/trigrams relative to a human base- line. The human baseline we for individual words is the Python library wordfreg (Speer al. 2018). For bigrams/trigrams, we com- pute human baseline from a mix of sources in- cluding large Reddit creative writing dataset, and a selection of public domain works from the Gutenberg Library (Project Gutenberg). For n-gram extraction, we remove stop-words.
Rank-Distance Slop Clustering
mistral-smlll-3 htw.s/mmlll..-22---itnttttteeeee
saappeee-chat--3--00
chatgpt 4a latest
mistral-mediumma3)
drepsnek-rr
kimi-2
0p1-5-2025-00-07
daudd-someet-4
daude-opus
cladee-3.-sonnet
tlaude-3 3.7-e
gemini-22.5 pro preview
deepseok chat-v3.1
deepseak-r1-0527
gemma-3-27b- 3-27b-it
gemma-3-126-4
gemmma-2-9bii
gemini flash 001
gpt-oss-120b
gpt-4-0314
gpt-3, ppt-3.5-turbo-0613
mistral 3atiillmalll33112424 1.224biinstruut2550
mistral httpaa/smmmii--d--irrtt---------
llama-3.--7bb-igg
Ilama-3. -.0.b------
"slop fingerprint" is collated from the top 120 most over-represented words and the top 40 most reererepreednted bigrams and tri- grams. To avoid over-indexing frequency words & phrases in single texts (e.g- character name), we require the pattern to oc- cur from at least 3 writing prompts indepen- dently. To examine the relationship of this fin- gerprint between models, we perform hierar- chical clustering on these top-200 lists per the average rank-distance between each model pair (Figure [6).
Ilama-3. 3.6---insttruct
llame 4-maverick
lama 4-scout
slón edward nourse
gemma-3- 3-1-mme----2b--ee- 3-12b--t--fpoo-k
high-
mark twain
jack. landan
h_g_ hel_wells
tolstoy
arthur cowacoocnanccmmm doyle
rudyard Kipling
andre norten
jane austen
frank baum
0.7 Cluster Distance
Figure 16: Top 200 over-represented words and bigrams/trigrams were extracted for each model relative to human baseline, for set of creative writing outputs. For included human authors, a selection of their works were used. A dendrogram was generated with cluster distance as the aver- age ranking distance of the top over-represented words & n-grams list between models. Our FTPO antislop finetune of gemma-3: 3-1bbb is highlighted, clustering closer to human authors than any other tested model.
It's important to distinguish between counting the frequency of words and n-grams in a text, and calculating their frrquency relative man baseline, as we are doing here. The for- mer simply surfaces patterns that are common in writing; the latter surfaces repetitive writing tendencies of model that begin stand out across multiple generations, leading to the per- ception of "slop". In some models this repeti- tion is extreme: mistral-small-33 3.1--2b-inssstrccc 2503 produced 102 "eyes never leaving" tri- grams and 62 "voice barely whisper" trigrams across just writing prompts.
hu-
We find a high correlation in words and n-grams found on the top most over-representee lists across the models tested, with "flickered" appearing on 98.5% of lists, and the trigram "voice harely whis- per" appearing 68.7% of lists. See Table 5 for the most commonly co-occurring word patterns across slop fingerprints, and 6orle trigram patterns.
We utilise this method for identifying over-represented usages to compile target list for slop duction with the Antislop Sampler and FTPO fine-tuning. should be noted that this method of identifying slop is domain-specific; the over-used patterns in creative writing will differ from pro- fessional writing, for instance. Here, we focus on creative writing, however the method can be applied to other domains by choosing different of prompts from which to derive the slop list.
25

---

The Published as a conference paper at ICLR 2026
L ASSESSING EMERGENCE OF NEW SLOP
A natural concern with any suppression method is a "whack-a-mole" effect: once set of over- represented patterns pushed down, the model might simply substitute different of equally extreme over-used patterns. Here we describe the experiments we ran to check whether FTPO is just trading one slop fingerprint for another, or whether it actually reduces the overall amount of slop.
L.1 SETUP
We focus on gemma-3-27b and its TTPO-fineturee variant pipeline is:
1. Generate creative-writing outputs from the Reddit Writing Prompts dataset for both models, using the same prompt set and decoding settings as in the main experiments.
2. For each model, compute token-level frequencies for words and n-grams (n E {2,3}), removing stop-words for n-grams as Section ??.
Compute over-representation ratios
fLIM(P)
p(p)
fuuman (p)
using the same human baselines in our primary experiments (wordfreq for unigrams, curated Reddit+Gutenberg corpus for bigrams/trigrams).
For each model separately. rank patterns by p(p) and extract the top-40 words and top-40 tri- grams. This is done without consulting the training banlist, that any new slop introduced by FTPO can surface.
All statistics reported below are computed on these "top-k" lists for the baseline vs FTPO model.
L.2 OVER- REPRESENTATION BEFORE AND AFTER FTPO
For the baseline gemma-3 3-27b, the top over-represented words show extreme usage relative to human text. Concretely, the top 5 most over-represented words (with human normaiier ratios and raw counts across our sample) is:
Gemma-3-27b (baseline):
elara: ratio 18,861 x, count 355
2. logline: ratio 3,739 x, count 109
worldbuilding ratio 3, 3,121 x, count 240
4. grimdark: ratio 2,819x, count 70
5. unsettlingly: ratio 2,275x, count 54
For gemma-3-22b--nntislg the corresponding top-5 looks like:
1. elara: ratio 1,552x, count 39
unusualnesse ratio 1,496x, count 18
3. lysandra: ratio 1,330 count 35
4. sutiissshessss: ratio 1,169x, count 12
logline ratio 1,028x, count 40
Two things are happening:
The same tokens that were severely over-used before (e.g.. elara, logline) are now much closer to human baseline.
26

---

Published as a conference paper at ICLR 2026
Some new tokens appear in the top-5, but their ratios and counts much smaller than the worst offenders in the baseline.
Aggregating across the full top-40 lists for these two models, we sec:
The mean over-representation ratio for the top-40 words drops by 73% (from 1,439x to 394x). The mean over-representation ratio for the top-40 trigrams drops by 36%.
This indicates that FTPO does simply swap new set of equally over-represented words: the overall mass of slop, as measured by the degree of over-use relative to human text, goes down.
This pattern is consistent with the design of the FTPO loss. At each suppression site, gradients are spread over multiple chosen alternatives instead of pushing probability into a single replacement token, and the margin-based switch-off stops training once the chosen tokens win by the specified gap. Both design choices make it hard for any single alternative to become the new "Flara."
L.3 ENTROPY AT SUPPRESSION SITES
We also measure how the local token distribution changes at the positions where banned pattern would have begun (i.e., the positions that generate FTPO training pairs). For each such position compute the Shannon entropy of the top-k token distribution.
For gemma- 3-27b,
Entropy suppression sites: 1.34 (baseline) 1.93 (FTPO). Entropy at random positions: 0.59 (baseline) -> 0.89 (FTPO).
So at the exact locations where slop pattern would be selected in the baseline model, our finetuned model has flatter distribution. At random positions, entropy also rises, but by smaller amount. This is consistent with FTPO increasing the probability mass a set of viable continuations instead of merely shifting probability from fixed phrase another.
L.4 SEMANTIC SHIFT VS. SEMANTIC COLLAPSE
To check for broader semantic drift, we embed outputs from the baseline and FTPO models and compare them in cosine space.
The procedure is:
1. Sample set of prompts.
For each prompt, generate one completion with the baseline model and one with the FTPO model under the same decoding settings.
3. Compute embeddings for each completion (using -eweriiiemmbediing--- 001) and measure:
The mean cosine distance between baseline and FTPO outputs.
The mean cosine distance between two independent baseline samples for the same prompt (as a "natural variability" baseline).
The mean cosine distance between baseline completions with and without a strong style in- struction (e.g.. "Write in the style of Hunter S. Thompson") reference for deliberate semantic shift.
We observe:
Baseline vs baseline (two temperature samples): average cosine distance z 0.079.
Baseline FTPO: average cosine distance 0.098.
Baseline vs baseline-sttyl--roonpt average cosine distance R 0.18.
So the FTPO model is slightly farther from the baseline than two random baseline samples are from each other, but much closer than deliberate style shift. Combined with the entropy results and the
27

---

Published as a conference paper at ICLR 2026
over-representation analysis, this suggests that FTPO is not pushing the model into completely different stylistic regime; is shaving down specific set of over-used continuations while keeping overall semantics in roughly the same region.
M EMBEDDING-BASER SEMANTIC SHIFT ANALYSIS
We also ran embedding-basee baaee check to see whether FTPO causes large shift in the model's stylistic or semantic content.
M.1
SETUP
We used the same prompt distribution as in our creative-writing experiments and drew 500 prompts. For each prompt, we generated using temperature sopperaauree1...
10 completions from the baseline model (gemma 3 2-27)),
10 completions from the
model
10 completions from the baseline model with added style instruction ("Write in the style of Hunter S. Thompson").
For each completion, we computed an embedding (via the OpenRouter embeddings API), All distances below are cosine distances, i.e. - cos(O)) between embedding vectors.
We then computed three quantities:
1. Within-model diversity: For each model separately, and for each prompt, we took all pairwise distances between that model's 10 embeddings and averaged them. Averaging those per-prompt values gives single within-model distance.
2. Baseline FTPO distance: For each prompt, took all cross distances between the 10 base- line embeddings and the 10 FTPO embeddings, averaged them, then averaged across prompts.
3. Baseline style-prompt distance: Same as above, but comparing baseline completions to bascline+style completions.
M.2 RESULTS
Table 7 summarizes the mean cosine distances:
Comparison
Mean cosine distance
Baseline (within-model)
0.079
FTPO (within- (within-model)
0.087
Bascline vs FTPO
0.098
Baseline vs baseline+style instruction
0.180
Table 7: Embedding- based distances averaged over 500 prompts and generations per prompt.
Lower is closer in embedding space.
The FTPO model shows slightly higher within-model diversity than the baseline (0.087 vs 0.079, about a 10.1% 1 increase), which is consistent with the entropy and diversity gains we see elsewhere. The baseline FTPO distance (0.098) is only modestly higher than baseline's 's within-model variability, and much smaller than the distance induced by a simple style prompt (0.18).
Taken together, this suggests that FTPO is not pushing the model into new semantic regime; it is making targeted, local adjustments while keeping responses roughly the same semantic neigh- bourhood as the original model.
28

---

Published as a conference paper at ICLR 2026
N INFERENCE- -IME BASELINES: DRY AND XTC
We also compared our methods against two decoding-tine approaches that target repetition and diversity: XTC and DRY (Weidmann) 2024b a). The goal here is see whether they reduce the same kind of global over-use that shows in our slop fingerprints, not just short-range repetition.
N.1 SETUP
We use gemma-3-2 ---mb and generate 2,000 creative- writing outputs under four settings:
baseline sampler (standard top-pitemperature))
FTPO-finetuned model (same decoding as baseline),
XTC sampler on the baseline model,
DRY sampler on the baseline model.
For each setting recompute the slop fingerprint: the top-40 most over-represented words and the top-40 most over-represented trigrams, measured as the ratio
(L
p(p)
Shuman (p)
against the same human baselines as in the main text. We then take the average p(p) over the top-40 words and top-40 trigrams.
N.2 RESULTS
Table 8 summarizes the average over-use of the top-40 patterns:
Method
Top-40 words avg ratio
Top-40 trigrams ratio
Baseline 1eemma-3-27b
1439x
173
FTPO-finetueed gemma- -3-278
394x
111x x
XTC sampler (on bascline)
1267 x
205x x
DRY sampler (on baseline)
1442x
168x
Table 8: Average over-representation ratios (model vs human writing) for the top-40 most over-used words and trigrams, under different inference-time methods. Lower better.
XTC and DRY do what they are designed to do locally: they reduce obvious near-term repetition and looping. But from the perspective of global over-use, they leave the slop fingerprint essentially in- tact. The average over-representation for the top-40 words stays in the same ballpark as the baseline, and the trigram ratios unchanged slightly worse.
By contrast, the FTPO-finetuned model shows a large drop in average over-use for both words and trigrams while using the simple sampler the baseline. This is consistent with the idea that decoding-time tricks are not enough to fix model-wide over-representation; you have to move probability mass in the weights you want the fingerprint itself change.
O REGEX BLOCKLIST USED FOR "NOT E, BUT "
regex_patterns;
29

---

Published as a conference paper at ICLR 2026
[ [?: ||
(?!ri)
btt.f//ahh....aaaoooo/////e/aeea (?!)
htgp://pah...aaaooooo.ooooooooar b)
"A httpah//eweiaataa//////o//////on |merely) \8* (?: (..11))
||
\bb) (?!
with\\b) http://:hh..............0.0.0.1.
thath httl:///chhhccccaaeeeeeeeee/atte [']
"I "bloo (o ||tp://aa_aaaaa..aaaaaaaaaaaaaa"
(B:it/they| you) As- secsis//reeleer were)
2",
[] (| [ (2 onl]
y/merely) 2\ (?: (?1 Wbbubbb [.?!]) A (?)
(I\bb) (?:also\\ss) ?*
30

---

Published as a conference paper at ICLR 2026
P AUTO-ANTISLOP CONFIGURATION FILE FOR GEMMA- ---MA-3--1E 12B-IT 2K BANLIST SIZE
MAIN ACTO-ANTISIOP CONF GUNFIGURTIIN
RUN SETUP
#
experiment_Das base dir: "results/auto
Base for timestamped
run directories
numan_profile_path "att://umannwwtttooppproroo. log_level: "INFO"
Iteration 0: Generates the baseline dataset computes slop
htrrings/ggrass
ban
Iteration 1: Generates a dataset using antislop, banning those strings & ngrams. Recomputes the slop strings/ngrams at the end & adds any new slop to the banlists Iteration 2+: Extra iterations catch slop that emerges after the initial
banned
num_iterations: 2 Minimum 2 iterations (this is enough to catch most slop)
model id: WAAAААCAAAAAAAAAA
Global model id for the pipeline. Can overridden on individual steps.
VLLM SERVER MANAGEMENT (Conditicnal: if -mangge-vlm is True)
manage_vllm:
vilm_model_id: null # Model served by VLIM (if unset, will use model_id!
vilm_port: 8000
vlim_ht_token: null # Optional: Your Hugging Face token if model is gated viim_ocuca__iiiiileeeeeeee devices:: "0" # set to e.g. "0 for multiple gpus Beim_ggu_memery_urt uttiitttinii 0.85 1eave some room for the refusal classifier: if you are using it (about 3gb)
vilnn_ax_mmdell1 4500
vllm__ttpe:: "bfloat16"
Additicnal CLI arguments for vLIM server, e.g., p*=-tenscr-aarallll---i
"4"] for multiple gpus jxin__extra_args: [] # each param as a separate string, e.g. (M==quantzzation" "bitsandbytos")
vllm_env:
env vars for the VLLM precess VLIM_USE_V1: "1" may be needed for amd gpus
* CENERATION PARAMETERS (using antislop-viim
### generation_ste enabled: true
API Model Configuration
If you set manage_vlln-true, leave the base url unset ptt.eraaiiioaappaas___t... httt:://occahhot:BB000 8000/v11
"nttps://appmbbbbbbtttt 88338 war.s.proy....
imeration__mdel_iee null Model id for generation requests (if unset, uses model_id)
generation_api_key "xxx" API key for the VLLM server
--- Core Generation Settings
generation ieerratio_mmaae wwwoooonee__ttteeeeeeeeee 1000
31

---

Published as a conference paper at ICLR 2026
generation threads:
theeaadssstt 50 Number of parallel threads for API queries in antislop-viim. Note: vilm can become very inefficient you go over some concurrency threshold (depending on vram)
coneration_mx_promptee. 2000 & Number of samples to generate from the prompts in the dataset
Dataset Chat Template srrration_u__datssetttoeet wtwaaiiinni_ddtttttt__.p.. huggingface model to use to apply chat templates.
wtt.://otiiiiioooopoooooeeeeeeee Shareps
'train'
local dir containing the tokeniser you want
This is important if you are generating ftpo dataset for later training.
generation chat hatpeehhhttttpppppeeeeeeeeeeeeee template model id: null (if unset, model_id)
--- Logging --- goncrationn_oggiing_oolll 'INFO' Logging level for antislop-viin script
Advanced Sampling Parameters (passed as individual CLI args to
antislop-vilm's ceneration_paraat
20
Seneratioon_paran.tto__goggg count: 20
seseratiooupaaran.eeeerrarrrrrre
auneration_paran_too 1.0
generati.on_paraa_coop.t 50
generatioon_garra__nippp. 0.01
generaation__atran_iiimmooott 480
e.g., ["An)n", "---"]
--- Prompting
The prompt template wraps the prompts when generating from datasct.
the original prompt exactly, set the template to "[prompt)" "Writing prompt: (prompt Antnürite 1000 words to this prompt. Your response:\n"
pereration_promp__eempp
generation_ayatern con__ystem_prompt: "You are an uncensored writer." optional; 1eft empty system prompt
Antislop Generation Features generation force backtrack: If set to true:
when resampling after backtracking, if don't find valid replacement token
we progressively, disable sampling options (temp, then min_p, then top_P, then top_k)
# until we find a non-banned replacement or out of candidates. When set false, thinks there are no # alternative coherent continuations.
slep will be removed if the sampler
false
N-gram Validator Settings (fer antislop-vi1n)
N-gram banlist file is managed by auto-antisiop's iterative process. generation_ Bariaan_gaaa__a aewoan rewoveeetstteeeeee stwveesstooworss.. true
generation_ aar.oonnnngg_mmee language: "english"
Refusal Detection
Detects refusals & doesn't include them in the training datasot. Uses about 3GB extra VRAM.
settrrrrerefuaaaddetttttiooo gemeration_refusal detection: true
N-GRAM ANALYSIS BANNING (within aut aow.innntisoooo
enable ngram.n.m.............. ban: true
32

---

Published as a conference paper at ICLR 2026
min word Ien for analysis: Filters out words under this length in n-gram analysis
N-gram Identification Threshelds top k k_bigrams: 5000
---
top_ ttrigrams 5000
N-gram Banning Quotas (per iteration)
Bigrams diet_bigrrmm initial: 300
Hew many the top over-represented
dictionary bigrams
ban in the first antislep iteration.
"Dictionary" means the bigrams were also found in the human
writing corpus.
dict Doigrams_subeequeee iteration
How many to ban in each subsequent
nodict_bigramm_ind iim_iiinial: 200 found at all in the
"Nedict" here means the n-grams were not
human corpus.
aodicc_biirams_suusepeeee Trigrams
dict_ weriigram_innitiin 300
diet diet_triggee wtt_ttiigrgmms__e subeeuuent:: :
nodict trigrars intiriaammiiiiitl initial: 200
nodicc_urrigraas__bbeeeeee 0
User-Defined N-gram Bans
User-supplied extra n-grams always ban (processed by auto-antislop) extra exxra_ngramm www.nggams___ ban:
# "voice barely whisper",
OVER-REERESPNTEE WORD ANALYSTS BANNING :emputee_vverrepuwerds::: true too__core___fooovvrrreaaaaaaagg 200000
--- Quotas for Adding rverssepdeeented Words to Slop Phrase banlist Lict_overrep_ing initial: 920 # Hew many of the top represepreee
dictionary words to
ban the first antislep iteration. "Dictionary" means the words were also found in the human
# writing corpus.
dist_ sett_overrep_suusequenttt iteration
Hew many ban in each subsequent
"Nodict" here means the n-grams were not found at all in the
80
human corpus.
nodilat_overrep_.obbeeeentt
SLOP PHRASE BANNING
*
slop phrases are over-rapresented whole phrases extracted from the generated texts.
mable_slcop_hhasee min_phrase_free_tt__eess
true Min frequency for new phrase from forenforsssccs to be considered
top n Sinitial_slo__a 0 New slep phrases from slop-torensics to ban
in iter 0
33

---

Published as a conference paper at ICLR 2026
top subsequent slop ban: New slop phrases from flop-forensics later iters
User-Defined slop Phrase Bans
User supplied list of strings to always ban case insensitive
To trigger a ban, the sequence must not have a word-like character (not punctuation or whitespace) directly on either side. That is to say, we
banning disallowed sequences that longer
substrings in words. The exception is if the banned string is already bookended by a non-werd character.
Examples:
banned string "cat"
won't trigger a ban for "cation"
will trigger a ban on "cat "cat [morecat]"
banned string "cat["
swill trigger
"cat [morecat]", because the banned string ends with a non-word character.
extra_ss extra_slopp_aa phlp__hrases_to_Dae ", "rain", "tapestry", "static", "regret", "rust"
#
Whitelisted Strings
These will be excluded from the list of slop strings that the pipeline finds.
Note: special tokens in the tckenizer and parts of the chat template are
autematically whitelisted.
whitelist__ srrellist_sttngg::
# "think", "thinking"
REGEX BANNING
User-supplied regex patterns to ban
Note: unoptimised regex patterns can slow down antislop generation, as
they will be called often on large texts.
ixtra_regex_patterns:
These ones ban "it's not x, it's y" type patterns:
d( |ttpee/eeeccaggaaao.oooooooaeaa"
(b) (b /|gpa///hhaaaaaaataoooooooaaaaaa
#* [ Inlly Imeroly) Inllyyy (2
# inute|/mintes//oorr hours day day Iyaar | | (?
!with\\b (?!even)\b) (?: ( ";,"::::::3))),,,,,222:::::::: (?:itttthhy
|aa youl that) (2:\Ne were isleli |
: ( \b 1
34

---

Published as a conference paper at ICLR 2026
\bno) Natoongerrrrsseeee is (?: just only (nerely) \ [1, (1 (M
s* (2:it/they yyou \ (?:18) aare was (weee) [== [==_]]] )
#" Ab\ (?:wasn| weren ian
(ttiiannnn/aannnnieeeee aann [t/ntttttt)))) \is (3: just on
1y mmmmmer) \s* (x.xxx33) (?: (B!\\bbut \bl [.?!]) -) (1, Masbbttast ?!T\/b) (?:also) 1s+)
F
FINETUNING GAA..A*ААААААААAИT finetune_eneen _nabbed: true
General Finetuning Setup
finetunc_usc_unslottee false
finetune_mode: "ftpo" # ftpo dpe-final-token (final token preference optimisation)
finetun__ftpoodatastt: ""
you can speci fy an existing ftpo dataset,
or leave unset to let the
# pipel ine use the one produced in the generation step
finetune_ base @atase_model___eeeeeeeeeee model id: mull Base model for DPO (if unset, model id) finetune_max_see_Jlem 2500 this may truncate some outputs Finetune_lcad_in_41e true & qlera
Early Stopping
finetune finetune_early__ee wins: 0.85 # Early stopping threshold for fraction of *chosen* completions that are selected over *rejecteds.
# More than 0.85 may be overtrained.
Set to 1.0 to disable early
stopping.
finetune_eard set.eeeyyyytttiinnllssssss null # Loss threshold for early ng. Set to mull to disable.
LORA Configuration
Lonetune_lora_c. 256 # the itpo trainer works best with high lora rank Seaetuee_lora Leatuue__orra_appaa 256
Sinetune_lorr_ddeee 0.05
finetune_wwiggh__eeea: 0.01
finetune_targgt_moollee ["up_proj", "down_proj", "Im_head"]
--- Layer Freezing ---
Leaetum__freere_early__yyyy true
finettuee___layyrsssuufroeeer 5
--- Training Process
finetune_ wttpeeearrriiiiiaaaaeeeeeee
"unsloth"
iieeunne_chat_templaeee "* e.g. "gemma-3" get the chat template from unsloth's helper required, otherwise leave the string blank to use the tokeniser's chat template
bateh 3
Ainetune_warmyy wttpwaawraaaaaeeeeee ratio: 0.1 finetune_num_opochhs
- Learning Ratc
finetune_learninggraee 0. 0.000001
Eneerune_autc_llammnnnn.r.
true: automatically determine
learning rate based on dataset size, effective batch size lora rank finetune lat_aautt___rrnnee Baaiing_rttt_eeee rwte_acjjstteet__eee scalingg 0.08 scale the auto-ir by this factor
35

---

Published as a conference paper at ICLR 2026
DPO/ETPO Specific
finetune_beta: 0.1
beta
--- Output & Saving Sinttue_cctpuut_dr__saff.. "_Etpo_exp01" Appended to experiment dir
seatuune_ave_merredd Linetune__ave__guu__22 false
true
Dataset Handling for Finetuning
mneture_max_trrin_exaagg 12000 adjust needed finetune_shuffle_sedd 42
---
FTPO Sample Regularization
off; 0.9 strongly downsamples overrepresented rule violations (this is useful because the raw generated dataset typically very skewed)
0.8
strenghh:: 0.2
ftpo_ rtpo_sample_min, wwwsshhttoeeeeeee
filter out ftpo samples that have fewer than this number the chosen tokens list
FTPO-speciřic hyper-parameters
Leave any of these out (or set null) to fall back to FTPCTrainer defaults.
Loss terms computed separately for the target (chosen rejected)
tokens the remainder of the vocab. This because we want to allow more freedom movement for the target tokens.
MSE loss term 1: light loss applied tokenwise target tokens stpo_lambda_moe saseamme__ttt ins_ttaree:: 0.05 Strength of MSE loss tether on the individual logits in the
# chosentrejected set vs reference.
sypo_tauumse_tave
Grace bandwidth (logits) before the above MSE loss kicks in.
MSE loss term 2: stronger term applied to remaining (non-target) vocab
ftpo_lambda_nse:
# For chosen token: "after winning vs rejected token by this margin, preference loss turns off"
tecii_epsslln__eee
36


Reviews:
[{"review_id": "XMZYeqd0TI", "reviewer": "Reviewer_wHcb", "summary": "The paper proposes an approach to discourage repetitive patterns generated from LLMs. It first gives an approach to identify repetitive pattern by sampling model outputs and comparing frequencies of ngrams with standard frequencies. Once identified they propose and compare a couple of ways to reduce their occurrence while maintaining generation quality. The first approach is to backtrack whenever a bad pattern is detected and then regenerate from that point by sampling again while discouraging the token that led to the bad pattern by renormalizing probabilities. To mitigate the decrease in throughput due to the backtracking, they propose to use finetuning approaches using synthetic data. They propose a new approach FTPO which precisely edits logits for preferred and rejected tokens, and is able to maintain the generation quality better than DPO finetuning while reducing the undesired n-gram frequency more substantially.", "questions": "- What was the value of n in your ngrams for the investigation in section 3 and how did you decide on a value for that?\n\n- I would suggest adding equation numbers to equations (e.g. the loss equations in FTPO) for ease of referring to them\n\n- In line 323, it says \"we freeze all layers except the last 5 and lm head\" This does not sound like standard practice.. do you do experiments with training all layers via LoRA and see how that works compared to only the last 5 layers? It is okay to maybe use a lower rank LoRA but use it on all layers if gpu memory constraints are there.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem of reducing repetitive patterns in LLM generation is well motivated and an important one to solve in order to get natural texts from LLMs.\n\n- The soft-banning approach proposed provides customizable adjustment strengths which are useful to add flexibility.\n\n- FTPO approach is well motivated with well thought out components to encourage the logit distribution you want while constraining the logits so that they don’t change too much, with different components of the loss encouraging different goals.\n\n- The results show that FTPO can provide better tradeoff between quality of output and reduction in slop than DPO and other approaches (Figure 3).", "weaknesses": "- The slop fingerprinting would be dependent on what queries are fed into the model to get the outputs, and so it might be difficult to get fingerprints (overrepresented n-grams) that are more universal (i.e. show up in wide-scale use in general domains).\n\n- Not enough attention is paid to detecting and filtering semantic fingerprints that are beyond specific ngrams and of more semantic nature. For example patterns like ”it’s not just X, it’s Y.” are discussed in the paper very briefly in section 6.5 which proposes using regular expressions to detect them, but it might lead to reduction in output quality if we prevent entire clause structures like that from being generated (which is not studied in the work).\n\n- The approach of soft banning would require tuning strength parameters to use for renormalizing probabilities and it is not clear to me that a single parameter setting will work well across multiple domains in a wide scale real-life deployment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eMR8PErR62", "reviewer": "Reviewer_HpSi", "summary": "This paper proposes a new method named Antislop to prevent the large language model (LLM) from generating repetitive or overused sequences while maintaining its generalization capability. More specifically, Antislop can build a preference dataset by identifying the overused patterns in the target LLMs and then regenerating the sequences from the starting point of the detected pattern with resampling. Based on the preference dataset, the finetuning method called final token preference optimization (FTPO) is applied to adapt the target LLM, such that it could generate repetitive sequences much less frequently. Experimental results show that the FTPO can achieve a higher suppression rate of overused patterns than DPO and logit-based token banning while maintaining its performance on the downstream tasks.", "questions": "Please refer to the weaknesses section for details.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The strengths of the paper are listed as follows.    \n1. It is great that the authors have a plan to release their dataset and codes for reproducibility.   \n2. The paper is well motivated. Generating repetitive patterns is a big issue for LLMs and could affect the output quality significantly. Therefore, an effective method to address this issue is important.", "weaknesses": "The weaknesses of the paper are listed as follows.\n1. The training-based method is model-specific. It requires building a different preference dataset for different models, which is costly and thus limits its application and impact.   \n2. The experimental results are not convincing enough. First, the authors only compare their method with DPO and logit-based token banning. There are many existing works mentioned in the related work section. However, the authors did not provide any justification for not comparing with the remaining existing works on this topic. Second, the authors did not provide an ablation study of the regularization terms defined in Section 5.2.    \n3. Some important technical details are not clear. I list some of them as follows.   \na. When quantifying the slop, it is unclear how to define the pattern $p$.   \nb. When detecting the banned patterns, Antislop will lower the initiating token’s probability. However, it is unclear how to adjust the probabilities of the remaining tokens.    \nc. In FTPO, it is unclear how to obtain the set of chosen tokens for each rejected token.    \nd. Many hyperparameters in FTPO are logit-space operations. Will it make the choice of hyperparameters tricky since the search space is large?    \ne. In equation of $L_{target}$, it is unclear how to decide the value of $y_{ref}$.     \n4. The writing and organization of the paper can be improved for better readability. First, it would be better to place Figure 1 on page 5 since the first time it is referred to is on page 5. Second, it would be better if the authors could provide a figure to illustrate the difference between the proposed method and the other types of existing works to highlight the technical novelty of the proposed method. Third, before you describe the loss function of FTPO, it would be better to clearly define the form of the dataset. For example, does each data sample have labels or pseudo-labels? How to get these labels? Is the loss only calculated on the response part of each sample? Last, when introducing the benchmarks, it would be better to describe the evaluated dataset and the metrics separately instead of merging them into the same part.   \n5. The FTPO can prevent the target LLMs from generating the overuse patterns detected by the preference dataset. However, it is unclear whether it will incur new overuse patterns in the finetuned LLMs.    \n6. It would be better if the authors could provide the cost analysis of the FTPO, which is important to show the scalability of the proposed method, especially when the target LLM is large.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SbWW6QoScs", "reviewer": "Reviewer_DMCK", "summary": "This paper introduces ANTISLOP, a framework designed to identify and mitigate repetitive, low-diversity “slop” patterns in large language model outputs. The authors propose Final Token Preference Optimization (FTPO), a novel preference optimization method that regularizes model logits through a margin-based loss to discourage redundant phrasing. Combined with the Antislop Sampler, which automatically generates preference pairs from repetitive outputs, the framework enables self-supervised fine-tuning to improve linguistic diversity and stylistic quality. Extensive experiments across creative writing and reasoning datasets demonstrate that ANTISLOP effectively reduces repetition while maintaining fluency and coherence.", "questions": "Why you choose mmlu and gsm8k? In my opinion, they are more related to scientific writing (maybe), rather than creative writing. But your dataset is based on creative writing.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "\"slop\" is a very interesting entry point. The experiments are comprehensive and well-executed, covering multiple datasets and comparing FTPO with existing training methods.", "weaknesses": "* The paper lacks qualitative examples illustrating how FTPO replaces repetitive phrases with new ones. Does the model exhibit “over-pruning” or semantic drift? Without such examples, the linguistic significance of the contribution remains unclear.\n\n* The theoretical foundation of FTPO’s uniqueness and generalization is underdeveloped. At present, it still feels quite similar to DPO. You could first address this from a writing and framing perspective — for instance, by clarifying why existing preference optimization methods fail to handle “slop” effectively. Highlighting this gap might help crystallize the underlying theoretical novelty of FTPO.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ghXmzaIwMA", "reviewer": "Reviewer_VzHt", "summary": "The paper presents Antislop, a system for reducing over-represented lexical and stylistic patterns in LLM outputs. It has three components: (1) an inference-time Antislop Sampler that detects matched strings or regex patterns and backtracks to resample with a tunable penalty, (2) an automated “slop fingerprinting” pipeline that builds pattern lists by comparing model outputs to human baselines, and (3) a training objective FTPO that localizes updates to the token where a banned pattern would begin. Experiments on creative writing tasks and standard benchmarks report large reductions in targeted patterns with small quality changes. Throughput costs of the sampler are measured, and FTPO is proposed to retain suppression without runtime overhead.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Improving the humanness of LLM generation is an important problem with a lot of real-world applications. This paper provides a complete recipe from decoding to training, including a backtracking sampler, an automated fingerprinting step, and a localized training objective.\n1. The decoding overhead is measured and a training procedure is designed to internalize the discouragement of undesired patterns.\n1. Experiments span multiple model sizes and families and include both creative writing outcomes and collateral checks on standard tasks, with ablations and throughput analyses that support the claims", "weaknesses": "1. Training to discourage specific tokens or sequences has been studied somewhat extensively [1]. The contribution here can feel incremental without a stronger theoretical or empirical separation from this line of work.\n1. Broader testing on recent creative-writing setups would strengthen the work. For instance, testing on poems, stories, and jokes could serve as a complementary setting [2]. \n1. Does suppressing the most prominent patterns surface other generation patterns? These patterns may not be human like either. I am curious about the results of the study in section 3 applied to the fine-tuned model. \n\n[1] Neural Text Generation with Unlikelihood Training. Welleck, Kulikov, Roller, Dinan, Cho, Weston. 2019.  ￼\n[2] Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity. Zhang et al. 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)