Improving Black-box Robustness with In-Context Rewriting
Authors: Kyle O'Brien
Nathan Hoyen Ng
Isha Puri
Jorge Mendez-Mendez
Hamid Palangi
Yoon Kim
Marzyeh Ghassemi
Thomas Hartvigsen
Venue: Accepted by TMLR ()

Abstract: Machine learning models for text classification often excel on in-distribution (ID) data but struggle with unseen out-of-distribution (OOD) inputs. Most techniques for improving OOD robustness are not applicable to settings where the model is effectively a black box, such as when the weights are frozen, retraining is costly, or the model is leveraged via an API. Test-time augmentation (TTA) is a simple post-hoc technique for improving robustness that sidesteps black-box constraints by aggregating predictions across multiple augmentations of the test input. TTA has seen limited use in NLP due to the challenge of generating effective natural language augmentations. In this work, we propose LLM-TTA, which uses LLM-generated augmentations as TTA's augmentation function. LLM-TTA outperforms conventional augmentation functions across sentiment, toxicity, and news classification tasks for BERT and T5 models, with BERT's OOD robustness improving by an average of 4.48 percentage points without regressing average ID performance. We explore selectively augmenting inputs based on prediction entropy to reduce the rate of expensive LLM augmentations, allowing us to maintain performance gains while reducing the average number of generated augmentations by 57.74\%. LLM-TTA is agnostic to the task model architecture, does not require OOD labels, and is effective across low and high-resource settings.

Published in Transactions on Machine Learning Research (07/2024)
Improving Black-box Robustness with In-Context Rewriting
Kyle O'Brien'
Nathan Ng
Isha Puri
Jorge Mendez
Hamid Palangi?
Yoon Kim
Marzych Ghassemi
Thomas Hartvigsen
EleatherA/
Google
University of Toronto
Vector Institute
MIT CSAIL
University of Virginia
Reviewed on OpenReview: https: //pp:://ppemreee opw.revveee.........mmme fttereeeewwwweeteooooooeee formm? id= e92dgUUfk0
Abstract
Machine learning models for text classification often excel on in-distribution (ID) data but struggle with unseen out-of-distribution (OOD) inputs. Most techniques for improving OOD robustness are not applicable to settings where the model is effectively a black box, such as when the weights are frozen, retraining is costly, or the model is leveraged via an APL Test-time augmentation (TTA) is simple post-hoc technique for improving robustness that sidesteps black-box constraints by aggregating predictions across multiple augmentations of the test input. TTA has seen limited use in NLP due to the challenge of generating effective natural language augmentations. In this work, we propose LLM-TTA, which uses LLM- generated augmentations as TTA's 's augmentation function. LLM-TTA outperforms conven- tional augmentation functions across sentiment, toxicity, and news classification tasks for BERT and T5 models, with BERT's OOD robustness improving by an average of 4.48 per- centage points without regressing average ID performance. We explore selectively augment- ing imputs based on prediction entropy to reduce the rate of expensive LLM augmentations, allowing us to maintain performance gains while reducing the average number of generated augmentations by 57.74%. LLM-TTA is agnostic to the task model architecture, does not require OOD labels, and is effective across low and high-resource settings. We share our data models and code for reproducibility.
1 Introduction
Text classification models deployed real-world settings must excel on in-distribution (ID) inputs sampled from their training distribution and be robust to unscen -uttof-distributionn (OOD) inputs. OOD robustness is important. for deploying safe and trustworthy models in real-world settings (Hendrycks et al.
2021 This
challenge is especially acute in high-stakes settings such content moderation (Ashish et al. 2023 Zhou
et al., 2021), spain detection (Dada et al.] 2019), and healthcare (Rasmy et al.) 2020). OOD robust.ess in
NLP is challenging in practice due to the complex nature of natural language data, adversarial examples
(Goyal et al. 2022), and shifting domains (Yuan et al. 2023 Koh et al. 2020 Yang et al. 2023
Most existing methods for improving OOD robustness in NLP require access to model weights by modilying
the training process (Howard & Ruder 2018 Ma et al. 2019 Ruder et al. 2019 Tu et al. 2020 Yaghoobzadeh
ett al. 2021) or adapting the model to new domains at test time (Wang et al. 2023b). Modifying the task model can be challenging in practice when retraining costly, the underlying model is abstracted away from the practitioner, or sufficient OOD labels are unavailable. These constraints render the task model effectively
black box. In this work, we study the NLP task of short-form text classification in black-box setting by turning altention lowards the inputs to the model.
Test-time augmentation (TTA) sidesteps the need for modifying the task model new labels by aggre- gating multiple predictions over augmentations of the test input, thus arriving at more robust predictions.
http:///maggiiaeeeccoo///lllelts http:///gthaa..coogggglll///////

---

Published in Transactions on Machine Learning Research (07/2024)
The choice of textual augmentation function is critical since augmentations must be diverse and semantic- preserving (Shanmugam et al.
2020), challenge with conventional augmentation functions such as word
insertion and synonym substitution (Xiong et al. 2023). LLM-driven advances in machine translation (Kocmi
& Federmann 2023 Wang et al. 2023a Zhu et al. 2023), paraphrasing (Witteveen & Andrews 2019 Wahle
et al. 2022; Cegin et al. 2023). and style transfer (Patel et al., 2022; Sungun et al. 2022 Roy el al. 2023)
indicate that higher-quality augmentations are now feasible. In this work, we study using LLMs as the
augmentation function for TTA (LLM-TTA).
Our study is composed of nine public datasets and one novel synthetic dataset across sentiment analysis, toxicity detection, and new topic classification. We consider the dataset used for model optimization as ID, while the other challenging evaluation sets are considered OOD. The experimental selup and its limitations are discussed in Sections 4. and 6
We experiment with two LLM-TTA methods: zero-shot paraphrasing, where we prompt the LLM to gener- ate paraphrases of the input text, and In mn-Context Rewriting (ICR), where the LLM rewrites the input to be more like a set of ID exemplars provided in the prompt. Both methods outperform TTA with conven- tional augmentation functions for BERT and T5 across averaged across nine datasets for sentiment, toxicity, and news topic classification. These results demonstrate that LLM-TTA is simple black-box robustness technique effective across multiple tasks. Our primary findings are:
1. LLM-TTA Improves OOD Robustness. ICR improves BERT classifier's absolute accuracy on OOD data by an average of 5.12% for sentiment, 7.18% for toxicity, and 1.15% for news topics, all with minimal regression to ID performance. For toxicity, BERT's ID accuracy improves by 2.99%, suggesting that LLM-TTA can improve both ID and OOD performance in some scttings. Ablating training set size (Section 5.5 shows that. LLM-TTA is useful in data-scarce and rich settings.
2. TTA with Conventional Augmentation Functions Often Hurts Performance. In contrast to LLM-TTA, TTA with conventional augmentation functions generally hurt both ID and OOD performance. Back-translation is the best-performing conventional augmentation functions, with BERT's OOD robustness improving by an average of 2.85 percentage points while word insertion regresses performance by -0.99 points and substitution by -0.53 points averaged across tasks.
3. Selectively Augmenting High-Entropy Test Inputs Improves Efficiency. We can reduce the rale of expensive LLM augmentations by only augmenting test inputs in which the task model uncertain in its prediction. For BERT as the task model and ICR as the augmentation function, we reduce the percentage test inputs requiring augmentation by an average of 57.74% while still improving robustness. The uncertainty threshold only determined from ID statistics.
2 Related Work
Text Augmentation. Data augmentation is strategy that enables practitioners to significantly increase the diversity and quantity of data available without collecting additional annotations (Feng et al.] 2021) Train-time augmentation has been found to improve performance in low-rcsource scenarios (Chen et al. 2021), mitigating harmful features such as gender bias (Zmigrod et al. 2019), and improving robustness
[Morris et al.] 2020 Ng et al. 2020).
Textual data augmentation can be performed at the character (Karpukhin et al. 2019 , word Zhang et al.
2015
Kobayashi 2018 Wei & Zou 2019), or whole-text (Vickrey & Koller 2008 Hou et al. 2018 Yu et al,
2018
Anaby-Tavor et al. 2019 Kumar et al. 2020) level. One concrete example is word substitution (Weil
& Zou 2019),, where cach word in the text has some probability of being replaced with a related word. Increasing the likelihood of replacement can result in more diverse augmentations but comes with the risk of
losing the original semantic meaning of the source example (Xie et al. 2019 Bayer et al., 2021). Conversely, augmentations that do not introduce sufficient diversity are unlikely to be effective for large pretrained models (Longpre et al. 2020).
Test- Best-Time-To Adaptation. This approach extends TTA by updating the weights of the source model test- time (Liang et al. 2023). Adaptation is commonly implemented in an unsupervised manner by minimizing proxy for supervised loss such as entropy (Wang et al. 2021 Zhang et al. 2021), distribution alignment
2

---

Published in Transactions on Machine Learning Research (07 2024)
Training Distribution
Test Distribution
News
Social Media
Wrong X
News Model
BREAKing
#PRESIDENT just ANNOUNCED
Topic: Sports
LLM Test-Time Augmentation
Augmentations
Correct
LLM
News Model
Just in: The president formally announced...
Topic: Politics
Figure 1: LLM-TTA. In settings where the task model is effectively black box, intervene the input data to improve robustness. We propose rewriting OOD imputs at test time using an LLM to improve robustness (LLM-TTA). Our experiments find that LLM-TTA improves performance without requiring task model access or OOD labels.
Hassan et al. 2023 or through perturbing model internals such as with dropout (Liang et al. 2023 Crucially, this technique assumes the ability to modify the source model. Singh & Ortega (2022) extended the NLP TTA evaluation from Lu et al. (2022) to include adaptation and found that entropy minimization- based adaptation can improve performance accuracy on OOD toxicity detection.
Test-Time Augmentation. TTA refers to the practice of aggregating predictions across multiple aug- mentations an original test input during inference. This technique has been widely used in computer vision
to improve ID performance (Moshkov et al. 2019 Ashukha et al. 2021 OOD robustness (Enomoto et. al.
2022
Kim et al.
2020
Molchanov et al.) 20200) adversarial robustness (Song et al. 2017 Prakash et al.
2018
Cohen et al.
2019) and uncertainty estimation (Ayhan & Berens 2018 Gawlikowski et al. 2021). Lu
et al.
2022) provided the initial foray into studying TTA for NLP. They studied how common word-based
angmentations improve the robustness of a distilBERT (Sanh et al.,
2019) on the WILDS CivilComments
[Koh et al., 2220)) dataset. Matiana et al. (2021) combines cosine similarities of model representations across multiple paraphrases of the input to classify stories. Xiong al.) (2023) explored dynamically assigning different weights to augmentations during aggregation to reduce the influence of potentially noisy augmenta- tions. These works found that incremental improvements in accuracy are possible but that the word-bascd augmentation functions are a bottleneck for performance. Our study differs from these works in that we focus our evaluation on LLM- based TTA and across a more diverse set of tasks, models, and augmentation functions, as well as measure ID performance alongside OOD robustness.
3 LLM-TTA: Generating Faithful Augmentations With LLMs
In this section, we describe LLM-TTA, method for performing test-time augmentation for natural language classification using the rewriting capabilities of large language models.
3.1 Augmenting Test Inputs
In this paper, we consider a supervised classification task from an input space X E R" to a discrete output space Y with K classes. We assume access to a model f trained on a dataset Dtrain = from a distribution Ptrain (X, Y). test-time, we are given a point x sampled from an unknown distribution Pood X, Y), which may or may not be the same as our original training distribution.
sampled
TTA is a simple post-training approach for improving the generalization accuracy of f on arbitrary inputs x. The main aim of TTA to reduce the effects of poor prediction on a single point by aggregating them across many similar augmented points. TTA involves three steps: augmentation, inferences, and aggregation.
3

---

Published in Transactions on Machine Learning Research (07 (024))
1) Augmentation
2) Inference
Aggregation
In Unfaithfal, Diane Lange impresses.
Diane Lane shines "Unfatthful"
Classifer
Dione impreasivelly performs in Unfaithful.
LLM
Distributton Examples
Negative
Neutral
Positive
Diane Unfalthful.
In-Context Rewriting
Diane Lane impresses Unfalthful.
Figure 2: TTA Inference Steps. This figure shows the three stages of TTA. The process begins with augmentation function generaling multiple altered versions of the current test input. For ICR, the input to the LLM also contains ID examples. The task model then makes predictions over the test input and its augmentation. Lastly, we aggregate the predictions to arrive at a "smoothed" judgment. Standard aggregation methods include mean probability aggregation (demonstrated in this figure) and aggregation.
As described in Section 2 TTA differs from lest-time adaptation (Wang et. al., 2023b) in that the weights
of the source model remain frozen. Unlike train-time augmentation (Shorten & Khoshgoftaar 2019 Bayer et al. 2021) TTA does not require practitioners to modify their training regimes. Section 2 provides further description of these alternative techniques. We use terminology inspired by Shanmugam al. (0020) and notation from Goodfellow et al. (2016).
Augmentation. We define an augmentation a, as stochastic transformation of an input x from which we can sample another augmented point x' (xxxxxx.. Given a set of such augmentations set of single samples from cach of the
M {a.]m define M(x) = fx/ (x(x' (x augmentations in M. For ease of notation, assume one of the transformations in M is the identity transformation 7(x) such that the original point is always in Mxx).
2. Inference. Each of the points in (x) is passed into the model f to generate a set of predictions f(M(x)) [
Aggregation. A final prediction ý Gx)(M(x)) is derived from an aggregation function G that combines the set of predictions f fMM(x)). Common aggregation methods include vote-based aggre- gation, where the most commonly predicted class is chosen, and mean-based aggregation, where the probabilities for each class are first averaged across augmented samples before prediction is made.
3.2 Rewriting Test Inputs with Language Models
We study whether employing TTA with augmentations generated by an LLM (LLM-TTA) will improve task-specific classifier's robustness. LLMs have achieved strong performance on adjacent tasks that require faithfully rephrasing a target text. We hypothesize that the LLM-generated augmentations will outperform conventional a augmentation functions. so, these results suggest that L.MM-TAA a promising general non- parametric technique that NLP practitioners can leverage without depending on the task model architecture access weights.
In LLM-TTA, LLM P. an inference process where an augmentation of the natural language test input generated by language model (LLM) conditioned with a natural language prompt as well the original input x. We study two prompt templates for P detailed in Figure

---

Published in Transactions on Machine Learning Research (07/2024)
LLM-TTA: Paraphrasing
LLM-TTA: In- n--Cotexxt Rewriting (ICR)
#** Instructions ###
Instructions
The assistant is to paraphrase
The assistant is to paraphrase the input text as if was one the examples.
the input text.
Change the details of the text if necessary.
#** Input Text ###
Now paraphrase {t(" ((("<aayll_inppuu input>"]]]
#** Style Examples #*# <style_ twanstrrnneeeeee _raasserreeeeplarss
Return the text in the format: (((Paraphased Texth}}
### Input Text ###
Paraphrased Text
Now paraphrase "erretyll_nnput""""""""" as if was one the details of the text if necessary.
Paraphrased Input Text: {[[a## Assistant:
examples. Change
Return the text in the format: "'Paraphrased Text rre
#** Paraphrased Text Paraphrased Text:
Figure LLM-TTA Prompts. We evaluate LLM-TTA with two prompting methods. "<style imput>> is replaced with the test input and "<style transfer eremplors>' with the ID examples during inference. During the course of prompt engineering, find that instructing the LLM to generate text in specific formats surrounded by brackets and to change the details of the text while preserving semantics leads to the best performance.
1. Paraphrasing. P contains instructions for the model to generate semantic- paraphrase of the test input. This approach is zero-shot in that P contains no examples, thus not requiring access to ID data. We expect simple paraphrasing done by capable LLM can generate diverse and semantic preserving augmentations, even for subtle labels such as sentiment.
2. In-Context Rewriting (ICR). In this approach, we prompt the LLM to rewrite the given OOD example to be more like a set of ID exemplars. ICR does not require practitioners to articulate the semantics or writing style of the original distribution. Instead, the LLM prompted to infer the differences between examples and via in-context learning. We hypothesize that rewriting OOD imputs to be more like ID data can further improve performance over simple paraphrasing since the task model was trained to excel on ID data.
3.3 Entropy-Basee Based Selective Augmentation
LLM inference is inefficient when the model's predictions are invariant across the test input and its aug- mentations. This invariance is a blocker for overruling an incorrect prediction for the test input. To reduce the rate of expensive LLM inforences, we explore whether the entropy of the task model's class probability distribution predictive heuristic for whether the model will make incorrect prediction. We se- lectively augment test examples that are likely to benefit from TTA while deferring to the model's original prediction otherwise. Lower entropy has been observed to be correlated with correct predictions in machine learning models (Grandvalet & Bengio 2004). This observation has motivated entropy to be leveraged as an unsupervised loss metric in test-time adaptation techniques (Wang et al. 2021 Zhang et al. 2021 Singh
& Ortega 2022
Entropy-basee based selective augmentation (ESA) can be formally defined as follows. Let x the test input, (x) be the entropy of the output probability distribution (y()) of the model f, and e be an entropy threshold. For ease of notation, let ATA(x; f: M) be the aggregated prediction for f using TTA with an augmentation function M. We then define the model prediction ý

---

Published in Transactions on Machine Learning Research (07 (07/2024)
TTA M) arg magy Ps (v/x) otherwise
if 2x
y
(1)
TTA is only used when the original prediction entropy is above a predetermined threshold. The percentage of examples above this threshold (thus requiring augmentation) is referred to as the augmentation rate. We assume access to the labels for the ID evaluation set to determine optimal threshold. Within this evaluation set, optimal threshold is calculated that balances gains in accuracy while minimizing the augmeniation rate, details of which are described in Appendix A.4
4 Experimental Setup
We study how well LLM-TTA can improve the robustness of various lask models across three short-form text classification tasks. We simulate a black-box setting by keeping the weights of the task model frozen test time and study methods that are task model architecture independent. The following sections describe our datasets, task models, and TTA augmentation functions.
4.1 Datasets
We use three ID evaluation datasets and seven OOD datasets across sentiment analysis, toxicity detection, and news topic classification. Each task model is optimized for the ID evaluation set either through fine- tuming for BERT and T5, or prompting with Falcon. The training splits for the OOD datasots are not used. Dataset statistics are listed in Appendix A.6
We refer to the dataset that optimize the model for ID. The challenging splits from other datasets, which the ID model struggles generalize, are considered OOD. We do not investigate the specific properties that make these dalasets challenging This abstract notion of OOD contrasts with other works (Hendrycks 2020a Koh et al. 2020 See Section 6 for further discussion.
et al.
Sentiment Classification. We consider three sentiment classification datasets from the BOSS benchmark (Yuan et al.
2023). Each is three-way classification task where the labels are positive, neutral, and negative. We keep this benchmark's selection of ID and OOD splits. The ID dataset consists of Amazon reviews (McAuley & Leskovec 2013), while the three OOD datasets are DymaSent (Potts et al.] 2020 SST-5 (Socher et al.j (013)), and SemEval Nakov et al., 2016). Yuan et al. (2023) selected these three OOD shifts since their centroids had low cosine similarity with the ID (Amazon) centroid.
Toxicity Detection. We also leverage the toxicity task in the BOSS benchmark. Toxicity detection is framed as a binary classification task between non-toxic (negative) and toxic (positive). We similarly rely the ID and OOD dataset selections from Yuan et al. (2023) The ID dataset is Civil Comments (Borkan eet al. 2019), collection of users' comments on articles on the Civil News platform. The OOD datasets are AdvClvil, an adversarial version of Civil Comments introduced in the benchmark, as well as existing datasets ToxiGen (Hartvigsen et al. 2022) and Implicit Hale (Elsherief et al.] 2021). As with sentiment, these shifts were selected by Yuan et al. 2023 since they have low cosine similarity with the ID centroid.
News Topic Classification. AG News Zhang et al. 2015 a four-class news topic classification problem where the model is tasked with determining whether a given news article pertains to "World," " "Sports," " "Business," " or "Sci/Tech."" This task has a single OOD dataset, a novel datasot that is composed of the AG News test style-transferred to resemble social media posts. Since each entry in the ID evaluation set has corresponding style transferred entry in the OOD set, style have performance. Additional details on how this dataset created can be found in Appendix A.7
isolate the effect that differences in writing
4.2 Task Models
We study black-box robustness for three task models. First, we fine-tune BERT (Devlin et al., 2019) and T5-Large (Raffel al. 2020) for each ID dataset. Second, we consider Falcon-7b, an instruction tuned LLM
Almazrouui et al. 2023) To make predictions with Falcon, we 16-shot prompt with randomly selected

---

Published in Transactions on Machine Learning Research (07/2024)
ID exemplars with equal numbers per class. Fine-tune details are in Appendix A.5 and Falcon prompting details are in Appendix A.9
4.3 Baseline Augnmention Functions
We compare our LLM-TTA augmentation functions with three representative functions studied in the NLP literature.
Word-Level Augmentations. Existing works studying TTA for NLP have focused on world-level aug- mentations (Lu et al., 2022] Xiong et al. 2023). We use the word insertion and substitution methods implemented in nipang recommended by Lu et al. (002). We use this library's default parameters, where each word in the lext has a 30% chance of being auginented for a maximum of 10 words. Insertion adds new word after a target word in the text based on the word BERT predicts will come next based on the preceding and subsequent words. Substitution follows similar approach, with the target word being replaced. Using BERT's predictions reduces the chance of adding nonsensical words that may change the semantics of the text.
Back-Tnanslttion. We select English+ German back-translation as a representative whole-text augmen- tation function. While not studied in the NLP TTA literature, translation is a common data augmentation technique (Edunov et al. 2018: Xie et al. 2019 Ng al. 2019). Stochastic translations act as paraphrases of the original text. We select German as the target language since English+ is common pairing in the literature (Edunov et al. 2018 Sennrich et. al., 2015 Hoang et al. 2018). Including a whole-text augmentation function in our experiments allows us to study how well paraphrasing with and without an LLM affects performance. Model and decoding specifics are included in Appendix A.9
4.4 Multiple Evaluation Runs
To ensure that performance is robust across generated augmentation, we run our OOD evals four times using various seeds: 3, 16, 46, and 58. The results reported for OOD shifts are the mean values derived from these four runs, as well as the standard deviation. Each augmentation function is non-deterministic, which results in differing augmentations being generated across runs. This approach allows us to assess how consistently each augmentation ion function affects performance. We conduct a single run for the ID datasets.
4.5 TTA Settings
A Aggregation. Following Lu et al. (002)) each experiment uses the test input and four augmentations to arrive at final prediction. We only mixing augmentation functions to better study the effect that specific augmentation functions have on performance. We use two different aggregation methods for mapping the predictions over the augmentations to final class prediction. For BERT, we average the class probability distribution across the test input and its four angmentations and sclect the class with the highost probability. T5 and Falcon use a voto-basod aggregation method where a verbalizer function maps a set of lask-specific valid tokens (such as "1', "pos," and "positive") class label. We then select the most commonly predicted class across the five inputs the final prodiction.
single augmentation function per experiment rather than
LLM-TTA. We use Stable Beluga 2-7B (SB2) to generate augmentations. SB2 is a L.Lama 2 Touvron ct al. 2023 model finc-tuned with additional instruction tuning. Figure 3 details the prompts for the LLM- TTA paraphrasing and ICR methods. These prompts and the test input are passed to the model, resulting in four stochastic augmentations. ICR uses 16 randomly selected unlabeled exemplars balanced across classes sourced from the IDD training set. We further describe the decoding details in Appendix A.9
we study whether only augmenting test inputs in which the predicted class probability distribution is above a predetermined threshold still improves performance while reducing the augmentation rate. We find the optimal threshold for the ID test set that strikes a balance between performance gains and augmentation rate. We do not rely on access OOD labels. We further describe this methodology in Appendix A.4
ESA Threshold. As introduced in Section 3.3

---

Published in Transactions on Machine Learning Research (07/2024)
Sentiment
Turkitt
Toret
Augmentation
BERT
T5
Palaon
BERT
T5
Falron
BERT
T5
Falron
Noms
52.0555
58.00%
46.71%
53.85%
56, 6355
59.11%
08-55756
88.90%
25.8658
Insert [Lu
52.06%
06.11%
45.82%6
68.11%
57 37.5996
67.30%
88.800 08.809
89.69%
24.96%
Sobsttruse
51.44%% 16.3
54.15%
16.3
19.3
58.87%
09.777%
16.1
10.1
Trundate
Semnzcch
2015
52.4155 45-1
66.00%
44.0255
85.0
62.115
63.4055 30.0
60.05%
20.0
88.55555
26.1
88.13%
=aa
26,8855
400
LLM-TTA: Paraprrae
55.24% 16.2
30.01%
10.3
45.139%
a64
64.67%
10.3
67.20%
10.5
58.00%
10.0
88 97.997
10.0
80.06%
20.1
27.307
10.2
LLM-TTA: ICR
37.17% 20.5
50.75%
20.3
43.79%
20-3
61.00%
2n.2
03.80%
20.8
58.115
20.3
89.72%
20-1
90.30%
20.1
27.82%
20.9
Table 1: OOD TTA Performance. We report the mean and standard deviation of task model accuracy with TTA using various augmentation functions across four runs. Sentiment and toxicity results are averagod across the three OOD shifts for each task. Results are divided between TTA with conventional augmentation functions and LLM-TTA (our method). LLM-TTA is the best-performing augmentation function for BERT and T5 across all tasks. Larger models tend to benefit less from TTA, with all TTA methods hurting Falcon's 's sentiment and toxicity performance. These results suggest that LLM-TTA can be useful technique for improving task model robustness.
Sentiment
Toxicity
News
Tweets
Augmentation
BERT
T'
Falcon
BERT
T5
Falcon
BERT
Palcon
None
90.38%
90.12%
90.49%
88.46%
90.57%
17.01%
91.13%
94.87%
30.25%
Insert
[L et al. 2022
90.48%
88.40%
89.41%
89.25%
90.92%
16.94%
94.89%
92 22%
26.84%
Subalitute LLu el al 2022
89.96%
85.78%
86.58%
89.90%
91.33%
19.99%
94.86%
91.91%
26.34%
Translate Sennrich al. 2015
89.05%
87.22%
87.14%
89.64%
90.65%
17.39%
93.92%
92.70%
30.12%
LLM TTA: Paraphrase
90.54%
89.90%
90.35%
90.58%
91.32%
18.43%
93.72%
92.05%
28.78%
LLM TTA: ICR
90.85%
90.78%
90.35%
91.45%
91.87%
19.00%
93.33%
91.99%
29.17%
Table 2: ID TTA Performance. We continue the format from Table to report ID accuracy. Results are from a single experiment run. While TTA works best as an OOD robustness technique, it can also improve ID performance. LLM-TTA improves BERT and T5 performance on sentiment and toxicity yet regresses performance news. Similar to OOD robustness, TTA is most effective for BERT and can often hurt Facon's 's performance. These results suggest that LLM-TTA can improve OOD robustness without regressing ID performance, but the degree of gains depends on the model and task.
5 Results
5.1
LLM-TTA Improves OOD Robustness
Generating augmentations with LLMs outperforms all other augmentation functions for BERT and T5 across each OOD task, as shown in Table We report the average accuracy across the multiple OOD shifts for sentiment and toxicity which is then average across four experiment runs. ICR is the best-performing augmentation function, with BERT's OOD performance improving by an average of 4.48 percentage points and T5's by 2.66 percentage points. Falcon benefits less with LLM-TTA regressing performance on sentiment and toxicity for an overall net regression of 0.67 percentage points.
ICR is the overall best-performing augmentation function. Augmenting test inputs to be more like ID exemplars outperforms 0-shot paraphrasing. These gains are achieved without requiring the entire ID dataset at test time, OOD labels, or explicit descriptions of the original distribution within the prompt. O-shot paraphrasing also outperforms all conventional augmentation functions and ICR for the toxicity task. These results suggest that I.LM-generaeed augmentations generally out putperorr conventional augmentations, and that performance can be further improved for some tasks by leveraging ICR over simple paraphrasing.
LLM-TTA can improve task model robustness without modifying the task model's weights or training rogime. TTA's 's flexibility allows it to be slotted into existing systems. The degree to which TTA improves robust.ness is contingent on the augmentation function, with whole-text augmentation functions outperforming word-level augmentations. The modest gains, even with a powerful augmentation function, suggest that augmentation quality may not be the only bottleneck for improving black-bor robustness. While we observe improvements, further progress is needed to make models more robust to OOD shifts as test. time.
8

---

Published in Transactions on Machine Learning Research (07/2024)
Sentiment
Toxicity
News + Tweets
Augmentation
ID
OOD
ID
OOD
OOD
LLM-TTA ICR: BERT
90.85%
57.17%
91.45%
61.06%
93.53%
89.72%
LLM-TTA ICR: T5
90.78%
59.75%
91.87%
63.83%
91.99%
90.30%
LLM-TTA ICR: Falcon
90.35%
43.79%
19.00%
58.11%
29.47%
27.82%
LLM
86.53%
50.63%
76.32%
71.76%
81.39%
80.50%
Table 3: Direct LLM Performance vs LLM-TTA. We report ID accuracy and mean OOD accuracy across shifts and seeds for the augmentation LLM (SB2) and the task models with LLM-TTA, BERT and T5 outperform the LLM on all ID datasets and 2/3 OOD shifts. Falcon generally underperforms SB2. SB2 outperforms all lask models in terms of OOD toxicity performance. These results suggest that LLM-TTA can be effective even when the LLM underperforms the task model, but there may be instances where using the LLM directly is more efficient.
Sentiment
Toxicity
News - Tweets
Augmentation
Accuracy
Aug Rate
Accuracy
Aug Rate
Accuracy
Aug Rate
None
52.05%
53.88%
88.57%
Paraphrase: Default
55.24% #0.2
100.00%
64.67% +0.3
100.00%
88.97% =0.0
100.00%
Paraphrase: ESA
54.77% +0.1
50.51%
61.24% +0.3
66.50%
88.80% -0.1 1.62%
ICR: Default
57.17%
100.00%
61.06%
100.00%
89.72%
100.00%
ICR: ESA
56.38%
56.53%
58.33%
10.1
66.50%
89.21%
3.76%
Table 4: BERT Performance with Entropy-Based Selective Augmentation (ESA). Metrics are mean accuracy across runs and augmentation rate: the percentage of test inputs that are angmented. The augmentation rate is identical across runs. Default refers to when every test input is augmented. Selective augmentation can improve performance while augmenting significantly fewer inputs, thus reducing the cost of expensive LLM-based augmentation.
5.2 LLM-TTA Can Improve ID Performance
Preserving ID performance is essential in settings where the overall distribution has only partially shifted. Table reports ID gains using TTA. LLM-TTA improves ID performance for BERT and T5 for sentiment and toxicity but not for news. ICR is the best-performing augmentation function for these models and distributions. All augmentation functions regress Falcon's ID performance results follow a similar pattern to the OOD results reported in Table ! where LLM-TTA performs the best for BERT and T5. These results suggest that LLM-TTA does not seriously regress ID performance and can even improve ID performance in some settings.
sentiment and news. ID
5.3 LLM-TTA Can Outperform the LLM
LLM-TTA augments every test input by default. In practice, using the augmentation LLM directly for the task may be advantageous if it outperform the task model. In this section, evaluate Stable Beluga 2 (SB2), the LLM we use for augmentation in our experiments, directly the tasks. SB2 is prompted with the same task templates and exemplars as Falcon detailed in Appendix A.9
We report our results in Table 3 SB2 underperforms BERT on all ID datasets and most OOD sets. SB2 overall outperforms BERT and T5 on the OOD toxicity detection and outperforms Falcon across most tasks. These results show that LLM-TTA can still improve a task model's robustness even when the LLM itself underperforms the task model. However, using the L.M directly may be practical in some settings.
9

---

Published in Transactions on Machine Learning Research (07 (07/2024)
5.4 Selective Augmentation Improves Efficiency
Tables and demonstrate that LLM-TTA can improve performance when every test input is augmented. However, augmenting each imput can be costly when using L.Mss. We introduced entropy-based selective augmentation (ESA) in Section 3.3 to address this issuc. ESA only augments test imputs in which the model is uncertain in its prediction. We use the entropy of the OOD test inpul's predicted class probability distribution as the confidence measure. The intuition behind this method is that TTA is unlikely to be effective when the task model is confident in its prodiction for the unaugmented test inputs and is thus unlikely to change its prediction across faithful augmentations. We use BERT as our task model since this model explicitly outputs a class probability dist ributiion and benefits the most from TTA.
Table | demonstrates that selective augmentation can preserve most performance gains while drastically reducing the number of expensive LLM calls. ICR tends to benefit more than paraphrasing. BERT's robustness on sentiment is improved by 4.33 points when only augmenting 56.53% of inputs compared to 5.12 points when augmenting every input. Similarly, the majority of gains are still realized for toxicity while reducing the augmentation rate by roughly a third. News only preserved half of the performance gains yet reduced the augmentation rate by over 95% for ICR and paraphrasing.
That performance can still be improved while augmenting drastically fewer OOD inputs suggests that TTA does not change most model predictions. Selective augmentation is a promising direction for narrowing in on test inputs likely to benefit from augmentation while avoiding those that will not benefit. The entropy- based approach we studied can meet this criterion without requiring OOD labels. Future techniques can improve upon this approach by leveraging other signals beyond entropy.
5.5 LLM-TTA Is Effective in Both Data Scarce & Rich Settings
The BERT models studied previously are trained on the full training set numbers tens of thousands of examples. Whether LLM-TTA is effective across data scales is of interest since many practitioners operate in data-scarce regimes. It is unclear if LLM-TTA's 's performance generalizes to models trained on far fewer examples.
In this experiment, we study whether LLM-TTA can improve task model robustness across data scales. We train 5 BERT models on 5%, 10,, 20%, 40%, and 80% of the ID training set for cach of our three tasks. The base models and hyperparameters are identical across runs and follow the training regime outlined in appendix section Appendix A.5 across classes.
We build each balanced training subset via stratified random sampling
Figure 4 shows the average performance improvement over the no- baseline averaged across OOD shifts. LLM-TTA's performance peaks at 10% of the training set for sentiment, 100% for toxicity, and 20% for news. The deltas between the best-performing dataset size and the fully trained sets are generally small, suggesting that TTA is only marginally more useful in low-resource settings. The takeaway for practitioners is that, given identical base models, LLM-TTA's performance in high-resource settings broadly generalizes to low-resource settings and vice versa.
5.6 LLM-TTA Affects Some Classes More Than Others
Whether TTA affects some classes more than others is of interest since many ML techniques can increase net performance while hurting specific classes or subgroups (Shanmugam et al., 2020 Yang et al. 2023 Kirichenko et al., 2023). Figure 5 shows the percent of all changed judgments broken down by class for BERT using ICR as the augmentation function. There meaningful variance across classes depending on the task. For sentiment, there are far more new corrections than new mistakes for the positive and neutral classes, but there are slightly more mistakes for the negative sentiment. LLM-TTA only benefits non-toxic examples in the toxicity task. Fewer predictions change overall for news task classes than sentiment and toxicity.
New mistakes dampen the performance gains introduced by TTA. Even classes where TTA overall improves performance can have many new mistakes. Mitigating the trend of two steps forward and one step back is promising direction for further improving TTA's effectiveness.
10

---

Published in Transactions on Machine Learning Research (07 (024)
Sentiment
Toxicity
News
1.0-
Accuracy Gain
4.0
Accuracy Gain
7.5
Accuracy Gain
5.0
0.5
2.0
2.5
0.0
0.0
0.0
1500
3000
6000
12000
24000
3000
6000
12000
24000
48000
4800
9600
19200
38400
76800
Training Set Size
Training Set Size
Training Set Size
Insert
Substitute
Translate
LLM-TTA: Paraphrase
LLM-TTA: ICR.
Figure 4: TTA Effectiveness Across Data Scales. This figure shows the absolute improvements in OOD accuracy averaged across shifts and experiment runs with standard deviations. We train five BERT models on 5%, 10%, 20%, 40%, and 80% of the ID training set. We find that LLM-TTA improves robustness across data scales. These results suggest that LLM-TTA can still be helpful for practitioners operating in data-scarce regimes.
Sentiment
Toxicity
News
3
Percent Outcomes
Percent Outcomes
Percent of Outcomes
0.6
4
2-
0.4
2
0.2
0.0
Negative Neutral Positive
NoxiToxic
Toxic
World Sports
Business Sci Tech
New Correct
New Mistake
Figure 5: Changed Predictions Across Classes. Results from BERT with ICR the TTA augmen- tation function across all OOD inputs. Variance across classes indicates that TTA affects some classes more than others. TTA can hurt the performance of some classes while improving overall performance.
5.7
The Optimal Number Augmentations Varies by Task
The number of augmentations generated per test input an important hyperparameter in TTA. Determining optimal augmentation count that balances performance improvements and efficiency is critical when augmentation is expensive, as with LLM-TTA. In this experiment, study BERT's OOD robustness averaged across OOD shifts using the test input with varying numbers of augmentations.
Figure c reports performance across augmentation counts. LLM-TTA's performance largely plateaus af ter two augmentations, whereas word-level augmentation functions can benefit from larger augmentation batches. These results demonstrate that practitioners may be able improve efficiency by using fewer augmentations without compromising performance.
11

---

Published in Transactions on Machine Learning Research (07 2024)
Sent inmett
Toxicity
News
5.0
1.0
10.0
Accuray Gain
Accuracy Gain
Accuracy Gain
2.5
0.0
5.0
0.0
1.0
-2.5
0.0
1
2
3
4
2
3
2
3
4
Number of Augmentations
Number of Augmentations
Number of Augmentations
Insert
Substitute
Translate
LLM-TTA: Paraphrase
LLM-TTA: ICR
Figure TTA Effectiveness Across Augmentation Count. We report the improvements in OOD accuracy averaged across shifts for each augmentation function and the number of augmentations used per inference. LLM-TTA's performance generally plateaus after two augmentations per inference.
6 Limitations
We study how well LLM-TTA can improve performance without investigating the specific factors contributing to the datasets being OOD just that models struggle to generalize to them. Other works have proposed a more nuanced view of OOD robustness (Hendrycks et al.J 2020a Koh et al.) 2020: Taori et al. 2020).
A common alternative approach to studying OOD robust is to control for the shifts by modifying specific propertics of the training and evaluation sets, such as increasing length (Varis Bojar 2021 Ruoss et al. 2023 Zhou al.) 2024), intruiign perturbations (Hendrycks & Dietterich 2019), and evaluating evaluation sets are
examples harder than those seen in training (Hase et al., 2024). In contrast, selected based on low cosine similarity with ID centroids or synthetically created writing style shifts. These criteria provide an opportumity to study generalization more broadly but do not rule out the possibility that the evaluation data contains challenging ID) samples. Future work could further isolate and measure the properties that make datasets OOD to better understand which types of shifts LLM-TTA can most effectively improve.
LLM-TTA outperforms TTA with conventional augmentation functions across tasks for most task models we study. However, these gains come at the cost of incrcasing the computation and latency required for gener- ating augmentations. This tradeoff diminishes the LLM-TTA'S 's utility in low-compute settings. LLM-TTA 's performance improvements are generally modest and insufficient for undoing the performance regression caused by OOD shifts, even when leveraging a capable LLM for augmentation. Furthermore, it may be practical in some settings use the LLM directly for the task instead of augmentation if it is superior the task (Section 5.3
Larger task models tend to benefit less from LLM-TTA. It remains unclear whether the size of the pretraining corpus, model architecture, or another factor influences how well a task model will respond to LLM-TTA. This trend suggests that LLM-TTA is unlikely to be effective for practitioners using LLMs as their task models. We study the effect that parameter count has in isolation in Appendix A.2 An avenue for futurre work is better understand what factors in task model's training architecture influence TTA's effectiveness.
We study ESA to avoid augmenting examples that are unlikely to benefit from TTA. This selective augmen- tation reduces the rate of expensive LLM augmentation while still improving robustness. While entropy is a predictive heuristic of whether the task model will change its prediction, accuracy with selective augmenta- tion underperforms the default approach of augmenting every test input, thus producing a tradcoff between classification performance and efficiency. More work necessary to better classify which examples will likely benefit. from augmentation and explore alternative metrics to entropy.
12

---

Published in Transactions on Machine Learning Research (07/2024)
Our study uses realistic selection of models yet has limitations. We leverage extensively pretrained BERT, T5, and Falcon models. There is risk that our evaluation datasets were present in these model's pretraining corpora. It is unclear whether if a model was pretrained or not improves or regresses LLM-TTA's perfor- mance. There is some evidence that diverse pretraining can improve model's natural robustness (Hendrycks et al. 2020b). better understanding of the eflect that pretraining has on TTA is an opportunity for future study.
This work studies TTA solely for short-form text classification. While LLM-TTA outperforms baselines across multiple differing domains, it is unclear how well our results generalize to other NLP tasks. For instance, lasks such as extractive QA (Rajpurkar et al. 2016), which require the original structure of the text to remain unchanged, are unlikely to benefit from augmentation. Studying LLM-TTA other settings is a promising direction for future work.
7 Conclusion
This work studied whether test-time augmentation with LLMs improve the robustness of task-specific models. We observed positive results - LLM-TTA can improve the robustness and, in some cases, ID performance of lask models. LLM-TTA is useful in low and high-resource settings across multiple tasks. These gains are realized without assumptions with respect to the task model's architecture or the ability to train the model. We can use entropy statistics to reduce the number of LLM augmentations required via selective augmentation. These results demonstrate that LLM-TTA can be a simple method for improving the robustness of task-specific models, an important problem in applying machine learning to real-world settings. While LLM-TTA does improve robustness, more work is necessary make task models fully robust to distribution shifts.
8 Acknowledgement
We are grateful to EleutherAl for permitting access to their compute resources for initial experiments. The welcome and open research community on the EleutherAI Discord was especially helpful for the literature review, debugging PyTorch issues, and information necessary conduct the parameter count ablation experiment (Appendix A.2 In particular, we would like to thank Stella Biderman, Nora Belrose, and Hailey Schoelkopf.
We are grateful to the University of Virginia Research Computing team for providing access to excellent high-performance computing resources.
Lydia O'Brien provided copy editing and feedback on figure design and engaged in extensive discussions that shaped the direction of this project.
M. Ghassemi's work supported in part by Quanta Computing and the Gordon and Betty Moore Foundation.
The research of J. Mendez-Mende is funded by an MIT-IBM Distinguished Postdoctoral Fellowship.
References
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mer- ouane Debbah, Etienne Goffinct, Daniel Heslow, Julien Launay, Quentin Malartic, Badroddine Nounc, Baptiste Pannier, and Guilherme Penedo. Falcon-40B: an open large language model with state-of-the-art performance. 2023.
Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, N. Tep- per, and Naama Zwerdling. Do not have enough data? deep learning to the rescue! AAA/ Conference on Artificial Intelligence, 2019. URL
Ashish, Aakanksha Rani, and Hatesh Shyan. A comparative study and analysis on toxic comment clas- sification. 2023 International Conference on Sustainable Computing and Smart Systems (ICSCSS), pp. 783 -33,787, 2023. URL https://app: semaniiscchooa..rrcc........a... wtw.aatttcccco....ooooooaaaaaaaa
13

---

Published in Transactions on Machine Learning Research (07/2024)
Arsenii Ashukha, Andrei Atanov, and Dmitry P. Vetrov. Mean embeddings with test-time data augmentation
for ensembling of representations. ArXiu, abs/2116.08003, 2021. URL http:ps/ppp...smmttttccco.o.ooop
:Bs/CorppuID:23548800
Murat Seçkin Ayhan and Philipp Berens. Test-time data augmentation for estimation of heteroscedas-
tic alcatoric uncertainty in deep neural networks. 2018. URL https://api http://ppp..ttaattccccccoo......
(orpusIDD1339933:;
Markus Bayer, Marc-André Kaushold, and Christian Reuter. A survey on data augmentation for text classification. ACM Computing Surveys, 55:1 39, 2021.
Stella Biderman, Hailey Schoelkopf, Quentin Authony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mo- hammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. arXiv preprint arXio:2304, 01373, 2023.
Daniel Borkan, Lucas Dixon, Jeffrey Scott Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. Companion Proceedings of The 2019 World Wide Web Conference, 2019.
Tom Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Aduances in neural information processing systems, 33:1877-1901, 2020.
Ján Cogin, Jakub Simko, and Peter Brusilovsky. Chatgpt to replace crowdsourcing of paraphrases for intent classification: Higher diversity and comparable model robustness. ArXiv, abs/2305.12941, 2023. URL
https://api. /
www.ntscccmooooarr.....ooooc wrt.:oopaar.rgg/rouuuusslusl : 258832868
Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, and Diyi Yang. An empirical survey of data augmen- tation for limited data learning in nlp. Transactions of the Association for Computational Linguistics, 11: 191 211, 2021. URL attppe///pp..emmasiccccooo.r.rr. ott.aaattccccc.....ooooooooooooa 235422222
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. What does bert look at? an analysis of bert's attention. In BlackbozNLP@@CII 2019. URL https://api swmantccscoolar..coommmmmm org/
:1144670
Jeremy M. Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via ran- domized smoothing. In International Conference Machine Learning, 2019. URL https://api. semanticscholar wtgg/GorpuuID:55842988
Emmanuel Gbenga Dada, Joseph Stephen Bassi, Haruna Chiroma, Shafi'i Muhammad Abdulhamid, Ade- bayo Olusola Adetunmbi, and Opeyemi Emmanuel Ajibuwa. Machine learning for email spam filtering: review, approaches and open rescarch problems. Heliyon, 5, 2019. URL URL/https:// httpps://pp. sem.acccco.emmiiicccooo..o.r ort/CoorpusI::119937775
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert:
Pre-training of deep bidirectional transformers for language understanding, In North American Chapter of the Association for Computational Linguistics, 2019. URL
wow.::ollrrrrrr/rroooooooo CrggCorrrsIDD 52967399
Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. Understanding back-translation at scale. arXiv preprint X80: 09381, 2018.
Mai Elsherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Mummun De Choud- huy, and Diyi Yang. Latent hatred: A benchmark for understanding implicit hate speech. ArXiv, abs/2109.05322, 2021.
Shohei Enomoto, Monikka Roslianna Busto, and Takcharu Eda. Augnet: Dynamic test-time augmentation via differentiable functions. ArXio, abs/2212.046681, 2022. URL https://api semantccschola..c...m org/
ID:
14

---

Published in Transactions on Machine Learning Research (07/2024)
Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard H. Hovy. A survey of data augmentation approaches for nlp. In Findings, 2021. URL https:
/api.semanticccholar wttg/forrpus..2224000005.
Lco Gao, Stella Rose Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Auish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb datasel of diverse text. for language modeling. ArXiu, abs/2101.0000 00027, 2020. URL. https://api. ser.aaiiccemannccchooola...
orgg/Corpue1D:230337775
Tianyu Gao, Xingcheng Yao, and Danqi Chen. In Empirical Methods in Natural Language Processing (EMNLP), 2021.
Jakob Gawlikosski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseo Lee, Matthias Humt, Jianxiang Feng, Anna M. Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, M. Shahzad, Wen Yang, Richard Bamler, and Xiaoxiang Zhu. A survey of uncertainty in deep neural networks. Artificial Intelligence Review, 56:1513 1589, 2021. URL httph: //api. www.eeiicmaooooaaaa..........aae :ot.aaarrrrrrrroorrss:22:2:211 2357550022
lan Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, Cambridge, MA, USA, 2016.
Shreyansh Goyal, Sumanth Doddapaneni, Mitesh M.Khapra, and Balaraman Ravindran. A survey of ad- versarial defences and robustness in ulp. 2022. URL http:////api..ematicccchoooooooe Drr.rlpuurrg/
247447518
Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. In Conférence francophone sur I' l'apprentiasage automatique, 2004. /ttpss//://ppp httppp/s//ppi.emmtttcccchhhlaa semantccscholarr wrwparrppsoCCCCCCDD. 7890982
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarlen Sap, Dipankar Ray, and Ece Kamar. Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3309-3326, 2022.
Peter Hase, Mohit Bansal, Peter Clark, and Sarah WiegrefTe. The unreasonable effectiveness of easy training data for hard tasks. ArXiv, abs/ 21s/2401.06751,, 2024. URL http:://pppp.eenntttccccccoooooo 266977266
Jameel Hassan, Hanan Gani, Noor Hussoin, Muhammad Uzair Khattak, Muzammal Nascor, Fahad Shahbaz Khan, and Salman Khan. Align your prompts: Test-time prompting with distribution alignment. for zero- shot generalization. ArXiv, abs/2311.01459, 2023. URL stttp:://app.seeaatticccooo.. ora.ilaarrrr//rrrpsD
264935246
Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robusttess to common corruptions and perturbations. ArXiv, abs/1903. 12261, 2019. URL
D
56657912
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Lixuan Zhu, Samyak Parajuli, Mike Guo, Dawn Xiaodong Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-listribution generalization. 2021 IEEE/CVF International Conference Computer Vision (ICCV), pp. 8320 8329, 2020a. URL https: //api. wwmanticscholatt .aweelorgrrr.ee org/CorppssI::22255555
Dan Hendrycks, Xiaoy Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. Pre- trained transformers improve ont-of-distribution robustness. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 27511, Online, July 2020b. Association for Com- putational Linguistics. doi: 10.18653/vv http//112222.al11// acl-0.cclmminn 244. URL hctps://aclanthooogg.coo orgg/200. acl-main. 244
15

---

Published in Transactions on Machine Learning Research (07/2024)
Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety.
ArXio, abs/2109.1391 13916, 2021. URL https://api.eemaantt www.seemmtmttoollaa......c http:://ppp..saaattcccccooooorrr org/CorpusID: 22t 238199240
Cong Duy Vu Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. Tterative back-translation for
neural machine translation. In NMT@ACL, 2018. URL https://api. 51880064
Yutai Hou, Yijia Liu, Wanxiang Che, and Ting Liu. Sequence-to-sequence data augmentation for dialogue language understanding. In International Conference on Computational Linguistics, 2018. URL https:
7/api. seap.ssemnttcschoolaa ortp/oorpusDD44957755555
Jeremy Howard and San Ruder. Universal language model fine-tuning for text classification. In Annual
Meeting of the Association for Computational Linguistics, 2018. URL sttt:::/aap..emmanticccctt org/CorpusID:401000
Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, and Marjan Ghazvininejad. Training on synthetic noise improves robustness to natural noise in machine translation. ArXiv, abs/1902.01509, 2019. URL https
//api. Saapi.semanticscholarr
http//oorpsII:::666000 59604474
Todoo Kim, Younghoon Kim, and Sungwoong Kim. Learning loss for test-time augmentation. ArXin, abs/2010.111222, 2020. URL https///app.semaniisssooooo wttp://ww.aaa...eorrrraaaaaa::oa :220039911
P. Kirichenko, Mark Tbrahim, Randall Balestriero, Diane Bouchacourt, Ramakrishna Vedantam, Hamed Firooz, and Andrew Gordon Wilson. Understanding the detrimental class-level effects of data augmenta-
tion. ArXiv, abs/2400.01174, 2023. URL http::/pppp..eemmmcccccccoaorrrs
266741225
Sosuke Kobayashi. Contextual augmentation: Data augmentation by words with paradigmatic relations.
Arxis, abs/ has/1805506201, 2018. URL.
Tom Koemi and Christian Federmann. Large language models are state-of-the-art evaluators translation quality. In European Association for Machine Translation Conferences/ Swnferences/Workshopps 2023. URL https:
//api. seatticematt httg/oorppaID:25723300
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubra- mani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, 2020. URL https:
//api. www.smmccccccchlaar.... www.aaaww.ccc.........ooooaaaaaa 22911166222
Varun Kumar, Ashutosh Choudhary, and Eunah Cho. Data augmentation using pre-trained transformer models. ArXie, abs/ Abs/2203.002445, 2020. URL http:::/ppp..emmmiiccccoooo org/CorpuslDDrDDC.oooo 2111197776
Jian Liang, Ran He, and Tien-Ping Tan. A comprehensive survey on test-time adaptation under distribution
shifts. ArXie, abs/2303.1555 15361, 2023. URL tttt::://api..mmmatticcccee www.://aa...rrrooo.uss.e... : 257767040
S. Longpre, Yu Wang, and Christopher DuBois. How effective is task-agnostic data augmentation for
pretrained transformers? In Findings, 2020. URL https: semaaccccccannc.c..c...... wwwp:/aarrrrrr/oouuuoeee Crg/Corp 222132977
Helen Shiyang Lu, Divya Shanmugam, Harini Suresh, and John V. Guttag. Improved text classification via test-time augmentation. ArXiu, abs/2206.13600 2022.
Xiaofei Ma, Peng Xu, Zhiguo Wang, and Ramesh Nallapati. Domain adaptation with bert-based domain classification and data selection. In Conference Empirical Methods in Natural Language Processing, 2019. ottpp//rrrr//ggi....aaaaatoooooo hrt.
Shahbuland Matiana, Jr. Allen Richard Smith, Ryan Techan, Louis Castricato, Stella Biderman, Leo Gao, and Spencer Frazier. Cut the carp: Fishing for zero-shot story evaluation. ArXio, abs/2110.03111, 2021. URL ttt.hhtpr.e/pppp....aaattcttooot httpp//.aa..t.aaaaa..a..ooooaaaa
swwanticscholar
org/CorpuuID::28841155
16

---

Published in Transactions on Machine Learning Research (07/2024)
Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. Proceedings of the 7th ACM conference on Recommender systems, 2013.
Dmitry Molchanov, Alexander Lyzhov, Yuliya. Molchanova, Arsenii Ashukha, and Dmmityy P. Vetrov. Greedy
policy search: A simple baseline for learnable time augmentation. ArXiv, abs/2002.09103, 2020. URL
https://api www.ssssccccccon.rrrro..o.o. wrwp:aroso///OOOo//000
John X. Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp. In Conference Empirical Methods in Natural Language Processing, 2020. URL
220714040
Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow. On the stability of fine-tuning bert: Misconceptions, explanations, and strong baselines. ArXie, (bs/2006.048844 2020.
Nikita Moshkov, Botond Mathe, Attila Kerlés--Parkas, Réka Hollandi, and Péter Horváth. Test-time aug-
mentation for deep cerrinn-based cell segmentation microscopy images. Scientifie Reports, 10, 2019. URL https://apll set..ppp..smmatccccoolaaoa..r. Seeeaattccccooaarrrrr//oppoooooo htgg/CorpuSDD:20688333
Preslav Nakov, Alan Ritter, Sara Rosenthal, Fabrizio Sebastiani, and Veselin Stoyanov. Semeval-2016 task 4: Sentiment analysis in twitter. ArXiv, abs/1192.011973, 2016.
Nathan Ng, Kyra Yee, Alexii Baevski, Myle Ott, Michael Auli, and Sergey Edunov. Facebook fair's wmn199
news translation task submission. In Conference on Machine Translation, 2019. URL https://api.
semanticscholar.co org/
Nathan Ng, Kyunghyun Cho, and Marzyeh Ghassemi.
Ssmba: Self-supervised manifold based data ArXiu, abs/2009.10195 2020. URL https:
augmentation for improving out-of-domain robustness.
s/api.semanticschollar wtw.:hoo....ooog/ooorosai::z:22. 221836078
Ajay Patel, Nicholas Andrews, and Chris Callison- Ballison-Burch. Low-resource authorship style transfer with in- context learning. ArXiv, abs/2212.08988, 2022. URL httpp:://ppp..semattcccccchh// www.:olarrrrrrrroooeoooe/eooeo Tang/CorpusID:
263859161
Christopher Potts, Zhengxuan Wu, Atticus Geiger, and Douwe Kiela. Dynasent: dynamic benchmark for sentiment analysis. ArXie, abs/2012.15349, 2020.
Aaditya (Adi) Prakash, Nick Moran, Solomon Garber, Autonella DiLillo, and James A. Storer. Deflecting adversarial attacks with pixel deflection. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8571-8580, 2018. URL https://app. www.sticscoolla....o.o.. sew.attttcccolaaa..ooo//oorooooa
Colin Raffel Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 21110)):1-67, 2020.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Conference on Empirical Methods in Natural Language Processing, 2016. URL https://api. http:p////iii..eaaacccccoooo... wtt.://oo........oooooopaaaaaaaa
Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi. Med-bert: pretrained contextualized embed- dings on large-scale structured electronic health records for disease prediction. NPJ Digital Medicine, 4, 2020. URL httph//ttt//ppppp.eaaatttccoooot wttp://cccc.....oooooooooooooiaa ort//oorpusD::211887777
Shamik Roy, Raphael Shu, Nikolaos Pappas, Elman Mansimov, Yi Zhang, Saab Mansour, and Dan Roth. Conversation style transfer using few-shot learning. ArXiv, /bs/2302.08662, 2023. URL https://api,
Scbastian Ruder, Matthew E. Peters, Swabha Swayamdipta, and Thomas Wolf. Transfer learning natural language processing. In North American Chapter of the Association for Computational Linguistics, 2019.
URL httppp://ppp..seaamiiccchhoooo/ wtt.ratcoot.ttooooooooooooaaaaad
17

---

Published in Transactions on Machine Learning Research (07/2024)
Anian Ruoss, Gr'egoire Del'ctang, Tim Genewein, Jordi Grau-Moya, R. Csordás, Mehdi Abbana Bennani,
Shane Legg, and Joel Veness. Randomized positional encodings boost length generalization of transformers. ArXiu, abs/2305.16843 2023. URL ottpp:///pp..eeaatttcccoooo.... http://aaa....toooooooooooooo/oa 0T5//0rppu81::2228477777
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distillert, a distilled version of bert:
smaller, faster, cheaper and lighter. ArXíe, abs/1910.01108,, 2019. URL http
Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. ArXio, abs/1611.06709, 2015. URL http:///pp..semannicccc..o.o...e http:/ecccccaaaoroorrrorreeeeeee
15600925
Divya Shanmugam, Davis W. Blalock, Guha Balakrishnan, and John V. Guttag. Better aggregation in
test-time augmentation. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp-
1194-1203, 2020. URL attps://apt) semantcccccccc..l............... Drrrr 238634826
Connor Shorten and Taghi M. Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 6:1-48, 2019. URL
htt.a//aaa.aoargrrrouuuuiDa:-a-. 1968111884
Ayush Singh and J. Ortega. Addressing distributiion shift test time in pre-trained man-trainedlannuaaee language models. ArXin, abs/2212.02384, 2022. URL https://apts www.eeeiinaaenllaaeeeeeee httpaarrrrree/oopss::2222222211
http::::/pp...enaaa.cccclloo...
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, A. Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Conference on Empirical Methods in Natural Language Processing, 2013.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. ArXio, abs/1710.100 2017. URL atcttp::/app..smmattcccatte watpaatiiiiccclaaooooor/rpppupee wrgp/ooppoaD: 3313632
Mirac Sungun, Luke Melas-Kyriazi, and Dan Jurafsky. Prompt-and-rerank: A method for zero-shot and few-shot arbitrary textual style transfer with small language models. ArXiv, abs/ abs/2205. 11503, 2022. URL
htt.:appi.ssmaannccccchhoaa.aa Sttaaa/niiccccooo.ooooooooooooro 248987526
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Mea- suring robustness to natural distribution shifts in image classification. ArXie, hhss2007.006444 2020. URL
www.aiastaaannicchhlaeeeoe. wwtpi/oolll.rr///ooooooo wrw/ Cogg/orr 220280805
Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cris- tian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony S. Hartshorn, Saghar Hosscini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Ko- renev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poul- ton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia. Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. ArXie, abs/ 09988,, 2023. URL /tt.htpp:///ppiieeaattcccooobbbo www.sseestniiiools...mmmmm
org ttg//orrpusDD.. : 259950998.
Lifu Tu, Garima Lalwani, Spandana Gella, and He He. An empirical study on robustness to spurious corre- lations using pre-trained language models. Transactions of the Association for Computational Linguistics, 8:621 633, 2020. URL
Dusan Varis and Ondrej Bojar Sequence length is domain: Laneth- overitttin in transformer models. Arxio, /ts//2219.077776, 2021. UR
18

---

Published in Transactions on Machine Learning Research (07/2024)
David Vickrey and Daphne Koller. Sentence simplification for semantic role labeling. In Annual Meeting of the Association for Computational Lánguistics, 2008. URL ptttp:://api..emmaticccos
org/CorpusID;
2382276
Jan Philip Wahle, Terry Ruas, Frederic Kirstein, and Bela Gipp. How large language models are transforming machine-parapbrase plagiarism. ArXiv, abs/2210.033688, 2022. URL httpss//app. sot.apipi..mmmtiicccccocoo.lll
org/CorrpusDD:257722277
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021.
Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, and Zhaopeng Tu. Document-level machine translation with large language models. ArXiv, abs/2304.02210 2023a. URL
Zixin Wang, Yadan Luo, Liang Zheng, Zhuoxiao Chen, Sen Wang, and Zi Huang. In search of lost online test-time adaptation: A survey, ArXiv, abs/ abs/2310.20199, 2023b. URL secre:://pi..emmattiiccc...... wemantccchoollll....
org/CorpusID:264813775
Jason Wei and Kai Zou. Eda: Easy data augmentation techniques for boosting performance on text
classification tasks. In Conference on Empirical Methods in Natural Language Processing, 2019. URL
httpeeaaaaaatttaallloooorrrrraaa OrggCooppsIDD:5522222
Sam Witteveen and Martin Andrews. Paraphrasing with large language models. In Proceedings of the 3rd Workshop on Newral Generation and Translation, pp. 215 220, Hong Kong, November 2019. Association for Computational Linguistics. doi: 10.18653/v1 /1/33///11195622.
URL http:ttt//caaatttolooo http:://acccaatoooooooooo//////s orw.:/99or/DD19522222
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface's transformers: State- of-the-art natural language processing. ArXiv, abs/1190.003771, 2019.
Qizhe Xie, Zihang Dai, Eduard H. Hovy, Minh-Thang Luong, and Quoe V. Le. Unsupervised data aug- mentation for consistency training. arXiv: Learning, 2019. URL https: /ppi. swmanticscholar.oorggo org/
CorpusID:196873888
Haoyu Xiong, Xinchun Zhang, Leixin Yang, Yu Xiang, and Yaping Zhang. Stta: enhanced text clas-
sification via selective test-time augmentation. PeerJ Computer Science, 2023. URL https://api. semanticscholar Senaaattcccccbbbr..oooo//oppoppo org/CorppuID::264168..2
Yadollah Yaghoobzadeh, Soroush Mehri, Remi Tachet, Timothy J. Hazen, and A Alessandro Sordoni. In- creasing robustness to spurious correlations using forgettable examples. In Conference of the European Chapter of the Association for Computational Linguistics, 2021. URL htpps://ppp..emmmtticccoo.. org//orpuulD:223158881
Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzyeh Ghassemi. Change is hard: A closer look at subpopulation shift. ArXiu, abs/2302.122244, 2023. httpss/:: http:/tp::/pppiieeaattttccaaaiae semanticscoolar http://cccc.aaooooooorooroo////p org/CorpusID: 257102790
Adams Wei Yu, David Dohan, Minh-Thang Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoe V. Le. Qanet: Combining local convolution with global self-attention for reading comprehension. Ar abs//184400944 2018. URL http::///ppp.sssanisseeeeeee Set.aaawww.ccclaa....ooooooooaaa 48849909
Lifan Yuan, Yangvi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, and Maosong Sun. Revisiting out-of-distribution robustness in nlp: Benchmark, analysis, and llms evaluations. ArXiv, abs/2306.0461a. 04608, 2023.
Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustness via adaptation and augmen- tation. ArXiv, abs/2110.08 10.05506, 2021.
19

---

Published in Transactions on Machine Learning Research (07/2024)
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. Character-leves convolutional networks for text classifi- cation. In NTPS, 2015.
Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. Challenges in au- tomated debiasing for toxic language detection. ArXin, abs/2102.00086, 2021. URL https://api.
semanticscholar. wwwpiiarrrrr/oooooo wtrg/CorppssDD:237411399
Yongchao Zhou, Uri Alon, Xinyun Chen, Xuezhi Wang, Rishabh Agarwal, and Denny Zhou. Transformers can achieve length generalization but not robustly. ArXiv, abs/ 0bs/2442.09931,, 2024. URL https:://pp.
semanticscholar. wwwpoollarrrrrooorrarreeee org//orrpuaDD::27661166
Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Jiajun Chen, Lei Li, and Shujian Huang. Multilingual machine translation with large language models: Empirical results and analysis. ArXiv, 0t5/2230404677 2023. URL Utt.htpp:://pp..emmmtttiooo sca.ccccclam
:ttg/CorpuuID:250445005
Ran Zmigrod, Sabrina J. Mielke, Hanna M. Wallach, and Ryan Cotterell. Counterfactual data augmentation for mitigating gender stereolypes in languages with rich morphology. ArXiv, abs/ 0ass/9966044577 2019. URL
https: //ppi. http://ccchoooa....oooo/oopppaaa
20

---

Published in Transactions on Machine Learning Research (07 (07/2024)
A Appendix
A.1
Detailed Main Results
Boss Sentiment
Boss Texicity
AG News - Tweets
Model
Augmentation
AZ
SST
SE
DS
CC*
AC
TG
IH
News
Tweets
BERT
Paraphrase
90.54%
72.10%
47.95%
46.48%
90.58%
63.29%
66.14%
64.96%
93.72%
89. 89.03%
ICR
90.85%
73.60%
49.02%
48.56%
91.45%
51.45%
64.14%
65.64%
98.53%
89.66%
T5
Paraphrase
89.90%
77.06%
50. 80.
50.30%
91.32%
71. 71.62%
66.35%
64.42%
02.03%
89.84%
ICR
90.78%
75.75%
51.11%
52.27%
91.87%
59.90%
66.24%
64.93%
91.99%
90.23%
Falcon
Paraphrase 90.35%
53.93%
40.45%
41.57%
18.43%
79.95%
52.97%
43.53%
28.78%
27. 65%
ICR
90.35%
51.03%
39.74%
40.42%
19.00%
77.90%
52.44%
44.45%
29.47%
27.60%
Table 5: Seed = 3. LLM-TTA Accuracy across augmentation functions, datasets, and models. u** indicates the ID split the task models are optimized for.
Boss Sentiment
Boss Toxicity
AG News + Tweets
Model
Augmentation
AZ'
SST
SE
DS
CC*
AC
TG
III
News"
Tweets
BERT
Paraphrase
90.54%
71.54%
47.98%
45.86%
90.58%
62.44%
66.03%
65.10%
98.72%
88.95%
ICR
90.85%
73.03%
49. 19.23%
48.80%
91.45%
51.57%
65.92%
65.74%
93.53%
89. 73%
T5
Paraphrase
89.90%
75.94%
50.84%
49.75%
91.32%
71.98%
66.56%
64.57%
92.05%
89.90%
ICR
90.78%
75.19%
51.16%
52.22%
91.87%
60.63%
66.67%
65.14%
91.99%
90.25%
Falcon Paraphrase
90.35%
53.00%
40.64%
41.48%
18.43%
80.31%
52.76%
43.62%
28.78%
27.56%
ICR
90.35%
51.22%
39.70%
40.56%
19.00%
77.78%
52.12%
44.31%
29. 29.47%
27.83%
Table 6: Seed = 17.
Boss Sentiment
Boss Texicity
AG News -
Tweets
Model
Augmentstion
AZ*
SST
SE
DS
CC*
AC
TG
IH
News*
Tweets
BERT
Paraphrase
90.54%
71.63%
47.94%
46.00%
90.58%
64.13%
65.92%
65.03%
93.72%
88.95%
ICR
90.85%
73.50%
49.21%
48.96%
91.45%
51.69%
64.97%
65.75%
98.53%
89.81%
T5
Paraphrase
89.90%
76.59%
50.51%
49.61%
91.32%
70.17%
66.67%
64.44%
92.03%
90.07%
ICR
90.78%
76.12%
51.29%
52.15%
91.87%
59.54%
66.24%
64.91%
91.99%
90.53%
Falcon
Paraphrase
90.35%
53.09%
40.71%
41.11%
18.43%
80. 80.43%
52.97%
43.70%
28. 78%7
27.26%
ICR
90.35%
50.94%
39.73%
41.09%
19.00%
77.17%
52.44%
44.26%
29.47%
27.83%
Table 7: Seed = 46.
A.2 Isolating Parameter Count's Influence on TTA Performance
Table Ishows that BERT, the smallest model evaluated in terms of parameter count, benefits the most from TTA. That T5 and Falcon did not benefit as consistently raises the question of whether TTA becomes less effective with model scale. This effect can only be imperfectly studied with BERT, T5, and Falcon since there are confounding factors such as architecture and training datasets beyond parameter count.
The Pythia model suite (Biderman et al.) 2023) is a suite of decoder-only transformer languages models trained on The Pile (Gao et al.) 2020). Each model in the suite is identical in architecture, training data, and data ordering, with parameter count as the salient difference. Although Pythia is not of the reduced confounders help isolate the influence that model size has on TTA performance. We evaluato TTA performance on 2.8b, 6.9b, and 12b Pythia models trained with duplicated examples.
Table reports performance across model sizes. We do not obscrve a clear relationship between parameter count and the degree which TTA improves performance. This result is observed for traditional and LLM- based TTA methods. Pythia (with duplicated training example) was trained 300 billion tokens, while
21

---

Published in Transactions on Machine Learning Research (07/2024)
Boss Sentiment
Boss Toxicity
AG News
Tweets
Model
Augmentation
AZ*
SST
CC*
AC
TG
1I1
News"
Tweets
BERT
Paraphrase
90.54%
71.54%
18.11%
45.74%
90.58%
62.08%
66.03%
64.93%
93.72%
88.94%
ICR
90.85%
73.78%
49.24%
49.05%
91.45%
52.42%
65.82%
65.65%
93.53%
89.70%
T5
Paraphrase
89.90%
76.12%
50.53%
49.98%
91.32%
69.08%
66.24%
64.35%
92.05%
90.12%
ICR
90.78%
75.66%
50.97%
52.78%
91.87%
59.78 78%
67.20%
64.81%
91.99%
90.20%
Falcon
Paraphrase
90.35%
53.75%
40. 72%
41.37%
18.43%
80.07%
52.87%
43.61%
28.78%
27.55%
ICR
90.35%
50.94%
39.52%
40.58%
19.00%
78.14%
52.02%
44.23%
29.47%
27.99%
Table Seed 58.
SST-5
ToxiGen
AG Tweets
Augmentation
2.8b
6.9b
12b
2.8b
6.9b
12b
2.8b
6.9b
12b
None
39.27%
51.96%
46.18%
57.63%
57.63%
58.90%
25.16%
25.05%
25.14%
Insert
39.37%
51.12%
45.99%
57.61%
57.63%
59.00%
25.04%
24.99%
25.03%
Substitute
39.46%
49.81%
46.55%
57.63%
57.63%
57.42%
25.03%
24.99%
24.99%
Translate
40.86%
51.49%
45.52%
57.63%
57.63%
58.79%
25.17%
25.04%
25.04%
Paraphrase
41.20%
51.12%
43.82%
57.55%
57.55%
58.40%
25.30%
25.11%
25.11%
ICR
40.86%
53.08%
45.05%
56.63%
57.63%
58.16%
25.18%
25.12%
25.18%
Table 9: Parameter Count Ablation TTA accuracy across Pythia model scale is reported. The degree to which TTA improves performance over the baseline is not strongly correlated with parameter count. These findings suggest that other factors, such as the training distribution and model architecture, are more likely to influence TTA's 's performance.
BERT trained 3.3 billion tokens CClrr al. 2019). We conjecture that the pertaining objective and corpus size may play larger role than the learnable parameter count in isolation.
A.3 Analyzing In- Context Rewriting Augmentations
In-Context Rewriting (ICR) prompls the LLM to rewrite OOD examples such that they are more like a set of ID examples provided in the prompt. The intuition behind this technique is that the strong ID performance can be generalized to OOD examples, which are rewritten such they are ID while preserving semantics. Does ICR successfully rewrite OOD examples such that they'r re ID? We study this question by evaluating the embeddings of the ID evaluation set for Boss Sentiment and the SST-5 OOD shift. The mean representation of the four augmentations for each OOD input is used. We use the RoBERTE model introduced in Gao ct al. 2021 for emboddings.
ICR's augmentations generally do not bridge the gap between distributions. The cosine similarity between the ID evaluation set's 's centroids and the SST-5 original examples centroid 0.3285. The similarity between the augmentation's centroid and the ID eval set is 0.3445 only slightly more similar. Figure demonstrates that most augmentations remain OOD.
These results suggest that prompting may be insufficient for entirely bridging distribution gaps. However, the fact that ICR consistently outperforms 0-shot paraphrasing suggests that ID examples in the prompt still positively influence performance. Additional work needed to bridge domain gaps fully.
A.4 Selecting Entropy Thresholds: Expanded
Section 5.4 describes entropy-based selective augmentation. Selecting the entropy threshold involves deciding which distribution to sample from and the tradeoff between augmentation rate and performance. We use the ID evaluation set to find an optimal threshold. We selected the entropy based on generalization of the F-score metric. We treated accuracy as precision and the augmentation rate as recall.
22

---

Published in Transactions on Machine Learning Research (07/202) 2024)
In-Distribution
Out-of-Distributii
Ont-of-DDisribattion Augmented
Figure 2-D UMAP Representations of Text Embeddings. Embeddings for the Boss Sentiment ID evaluation set (Amazon reviews), the OOD (SST-5) test examples, and the mean embeddings for each test imputs augmentation batch are plotted. Suppose ICR was successfully rewriting OOD examples to be ID. In that case, we would expect to see far more angmentations in the same region of embedding space the ID evaluation. The fact that augmentations remain largely in the same space as the OOD inputs they are sourced from suggests that ICR is not bridging the domain gap, even in cases where ICR meaningfully improves performance.
Let rateang be the augmentation rate (the percent of examples augmented) and a by the accuracy of the given distribution.
acc (1 rateaug)
ratcaug (1 +
(3)
(2)
(( 1 )2. acc) + (1 rateaug)
A.5 Task Model Training Details
We opted to train our own task-specific models for the experiments to have maximal control over the training process. We use an uncased BERT base model available through the HuggingFace Transformers library (Wolf et al. 2019). We trained a separate model on each separate in-distribution dataset for a total of four models. Following the suggested baseline in Mosbach decay of 0.01, and lincar learning rate schedule peaking at 2c-5. We took the checkpoint that scored the highest average class Fl on the ID test set. T5-Large followed the same training procedure and identical hyperparameters.
al. (2020), we used a batch size of 32 examples, weight
A.6 Dataset Statistics
23

---

Published in Transactions on Machine Learning Research (07/2024)
Amazon (TD)
88T-55
Sem Eval
Dynnsen
Nag
Neutral
Pos
Total
Neg
Nentral
Total
Neg
Pox
Total
Neg
Neutral
Pow
Total
2151
32862
3801
38904
282
400
390
1072
3229
7009
10336
20624
1440
1440
1440
4320
Table 10: Sentiment Task Dataset Statistics
Civil Comments (ID)
Adv. Civil
ToxiGen
Implicit Hate
Renign
Toxie
Total
Renign
Toxie
Total
Renign
Trxcie
Total
Benige
Tooii
Total
89643
7777
97320
162
672
824
544
400
944
13201
8189
21480
Table 11: Toxicity Task Dataset Statistics
News (ID)
Tweets
Worlds
Sports
Business
Sci/Tech
Total
Worlds
Sports
Business
Sci/Tech
Total
1900
1900
1900
1900
7600
1900
1900
1900
1900
7600
Table 12: News Task Dataset Statistics
A.7 AG News Tweets Dataset
Motivation AG News is four-way topic classification task introduced in Zhang et al. (2015) In this setup, task model must classify whether given news article is about world events ( (World), sports and athletics (Sports), business and economics (Business), and scientific developments (Sci/Techl) The test set on HuggingFace (huggingface. coceaccee.c/eeeeeee datasstss/aggnees is composed of 7,600 examples equally balanced across the four classes.
News topic classification presents a promising opportunity for largely isolating the effect writing style shifts. Existing deep learning methods also perform well on this dataset with accuracy reaching higher than 90% (paperswithcode.. wtt.cooo/coaa. commsotaattext wwwwssaicaaaaiccccco.aa.ooaa wwweratiooooooonnneeeee- anwwg-news)
Another motivation for this particular task is the common risk of data augmentation inadvertently flipping the label/semanties of the text Bayer et al. (2021). Unlike other tasks such sentiment classification subtle hate speech, the topic of news article is unlikely to change during augmentation, thus preserving the original label.
(6/7/23 version) for style transfer. We did an initial pass through all 7,600 examples using a conservative 'V1' prompt and greedy decoding. Calls were made using the OpenAl Python SDK with top and temperature set to zero. The data was then lightly preprocessed to reduce the number of examples that began with BREAKING NEWS flanked my emojis. The V1 prompt Write the following news summary in the style of Twitter Twitter/socia media. Maintain all the relevant information and sentiment", and the V2 prompt Write the following news summary the style of Twiller/- social media. Mainlain all the relevant information and senti- ment. Add some flare with humor, anger, sarcasm".
Creation We used GPT-3.5 Turbo Brown et al. 2020
512 of the initial model responses did not result in satisfactory generations. These were typical cases where the generated lext was almost indiscernible from the original text or the generation was entirely emojis. We called GPT-3.5 Turbo again with an updated prompt and hyperparameters (temperature=0.7, top 00p_, presence__peneeeeeee ponalcy__o for these examples. Whereas all the first-pass generations did not. have any instructions the model as to the sentiment mooo of the hypothetical post author, we purposefully instructed the model to 'Add some flare with humor, anger, or sarcasm.' in the generation.
It's important note that we did not enforce Twitter's character limit. These sequences should be considered more broadly inspired by social media posts rather than following the exact. specifications of Twitter
24

---

Published in Transactions on Machine Learning Research (07/2024)
posts. We also did not manually review every sequence in the dataset to confirm that the original label was preserved. GPT 3.5 Turbo also hallucinates facts, such as adding the hashtag #Olympics2021 even though the original dataset was created in 2015.
A.8
Examples
Version
Prompt Examples
Label World
Original
Generated
Venezucla Prepares for Chavez Rocall Vote Sup- porters and rivals warn of possible fraud; govern- ment says Chavez's defeat, could produce turmail in world oil market.
# #Breaking: Venezucla readies #ChavezRecal- IVote amidst cries of #frand from both sides, Gov- ernment cantions defeat could shake up #oilmar- ket.
Sports
Natalie Coughlin Wins 100M Backstroke (AP) AP American Natalie Coughlin won Olympic gold in the 100-meter backstroke Monday night. Cough- lin, the only woman ever to swim under 1 minute in the event, finished first. in minute, 0.37 onds. Kirsty Coventry of Zimbabwe, who swims Auburn University in Alabama, earned the sil- ver in 1:00.50. Laure Manaudou of France took bronze 1:00.88.
Natalie Coughlin makes history as she wins gold in the 100M backstroke at the #Olympics2021. She finishes in minute 0.37 seconds her incred- ible performance leading her to become the only woman swim under minute in the event. Con- grats, Natalie! #TeamUSA
Business
Dollar Near Recent Lows, Awaits ZEW NCW/CPILLC LON- DON (Reuters) - The dollar held steady near this week's four-week low against the eura on Tuesday with investors awaiting a German investor confi- dence survey and U.S. consumer inflation numbers to shed light on the direction.
Dollar stays near recent lows against the curo In- vestors await German survey & US consumer in- flation data Info to give direction to the currency # #fnannce #cureencyxcchange #investing
Sei/Tech
Intel Delays Launch of Projection TV Chip In an- other product postponement, semiconductor giant Intel Corp. said it won't be offering chip for pro- jection TV's by the end of 2004 as it had announced carlier this year.
# NeehhNwws: Intel announces delay in lauuch of projection TV chip. Chip won't be available by end of 2004 as previously announced. #Tntel #De- lay #VCChip
Version 2 Prompt Examples
Label World
Original
Generated
Venezuela vote keeps Chavez as president CARA- CAS Venezuclans voted resoundingly to keep firebrand populist Hugo Chavez as their president victory that drew noiay reactions yesterday from both sides in the streets. International ob- servers certified the results as clean and accurate.
Hugo Chavez remains as the President of Venezuclal The people have spoken, and they want their firebrand populist leader to continue leading the country. *Cue street celebrations* Oh, and just a little side note, international observers gave the results a for being clean and accurate. # #ChavezForever # Venezulla
Sports
St. Louis Cardinals News Right-hander Matt Mor- ris threw seven solid innings, but the Cardinals needed bases-loaded walk to second baseman Tony Womack and a grand slam from new right fielder Larry Walker to key six-rum eighth inning for ...
Matt Morris doing the heavy lifting for the @Car- dinals, but it was Tony Womack's patience and Larry Walker's big swing that sealed the deal with SIX eighth inning! #OpeningDay #Red- hirdsRising
Business
Yahoo! Ups Ante for Small Businesses Web giant Yahoo! is gambling that price cuts on domain name registration and Web hosting products will make more competitive with discounters in the space which means that small businesses looking to move online get a sweeter deal through
NEWS ALERT Yahoo! stepping up game small businesses! The web giant is slashing prices on domain name registration and web host- ing products to keep up with competitors. Now, small bix owners can get a sweeter deal through Yahoo!. # #ahhoo #SmallBusiness # #WbHosstnng Good news, America! Most homes now have broadband connections! While the total number of internet users has plateaued, those online are adopting broadband at lightning speed. Accord- ing to Marc Ryan at Gandmessurement, it's time to kick dial-up to the curb! #internetUpprrade # NNoMoreDialUp
Sec/Tech
Most US homes have broadband connections While the total number of home internet users has reached a plateau in the US, those who do use the internet are adopting broadband at rapid pace, according to Mare Ryan, senior director of a analysis at the audience measurement company.
A.9 LLM Inference Parameters
TTA: OOD Generations. For LLM-TTA and back-translation, we generate four augmentations for each test input using temperature-based decoding with a temperature of 0.3.
25

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Sentiment Prompt
Dusk: The Jollowing bext bas vither a augelave (01, avaical (2), us protlive (1), Revara tav pualiaues label For Lay test.
twostaed ut, kiad pricey" Labsing
téfacy deing thiing actwepe groww gaeing utility. "ganing contane"
yeu argacting
publicreyy
stactey your phona Labolo0
Aege-ptblssber repencadll trying
you will charged the
Nasspolintas
tayy exeuielly
"Mell zepear thed book. I belteved of aithongh could and otd during sone details."
Labeled
"Fatt being suid, it's ponsibalativa deupped WAT dowa und ita going to retura it. The plagrin miczophous veuns Like aztaation just ssiiing for "beviaaa" piace. The zoind qualiny average, at bert, and i enill catai my after Glos anadees te usy batter. Ie you're looking for coeplate 1002, stvalese arperience, think Turtie Deact may be the way you aeed to go." labaiig
*Oood eovie." lebele2
"Vazy nice clous. fveling after use"" Labelei
*Gooot buy Lebel-1
saybe erigieally The very intaresting you have trsuble taking anti-blasios will harbe iteted sgaai their affiracy." labeled
spienty
find book utile
hatgeut. parasnal) ergeriancs
"Tayy avi look Like Lbe pictize, 1 know jesvley,
good quility. may zeture then"
Label-0
'927 30" deep. Naybe 2617 Taize advertinteg!" Labeled
"Good prodect, great peice. I've bought about 20 of this sou... eill vell made." Lobel-i
"Ostreesly strong dirable pouer cabla, Mich better quality chan the iactuded appla lightning cablas that T'ee uead. -Worke uall eith pozertanks 2:1a thargeza, thy cuble downn' get guebled up with the Sylou besiding clurgeo ay Ipod 2 fall upeet. Greas product." Labeles
disggointed easily the vegetables oct holder bettore. thn
really it very much" Labele?
saper
gevory
fall
Pata doneats lat that bother his, just singtng his 1istie nong,
1ittla
Tabalel
'I have shouse with uhite-tt vary light slab grantts, and an not Laprossed, let no preface comsusta soying that I aqueogos AND vipe dovn all the grasite 2008 of the time. Right off the bas, the vub dack had couple of plaose shere she oenter di caen to de ench of anysaing- After a conths, sore egess are deceloping that, show this wviature-intteee darisnning The sualer is cleasly losing its offectirensss, afver only 3 scotha. Very dissepoiatine," Labalno
tub dack of the sana. Thie grantte absorbe astature and darkane. used this nealer,
"OtverBox mokes u greas product." Lobel-1
What de the label, the following best? You wast dectde shict labal the test ia."
ingurt""
TTA:: ID Generations. LLM-TTA uses beam search decoding four return sequences, four beam groups, four beams, and a diversity penalty of 0.5. Back-translatton uses the HuggingFace generation pipeline API. Text translated from English to German (facebook/ -endo- and then back into English The generation parameters are four return sequences, temperature of 0.7, four beams, four beam groups, top-P of 0.95, top_ k of 0, repetition penalty of 10.0, diversity penalty of 1.0, and repeating n-gram size of 2.
LLM Classifier Inference. We use the following prompt template for LLM inference. Each prompt contains 16 in-distribution training examples selected and ordered randomly within the prompt with a random seed of 42. Examples in the prompt are balanced across classes. Greedy docoding is used with a max of 10 new tokens. Verbalalizers are used to map generated tokens to labels.
26

---

Published in Transactions on Machine Learning Research (07 (07/2024) 2204)
Toxicity Prompt
Task: Brown place
clansity stather is cuntaion tostc
If the twer toxic, O. Tf is tocio, Yoters
"Thu vecuenie posezhouse for halorsa Counlyy, Lubecto, voul off shoce by the las suilew uxscaled by tax sill Clinton DCJ. How, th tobures is still juse in Braast,, not to hone tom of Fatrmont. tillioon of Six west to the larper busdiar or citston, het case har late Kartem Noeth Carolins dastitute. in used to make sta/ye, off tobacto on our rarn curtng the "D0 is Vast to ince har that iand in used soday? It in aecransde for naturni dove tyes, birds) basttst. Just para the tazes." labeleg
"I aee our contrr vacreating into tealationien. The aorld won't awlow it. argect Thee will he teeted by our advenaaties axrty into his pracidency. nen shola lot of back up happening. by grase in the world will atcher come to respact hin faxr hin. the world of aorpelitiso both appear to have equal volue." Label-0
Ter reacts te anybeetes guace. bit
hepublican
right Repablicans. spoenst
kooendvel. TF you sepport isn't just
foot
don't to
thone sypee are bafore
take Labalei
"Mell He, Biike, at sucks to you. I have vocaod in the office building wil inbwley for decodes and only us the curest of ocomisu heve sucousterve. the poison type cncizemett you deaceibe Sure, Shore are loods of presoniliiinn but the baok stalbing and blasant labier clisbing is the eccegtion rather than the rulo. Bire, the stakes are highers your capahilities are being onliatted You are hired for role: bos uell do fill 15.7 Narbe there as is listle seanvork in journaliin; that would sxplain your axperieace. à tree quickly becoess dystanctions if eversbody is cut for Labele1
"Yon tett nontthing out. Whxt dons the russtion dighirat offer in naturn the repant of conettone?* Labaleo
"Thay '12 any say did ie take then no long to tackle greantouze gaz axinsione when it wan obetous moch sartter that nonething pectous bad to dona." labeleo
night stop to give comttdy
change
you
by yournelf you
driving.
"You, bund. unfortunntely
taless....
buckstabbengg
up, cuiting
Labele1
rBoas it take cartatn spectal polieical to aiready present typorriton? br shat a leartal teals once Shoy ges vated into the pig trough and see avoryone else vanstning tax winsy and anl lin thementine to the hightt biddar? Thee atunys contered." Label ot
that maker the altmy. opportuntatic and glant
"It's skay. Is seees Canedians are contest with misicum und mediccse vugus, simple and bering life styles. let the cest of the woeld rvup the drugso. to becoer cish uni pecspecous, We axv zut kavvn for bering globel. perapectives, anbisavus evosomie pluas, Our gut Loue of un ideeloyy different live sur em maber ue in intrenarted brach. After all, lite would be all good all of un tne lsve just sligitly abore the porerty lize. Ne don't need your Ostasss money." Labelel
good wtilling
Teep
together
grord
"He ucid logal
gorezzment
includoo youe
Label-0
"Ie azyone in Teump's odatstatzation had said that the Daney firine "ureng', they vould be fired That juzs the be operates. Be's sulfish, arzegant, clasisus, and complstely unquliiiied politiccian in and the sonner tonal Troup is recine free oftice, the sooner us can gen our counery back on track, from she sajority of Troup vottrr mow that they aade hage wixtsbe to giving ain shate vata." tabale)
it's und the bistuzy countsy,
"You are on adiot, You thisk Pacebook elected Traegh" Labeled
"Hee calla Trepptter bloe hand, Hillary a Tikr and people sho vote by what the polla any idiote.
Tabnlei
*Aa Trump's tauds cour, bead Bestiah Commueelll, u lyposeiticel pap our husoured conserrutive Lau't about Librrul privslaged taring adrastage loeghoine they tega)
folloring
"ctaet. Tegert" Label=
27

---

Published in Transactions on Machine Learning Research (07 (07/2024) /004)
News Topic Prompt
Task: Grven nou tect, clussity thether topte of the article following. Jatern the to for each categery.
World Sporsent, Bosineos + 2, Sel/Tecara
"Sscurity sight for Adghan inceuuratio kiglus and internationel fereso luid pecurity ring ascurd label us Koaday is pesperation for the insgureiion of Bonud lurses es Algamisten #595s lisst poyslurly slecved" Labelog
"Utraine Oppaattion Fleces Muncies kfter Victory RIY (hautars) Opporttion leader Viktor Testchmmta, fresh from nacering cerzen Utraine's dispoted elaction, sp deal faturday to redace the prusident's powsrs and bold his exppesters
keep ip praseine
Earnings
agrthuntsses trongartation coepany. pronteing.
Giasto Thardday teied Bren rockie Hamning tried
*Oleato
every possible of quarterbaok
part." label=]
"Clensua considers zetirement,
decizion yut quetii 450.a very closs cetirensat, Clesues told reporters Friday. guot; Uevieulyy baven s3Ree made the dectition for the spenning searon you." labalei
"Illiowi io Geveruse vasta in Bea Voolent Vidlo gamss Yosterday Illinois Owr, and Bluueeerch propoued stale lee that would bus the welling und zeutel of violent and or occullyy axplonit vides giase to Children unior the ago of 18, Labeled
"Pelota kirvaru Vote to Out 2ay and Busefita The pilota will vose 29 Airway's demand VESCO millicn vage and bunedit concesaions, Lebel-2
their usion suid Turaday night."
sight dodgne apyeara change
Accorsing security
glant
scqutettter apyeane company
upplications."
Medtrocie Wine The Agproval tespwhhngg ///re, Sunday suid U.S. regalators appeoved Imagu Sintey iaplentbble defabrillator (100) muttl varliur Las it expectud," Label=5
Sentery (m) CHEOAEO (heutere) Nettronte blsid ow..t//ttannnccce www.:///nnnnaaaaaaaaae..
"Narst genaral connnnl to ratige antd probs HDI YoR The groaral conneal for brcker Mareh sr;; Netanna Cas. sill step domn nast usaz as the cacastled cire seeks to clean house and referm its bestzere practiceo, the Nall Street Jeursal reposted on Friday. Labsl=2
Maborr, Hikd agnas on intry groemneets, aight Peterntaiios ki aloshigs: Abtgt munua (1PP) Srina Ministar tvial Starmn #18 Lind party and the appoatatio Labarr agrand to farm nacional mity givereeert, shile at Teat night Palastiviine vace killed Buring an lacueli une incuraion in souibure
Labul=0
"Pantbere' Hirgen but (AP) Caroline ronning Stephes Devio sixeed sixth gawe pesssn Badda becosse a kiee injary, gass bazland."
*Dsiro botting Arstas fumeral untteary funaral for Palaetisian Peattdent, Yasser Aratas, eill hald is the Egyptian capital, Ostin shtee warring. after funeral Pracident tratst 438 coffie be fiom ballcagtar taealaah bertal." Labeted
"How Oas Ve Buy IK Marms? Troditionnl untivirus techmllogy
too slow to presect users, zesearchers pay. Lobole8
"GooCorp retects Steel Partoses offer Centorp (CT.3: Baoce, Profile, Rezearch) in surospace and real estate coppany, Honday suid its board bas refected 1827 per abace offer frem 10 investeaut fund Stesi Pactours 11, saying the price vas aush:inadequase: Labelr2
*1 latter Trust (Forbee
Perber ior There zora corporate parfermance than shat you zan to the exrntage reporte. Goold inewetor anticiparad
the trechin at coapantes like teren, Malpass, Vorldces and Tyce by lookingg more clonely at hoe they vare geverned zoz they kept their books? Thatr prohimms, oaze, Baron RocldDas
visible in hindsighs, monetheleso Adelghil
tolitale sigas that was ssacited diagany Iosiders surned
reported carningo growth as
expported by
the folloving bess? decide
"stast_ Label=
28


Reviews:
[{"review_id": "AvPbBM2qzx", "reviewer": "Reviewer_aT3v", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strength\n1. This work is the first to consider LLM augmentation in the test time augmentation scenario\n2. Detailed analysis is provided on the impact of augmentation on finegrained performance of different tasks, and the impact of other factors in the process.\n\n## Weakness\n1. From the novelty perspective, there are previous work in both using test time augmentation to improve performance, and to use LLM for augment data generation. The paper 's proposal of using LLM for test time augmentation seems to be a straightforward extension on existing methods\n2. The effectiveness of the proposed method is in doubt. From the experiment results, using the test time augmentation does not consistently improve model performance, and the use of LLM for augmentation does not consistently outperforms previous augmentation methods. This suggests a missing link in the proposed method, which is identifying whether test time augmentation is helpful for the scenario. Though the paper proposes a entropy-based selection criteria, apperantly it is not effective enough\n3. Using LLM is costly, and with the power of LLM in hand, the use of the original black-box model may not be needed any more. More detailed analysis on if using pretrained LLM or LLM + in-context learning would be adequate to outperform the blackbox models used in this paper, especially in the \"out of distribution\" cases.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method to improve the model performance on language tasks of both in-distribution and out of distribution scenario of  a black-box model. The proposed method utilizes the scheme of test-time augmentation, and proposes to use a LLM to rewrite the input as a type of augmentation. Additional analysis is made to reduce the portion of test data that needs LLM augmentation. Performance improvement is observed emperically in multiple scenarios.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Need a more systematic way to decide when using LLM augmentation is helpful, and when using the LLM directly for the task is better.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "syFxa6lkd8", "reviewer": "Reviewer_LfbM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\nThis paper is well written and I am glad to read it. The description is clear and the claim is well supported by evidence. The idea using in-context rewriting for TTA is new to me, and also make sense, since the LLM models are powerful in language understanding and generation. \nThe Entropy-Based Selective Augmentation (ESA) for selectively augment test examples is also a good method to save the computational cost for TTA in practice. This paper also well addresses its limitations. \n\n\n**Weaknesses:**\n\nWhen the effectiveness of the proposed methods is supported by experiments, I am not confident whether the experiments on classification tasks are enough, especially under the setting that the models are fine-tuned from BERT/T5. Is it better for training from scratch for supporting the claims? Or considering the other generation tasks (even though the evaluation is more complicated)?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates how to improve the out-of-distribution (OOD) robustness by Test-time augmentation (TTA) for NLP tasks. It proposes LLM-TTA, which uses LLM-generated augmentations as TTA’s augmentation function. In particular, it uses two prompt templates including Paraphrasing and In-Context Rewriting (ICR). It also proposes Entropy-Based Selective Augmentation (ESA) for selectively augment test examples that are likely to benefit from TTA while deferring to the model’s original prediction otherwise. The effectiveness of the proposed method and the claims are well supported by experiments.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "NA", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "PdXsTihB7M", "reviewer": "Reviewer_M6U3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I appreciate the simplicity and accessibility of the method. It is well-motivated, combined with a useful set of experiments, and is likely to be a useful addition to the field.\n\nWhile I am basically ready to recommend acceptance, I also remarked some issues with the reliability of the experiments (namely, and especially, the lack of error bars), and a somewhat reductionist view of OOD generalisation. I will dive into more detail on both of these points in the Requested Changes section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the Authors propose to improve test-time augmentation methods by using an LLM to paraphrase the test example in various styles, then aggregating the results across paraphrases to predict the final result. Through several standard benchmarks on base embedding models like BERT, it is demonstrated that the system tends to improve OOD performance, while not incurring a significant penalty in IID performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Firstly, and critically, it is important to obtain confidence measures around these numbers. While the model weights may remain fixed, we should still be able to measure the performance across different sample collections paraphrased by the LLM, and report the variance in that way. This is, in my opinion, critical for practitioners to understand the robustness of the proposed method.\n\nSecondly, while the work's objective of ameliorating OOD generalisation issues is certainly very welcome, I also find that the work focusses only on OOD splits of freeform text tasks. This in my opinion offers a limited view of OOD; many established OOD robustness papers will instead focus on tasks for which controllable distributions at test time can be generated in certain parameters (e.g. problem size). Such papers will tend to focus on reasoning tasks, which are often not hard to synthetically generate (they could be as simple as, e.g. elementary arithmetic tasks). To name a few possible examples:\n\n* Ruoss et al., Randomized Positional Encodings Boost Length Generalization of Transformers\n* Zhou et al., Teaching Algorithmic Reasoning via In-context Learning\n* Zhou et al., Transformers Can Achieve Length Generalization But Not Robustly\n* Veličković et al., The CLRS Algorithmic Reasoning Benchmark\n\nIf the authors want to maintain the OOD narrative as they presently have it, I would strongly recommend they evaluate on a reasoning benchmark in the style of the above, and properly contextualise such works in their related work.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a new approach to improving the performance of text classifiers through the use of test-time data augmentation via an LLM. The main application is out-of-distribution classification, where the target text does not necessarily match the distribution that the classification model is fit to.

Two strategies are introduced: paraphrasing and in-context rewriting. Paraphrasing is a zero-shot approach that generates a diverse set of semantically equivalent inputs to the test input, whereas in-context rewriting uses a few-shot approach to rewrite the input in a similar style to provided examples. A strategy called entropy-based augmentation is introduced to select when to use the LLM for augmentation, saving time and cost while still yielding the benefits of test-time augmentation.

A few questions and concerns were raised by the reviewers around the novelty, scope, error bars, and comparisons to a fully in-context LLM. These were satisfactorily answered.

The remaining concerns raised by the reviewers include:
A request around a baseline that trains BERT or T5 from scratch (as opposed to fine-tuning). The statements made by the authors in their response (they are trained from scratch), and the text in the paper (they are fine-tuned, section 4.2) do not seem to agree.
A disagreement on the definition of OOD. There is no controllable, or quantifiable measure of OOD used in this paper, and one reviewer believes that the writing should shift focus towards generalization rather than OOD.

On this last point I would tend to agree. The term OOD is being used to mean “something that comes from a different distribution than the training set” without really specifying what is different about this distribution, or how different it is. Perhaps they are in-distribution, but simply challenging examples?

The reviewer suggested modifying the writing to focus less heavily on the concept of OOD. I would alternatively suggest writing a short section discussing this point, with some possible ideas on how to assess OODness in a measurable way in future work. The reviewer has already provided some suggestions around reasoning.

I think that with these changes, the paper satisfies the criteria for TMLR and should be accepted.
Decision: Accept (poster)