Test-time Augmentation for Factual Probing
Authors: Go Kamoda
Benjamin Heinzerling
Keisuke Sakaguchi
Kentaro Inui
Venue: EMNLP 2023 Findings (2023)
Keywords: Factual Probing
TTA
Calibration
TL;DR: Application of test-time augmentation calibrates model confidemce, but challenges of yielding high-quality paraphrases result in inconsistent accuracy improvement.

Abstract: Factual probing is a method that uses prompts to test if a language model ``knows'' certain world knowledge facts.
A problem in factual probing is that small changes to the prompt can lead to large changes in model output.
Previous work aimed to alleviate this problem by optimizing prompts via text mining or fine-tuning.
However, such approaches are relation-specific and do not generalize to unseen relation types.
Here, we propose to use test-time augmentation (TTA) as a relation-agnostic method for reducing sensitivity to prompt variations by automatically augmenting and ensembling prompts at test time.
Experiments show improved model calibration, i.e., with TTA, model confidence better reflects prediction accuracy.
Improvements in prediction accuracy are observed for some models, but for other models, TTA leads to degradation.
Error analysis identifies the difficulty of producing high-quality prompt variations as the main challenge for TTA.

Test-time Augmentation for Factual Probing
Go Kamoda'
Benjamin Heinzerling
Keisuke Sakaguchi!? Kentaro 3,12
'Tohoku University
RIKEN MMBZUAI
go. kakkdamoadcc.... tohoku. ac. jp
benjamin. heineeenineeernerrkeennnn
keisuke. sakaguchi@tohoku. jp.
kentaro. inui@mbzuai. aa. ae
Abstract
Where
Albert Einstein die?
Albert Einstein die?
Factual probing is method that uses prompts to test if a language model "knows" certain world knowledge facts. problem in factual probing is that small changes to the prompt can lead to large changes in model output. Previous work aimed to alleviate this problem by opti- mizing prompts via text mining fine-tuning. However, such approaches are relation-specific and do not generalize unseen relation types. Here, we propose to use test-time augmentation (TTA) as aggnosi- method for reduc- ing sensitivity prompt variations by automat- ically augmenting and ensembling prompts at test time. Experiments show improved model calibration, i.c., with TTA, model confidence better reflects prediction accuracy. Improve- ments prediction accuracy are observed for some models, but for other models, TTA leads to degradation. Error analysis identifies the dif- ficulty of producing high-quality prompt varia- tions as the main challenge for TTA.
Augmenter
location did Albert Einstein pass away?
site of Albert Einstein's death?
Pre-tralined
Pre-trained
Prendce
Preedoon Exetil
Proees Eris
Aggregator
Zurich
Dunh
Princeton
Figure 1: Factual probing with a single prompt (left) and with test-time augmentation (TTA). The orange com- ponents are added when using TTA. The Augmenter automatically augments the original prompt. The aggre- gator takes the generations from all prompts as input and outputs the generation with the highest score.
Prior work has aimed at finding better prompts for factual probing'. typically by employing su- pervised learning to find an optimal input token sequence for a given relation (Shin et al.. 2020; Jiang et al., 2020; Zhong et al., 2021). I). Since these approaches require supervision for each relation. they do not generalize unseen relation types, which reduces practical applicability.
1 Introduction
Pre-trained language models (LMs) such as BERT (Devlin et al., 2019) and T5 (Raffel et al., 2020) im- plicitly encode world knowledge from the training corpus in their parameters (Roberts et al., 2020). Encoded knowledge can be retrieved from an LM via a suitable prompt (Petroni et al., 2019). For example, a prompt such as "Where did Albert Ein- stein die?" is designed to retrieve the fact that "Albert Einstein died in Princeton." However, this capability is not robust since small changes to the prompt lead to drastic output changes (Heinzer- ling and Inui, 2021; Cao et al., 2022). If the model fails to answer correctly, it is thus difficult dis- tinguish if it did not learn the corresponding fact during pre-training or if it actually did but did not produce the correct answer with the given prompt.
Here, we propose to use test time augmentation (TTA) as an unsupervised, relation-agnostic ap- proach for improving prompt-based factual probing. TTA originates in computer vision, where given an input image test time, the idea is to 1) apply augmentations such flipping, cropping, or con- trast changes, 2) have the model make predictions for all augmented versions of the image, and then 3) aggregate these predictions to obtain the final prediction. TTA has been found to increase robust-
WWhile "probing" more commonly refers to model analysis via light-weight classifiers, we follow prior work in using "factual probing" to denote model analysis via knowledge- eliciting prompts.

---

ness and improve model calibration by reducing overconfidence in wrong predictions (Krizhevsky et al., 2012; Wang et al., 2019; Perez et al., 2018; 8;; Matsumaga et al., 2017). The benefits are also desir- able in factual probing, where LMs should exhibit robustness to paraphrases and should generate well- calibrated predictions. However, TTA has found little use in NLP so far. While it is relatively easy to create feature-preserving augmentations of images (e.g., flipped image of cat is still an image of a cat),
For each relation, we manually created an English prompt template, e.g., "Where did [subject) die?"
Augmenter To yield paraphrases with variety, the augmenter uses three types of prompt augmen- tations. The first type is synonym replacement, which replaces words in the input prompt with a synonym. For instance, this type of augmentation replaced the word "buried" with "inhumed". Syn- onyms are selected via word embedding similarity using GLoVe (Pennington et al., 2014) or via Word- Net (Saedi al., 2018). The second augmentation type is translatonn,,,, with French, Russian, Ger- man, Spanish, and Japanese as target languages.
augmentation of text is a challenging problem since even small edits, such as replacing single token, can completely change the meaning of a sentence (Rethmeier and Augenstein, 2023). In this paper, we empirically demonstrate the benefits and challenges of TTA for factual probing.
The third augmentation type is stopword filtering, which removes stopwords from the prompts.
From a single original prompt. the augmenter produces one prompt via stopword filtering and four prompts for each of the seven other augmenta- tion methods, resulting in a total of 30 prompts.
As a similar approach in the context of chain- of-thought reasoning, Wang et al. (2023) prepare a single prompt as model input and aggregate multi- ple outputs ("reasoning paths") to improve model performance. TTA differs from this method in that it automatically prepares multiple inputs, i.c., prompts.
Model We ran experiments on the following pre-trained language models: T5 for Closed Book Question Answering (Small, Large, 3B, 11B) (Roberts et al., 2020), FLAN-TS (Small, XL) (Wei et al., 2022), and TO_3B (Sanh et al., 2022).
To apply TTA factual probing, we add prompt augmenter and prediction aggregator to the prediction process (Figure 1). First, the in- put prompt automatically augmented by the aug- menter. The augmented prompts are then individ- ually fed to an LM. The aggregator collects the outputs for each prompt and determines the final prediction. Our evaluation of this setup consists of two parts: We 1) evaluated the overall prediction accuracy and investigated the impact of the num- ber of augmented prompts on the accuracy, and 2) inspected the change in the confidence of model predictions.
Models decode with beam-search where the beam size fixed 10 and return generated se- quences with scores. Scores are in the order log-likelihood (negative), and the exponentiated scores are in the order of probability.
Aggregator We aggregate generations by taking the sum of generation probabilities. The model output with generation probabilities (PLM) for each of the K augmented prompts (p;) will be fed into the aggregator to choose one final prediction. The aggregator recalculates the generation score (s) by taking the sum of the generation probabilities of identical generations (Eq.1). The final prediction of an object for the fact with subject and relation is the one with the highest score (Eq.2).
Results showed that the greater the number of augmented prompts, the better the performance when implementing TTA. TTA was also effective in reducing the number of overconfident and incorrect outputs. However, in terms of overall accuracy, TTA was only effective on smaller models.
= Σ PLM(s/PP))
(1)
2 Setup
Dataset We constructed a dataset of 12,500 rela- tional facts from Wikidata. Each fact is composed of subject, a relation, and an object. We filtered out facts with multiple possible objects and col- lected 500 unique facts for each of the 25 r
y argmax((((122,) ()))))
(2)
Evaluation Metric We quantify the effect of TTA as the relative change in exact match accu- racy compared to not using TTA:
relative effect (#correct w/ TTA) (3)
"The relations we selected are shown in the appendix.
(#correct w/o TTA) +

---

Small
FLAN Small
T0_38
T5-Small
T5-Large
15 Large
TS 13-119
FLAN TL-XL
1.0
0.8
wiett TIA
WoLt T1A
1.10
0.6
Relative effect
1.05
0.4
1.00
Accuracy
0.0
T5-3B
T5-11B
1.0
Wr TIA
w TIA
wieut TTA
WHOLE TTA
0.6
Number of prompts (1 s 30)
0.4
Figure 2: The relation between the number of prompts
and the average relative effect (Eq.3) of 'TA. A relative effect of 1.0 means no change in accuracy between with and without TTA.
0.0
0.0
0.2
0.4 0.6
0.8
1.0 0.0
0.2 0.4
0.6 0.8 1.0
Normalized Confidence
Figure 3: Comparison of model calibration with TTA and without TTA improves the model calibration of all four models in the comparison. Dotted lines rep- resent ideal calibration, i.e., model confidence equal prediction accuracy.
To prevent division by zero, one is added to the numerator and the denominator. The metric judges correct only if the output is perfect match. How- ever, observed cases where FLAN models indifferent capitalizations, e-g., the model gen- erated "africa" instead of "Africa. For this reason, we exceptionally adopt case-insensitive match ac- curacy for the FLAN models.
the result between K - 1 and K 30 shows that TTA leads better performance for three models, namely T5-Small, T5-3B, and TO_3B.
3 Results
Calibration In addition to increased accuracy, a second potential benefit of TTA to improve model calibration by reducing overconfidence incorrect answers. In this section, we investigate the effect of TTA on model calibration.
3.1 Positive Effects
Accuracy The top half of Table shows an exam- ple TTA increasing the accuracy on the T5-1 IB model. The model gave an incorrect answer with the original prompt but produced the correct answer with TTA.
In setup, the aggregator re re-ranks generations by calculating the sum of the generation probability of identical generations for each fact instance. Be- cause the original generation scores cannot used as confidence after aggregation, we define the con- fidence of the aggregator as the ratio of the score to the final output and the sum of calculated scores for all candidate generations from all prompts for each fact instance.
Here, we assess the impact of the number of prompts (K) on TTA performance. As described in $2, the augmenter produces 29 paraphrases from an original prompt. For each subset K prompts containing the original prompt and K K-1(1 < K < 30) randomly sampled paraphrases, we measure the change in model accuracy in terms of relative effect (Eq.3))
confidence scortriial output
(4)
We ran five iterations of random sampling and report the result Figure 2. The sudden drop from shows that models are highly sensitive to prompt variations. The relative effect appears to stabilize in the range of 20 < K < 30 prompts, suggesting that aggregation of multiple prompts via TTA mitigates model sensitivity to variation of individual prompts. A comparison of
candidates score
To enable a comparison of the relationship be- tween the defined confidence and accuracy with and without TTA, we normalize scores to a max- imum of 1. We then calculate the accuracy of model prediction with confidence between 0.14 and 0.1(i 1) for each
K to
10 N).
Figure shows the relation between the calcu- lated confidence and accuracy before and after ap-
3 = 1 setting indicates the baseline setting without ITA.

---

Type
Prompt
Generation
Original
What continent is Para District located on?
Africa
WordNet
What continent is Para District based on?
North America
bt-fr
What continent is the Para District located on?
South America
bi-ru
What continent is Pará County on?
South America
bt-de
On which continent the Para District located!?
South America
Original
Where is Hans-Georg Gadamer buried?
Heidelberg
GLoVe
Accordingly is Hans- Georg Gadamer buried?
in Bonn
WordNet
Where Hans- Georg Gadamer inhumed?
Erlangen
bt-fr
Where's & Hans-Georg Gadamer buried.
Frlangen, Germany
bt-ru
Where's 's Hans-Giorge Gadmer buried??
Wieshaden, Baden- Württemberg
bt-de
Where's Hans-Gcorg Gadamer buried?
Erlangen, Germany
bt-es
Where Hans- Georg Gadamer buried?
Heidelberg
bt-ja
Where are the goodly places? where is the plac...
Mount of Olives
Stopword-fiile -ilterinn
Where Hans-Georg Gadamer buried?
Marburg
Table 1: Examples of TTA improving and degrading performance. While Para District is located in South America ", the model gives a wrong answer under the original prompt. Applying TTA yields the cor- rect answer (top half). The bottom half of the table shows an example of TTA degrading performance. The correct answer is Heidelberg but TTA yields Erlangen, Germany ". The Type column specifies the augmentation type, with "GLoVe" and "WordNet" referring to synonym replacement and the "bt-" prefix indicating back-translation. Further examples are given in Appendix Table 3.
plying TTA. Without TTA, models tend to exhibit overconfidence by generating high-confidence swers with relatively low accuracy. This effect is es- pecially pronounced in larger LMs, i.e., T5-3b and T5-11B. contrast, after aggregation of multiple answers with TTA, confidence much better reflects accuracy since the accuracy of high-confidence pre- dictions improved. These results show that TTA can improve model calibration.
prompts, such as those shown in the bottom half of Table 1, do not meet these criteria. For example, not all augmented prompts preserve the meaning of the original prompt.
To better understand the impact of the quality of automatically augmented prompts, we conducted additional evaluations. The first is to remove tremely poor paraphrases. Inspection of TTA errors revealed that one the relations, namely "follows". was particularly error-prone in augmentation step, likely due to the large variety of different en- tity and event types* for which this relation is used in Wikidata. We thus analyze the impact of TTA after removing all instances of the "follows" rela- tion from the dataset. The second evaluation aims to improve the quality of paraphrases by using a large language model, namely GPT-3 text-davinci- 003 (Brown al., 2020), assuming that this larger LM produces better paraphrases than the simple augmentation methods used our original setup. Figure 4 shows how GPT-3-based augmentation changed the effect of TTA on the T5 models. With the exception of 15-11B augmentations produced by GPT-3 show positive effect for all models.
3.2 Negative Effects
Accuracy Accuracy declines when the original prompt elicits elicits the correct answer, but the TTA re- sults in an incorrect answer. The bottom half of Table shows an example of this. The 30 prompts yielded 18 unique model answers, among which seven prompts yielded the wrong answer "Erlangen, Germany"", while only four prompted the correct answer "Heidelberg" (Table shows only subset of these answers). Overall, TTA led to performance degradation with the TarLare T5-11B, and the FLAN-TS models (Figure 2).
Error Analysis Manual inspection suggests that the negative effects of TTA are mainly due to the low quality of the augmented prompts. Ide- ally, paraphrases should be grammatical, meaning- preserving, and exhibit variety in vocabulary choice and sentence structure. However, many augmented
'Arguments of the "follows" relation include: numbers (2 follows l), months (Fehruary follows lanuary) and events (the 2024 Paris Olympics follows the 2020 Tokyo Olympies).

---

T5-Small (CPT3)
T5-5mall
T5-38 (CPT3)
TS-38
Limitations
La-Large (GPT3)
La
T5-118 (EPT3)
TS-118
1.25
The TTA setup we present in this paper can be applied to short-answer generation tasks such as factual probing or classification tasks. A different setup would be necessary to apply TTA to tasks involving more complex model outputs, such summarization long-form question answering, since the aggregation step in our selup is not suited for such outputs.
1.20
Relative effect
1.15
1.10
1.05
1.00
In factual probing, TTA showed improvement in overall accuracy on three out of seven models. This means that whether or not TTA would improve the accuracy is unsure beforehand in practical uses. Deeper analyses of what improves/degrades the performance needed to judge whether to TTA or
Number of prompts (1 11)
Figure 4: The relation between the number of prompts and the average relative effect (Eq.3) of TTA when re- moved "follows" relation and paraphrased using GPT-3,
4 Conclusions
Ethics
Our experiments involved pre-trained language models and Wikidata, both of which are charac- terized by various forms of social biases. Conse- quently, experiments and conclusions inherit
We applied the idea of test-time augmentation (TTA) to language models, motivated by the ob- servation that models are robust prompt vari- ations. Specifically, we examined the effect TTA on model accuracy and calibration in a factual prob- ing setting.
and possibly amplify these biases.
Acknowledgements
Out of the seven models investigated, TTA had a positive effect on the accuracy of smaller models, namely T5-Small, T5-3B, TO_3B. When controlling for the quality of augmentations, TTA also improved the accuracy of one more model (T5-Large). On other models, TTA had a negative effect in terms of accuracy. In terms of model calibration, we observed a positive effect since TTA reduced the number of high-confidence incorrect model answers.
This work was supported by JST CREST Grant Number JPMJCR20D2, Japan, and JSPS KAK- ENHI Grant Numbers JP2IK17814, JP21K21343, and JP22H00524. We would like thank the mem- bers of TohokuNLP for their frequent participation in discussions during the course of this research.
The main remaining question is why the effect of TTA is inconsistent. We hypothesized that the inconsistent effect of TTA is due to the poor quality of automatically augmented prompts, and our anal- ysis showed that the high quality of paraphrases is one of the important conditions for TTA to be effective. related question we left to future work is how to enable TTA for relations that are difficult to paraphrase automatically, such as the "follows" relation (See Section 3.2).
While the ultimate goal is arguably make lan- guage models robust against paraphrases without extensions like TTA, the results presented in this work show the potential benefit TTA, especially for smaller LMs.

---

References
on Empirical Methods in Natural Language Process- ing.
Tom Brown. Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler. Jeffrey Wu. Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz. Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in neural information processing systems.
Fábio Perez, Cristina Vasconcelos, Sandra Avila, and Eduardo Valle. 2018. Data augmentation for skin le- sion analysis. In Lecture Notey in Computer Science. Springer International Publishing.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowl- edge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Process- ing and the 9th International Joint Conference on Natural Language Processing.
Boxi Cao, Hongyu Lin, Xianpei Han, Fangchao Liu, and Le Sun. 2022. Can prompt probe pretrained language models? understanding the invisible risks from a causal view. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. Journal of Machine Learning Research.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies.
Nils Rethmeier and Isabelle Augenstein. 2023. A primer on contrastive pretraining in language processing: Methods, lessons learned, and perspectives. ACM Comput. Surv.
Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the param- eters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.
Benjamin Heinzerling and Kentaro Inui. 2021. Lan-
guage models knowledge bases: On entity repre- sentations, storage capacity, and paraphrased queries. In Proceedings of the 16th Conference of the Euro- pean Chapter of the Association for Computational Linguistics.
Chakaveh Saedi, António Branco, João António Ro- drigues, and João Silva. 2018 WordNet embeddings. In Proceedings of the Third Workshop on Represen- tation Learning for NLP.
Dieuwke Hupkes, Mario Giulianelli, Verna Dankers. Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Chris- tos Christodoullppulls, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schotmaann Khuyagbaatar Batsuren, Kaiser Sun, Koustuy Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske, Ryan Cotterell, and Zhijing Jin. 2022. State-of-the- gen- eralisation research in NLP: a taxonomy and review. CoRR.
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker. Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim. Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang. Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj, Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Fevry. Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In International Conference on Learning Representations.
Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. How we know what language models know? Transactions of the Association for Computational Linguistics.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hin- ton. 2012. Imagenet classification with deep convo- lutional neural networks. In Advances in Neural In- formation Processing Systems 25. Curran Associates, Inc.
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.
Kazuhisa Matsunaga, Akira Hamada, Akane Minagawa, and Hiroshi Koga. 2017. Image classification of melanoma, nevus and seborrheic keratosis by deep neural network ensemble. arXiv.
Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS- MT - building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word rep- resentation, In Proceedings of the 2014 Conference

---

Guotai Wang, Wenqi Li, Michael Aertsen, Jan Deprest, Sébastien Ourselin, and Tom Vercauteren. 2019. Aleatoric uncertainty estimation with test-time aug- mentation for medical image segmentation with con- volutional neural networks. Newrocomputing.
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoc Le, Fed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models.
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M. Dai, and Quoe V. Le. 2022. Finetuned language models are zero-shot learners.
Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021. Factual probing is [MASK]: Learning learning recall. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies.

---

A
Dataset
Property ID
Property Name
P17
Country
P19
Place of birth
P20
Place of death
P27
Country of citizenship
P30
Continent
P36
Capital
P37
Official language
P50
Author
P69
Educated at
P103
Native language
P119
Place of burial
P131
Located in the administrative territorial entity
P140
Religion or worldview
P155
Follows
P156
Followed by
P159
Headquarters location
P407
Language of work or name
P495
Country of origin
P641
Sport
P740
Location of information
P937
Work location
P1365
Replaces
P1366
Replaced by
P1376
Capital of
P1412
Languages spoken, written, or signed
Table Wikidata property IDs and English labels used for constructing our dataset.
We collected 500 relational facts for each of the 25 relations selected from WikiData. Table 2 shows the corresponding Wikidata property and English property names.

---

B Augmentation Methods
Synonym Replacement We use the Python library "TextAttack" to replace words with their synonyms. Specifically, for synonym replacement using WordNet, we use the WordNetAugmenter, and for synonym replacement using GloVe embeddings, we use the WordSwapEmmeddinn class.
Back-translation We first translate the original prompt to eight candidates in the target language. Each candidate is then translated back into eight candidates in the source language, resulting in total of 64 back-translated prompt candidates. We adopt the round-trip probability as the score of the back-translated prompt candidates and select four candidates using the aggregation method mentioned in Section 2. For translations, we used OPUS-MT models (Tiedemann and Thottingal, 2020) The OPUS-MT models occupy roughly the same memory size as the T5-Small model.
Stopwords-fittrree This method removes stopwords and diacritics from the original prompt using the Python library "Texthero".

---

C Count-based Aggregation of Model Answers
1.05
Relative Effect
T5-small (count)
1.00
T5-large (count)
T5-3B (count)
0.95
T5-11B (count)
0.90
0.85
0.80
5
15 20 25
30
Number of prompts
Figure 5: The relation between the number of prompts and the average relative effect of TTA when aggregated using count-based method. relative effect of 1.0 means no change in accuracy between with and without TTA.
Counting the number of appearances the generations one method aggregation. We did not use count-based aggregation our main experiments because the possibility of having multiple generations with the same counts is high. The phenomenon is predicted to occur more often when we increase the number of sequences the model outputs. In addition, this method cannot take confidence into account as all generations by beam-search are equally weighted.
The result of using count-based aggregation is shown in figure 5. TTA degrades model performance with count-based aggregation.

---

D Additional output examples
Table gives a complete example of all augmented prompts and corresponding model outputs. (A subset of these is also shown in Table in the main part of the paper).
Type
Prompt
Generation
Original
Where is Hans- Gadamer buried?
Heidelberg
GL
Accordingly Hans- Hans-Georg Gadamer buried?
in Bonn in Bonn Erlangen
GLoVe
Consequendy is Hans- Hans-Georg Gadamer buried?
GL
Where poses Hans Georg Gadamer buried?
GL
Where represents Hans Hans-Georg Gadamer buried?
Erlangen, Germany
WordNet
Where is Hans-Georg Gadamer immerse?
German philosophical neo-Kantia
WordNet
Where Hans-Goorg Gadamer inhume?
Frlangen, Germany
WordNet
Where Hans-Georg Gadamer inhumed?
Erlangen
WerdNet
Where is Hans-Georg Gadamer swallow?
the throat
bt-fr
Where Hans-Georg Gadamer buried?
Heidelherg
bt-fr
Where's & Hans-Georg Gadamer buried?
Frlangen, Germany
bt-fr
Where's Hans-Georg Gadamer buried.
Erlangen, Germany in the garden
bt-fr
Where's Hans-Georg Gadamer buried in the house?
bt-ru
Where Hans-George Gadmer buried?
Basel
bt-ru
Where Hans-George Gadamer buried?
Erlangen, Germany
Where's 's Hans-George Gadmer buried?
Wiesbaden, Baden- Württemberg
bt-ru
Where is Hans-Gorg Gadmer buried?
Wieshaden, Wirtcn--ürttebberg
ht-de
Where's 's Hans-Cicorg Gadamer buried??
Firlangen, Germany
ht-de
Where Hans-Goorg Gadamer buried?
Heidelherg
bt-de
Where was Hans-Georg Gudamer buried?
the Munich Waldfriedhof
bt-de
Where will Hans- Hans-Georg Georg Gadamer be buried?
in the Cathedral of Speyer Heidelberg
Where Hans-Georg Gadamer buried?
bt-es
Where's 's Hans-Georg Gadamer buried?
Erlangen, Germany
bt-es
Where is Hans-Georg Qadamer buried?
In Munich cemetery
bt-es
Where Hans-Georg Gadhamer buried?
Innsbruck
bt-ja
Where are the goodly places? where is the plac...
Mount of Olives
bt-ja
Where are the goodly places? Where the plac...
Bethel
bt-ja
Where the goodly places? where
pla...
the mountain God
bt-ja
Where are the goodly places? where is the plac...
the place of his fathers
Stopword-filterine
Where Hans-Georg Gadamer buried?
Marhurg
Table 3: An example of case in which TTA degrades the accuracy of LM answers. The correct answer to the question "Where is Hans-Georg Gadamer buried?" "eidelhergg" ", but the aggregator returned Erlangen, Germany

---

E GenBench Evaluation Card
To situate our work in the broader context of efforts to understand and improve the generalization machine learning models for natural language processing, Table 4 provides a GenBench evaluation card (Hupkes et al., 2022).
Motivation
Practical
Cognitive
Intrinsic
Fairness
Generalisation type
Compositional
Structural
Cross Task
Cross Language
Cross Domain
Robustness
Shift type
Covariate
Label
Full
Assumed
Shift source
Naturally occuring
Partitioned natural
Generated shift
Fully generated
Shift locus
Train test
Fineture train test
Pretrain- train
Pretrain test
Table 4: GenBench Evaluation Card


Reviews:
[{"review_id": "0GuQLSky79", "reviewer": "Reviewer_nPms", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are pleased that the reviewer recognizes the significance of the problem addressed in our paper. We hope our subsequent responses adequately address the questions and concerns raised.\n\n\n## 1. Wang et al. (2022): Self-Consistency Improves Chain of Thought Reasoning in Language Models (Reasons to Reject 1):\n\nThank you for your reference. While Wang et al.'s motivation to generate reliable outputs aligns with ours, their approach differs. Wang et al. suggest generating multiple outputs from a single prompt and then taking the majority votes. In contrast, our method involves generating outputs from multiple prompts and then taking the majority votes.\nOur contribution lies in empirically demonstrating the effectiveness (or lack thereof) of the test time augmentation method for the factual probing task. Instead of merely chasing the state-of-the-art, we believe that it is crucial to explore similar yet distinct approaches and share the results with the community.\n\n## 2. Small changes to the prompt can lead to large changes in model output (Reasons to Reject 2):\n\nWe are emphasizing that a **single** alteration of a prompt can influence a model's outputs. Through test time augmentation, we can produce multiple outputs from **several** slightly modified prompts. By taking the majority/average, we can mitigate the impact of one minor alteration.\nThe sentence in the abstract is a little unclear. We will rectify this in the final draft.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper works on an important problem in factual probing, i.e., models could give quite different results for the same sample due to the small changes in prompts.", "reasons_to_reject": "The proposed augmentation method is not very novel. A similar approach has been proposed in \"Self-Consistency Improves Chain of Thought Reasoning in Language Models.\"\n\nThe experiments show the \"impact of the number of prompts\" and the \"relation between the calculated confidence and accuracy\". However, in the abstract, the motivation is alleviating the problem that \"small changes to the prompt can lead to large changes in model output.\" I do not understand how the proposed method helps to solve the above \"small change ...\" problem.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The factual probing results could change as the prompts change, making the probing results unstable. This paper mitigates this problem by using more prompts to query each fact. The additional prompts are obtained by synonym replacement. The proposed methods can somewhat improve the factual probing accuracy of some datasets.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8W4LqYacV8", "reviewer": "Reviewer_HiLF", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are pleased that the reviewer acknowledges that the contribution of our work is in improving model reliability by applying TTA and improving model accuracy or calibration of model confidence to accuracy. We hope our following answers address the raised questions and concerns satisfactorily.\n\n## 1. Impact of our approach to large models (Reasons to Reject 1):\nIf we assume that larger LMs are generally \"better\", then we can also assume that they are more robust to meaning-preserving input variations, so it is expected that the impact of our method becomes smaller as model size increases. In this study, we empirically assess the effectiveness of the test-time augmentation method for factual probing tasks across various model sizes. Rather than solely pursuing state-of-the-art results with larger models, it is also important to improve smaller LMs (environmental impact, financial constraints, hardware constraints when running LMs on edge devices, etc) and this is exactly where our method works best.\n\nWe would also like to remind the reviewer that the approach is effective in calibrating model confidence to model accuracy as explained using Figure 3, even (or especially) on large models. For example, before implementing TTA, T5-11B model had low accuracy when it generated the answers with high confidence(right part of the orange line in the lower right graph in Figure 3). This means a considerable number of confident answers were incorrect (= over-confident). After TTA, the relationship between the model accuracy and the confidence aligns better to the ideal line (the diagonal dotted line in the figure). This indicates that TTA reduced over-confident and incorrect answers.\n\n## 2. Why TTA is hardly used in NLP tasks (Question 1):\n\nWe will add an explanation to the paper in the camera ready. Our explanation on why TTA is hardly used in NLP tasks would be the following.\nTTA is easy to use with visual data since there is basically an infinite amount of augmentations that are almost guaranteed to preserve the input features in question. For example, a flipped image of a cat is still an image of a cat, the same for rotation, color variations, adding noise, blurring, etc. In contrast, automatic augmentation of textual data is much more difficult.\n\n## 3. About $K$ in Equation 1 (Question 2):\nWe appreciate the reviewer for noticing the oversight. The $K$ stands for the number of prompts we feed to the model for each instance. For example, if we obtain 3 paraphrases from an original prompt, $K$ equals 4, which is the sum of 3, the number of paraphrases, and 1, the number of original prompt.\n\n## 4. About the reason we adopt case-insensitive match accuracy on FLAN models (Question 3):\nWe will add an explanation to the paper when it is accepted.\nThe reason why we adopt case-insensitive match accuracy in FLAN models is because we found numerous outputs that ignored capitalizations. For example, there were cases where the model generated “africa” instead of “Africa”, which was not observed in other models we tested.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Experiments show model confidence better reflects prediction accuracy by improving model calibration. The results presented in this work show the potential benefit of TTA, especially for smaller LMs.", "reasons_to_reject": "Adaptability to large models limits the potential value of this technique. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper aims to address the problem in the factual probing  task, which is that small changes to the prompt can lead to large changes in model output. The authors propose to use test-time augmentation (TTA) as a relation-agnostic method for reducing sensitivity to prompt variations by automatically augmenting and ensembling prompts at test time.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1）Line 061 \"Nevertheless, TTA is hardly used in NLP tasks.\" It would be better to explain why \"TTA is hardly used in NLP tasks\".\n2）K is not explained in Eq 1.\n3）Line 135-136 \"with the exception of the FLAN models, for which we adopt case-insensitive match accuracy\". It would be better to give the reason for the exception.\n"}}, {"review_id": "lzNuuZFDb3", "reviewer": "Reviewer_jtoE", "summary": "", "questions": "- How many relations are there in the dataset evaluated (what are them), and how many instances does each relation has? \n- Does the x-axis of Figure 2 starts at 0 or 1?\n- What are the order of prompts in the x-axis of Figure 2, 4 and 5? Since there are prompts from different augmentation approaches, it is hard to identify the performance of each category. Especially when there is a performance drop immediately after the start.\n- What are the candidates generated in line 170, the generation from all prompts, or the generations from the same beam of individual prompts?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are pleased that the reviewer finds our test time prompt augmentation approach both interesting and appreciable in its simplicity. We hope our following answers address the raised questions and concerns satisfactorily (Reasons to Reject 2).\n\n\n## 1. About Jiang et al. (2020)’s work (Reasons to Reject 1):\n\nWe acknowledge that the back-translation method is well-established, having been used in the MT community for years [1][2]. Our contribution lies in the utilization of it to evaluate the test-time augmentation method for factual probing tasks. \nUnlike Jian et al. (2020), our approach does not rely on pre-selecting paraphrases using a pre-gathered dataset. Our TTA overcomes this constraint, offering spontaneous, relation-agnostic augmentation.\nWe will improve the writing to clarify this in our final draft. \n\n[1] Ondrej Bojar and Ales Tamchyna. 2011. Improving translation model by monolingual data. In Workshop on Statistical Machine Translation (WMT), and Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016.   \n[2] Improving neural machine translation models with monolingual data. Conference of the Association for Computational Linguistics (ACL).)\n\n\n## 2. About the impact of our approach to large models (Reasons to Reject 3):\n\nIf we assume that larger LMs are generally \"better\", then we can also assume that they are more robust to meaning-preserving input variations, so it is expected that the impact of our method becomes smaller as model size increases. In this study, we empirically assess the effectiveness of the test-time augmentation method for factual probing tasks across various model sizes. Rather than solely pursuing state-of-the-art results with larger models, it is also important to improve smaller LMs (environmental impact, financial constraints, hardware constraints when running LMs on edge devices, etc) and this is exactly where our method works best.\n\nWe would also like to remind that the approach is effective in calibrating model confidence to model accuracy as explained using Figure 3, even (or especially) on large models. For example, before implementing TTA, T5-11B model had low accuracy when the it generated the answers with high confidence(right part of the orange line in the lower right graph in Figure 3). This means a considerable number of answers were over-confident and incorrect. After TTA, the relationship between the model accuracy and the confidence aligns better to the ideal line (the diagonal dotted line in the figure). This indicates that TTA reduced over-confident and incorrect answers.\n\n## 3. About the dataset we constructed (Question 1):\nThank you for noticing this oversight. We gave only the total number of relation instances in the submission and indeed did not include counts for each relation. We collected 500 instances from each of 25 relations we selected, giving the total of X * Y = 12,500 relational facts we stated on line 82 in the submission. All instances are collected from WikiData, and the selected relations (properties) are the following. The relations were selected under the condition that we can collect 500 instances with low distribution bias. We will also add the information to the paper when it is accepted.\n\n|Property ID | Property Name |\n| - | - |\n|P17  | Country |\n|P19  | Place of birth |\n|P20  | Place of death |\n|P27  | Country of citizenship |\n|P30  | Continent  |\n|P36  | Capital |\n|P37  | Official language |\n|P50  | Author |\n|P69  | Educated at |\n|P103  | Native language |\n|P119  | Place of burial |\n|P131  | Located in the administrative territorial entity  |\n|P140  | Religion or worldview |\n|P155  | Follows |\n|P156  | Followed by |\n|P159  | Headquarters location |\n|P407  | Language of work or name |\n|P495  | Country of origin |\n|P641  | Sport |\n|P740  | Location of information |\n|P937  | Work location |\n|P1365  | Replaces |\n|P1366  | Replaced by |\n|P1376  | Capital of |\n|P1412  | Languages spoken, written, or signed |\n\n\n## 4. About the starting value of x-axis in Figure 2 (Question 2):\nThank you for noticing this oversight. The x-axis in Figures 2, 4, and 5 all starts with 1. The points where the x (number of prompts) is 1 represent the accuracy of the model with no prompt augmentation. We will revise the Figures in our camera ready.\n\n\n## 5. About the order of prompts in Figures 2, 4, and 5 (Question 3):\nIf the “order” mentioned refers to the scale of the x-axis, it is in a linear scale. 1 means that one single prompt is inputted into the model, and 10 means that 10 prompts, including one original prompt, are fed to the model.\nIf the “order” refers to “how we chose the 5, for example, prompts”, it was randomly sampled from the pool of augmented prompts. In the setting of Figure 2, there are one original prompt and 29 augmented prompts for each instance of the dataset. The plot where the x value is 5 means one original prompt is ensembled with 4 other randomly sampled augmented prompts. \n\n## 6. About “candidates generated” in line 170 (Question 4):\nIt refers to “all the generation from all prompts”. Since we fixed the beam size to 10, if the number of prompts was $P$, there would be $10\\times P$ candidates.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The test time prompt augmentation is an interesting problem to improve the effectiveness prompt without training. The proposed method is simple to generalize.", "reasons_to_reject": "- The back-translation based prompt rephrasing is also used in [1].\n- The introduction of the configuration and results lack important details to fully understand the procedure and performance.\n- The pretrained model of larger sizes are observed to demonstrate stronger ability of language comprehension and therefore more robust to the variance in the prompt templates, which would undermines of importance of ensemble multiple prompts generated from linguistic perturbations, The results also show that models of larger sizes receive lighter improvement corresponds to the smaller counterparts.\n\n\n[1] Jiang et al. How can How can we know what language models know?", "questions_to_authors": "- How many relations are there in the dataset evaluated (what are them), and how many instances does each relation has? \n- Does the x-axis of Figure 2 starts at 0 or 1?\n- What are the order of prompts in the x-axis of Figure 2, 4 and 5? Since there are prompts from different augmentation approaches, it is hard to identify the performance of each category. Especially when there is a performance drop immediately after the start.\n- What are the candidates generated in line 170, the generation from all prompts, or the generations from the same beam of individual prompts?", "paper_topic_and_main_contributions": "This work proposes a prompt augmentation method TTA without training instances. The augmentation includes synonym substitution, back-translation and stop word filtering. They collected a relational dataset from Wikidata and evaluated several pretrained language models of different sizes, and observe that TTA introduces more positive impacts for pretrained models of smaller sizes. And TTA in general improves calibration of the model. Lastly, they find that prompt rephrasing from GPT-3 also has benefits to most model variants.", "excitement": 2, "reproducibility": null, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Paper Topic And Main Contributions:
* This paper aims to address the problem in the factual probing task, which is that small changes to the prompt can lead to large changes in model output.
* The authors propose to use test-time augmentation (TTA) as a relation-agnostic method for reducing sensitivity to prompt variations by automatically augmenting and ensembling prompts at test time.
* The augmentations include synonym substitution, back-translation and stop word filtering.

Reasons to accept:
* The problem of answer variability depending on the prompt is important.
* The test time prompt augmentation is an interesting solution to improve the effectiveness prompt without training. The proposed method is easy to generalize.

Reasons to reject:
* The proposed augmentation method is not very novel. A similar approach has been proposed in "Self-Consistency Improves Chain of Thought Reasoning in Language Models." The authors explain in the rebuttal: Wang et al. suggest generating multiple outputs from a *single* prompt and then taking the majority vote. In contrast, the proposed method involves generating outputs from *multiple* prompts and then taking the majority vote.
* Diminishing improvements for larger models. The authors explain in the rebuttal that smaller LMs are important (environmental impact, financial constraints, hardware constraints when running LMs on edge devices, etc).
Decision: Accept-Findings