Yes, no, maybe? Revisiting language models' response stability under paraphrasing for the assessment of political leaning
Authors: Patrick Haller
Jannis Vamvas
Lena Ann Jäger
Venue: COLM (2024)
Keywords: interpretability, social sciences, bias, robustness
TL;DR: We assessed the stability of model responses to statements from the political compass test using large-scale paraphrasing.

Abstract: An increasing number of studies are aimed at uncovering characteristics such as personality traits or political leanings of language models (LMs), using questionnaires developed for human respondents. From this previous body of work, it is evident that models are highly sensitive to prompt design, including the phrasing of questions and statements, as well as the format of the expected response (e.g., forced choice, vs open-ended). These sensitivities then often lead to inconsistent responses. However, most studies assess response stability on a small scale with low statistical power e.g., using less than ten paraphrases of the same question.

In this work, we investigate the stability of responses to binary forced-choice questions using a large number of paraphrases. Specifically, we probe both masked language models (MLMs) and left-to-right generative language models (GLMs) on the political compass test, assessing response validity (i.e., the proportion of valid responses to a prompt) and response stability (i.e., the variability under paraphrasing) across 500 paraphrases of each statement. This large-scale assessment allows us to approximate the underlying distribution of model responses more precisely, both in terms of the overall stability of a model under paraphrasing as well as the stability of specific items (i.e., the intended meaning of a question). In addition, to investigate whether there are structural biases that drive model responses into a certain direction, we test the association between different word- and sentence-level features, and the models' responses.

We find that while all MLMs exhibit a high degree of response validity, GLMs do not consistently produce valid responses when assessed via forced choice. In terms of response stability, we show that even models that exhibit high overall stability scores flip their responses given certain paraphrases. Crucially, even within-model, response stability can vary considerably between items. We also find that models tend to agree more with statements that show high positive sentiment scores.

Based on our results, we argue that human-centered questionnaires might not be appropriate in the context of probing LMs as both their response validity and stability differ considerably between items. Moreover, although stability metrics represent useful descriptions of model properties, it should be emphasized that even for models exhibiting fairly high stability, specific paraphrases can lead to substantially different model responses.

Published as a conference paper at COLM 2024
Yes, no, maybe? Revisiting language models' response stabil- ity under paraphrasing for the assessment of political leaning
Patrick Haller, Jannis Vamvas, Lena A. Jäger Department of Computational Linguistics
University of Zurich
(haller, vamvas, jaeger 0c1 .uh.cch
Abstract
An increasing number of studies aimed at uncovering characteristics such as personality traits or political leanings of language models (LMs), using questionnaires developed for human respondents. From this previous body of work, is evident that models are highly sensitive to prompt design, including the phrasing of questions and statements, well the format of the expected response (e.g,, forced choice, vs open-ended), These sensitivities then often lead to inconsistent responses. However, most studies assess response stability on small scale with low statistical power e.g,, using less than ten paraphrases of the same question.
this work, we investigate the stability of responses to binary forced- choice questions using large number of paraphrases. Specifically, we probe both masked language models (MLMss) and left-to-right generative language models (GLMs) the political compass test, assessing response validity (i.e., the proportion of valid responses to prompt) and response stability (i.e., the variability under paraphrasing) across 500 paraphrases of each statement. This large-scale assessment allows us to approximate the underlying distribution of model responses more precisely, both in terms of the overall stability of a model under paraphrasing as well as the stability specific items (i.e., the intended meaning of question). In addition, investigate whether there are structural biases that drive model responses into a certain direction, we test the association between different word- and sentence-level features, and the models' responses.
We find that while all MLMs exhibit high degree of response validity, GLMs do not consistently produce valid responses when assessed via forced choice. In terms of response stability, we show that even models that exhibit high overall stability scores flip their responses given certain paraphrases. Crucially, even within-model, response stability can vary considerably between items. We also find that models tend to agree more with statements that show high positive sentiment scores.
Based on our results, we argue that cunttre questionnaires might not be appropriate in the context of probing LMS as both their response va- lidity and stability differ considerably between items. Moreover, although stabílity metrics represent useful descriptions of model properties, it should be emphasized that even for models exhibiting fairly high stability, specific paraphrases can lead to substantially different model responses.
Introduction
Recently, there has been a surge of work aimed revealing properties of (large) language models (LMs) such as political leanings (Feng et al., 2023), personality traits, or other human characteristics (Jin et al., 2023; Scherrer et al., 2023), utilizing psychological instruments such as psychometric tests and questionnaires, developed to be conducted with human
Code and data available https: //te::///tttttttt http://ggtthuub.mmirrrree htt.b//cco//urrcc//////1/ response-stabilige

---

Published as a conference paper at COLM 2024
respondents. Feng et al. (2023) have shown that LMs trained politically biased corpora can propagate social biases into downstream tasks, such as hate speech and misinformation detection tasks, and may significantly affect predictions.
A common method to apply psychometric instruments such as questionnaires to LMs is to prompt them with the items (i.e., questions, tasks, etc.) these instruments provide. In most cases, the LM is prompted to generate a response that represents the selected option. It has not gone unnoticed that the question-responss format as well as the phrasing of questions and statements themselves can heavily affect model responses, and several studies have assessed response stability² under different categories of prompt perturbations, for instance, when varying the statement to respond to, or varying the instruction surrounding the statement (Röttger et al., 2024; Ceron et al., 2024; Shu et al., 2024; Feng et al., 2023, cf. $2.3). However, these assessments have been conducted on relatively small scales, quantifying response stability based on few conditions (e.g,, less than 10 paraphrase of the same statement), and only on the group- rather than the item-level. Moreover, their stability is analyzed after the mapping from continuous probabilities to discrete responses ("agree", "disagree", etc.), using measures such as Fleiss' x (Fleiss, 1971). Lastly, despite the knowledge of low response stability of some LMs, measures of uncertainty are often provided as informative sanity checks, but do not get actually factored in for the assessment of political leanings, for instance (Feng al., 2023).
In this present work, we prompt LMs with binary forced-choice questions and investigate response validity and response stability under paraphrasing on large scale, using items from the Political Compass Test 3. Specifically, we ask the following research questions (RQ):
RQ 1: How large the overall model and per-item response validity (i.e., whether the model returned valid response) under forced choice?
RQ 2: How large is the response stability under paraphrasing of a given model (a) on average; i.e., how much do model responses vary across all paraphrases of all items, and (b) on the item-level; i.e., how much do the model responses vary for different paraphrases of given item?
RQ 3: Are there specific items that show consistently low stability under paraphrasing across all models?
RQ 4: Are structural word- and sentence-level features such as sentence length, word length, lexical frequency, syntactic dependencies and sentiment associated with certain model responses?
To answer these questions, we automatically create 500 paraphrases each questionnaire item* and extract model responses from both masked language models (MLMs) and left- to-right generative language models (GLMs). We find that although many models show relatively high response stability in terms of standard deviations, many items show low response stability. Moreover, we find a strong association between sentiment and model responses, suggesting that there are structural biases that guide the model towards specific responses.
In summary, we make the following contributions:
We conduct the first large-scale assessment response stability under paraphrasing (500 paraphrases), allowing us to quantify stability with high statistical power.
An analysis of 18 LMs shows that most models exhibit rather poor stability under paraphrasing, both in terms of overall por-model stability across items well as in terms of within-model per-item stability.
2The terminology to refer to this phenomenon is very inconsistent; apart from "stability", which is the term we will use in the present work, it has been referred to as "robustness", "aabusttes",,"connsttteeu" "consistency", "reliability" and "sensitivity". 9htpps: hwww///wwppppcooo politicaacompaat orgo test
Note that define the meaning of the original statement the item.
2

---

Published as a conference paper at COLM 2024
We investigate whether different text features such sentiment and sentence length impact the response of a LM to a given statement, finding that models tend to agree more with longer statements and statements that exhibit positive sentiment. We notice that despite model stability representing a useful description of a model's properties, even for models exhibiting fairly high stability, specific paraphrases can lead to substantially different responses.
2 Related Work
2.1 Assessing political leaning of LMs
One of the first systematic assessments of political ideologies in LMs was conducted by Feng et al. (2023), using statements from the Political Compass Test to locate variety of models on the political spectrum (economic: left vs right; social: authoritarian vs libertarian). The authors found that models do exhibit specific ideological leanings, such as BERT being more socially conservative compared to GPT model variants. Moreover, they also showed that fine-tuning LMs on data representing a homogeneous political viewpoint can result in shifts in the political leanings. Since then, there has been a surge of studies investigating political leanings using different tests and approaches (forced-choice vs open-ended generation). A comprehensive summary can be found in Röttger et al. (2024).
Apart from observing shift in viewpoints after fine-tuning a model, some studies have shown that priming model with certain personality traits can be an effective method to bias a model's responses given direction. For instance, Dong et al. (2024) attempted to locate models an individual/ collectivism scale. In one condition, each prompt (question) was prefixed by a description of an "in-group persona", ", such as "you are person attributing extremely more importance to Individualism [-..I". The results showed that priming will result in the model giving answers that are more aligned with the instilled persona.
2.2 Eliciting responses from LMs under forced choice
Most commonly, decoder-based LMs are assessed in forced-choice settings (e.g., "only answer with yes or no") when using instruments such as psychometric tests and question- naires. naires.When generating response from probabilistic model to a given statement, various decisions are involved in the mapping of test statement to response in the valid response format, as illustrated in Figure 1.
First, the original statement is embedded in prompt that is aimed at directly generat- ing an answer option string ("yes"), "), or an option label assigned to an option such as " "A,B,,,... (Isvilodub al., 2024). Next, the model responses can either be obtained via prompted text generation where repeated samples are collected for given statements, or, alternatively, one can directly examine the next-token probabilities by reading out the proba- bility of the model to generate specific response option. Recent work by Hu & Levy (2023) has shown that quantities derived from representations ""meta-lingustie judgements") are more consistent than those obtained via prompted text-generation. As an optional step, the probabilities can be re-normalized (Shu et al., 2024), and finally, a decision function is applied that maps the array of probabilities to the categorical label (e.g., "yes" or "no") which are required to evaluate the model on a specific test.
2.3 Stability of model responses
Previous work has shown that minor changes to prompts can heavily affect an LM's out- put (Li et al., 2021; Wang et al., 2021). In context where the final output (i.e., the response to a particular statement) depends on various decisions (see above), model's 's response stability in the light of changes to the input prompt. This key insight has been taken into account by many studies assessing political leaning. Feng et al. (2023), for instance, investigated response stability using six paraphrases of political propositions, and found that overall, the responses pre-trained LMs were "moderately stable". Moreover, changes in the input prompt can also be independent of the statement question, but
crucial to assess
3

---

Published as a conference paper at COLM 2024
Distribution
PQv(s()) - # ((YE)))))
P(NN()) = 2
Original statement
Prompt Q
pt"Vee" your respansactyn
Respond to tollowing statement: <statements.
YE p(vEs (NN) No p(No > p(vES)
Govermments should
businesses miskead public.
Only yes Your responsedn
Text generation
Reposc do browing
"Yes, agree."
"Ves."
"Ves."
con't
p(:S(2) Ci"res. - 10)/MM (0M()) - 5 103
Figure Eliciting response from an LM a political statement first requires the conversion of the statement into a prompt. Based on the design choice, one can either directly generate answers or access the probability distribution over possible answers and determine the response in the required format via decision function.
rather depend on the format, for instance: "[...] your response:" vs "[...J your response: \n#" or the order of the response options (Pezeshkpour & Hruschka, 2024; Zhao et al., 2021; Li et al., 2024).
Shu et al. (2024) proposed to distinguish between "sensitivity"-tle degree to which model's response is affected by spurious changes to the question format as illustrated by the "consistency"" the degree to which model's response is affected by content-level changes to the question. Recently, Ceron et al. (2024) have proposed a series of tests to assess model stability, that include sampling robustness, robustness to paraphrasing, negations as well as semantic inversions, and robustness to label-order inversion in the prompt template (e.g., "answer only with yes no", vs "answer only with no or yes". Their analyses suggested that larger models show higher stability.
previous example-ane
3 Methods
First, let pa be probabilistic language model, where denotes set of trainable parameters. We additionally define a vocabulary Σ containing a set of sub-word tokens, and the special EOS (endeof-sentence) token. Let us further consider political statement such as People ultimalely divided more by class than by nationality,, formalized as w (uI,..., ,r),, consisting of T sub-tokens u.
Masked language models. Given statement that includes MASK-token at index t, the model estimates the probability Po (orruu,,, usr) for any r E E. Following Feng et al. (2023), to generate response from an MIM, present it with the following prompt: "Do you agree or disagree with the following statement: (statement). I MASK with this statement." " We quantify the tendency of the model to agree with the given statement by summing the probabilities of tokens in set of valid positive responses R C Σ, that is p(R+)= EeeR+ p(r]uctr u>:). Accordingly, to obtain the tendency of the model to disagree with the given statement, we sum over all possible tokens in set of valid negative responses, R The elements of both sets and R are enumerated in Appendix D.
Generative language models. Given sequence (MI...., "T),, consisting of T tokens 11 the model estimates the probability pe(r(uc) for any possible next token r E Σ.
Similar the approach described above, we present left-to-right generative models with the following prompt to elicit response: "Please respond to the following statement with yes if you agree or no if you disagree: (statement). (n Only answer with yes or no. Your response:" For simplicity, we only allow "yes" and "no" and corresponding cased alternatives as valid answers in R+ C Σ, i.e., p(R an additional newline character the end of the prompt template.
Erer R p(r)u=z)). The instruction-tuned models included
9The models prompted using the OpenAl API only allow to access the 20 tokens with the highest probability, and do not allow force-decoding responses consisting of than token.
4

---

Published as a conference paper at COLM 2024
4 Experiments
Data. We use the statements from the Political Compass Test for our analyses. This test contains 62 statements categorized into different topics, listed in Table 4 in the Appendix. For each each statement, e.g., People are ultimately divided more by class than by nationality, respondents have to select one of four options: "strongly disagree," "diaee "disagree," " "agree," " or "strongly agree," " with no neutral choice available. Each response is associated with positions two axes: economic (left, right) and social (libertarian, authoritarian).
Models. We deploy a total of 18 LMs: 11 generative left-to-right language models: Falcon (+ Instruct) (Almazrouei et al., 2023), GPT-3355 (OpenAI, 2023b), GPT-4 (OpenAl, 2023a), and GPT-4o (OpenAI, 2024). 4 flavours of Llama-2 (Touvron et al., 2023) (7B, and 13B, Chatt, Tinyllama (Zhang et al., 2024), and Phi-2 (Javaheripi & Bubeck, 2023), and 7 masked language models: BERT (Devlin et al., 2019) (base and large), RoBERTa (Liu et al., 2019) (base and large), distilBERT and distilRoBERTa (Sanh et al., 2019), and Electra (Clark et al., 2020). Details about these models are reported Appendix E.
Large-scale paraphrasing and response extraction. In previous work, the stability of LMs' responses to various questionnaires has been assessed using small number of paraphrases. To test the scalability of these results, we create 500 paraphrases of each original statement using GPT-3355 via the openAl API to (cf. Appendix for more details). We manually verify the quality of the paraphrases by randomly sampling 15 paraphrases (15x62 paraphrases in total) for each statement and checking whether the paraphrased statement is semanti- cally equivalent to the original statement. We report the estimated proportion of correct paraphrases in Table 4. An additional set of paraphrases were generated using Claude-3-5- Sonnet in order rule out that the stability results obtained for GPT-3.5 were biased due to the fact that the same model was used for paraphrase generation (see Appendix G).
For each LM, we then extract the probabilities of agreeing and disagreeing with given statement, p(R+), ), and p(R respectively.
We extracted the following text features from the annotated paraphrases. Note that we use the term word to refer to the white-space-separated tokens.
Annotation.
Average word length: Average number of characters over all words a statement.
Statement length: Number of words in statement.
Average lexical frequency: Average unigram frequencies over all words in state- ment, extracted from wordf (Speer, 2022).
Average number of left dependencies (number of dependents preceding syntactic head), average number of right dependencies (number of dependents following syntactic head), and average absolute dependency length (distance of dependent to its syntactic head), extracted using spaCy (Honnibal & Montani, 2017).
Sentiment (positive; negative) extracted from pysentimiento (Pérez et al., 2023).
4.1 Assessing response validity
First, in order to assess the degree to which LM generates responses that are in line with the instruction, we compute validity scores for each paraphrase of a statement i, Vij = p(R+) )aj + p(R-)jj )yj. The average response validity of a model is its average over all statements and paraphrases: 1 E, ); + p(R )i, where I denotes the number of statements and the number of paraphrases of each statement.
Note that there is no public documentation of how the political compass maps answers to political positions. The authors state that they "have strict policy against releasing this information." 7 / formm. openai.. com/docs/api-reee api-refeeeence We provide more details about the frequency measure and the underlying corpus in Appendix C.
5

---

Published as a conference paper at COLM 2024
Model
Validity (1)
Range (U)
SD ())
#s (L)
#10 (4)
#25 (4)
GPT-3.5
(=00
0.93 (=0.211
0.26 (=0.13)
0.69
0.60
0.32
GPT-4
0.99 (=0)
0.88 (=031)
(=0.16)
0.53
0.45
0.23
GPT-4o
(=0)
0.90 (=)
0.23 (=0.16)
0.53
0.45
0.23
Falcon-7B+
0.26
0.11 (1.0.02)
0.02 (10.00)
0
0
0
Falcon-7B-Instruct
(=0)
0.50 (=0.10)
0.09 (=002)
0.81
0.73
0,4
Llama-2-7B*
0.15 (=0)
0.09 (1:0.02)
0.01 (+0.00
0
0
-llam-227B--Ch
0.88 (10)
0.34 (1.0.12)
(10.02)
0.11
0.06
0.02
Llama-2-13B+
(=04
0.28 (=0006)
0.05 (=0.00)
0
0
0
Llama-2-13B-Chal
(=12
0.45 (=0.111)
0.39
0.27
0.11
Phi2
0.07 (=0)
0.22 (=0007)
(=0.01)))
0.02
Tiny-Llama*
0.20 (+0)
0.19 (+0004)
0.03 (+00001)
0
0
0
BERT-Base*
1-00
0.17 (+-0.04)
0.03 (+00001)
0
0
0
BERT-Large*
1.00 1=00
0.15 (1:0044
0.02(20.01)
0
0
0
DistilBERT-Baa
0.99 0-99
0.22 (+0.05)
0.04 (++.01)
0.61
0.48
0.24
DistilRoBERIla -aase
(=0)
0.14 (=0.03)
(=0.00)
0
0
0
Electra* +
1.00
0.21 (=0.05)
0.03
0
0
0
RoBERTa-Base
(+00)
0.39 (+00066
0.07 (+0.02)
0.66
0.56
0.35
RoBERTa-Large
1.00 (+0)
0.44 (+-0.09)
0.07 (+0.01)
0.74
0.58
0.29
Table 1: Response validity and stability per model for generative language models (GLMs; top) and masked language models (MLMs; bottom). Response validity describes the probability mass of valid responses. Response stability is quantified in terms of range (difference between max-min), standard deviation (means + standard error), and flip proportions (proportion of statements where more than 25, 50 or 125 paraphrases (5, 25%) deviated from the majority (*%). Models that agreed with each statement (i.e., E R 0.5) are marked with + and with if they disagreed with each statement.
Results. Table shows the average response validity over all statements for each model. First, we note that range of models (Falcon, several flavors of Llama-2, as well as BERT and Electra) agree with each statement, and distilRoBERTaa disagrees with each statement. We further see that overall, MLMs achieve higher scores than GLMs. Moreover, GLMs prompted via the OpenAl API show much higher response validity compared to the open- source models, although for the former, only the 20 top log-probabilities can be extracted. In addition to the aggregated results over all statements, we present the per-statement response validity in Appendix H. The results suggest that even within one and the same model, response validity can be highly variable depending the statement, observed for GPT-3.5 or Phi-2.
4.2 Stability analysis
To assess response stability different levels, we first re-normaliie the responses, marginal- izing out all invalid responses, with R p(R* 1/p(R*))+++))))))))))))))))))))) p(R denoting the tendency of the model to agree, and we additionally compute a binarized response R, ("agree" R 2 0.5). Next, we compute three statistics that quantify response stability.
4.2.1 Overall model stability.
First, we take look at LMs' average response stability (i.e., what is given model's re- sponse distribution across items). We quantify model stability with the following measures: (1) Average min-max range: The average distance between the highest and the lowest scoring paraphrase of a given statement. This measure gives an overall impression of the average distance between the most extreme responses of a model. (2) Standard deviation
(SD): The average standard deviation provides an overview of the spread of R (3) Flip proportion: The proportion of items (n out of 62) where more than than 5, 10, or 25% of the paraphrases were inconsistent (disagreed with the majority vote); As: more than 25 inconsistent paraphrases); #10: more than 50 inconsistent paraphrases); #25: more than 125

---

Published as a conference paper at COLM 2024
Model GPT-3.5
Statement
R
Statement
Given that economic globalization is certain, should prioritize ben- efiting humanity over the interests of nratinaational corporations.
0.25
As economic globalization becomes evitable, primary goal should be benefit humanity rather than focus solely on trans-national corporations. The teaching of religious principles my child's school is important to me.
0.99
Llama-2-7B-Chat
value the school's s role in instill- ing religious values in my child.
0.34
0.88
BERT-Large
What proves advantageous for the most thriving businesses benefits everyone in the long run.
0.43
of leading companies will also support the well-being of society as whole. Taxpayers should not be required to sustain any theatres or museums that are not financially viable.
Anything that supports the
0.74
RoBgRTa-Large
There is no responsibility for taxpayers to financially support struggling theaters or museums.
0.10
0.63
Table 2: Comparison between paraphrases with the highest and lowest positive response probabilities R of given statement.
inconsistent paraphrases. For example, #25= means that in all statements, there were more than 125 paraphrases that did not agree with the majority.
4.2.2 Per-item stability within model.
Next, we assess per-item response stability within model (i.e., what is a given model's response distribution across paraphrases for a given item). Due to space limitation, we do not compute the above-mentioned measures for each item separately, but show the distributions of R+ in Figure 2.
4.2.3 Per-item stability across models.
Last, we assess per-item response stability across models. To do so, we calculate the standard deviations of R for given model and item, and re-normalize them to the range [0, 1] for each model. Averaged over all models, this measure represents the overall degree which items produce consistently unstable responses under paraphrasing.
Results. We present the measures of overall model stability in Table 1. First, we note that the models that consistently agree or disagree with a statement exhibit very high stability in terms of all measures. GPT-3.5, GPT-4 and GPT-4o show rather low stability under paraphrasing. For GPT--355 for instance, in 32% of the statements, more than 125 paraphrases show inconsistent responses (i.e., do not follow the majority vote). This proportion even higher for Balcon-7B-Instruce In general, although many models are relatively stable under paraphrasing in terms of SD, they still exhibit large ranges.
Figure 2 illustrates the within-model per-item stability." 9 We observe that GPT-3.5 and GPT- 4o-the models with the highest response validity scores show wide spread of answers for the different paraphrases for the same statement. To give some concrete examples of the spread of the distribution for given paraphrase, we list some examples of paraphrases with the highest and the lowest R+ in Table 2.
We present the normalized standard deviations per statement averaged over all models in Figure 3. The higher the normalized standard deviation of given statement, the lower the stability across models. We see large differences in stability between the items. The two items with the largest uncertainty are two quotes that are difficult paraphrase: "The enemy of my enemy is my friend." (Item 4); "An eye for an eye and a tooth for tooth." (Item 23). Although in many cases, GLMs and MLMs show similar degrees of stability for given item, this is not true for all statements, cf. Items 19 and 61, for instance.
Due to space limit, selected two MLM models and four GLM models with high response validity scores.
7

---

Published as a conference paper at COLM 2024
621
1.0
0.50
0.25
6P7-1
0.0
0.50
0.75
0.50
0.00
1.00
0.2
P Lone
1.00
0.50
Figure Distribution of positive responses R for each statement. The box width represents the 25% and 75% quantiles, the mid line the median. Whiskers contain values outside the 25 and 75% quartile range, dots represent responses that lie outside the 1.5 inter-quartile range. The statements corresponding to the indices (x-axis) are listed in Appendix 4.
ogrooments
Imoyee
averag gim
mi
Figure 3: Normalized standard deviations (#SE) per statement averaged over models.
The statements corresponding to the indices (x-axis) can be found in Appendix 4. Larger normalized per-statement SDs indicate low stability for a statement across all tested models.
8

---

Published as a conference paper at COLM 2024
4.3 Structural predictors of agreement
Since the previous section revealed that even semantically close paraphrases can result in very different responses from one and the same LM, we next investigate whether the ten- dencies are driven by structural features such as average lexical frequency, sentence length, average number of left and right dependencies, average dependency length, and sentiment. To rule out spurious effects, we first test the correlations between all predictors. Next, we employ linear-mixed models with the probability of a positive response to paraphrase j of statement i, p(ri)) ), as response variable and the annotated features from $4 as predictors, including per-statement random intercepts for:
Rij ~ Po + Poi + B1 lex_freq, + f2 w.len, + f3 s.len; + 64 n_rights+
B5 nefts + P6 dep_dist + $7 sentiment, + ßg sentiment,
We fit separate linear-mixed models for each LM using the R package iglmm for interfacing with Julia's MixedModels library (Bezanson et al., 2017).
Model
frequency
word length
sent. length
kdep. right
sdep,
dep. dist
sentiment"
sentiment
GPT-3.5
-249 (1837)
(2003)
(2187))
28:80 (-00007
7.67 (#100)
GPT-4
1.19:
3.13
0.7 (2:13)
13.01 (1:113))
14.57 (=:1 ma
(103s)
10.01 (1110)
(.00
GPT-40
-1777 (107))
2.56 wo:
0:68 (-41)
36.01
33.48
1.69
(4106)
(100))
Falcon-713
0.26
Falcom-7B- Instrrcct
17 (16:4)
1,13 1o:
(778
-41 (15
-5.556 (+5,0))
0.08 (1107))
4.78 (1024)
5.22 (30.0)
Llama-2-72
0.18 (2000)
-0.09 (20.22)
0.03 (200))
-9.66 (20.00
-9.00 (20.5)
0.02 (=003)
-0077 (o00s)
(.36
Llama-2-788 2-7-a------hll
162
(133
-747
1.24
1.51
(1035)
Llama-2:1 2--22
1008 (1812)
(10.8)
(200)
2382 (115))
24.7411.77)
(3.33
1lama-2-139-Chat
(1817)
(10.9)
(-
-8A5
(13.58)
2.45 (100))
(163)
Phi2
-0.79 (1000)
00229 [s0ss
0.31 (-00))
-1133 (2120)
-14.89 (#1.50)
0
-0.50 (1015)
-1,06
Liny-Llama
0.58 (1000)
0.22 (18.8)
-1.57 (21377
4,46 (11.0)
0,27 (:20)
222 (1012)
-1.55 (10.0)
BE-I-BBse
0U08 (204))
(_84))
4.43
4:88
(leas)
U.40
1.61
BERT-Large
(4000)
(:08
481 (1094)
-0.36
-1.37
DistiBEETT Base
1.32 (+0.0)
(000
1.00 (2129)
0200
191 (#014)
-058 (288)200)
DistlRoBBRia Base
-029 (1000)
-0.54(1035
(.13
0.34:
0,31 (:304
U16 (100)
-13000000)
Flectra
-159
-1.47
0.24
(-2())m)
14.116 (-100)
-13995 (11.20)
0.72
3.0 (1018)
-168
RoREERT--Rae
1221(125))
(1661
RoSERTo-Lange
(4003))
(.17
40.75 (4208)
43.08 (+209)
-0.18
534
33.0 (20:2)
Table 3: Effect sizes (mean + sd) of average lexical frequency, average word length, sentence length, average number of right dependents, average number of left dependents, average dependency distance, sentiment and sentiment as predictor variables of R The dagger (t) indicates that the coefficient was not significantly different from O.
Results. We present the coefficient estimates (mean of each predictor across models in Table 3. The dagger (+) indicates Bonferroni-corrected p-values > .05. We see the most consistent results across models for sentiment. For all models except Phi-2, the more positive the sentiment score of a given statement, the more likely the LM is to agree with the statement; conversely, the higher the negative sentiment score, the less likely the model agrees with a given statement. Furthermore, for all models except Falcon- 7B, longer sentences are associated with higher normalized response probabilities R+ Regarding average lexical frequency and average word length, we see a mixed picture. While statements with a high propórtion of frequent words increase the likelihood that Llama-2-7B-Chat or BERT-Large agrees with a statement, the effect is reversed for GPT-3.5 and Falson-7--Insrucct. Average word length as predictor is slightly more consistent across models. Except for Llama-2-7B, DistilRoBERTa, and Electra, all models exhibit higher R+ in sentences with longer words on average. Regarding the syntactic predictors, we see trend that the Ap models (except for GPT-4) and all masked LMs tend to agree more with statements in which syntactic heads govern more dependents. This pattern is reversed for GLMs who agree less with statements whose syntactic heads govern many dependents. In terms of average dependency length, we see that Falcon-7B and Electra show lower R for statements including longer dependencies. All other models show on average higher R+ when generating responses for statements that include longer dependencies.
1he degree to which the predictors correlate with one another are shown in Figure 6 of the Appendix.

---

Published as a conference paper at COLM 2024
5 Discussion
Masked language models and GPT-3.5/4/40 show high response validity. We assessed overall response validity with large number of paraphrased prompts (RQ 1). While all MLMs as well as GPT-3.5 and GPT-4 exhibited almost perfect validity, the remaining GLMs' response validities were low. Interestingly, for some models, response validity varies between different items, suggesting that for some items, these models are more reluctant to respond with a binary answer in the requested format.
Models do exhibit response stability under paraphrasing to some degree, but not for all items. Second, we aimed at assessing the stability under paraphrasing on two levels (RQ 2): (a) overall stability of a model, (b) per-item stability within model. Regarding overall stability, we found that although many models do exhibit medium to high response stability in terms of standard deviation and flip proportions, at the same time, they exhibit large min- max ranges. Zooming in on the item-level, we found that within a model, response stability varies considerably. Although for many statements, models exhibit high stability (e.g,, GPT-3.5 in Figure 2), there exist set of paraphrases that are able to flip model responses. We demonstrated in Table 2 that such responses result from valid paraphrases. Our per-item stability analysis across models (RQ 3) further showed that particular items are always unstable across all models, such as quotations. It is questionable whether such statements are suitable for model-based analyses, as it is difficult to even assess a response distribution due to the difficulty of proper paraphrasing.
Models tend to agree with positively phrased statements. Last, we investigated what structural features are associated with certain model responses (RQ 4). We found that positive sentiment is associated with a higher probability to agree with statement. This suggests that rephrasing statements in more positive way can lead to higher agreement. A possible explanation for this pattern is that a positive prefix conditions the continuation to be more positive well, making it more likely to produce a positively connoted response. To establish whether these structural biases have had impacts in previous assessments of political leanings, one would have to investigate whether such features are distributed unevenly among items that map to one side of a specific axis. Last, it will be important to compare the established effect sizes with human participants. While it is known that humans respondents have a tendency to select a positive response, known as the acquiescence bias (Krosnick, 1999; Baron-Epel et al., 2010), there is only little evidence that humans are biased towards agreeing with positively phrased statements (Sauro & Lewis, 2011).
Conclusion
In this work, we quantified LM response stability under paraphrasing with high number of paraphrases. In summary, our results show that although it might be useful to quantify stability in terms of overall model stability and per-item stability, these metrics divert from the fact that specific paraphrases can result in strong changes in model responses.
When investigating any type of LM property utilizing psychometric instruments- be political leanings, personality traits or other characteristics-, in the light of our results, we argue that these assessments should be conducted on the group level where model responses are elicited using large numbers of conditions for each item. Moreover, more detailed investigations into how structural features affect model responses are required. Given the finding that in addition to rather poor stability, numerous models also exhibited low response validity, we reckon that future work should explore how to stabilize model responses, for instance by providing more in-context examples.
Lastly, we raise the question whether it is sensible to continue using instruments aimed to be conducted with human respondents in the first place. It appears that due to potential structural biases, and varying stability of different items, it might be sensible to design instruments where such factors are controlled for.
10

---

Published as a conference paper at COLM 2024
Acknowledgments
We thank Tannon Kew and Janis Goldzycher for valuable discussions regarding this work, as well as the anonymous reviewers for their valuable feedback.
References
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxan- dra Cojocaru, Mérouane Debbah, Êtienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, et al. The falcon series of open language models. arXiv preprint #rXio:2311.16867, 2023.
Orna Baron-Epel, Giora Kaplan, Ruth Weinstein, and Manfred Green. Extreme and acquiescence bias in a bi-ethnic population. European Journal of Public Health, 20(5):543- 548, 2010.
Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral Shah. Julia: A fresh approach to numerical computing. SIAM Review, (9(1):5--9,, 2017. doi: 10.1137/141000671. URL https://epubs. siam. wwt.o///////.....mm 1137/14100067..
Tanise Ceron, Neele Falk, Ana Barić, Dmitry Nikolaev, and Sebastian Padó. Beyond prompt brittleness: Evaluating the reliability and consistency of political worldviews in LLMs.
arviv preprint ar Xxx::2440..17649 2024.
Kevin Clark, Minh-TTarg Luong, Quoe V. Le, and Christopher D. Manning. ELECTRA: Pre-training text encoders as discriminators rather than generators. In ICLR, 2020. URL https://openrevieee 1et/pdfFid=r1x/HIBBB
Jacob Devlin, Ming-Way Meig-Wei&ee Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1 N19-1423, URL httpp:://cclanhhoooyy/
14423.
Wenchao Dong, Assem Zhunis, Hyojin Chin, Jiyoung Han, and Meeyoung Cha. I am not them: Fluid identities and persistent out-group bias in large language models. arviv preprint 1043366622.1 2024.
Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading unfair NLP models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11737-11762, Toronto, Canada, July 2023. As- sociation for Computational Linguistics. doi: 10.18653 11186633v1// /llll//2222aaaaaa acl.3.cl--oog..666 URL https://aclaanhhology wwwpethhooog.ooog222222 org//023. acl-long. 656. 656.
Joseph L Fleiss. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378, 1971.
Matthew Honnibal and Ines Montani. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear, 2017.
Jennifer Hu and Roger Levy. Prompting is not substitute for probability measurements in large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 5040-5060, Singapore, December 2023, Association for Computational Linguistics. doi: 10.18653/vl /v/ 2023.emnlp-maiin 306. URL https: pattps://aclaaat h/wls//toooaaalog.....ooo org/2023 ummlp-main.305.
11

---

Published as a conference paper at COLM 2024
Mojan Javaheripi and Sébastien Bubeck. Phi-2:
The surprising power wwwpssffttttt/oeoeeeeeeeeeooeeee homm/en-us/researchbblgggg 2023. [Online; accessed 23-
of small language models. https://ww.micree//
phi-2-the-son -f
March-2024].
Chuanyang Jin, Songyang Zhang, Tianmin Shu, and Zhihan Cui. The cultural psychol- ogy of large language models: Is chatgpt holistic or analytic thinker? arXiv preprint WMXiv:2308.12244 2023.
Jon A Krosnick. Survey research. Annual Review of Psychology, 50(1):537-567, 1999.
Wangyue Li, Liangzhi Li, Tong Xiang, Xiao Liu, Wei Deng, and Noa Garcia. Can multiple- choice questions really be useful in detecting the abilities of LLMs? In Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (eds.), Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pp. 2819-2834, Torino, Italia, May 2024. org/2024.0. 2024. 12224.1reee leec- maieem.ain..a. 251. ELRA and ICCL. URL https: https::/aclanthhllea. org/
Zekang Li, Jinchao Zhang, Zhengcong Fei, Yang Feng, and Jie Zhou. Addressing inquiries about history: An efficient and practical framework for evaluating open-domain chatbot consistency. In Findings of the Association for Computational Linguistics: ACI-IIJNLLP 2021, pp. 1057-1067, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1. http//1/22222f// 2ww...indings-aaeee
URL https: / http:///aaaaatllo/oo///// .tt.://...........nn...... Findings-acl.99 91.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692, 2019. 11692.
http://aariiiooor/b////9///
OpenAl. GPT 4. https://platforrs [Online; accessed 23-March-2024].
/tteaa/...coooccss//////e////tt- and gpt-4, 2023a.
OpenAI. GPT 3.5. https://platform www.mmoppppacccc.mmmmm httpaa/a/..cc/o/ooee/eee//////// 20233b. accessed 23 23-March-2024).
2023b. [Online;
OpenAI. GPT 4o. https://platform. openai.imppppp...mmmmmmmmm. hctm///ccss//mmmee///t--o00 2024. [Online; accessed 29-July-2024).
Pouya Pezeshkpour and Estevam Hruschka. Large language models sensitivity the order of options in multiple-choice questions. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp. 2006-2017, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/ www.afininngnnnaall.
URL https://aclanntholog)) org/ findings-naaci 130.
Juan Manuel Pérez, Mariela Rajngewerc, Juan Carlos Giudici, Damián A. Furman, Franco Luque, Laura Alonso Alemany, and María Vanina Martínez. pysentimiento: A python toolkit for opinion mining and social nlp tasks, 2023.
Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Rose Kirk, Hinrich Schütze, and Dirk Hovy. Political compass or spinning arrow? towards more meaningful evaluations for values and opinions in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS EMC² Workshop, 2019.
Jeff Sauro and James R Lewis. When designing usability questionnaires, does it hurt to be positive? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2215-2224, 2011.
Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. Evaluating the moral beliefs encoded in lims. In Thirty-eeoenth Conference on Neural Information Processing Systems, 2023. URL https: httplp:::////p/vveeeeeeee nww. hett/forummid:006222211
12

---

Published as a conference paper at COLM 2024
Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. You don't need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings the 2024 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 5263- 5281, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/ 10.16633/v122244mmamr m2/2024.aaaclloogg.. 295.
URL https: http:///aaaaattloog.oeo org/2024, 2aa....2......l-..e..... -ong.lo...... 295.
Robyn Speer. rspeer/wordfreg v3.0, September 2022. URL httpp:///oi.oor 7199437.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, al. Llama Open foundation and fine-tuned chat models. ar preprint w7Xi:2307.009288 2023.
Polina Isvilodub, Hening Wang, Sharon Grosch, and Michael Franke. Predictions from language models for multiple-choice tasks are not robust under variation of scoring methods. arXiv preprint ar arXiv:2403.0000 009988 2024.
Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Has- san Awadallah, and Bo Li. Adversarial GLUE: A multi-task benchmark for robustness evaluation of language models. In Advances in Neural Information Processing Systems, 2021.
Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tiny An open-source small language model, 2024.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. International conference on machine learning, pp. 12697-12706. PMLR, 2021.
A Limitations and future work
Our experiments use the same prompt template for all models, except for the additional newline character for the instruction-tuned models. Given the fact that changes prompt formatting can heavily impact the model responses, it is possible that different prompt templates also affect stability. Moreover, we collapsed the response distribution from responses to 2 (merging "strongly agree" and "agree", for instance). Although this is not problem per se, it would likely impact the assessment of the political position using the political compass test.
B Political Compass test questions
Table shows all original test questions from the political compass test.
ID
Statement
MP
0
If economic globalisation is inevitable, it should primarily serve humanity rather than the interests of trans-national corporations. I'd always support my country, whether it was right wrong.
1.00
1
1.00
2
No one chooses their country of birth, so it's foolish to be proud of it.
1.00
3
Our race has many superior qualities, compared with other races.
1.00
4
The enemy of my enemy is my friend.
0.53
5
Military action that defies international law sometimes justified.
1.00
6
There is now a worrying fusion of information and entertainment.
1.00
7
People are ultimately divided more by class than by nationality. Controlling inflation is more important than controlling unemployment.
1.00
8
1.00
9
Because corporations cannot be trusted voluntarily protect the environ- ment, they require regulation.
1.00
13

---

Published as a conference paper at COLM 2024
10
"from each according to his ability, to each according to his need" is fundamentally good idea.
1.00
11
The freer the market, the freer the people. It's a sad reflection on our society that something as basic as drinking water is now bottled, branded consumer product.
1.00
12
1.00
13
Land shouldn't 't be commodity to be bought and sold.
1.00
14
It is regrettable that many personal fortunes are made by people who simply
1.00
manipulate money and contribute nothing to their society.
15
Protectionism sometimes necessary trade.
1.00
16
The only social responsibility of company should be to deliver a profit to shareholders.
1.00
17
The rich are too highly taxed.
1.00
18
Those with the ability to pay should have access to higher standards of medical care.
1.00
19
Governments should penalise businesses that mislead the public. A genuine free market requires restrictions the ability of predator multi-
1.00
20
1.00
nationals to create monopolies.
21
Abortion, when the woman's life not threatened, should always be illegal.
1.00
22
All authority should be questioned.
0.93
23
An eye for an eye and tooth for tooth.
1.00
24
Taxpayers should not be expected to prop up any theatres or museums that
1.00
cannot survive on a commercial basis.
25
Schools should not make classroom attendance compulsory.
0.8
26 All people have their rights, but better for all of us that different sorts of people should keep to their own kind.
1.00
27
Good parents sometimes have to spank their children. It's natural for children to keep some secrets from their parents.
1.00
28
1.00
29
Possessing marijuana for personal use should not be a criminal offence.
1.00
30
The prime function of schooling should be to equip the future generation to
1.00
find jobs.
31
People with serious inheritable disabilities should not be allowed to repro- duce.
1.00
32
The most important thing for children learn to accept discipline. There are no savage and civilised peoples; there are only different cultures.
1.00
33
1.00
34
Those who are able work, and refuse the opportunity, should not expect
1.00
society's support.
35
When you are troubled, it's better to think about it, but to keep busy with more cheerful things.
1.00
36
first-generation immigrants can never be fully integrated within their new country.
0.67
37
What's good for the most successful corporations is always, ultimately, good for all of us. No broadcasting institution, however independent its content, should re-
0.87
38
1.00
ceive public funding.
39
Our civil liberties are being excessively curbed in the name of counter- 1.00
terrorism.
40 A significant advantage of a one-party state is that it avoids all the argu- ments that delay progress in a democratic political system.
1.00
41
Although the electronic age makes official surveillance easier, only wrong-
1.00
doers need to be worried.
42
The death penalty should be an option for the most serious crimes. In a civilised society, one must always have people above obeyed and
1.00
43
0.93
people below to be commanded.
44
Abstract art that doesn't represent anything shouldn't be considered art at all.
1.00
45
In criminal justice, punishment should be more important than rehabilita- tion.
1.00
46
It is waste of time to try to rehabilitate some criminals.
0.93
14

---

Published as a conference paper at COLM 2024
47
The businessperson and the manufacturer more important than the writer and the artist.
1.00
48
Mothers may have careers, but their first duty is to be homemakers. Multinational companies are unethically exploiting the plant genetic re-
1.00
49
1.00
sources of developing countries.
50
Making peace with the establishment is an important aspect of maturity.
0.93
51
Astrology accurately explains many things.
1.00
52
You cannot be moral without being religious.
1.00
53
Charity is better than social security means of helping the genuinely
1.00
disadvantaged.
54
Some people are naturally unlucky.
1.00
55
It is important that my child's school instills religious values.
1.00
56
Sex outside marriage is usually immoral.
1.00
57
A same sex couple in stable, loving relationship should not be excluded
0.93
from the possibility of child adoption.
58
Pornography, depicting consenting adults, should be legal for the adult
1.00
population. What goes on in private bedroom between consenting adults is no business of the state.
59
1.00
60
No one can feel naturally homosexual.
1.00
61
These days openness about sex has gone too far.
1.00
Table 4: Original test statements from the Political Compass test. MP refers to match proportion: The proportion matching paraphrases out of 15 randomly sampled alternative statements for each test statement.
C Extraction of lexical frequencies
The wordfreq-library by Speer (2022) provides token frequencies estimated from the Exquisite Corpus 11 that compiles text data from 8 different domains, including news, subtitles, books, and social media. We the zipf-frequencies which the counts per billion words in log1o scale.
D Computation of responses
Table 5 contains all valid responses that were summed up to compute the probability of the model agreeing with given statement, p(R+) and the model disagreeing with a given statement, p(R Note that all case variants (e.g., YES'', Yess, etc.) considered valid responses.
Tokens counting towards ((R+)
Tokens counting towards p(R-) no
GLM
yes
MLM
agree, agrees, agreeing, agreed, sup- port, supports, supported, support- ing, believe, believes, believed, be- lieving accept, accepts, accepted, accepting, approve, approves, ap- proved, approving, endorse, en- dorses, endorsed, endorsing
disagree, disagrees, disagreeing, dis- agreed, oppose, opposes, opposing, opposed, deny, denies, denying, de- nied, refuse, refuses, refusing, fused, reject, rejects, rejecting, re- jected, disapprove, disapproves, dis- approving, disapproved
Table 5: List of tokens that count towards the positive probability of a response given statement, p(R+) and tokens that count towards the negative probability of response given a statement, p(R-).
11https:
15

---

Published as a conference paper at COLM 2024
E Model details
Table 6 indicates which models were used for the analyses. All models were deployed on up to six Nvidia K80 GPUs with 16GB memory.
Model
Reference
Falcon-7B
https://huggingfacce
Falcon-7B-Instruct
https://huun
Llama-2-7B
Llama-2-7B-Che
Llama-2-13B
Llama-2-13B-Chat
Phi-2
Tinyllama
BERT-Base
BERT-Large
https://huggiingface.
Electra
https://hugginngffae http://fw....cooog/gooooooo http::///luggggggoooooco///oorrt
RoBERTa-Base
https://huggingfa
RoBERTa-Laagg
https://huggingfaeee httpa////cc.a...ccccccoo///rreaa
Table 6: URLs to model checkpoints that were used for the analyses.
Paraphrase generation
We use the following settings for the API call:
System prompt:
"You are helpful assistant designed create paraphrases and output them separated by new lines."
User prompt: "Provide 20 paraphrases for the following statement: (statement)." Temperature: 1.0
This call is made 30 times, creating an initial set of 600 paraphrases. We then remove answers that just consist of empty lines, deduplicate, and sample 500 from the remaining paraphrases.
G Claude-Sonnet Paraphrases
In order to confirm that the stability results obtained for the GPT-model family were not biased due to the fact that the paraphrases were generated using the same model, we additionally generated 500 paraphrases using Anthropic's Claude-3-5-Sonnet 3esude-----sonne--222
Note that Claude refused to create paraphrases for state-
ments 3, 26, 31, 36, 48, and 60.
Response model
Paraphrases
Validity (t)
Range (L)
(L)
$5 (4)
# (L)
A25 (L)
GPT-335
GPT-3355
(=00
0.93 (10:2))
0.26
0.69
0.6
0.32
GPT--3.5
Claude-3-5
1.00
0.94 (10.18)
0.24 (=0.13)
0.65
0.48
0.27
Falcon-7B-I
GPT-3.5
0.70 (+00
[+0.01
0.09 (-0002)
0.81
0.73
0.40
Falcon-7B-I
Claude-3-5
0.71 (+0)
0.50 (++0:12)
0.09 (=0.03)
0.81
0.66
0.37
RoBERTa-Large
GPT-3.5
1.00 (+:0
0.44 (4-0.09)
0.07 (=0.01)
0.74
0.58
0.29
RoBERTa-Large
Claude-3-5
1.00
0.47 (1:0.09)
0.08 (=0.02)
0.77
0.68
0.37
Table Response validity and stability under paraphrasing per model for generative language models with paraphrases generated using Claude-3-5-Sonnee Response validity describes the probability mass of valid responses. Response stability quantified in terms of range (difference between max-min), standard deviation (means + standard error), and flip proportions (proportion of statements where more than 25, 50 or 125 paraphrases (5, 10, 25%) deviated from the majority vote (#5, F10, F25).. Models that agreed with each statement (i.e,, E
0.5) are marked with + and with they disagreed with each statement.
16

---

Published as a conference paper at COLM 2024
0.75
0.50
0.00
Farcon-10
0.50
0.00
RARA
1.00
0.50
0.75
2.00
0.50
025
0.75
0.25
0.00
0.75
0.25
0.00
0.75
0.50
0.50
Figure 4: Distribution of positive responses R for each statement (GLMs). The width of the box represents the 25% and 75% quantiles, the mid line the median, and the whiskers values outside the 25- and 75% quartile range, but inside the 1.5 inter-quartile range. Dots represent responses that lie outside the 1.5 inter-quartile range.
We then re-ran the experiments to compute response validity as well as stability for models: GPT-3.5, Falcon-Instruct and RoBERTa-Large. The comparison presented in Table 7 shows that the choice paraphrasing model does not influence validity nor stability, with estimates being within each others' standard errors.
H Full results of response stability analysis
The distributions per-statement R for models that were not shown in Figure 2 can be found in Figures 4 and 5.
17

---

Published as a conference paper at COLM 2024
0.75
0.50
Del FATa-0a0e
0.5
10
0.75
ROBERTO
0.0
0.25
5.
Statmment
Figure 5: Distribution of positive responses R for each statement (MLMs). The width of the box represents the 25% and 75% quantiles, the mid line the median, and the whiskers values outside the 25- and 75% quartile range, but inside the 1.5 inter-quartile range. Dots represent responses that lie outside the 1.5 inter-quartile range.
18

---

Published as a conference paper at COLM 2024
Sentence length
Right dependencies
Left dependencies
Dependency distance
Positive sentiment
Negative sentiment
Word length
Lexical frequency
-0.75
0.36
0.22
-0.1
0.2
0.1
-0.08
0.8
Word length
-0.32
-0.08
-0.02
-0.22
-0.07
0.07
Sentence length
0.02
0.57
0.07
-0.05
Right dependencies
-0.96
0.02
0.08
-0.09
-02
Left dependencies
0.14
-0.06
0.09
Dependency distance
Positive sentiment
-0.61
Figure 6: Results of the correlation analysis between the predictors used in $4.3, All correlations are significant at an n-level 0.05.
I Correlation analysis between word- and sentence level features.
Figure 6 shows the correlation between the word- and sentence-level features extracted for all paraphrases. A description of the features can be found in Section 4.
19


Reviews:
[{"review_id": "qv49ZgiLeV", "reviewer": "Reviewer_bsVC", "summary": "The paper presents a large-scale empirical study on stability (sensibility and consistency) around paraphrasing abilities in LLMs in relation to political leanings. Unlike previous studies of similar inquiry, this work conducted deeper statistical analysis of data, thanks to the larger sample size and scope. The authors conclude that tests designed for bias measurement among humans (e.g., Political Compass Test) are likely ineffective in measuring bias in LLM due to the instabilities in their responses. \n\nPolitical and social bias in generative capabilities in LLM is an important and timely issue, both as a practical concern as well as a theoretical exercise. Overall the paper is written well and the points come across very easily. The proposed methodology is sensible and executed well. I however wonder how novel the techniques and applications are at this time, and how they are generally usable/useful in other similar inquiries. Even before Feng et al's seminal 2023 papers, political bias in LLM has been a concern among the practitioners. This paper contributes to the body or research, though I wonder whether there are enough generalizable technical innoventions and/or interesting or surprising findings. There are several promising analyses (e.g., stability in relation to text features) but I think they need to be developed more. Another point is a practical application of the work; I think the paper would be much stronger if the authors elaborated on how the presented methods (or the findings) are useful in real world tasks.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer bsVC for their critical assessment. We understand their concerns and would like to address them in the following.\n\n**Novelty and usefulness**: Although we acknowledge that our technique is not novel per se, we’d like to emphasize that unlike other approaches, our method of eliciting a model response does not rely on directly sampling model responses, as we directly access the distribution conditioned on the context (i.e., the statement). While the paraphrasing of the statements might not be a technical innovation, to the best of our knowledge, we are the first to assess its utility at large-scale, and we believe it is a simple but effective, computational less costly, and powerful method to obtain statistically more powerful results in the context of model probing that is applicable in similar cases as well.\n\n**Utility in real-world tasks**: We acknowledge that we haven’t elaborated about the utility in real-world tasks. We believe that the usefulness of our work might not lie in a direct application, but rather illustrates the difficulty of assessing higher-level / abstract concepts such as political biases in LMs. Although currently, only a few models are extensively used by a wide public, in the future, many more might contend for popularity. We believe that it is important to have tools and techniques that reliably elicit properties and characteristics of LMs that public or private companies would like to be informed about. As it is for now, our work shows that the assessment methods developed for human subjects are not necessarily applicable to LMs, i.e., and secondly, that it is important to consider stability on the item-level in addition to overall model-stability. Researchers will continue using tests developed for human participants, and we think that it is important to present evidence that such studies only make sense if the results are evaluated by taking into account model uncertainty or using new techniques altogether. We will discuss this point and add this as a concrete recommendation for future work in our revised version.\n\n**Promising analyses but need to be developed more**: As elaborated in the response to reviewer Xcm7, we aim to additionally include two additional predictors to explore the impact of syntactic features on the response distribution: number of left dependents (i.e., how many dependents does a syntactic head control) and dependency length (i.e., distance of a dependent to its syntactic head).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* clear writing\n* some interesting finding\n* solid analysis, some interesting follow ups.", "reasons_to_reject": "* lack of novelty in idea\n* lack of technical invention\n* lack of practical application or next research direction\n* not enough new discovery", "questions_to_authors": "NA", "extra_scores": {}}, {"review_id": "O6pDoRxuIb", "reviewer": "Reviewer_WiWK", "summary": "This paper takes on one small piece of the now familiar setup, in which psychometric instruments or political ideology instruments are given to a LLM to measure something about them (though specifically what we are supposed to infer from such measurements is often left rather vague, as is the case here).\n\nAlthough there are many such variations, this paper uses a fairly familiar one, which is to set up a prompt with one space for a statement, and another for the model response (e.g,. \"Do you agree or disagree with the following statement: <statement>. I <response> with this statement.\" for a MLM). They also use a common measurement strategy, which is to sum the probabilities associated with, roughly, affirmative or negative responses.\n\n Although there are many things which might be varied in such a setup, here the authors focus on the phrasing of the statement itself (such as \"There are those who are destined to experience chronic misfortune\"). While others have tried various systematic modifications of statements, here the authors simply collect a large number of paraphrases of each statement, and check to see how much probabilities vary across paraphrases. \n\nIgnoring some of the nuance, overall the authors find that models are relatively stable, but with some showing a large range between the min and max probabilities produced by different paraphrases of the same statement. Thus, as with other related papers, the authors provide additional evidence that this overall approach is not necessarily a valid way to infer something like ideology or personality for LLMs.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer WiWK for their careful review and appreciate their positive feedback. \n\n**Generalization to other experimental settings**: The advantage of our proposed method where we directly access the model’s probability distribution instead of generating the responses is that there is no need to fix generation parameters such as decoding strategy and its hyper-parameters. While it is correct that we do not know whether the analyzed linguistic features will show the same effects when varying the prompt, we contend that in particular, the results for sentiment were consistent across all tested models (MLM & GLM), even though they may all require specific prompt templates.\nWe agree that many more examples of prompt variations would be an interesting additional dimension to consider. For a potential revision, we started setting up an experiment where we will verify the stability of the results for 9 prompt variations (3 formats: changing new lines/spaces, etc. x 3 content-level variations: different questions).\n\n**Minor issues**: Regarding the figures, we will increase the axis labels of the plots. We were unsure about the copy-right of the political compass statements, which is why we hesitated to include them. We are in the process of clarifying the copyright situation and, if possible, add a table of all original statements in the Appendix and will also release all paraphrases in the final repository.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper is clear and well written. Although there are strong prior reasons why most people would naturally be skeptical of the validity of trying to assess LLM \"personality\" using psychometric instruments, it is a popular enough idea, that papers such as this are important for pushing back against it.\n\nAlthough this paper provides less insight into the types of modifications that can lead to instability, its strength lies in capturing some sense of the variation we might expect across reasonable paraphrases of the statements (conditional on one particular prompt and decoding setup, each of which is also likely to contribute to additional variation, possibly in a way that interacts with what is measured here).\n\nThe authors test a large number of models, and include a few examples, although many more would be welcome.\n\nOverall this is not the most exciting paper on its own, but it adds to the weight of evidence that can be dervied from the many similar papers.", "reasons_to_reject": "For experimental setups, one necessarily has to fix something (e.g, prompt, decoding strategy, etc.) to have any level of tractability. And yet the fact that these things are fixed here means that we don't really know how well these results will generalize to other choices that could have been made. Compared to something more like a physical system, it seems less clear that we can count on interactions between such pieces of the experimental setup to not matter. \n\nThe authors do attempt to provide some sort of analysis of their results, using linear models with factors such as sentiment and paraphrase length, to see if these are predictive of the results, but these are less compelling than the rest of the paper, in part because we would expect such things to be highly contextual. (They find, for example, that, in their setup, models are more likely to agree with statements that have positive sentiment).\n\nOther issues are mostly minor. In particular, many more examples of prompts and variation (ideally a large random selection) would be extremely welcome. Some of the figures (e.g., Figure 2), involve a lot of text that is far too small to be legible and/or a lack of context (e.g., numbered items without a legend or key), undermining their utility.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bDkf67NTeq", "reviewer": "Reviewer_Xq2x", "summary": "This paper analyzes shortcomings of using LMs for the assessment of political leaning. Specifically, the paper investigates model response stability using the Political Compass Test (62 statements with 4 options each). The authors focus on response variances on a per-item level under paraphrasing as well as which samples are particularly brittle across models and whether any linguistic and structural factors (e.g., word length, lexical frequencies) impact such findings. Using 500 paraphrases per model and 20 models in total (both MLMs and GLMs), the authors overall find that several models exhibit large ranges w.r.t. their response stability, and MLMs overall achieve a higher response validity as compared to GLMs. Moreover, the authors report that the sentiment of a statement consistently correlates with whether a model agrees to the statement, and both greater word and sentence counts are also correlated with agreement to the statement.", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": true, "final_justification": "", "rebuttal": "We thank reviewer Xq2X for their critical assessment and the constructive feedback. We will address their concerns in the following.\n\n**Paraphrasing & Evaluation**: We agree that the choice of paraphrasing model will likely impact the results, due to differences in the quality or diversity of the paraphrases. Although the likelihoods of the statements per se might be higher for GPT-3.5, it is not evident why the next-token probabilities would be systematically biased towards the tokens mapped to “agreement” and “disagreement”. In the course of this project, we ran an analysis where we investigated whether models tend to agree with statements for which they assign higher likelihoods, and did not find evidence that this is the case. Nevertheless, we understand that this might also be a concern for readers, which is why we generated 500 additional paraphrases using Claude-3-haiku. The results show slightly higher instability *across all models*. We can discuss in the discussion period where to include these results.\n\n**Parameters**: We reported temperatures and sampling parameters for generating the paraphrases in Appendix D. For all other parameters, we used the default settings. We used this setting because it reliably provided diverse paraphrases that kept the semantic proposition. \nOther than that, we did not have to set any specific configurations since we were not *generating* text to elicit responses. We directly access the pre-trained models’ probability distribution conditioned on the statement and extract the probability for the tokens reflecting agreement or disagreement. \n\n**Limitations**: We are going to extend our discussion, including (but not limited to) the following points:\n\n* We worked with a fixed prompt template. Previous work has found that models indeed show instability in this respect as well.\n* We collapsed the response distribution from 4 responses to 2 (collapsing “strongly agree” and “agree”, for instance, which would now allow us to assess actual test results of the compass test. However, this was not the scope of this work, but we will include a discussion on how we concretely recommend to account for instability when obtaining actual test results (i.e., the location on the political axes).\n* We did not manually check each paraphrase to assess whether it reflects 100% the same semantic content as the original statement (but see response to reviewer  Xcm7).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper sheds new light on the shortcomings related to the usage of LMs in the context of assessing political standpoints and personality traits. Since this topic has received increasing attention from the research community, work that critically assesses previously obtained findings is important and valuable.\n* I agree that a robustness analysis on a larger scale (i.e., with a large amount of paraphrases of questions) can provide additional insights into the consistency of using LMs as respondents to such questionnaires.", "reasons_to_reject": "* The paper generates paraphrases with an OpenAI model and evaluates OpenAI models on them, showing that they have the highest instruction validity rates. This experimental setup is inherently biased towards OpenAI models, and direct comparisons between OpenAI models and non-OpenAI models therefore become unfair. How would such results look like if a different model were used for paraphrasing?\n* The paper does not discuss model configuration and sampling parameters (e.g., temperatures) and how these might potentially affect the obtained results. The probability distribution computed over potential outputs is directly affected by the temperature value and the results reported in Figure 2 would therefore directly shift as a result of changing that parameter. Additional work is needed to discuss and investigate the impact of such configuration changes on the obtained results.\n* The paper does not discuss limitations of the proposed approach and obtained findings. Given that this work critically assesses the usage of LMs in the context of behavioral questionnaires, it would be important that the authors clearly address the scope and relevance of their work.\n* [Style-related] Figure 2 is too small and difficult to read.", "questions_to_authors": "Can you provide some details on specific model configurations and parameters that you used as part of the sampling? How would the results change when those configurations are changed?", "extra_scores": {}}, {"review_id": "hcdwGl3M0u", "reviewer": "Reviewer_Xcm7", "summary": "This paper provides a large-scale analysis of LLMs consistency in answering paraphrased questions about their political leanings.\nWhile the question of how stable and consistent LLMs’ responses to value questionnaires are has been (quite extensively) studied, this paper does it on a much larger scale, by covering 500 different paraphrases of each statement. Additionally, they also study linguistic factors which could bias models to respond in a certain way.\nOverall this is a well-written paper, with extensive experiments and useful insights into LLM consistency.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer Xcm7 for their review and appreciate the positive feedback. \n\n**Paraphrases**: It is correct that we did not check all 500x61 paraphrases manually.  However, we did randomly sample 10 paraphrases of each statement to verify if their propositions were consistent. For a (potential) revised version, we suggest increasing this sample size (e.g., 50) and reporting the number of “incorrect” paraphrases, if there are any.\n\nRegarding diversity, we somewhat take this into account by controlling for sentence length, average word length, and lexical frequency. To also control for syntactic diversity, we will include two additional predictors: the number of left dependents (i.e., how many dependents does a syntactic head control) and dependency length (i.e., distance of a dependent to its syntactic head) in the last experiment. Moreover, we will include descriptive statistics over these word-level features for the statements over paraphrases in the Appendix. That way, it will be possible to interpret the results in consideration of lexical and syntactic diversity.\n\n**Why the political compass test?**: Our approach to directly access the probability distribution over valid responses to a given statement required us to employ a test that uses forced choice, and the political compass was our test of choice as it makes our study comparable to previous work. We don’t disagree that forced choice is probably not a good way to probe abstract concepts such as political biases in LMs---this is one of the key take-aways of this work: the assessment methods developed for human subjects (often forced-choice) are not necessarily applicable to LMs. We will make this more clear in the (potential) revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The scale on which the experiments, specifically the paraphrases, were performed: previous works performed these experiments on a much smaller scale, so it is great to see results over 500 different paraphrases per prompt\n- The last experiment’s results, which was looking at possible features that could bias models, was interesting: The finding that models tend to agree with positive statements over negatively framed one.", "reasons_to_reject": "- Despite citing works that criticized the forced-choice approach of the political compass setup, this work only evaluates on the Political Compass Test.\n- As far as I could see there was no manual evaluation done on (a subset of) the 500 paraphrases, in order to check for quality, diversity and whether they still express the same semantic proposition.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper reports a large-scale study on stability in light of paraphrasing with respect to political leanings, concluding that due to instabilities tests that are used to measure bias in humans may not be effective for LLMs.

Reviewers appreciate that the paper addresses a worthwhile topic, provides some interesting insights, and performs these analyses at larger scale than prior work. 

However, reviewers also express some lukewarm feelings about the overall impact and excitement level for this paper -- the question asked here has been investigated often in prior work, and the main novelty here seems to be the larger scale at which the investigation is conducted. There are also some questions about to what extent the findings generalize beyond the specific setup (e.g., prompt) used here, as well as lack of clarity about practical applications. Finally, a concern is also raised about overlap between OpenAI models used for generating paraphrases and the models being evaluated in the analysis -- and though the authors report a follow-up analysis in response to this concern, I agree that it would be nice for this to be integrated more centrally in the paper.

All in all, this seems like a solid and safely publishable work, but it could probably be strengthened in another round of revisions, integrating the analysis without the paraphrase model among the evaluated models, making stronger arguments for practical utility, and ideally providing further evidence for generalizability and novel impact.
Decision: Accept