Yes, no, maybe? Revisiting language models' response stability under paraphrasing for the assessment of political leaning

Authors: patrick  haller
jannis  vamvas
lena a jäger
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-22 18:39:49

Abstract: An increasing number of studies are aimed at uncovering characteristics such as personality traits or political leanings of language models (LMs), using questionnaires developed for human respondents. From this previous body of work, it is evident that models are highly sensitive to prompt design, including the phrasing of questions and statements, as well as the format of the expected response (e.g., forced choice, vs open-ended). These sensitivities then often lead to inconsistent responses. However, most studies assess response stability on a small scale with low statistical power e.g., using less than ten paraphrases of the same question.

In this work, we investigate the stability of responses to binary forced-choice questions using a large number of paraphrases. Specifically, we probe both masked language models (MLMs) and left-to-right generative language models (GLMs) on the political compass test, assessing response validity (i.e., the proportion of valid responses to a prompt) and response stability (i.e., the variability under paraphrasing) across 500 paraphrases of each statement. This large-scale assessment allows us to approximate the underlying distribution of model responses more precisely, both in terms of the overall stability of a model under paraphrasing as well as the stability of specific items (i.e., the intended meaning of a question). In addition, to investigate whether there are structural biases that drive model responses into a certain direction, we test the association between different word- and sentence-level features, and the models' responses.

We find that while all MLMs exhibit a high degree of response validity, GLMs do not consistently produce valid responses when assessed via forced choice. In terms of response stability, we show that even models that exhibit high overall stability scores flip their responses given certain paraphrases. Crucially, even within-model, response stability can vary considerably between items. We also find that models tend to agree more with statements that show high positive sentiment scores.

Based on our results, we argue that human-centered questionnaires might not be appropriate in the context of probing LMs as both their response validity and stability differ considerably between items. Moreover, although stability metrics represent useful descriptions of model properties, it should be emphasized that even for models exhibiting fairly high stability, specific paraphrases can lead to substantially different model responses.

Main paper: 
 introduction: 
 Recently, there has been a surge of work aimed at revealing properties of (large) language models (LMs) such as political leanings (Feng et al., 2023) , personality traits, or other human characteristics (Jin et al., 2023; Scherrer et al., 2023) , utilizing psychological instruments such as psychometric tests and questionnaires, developed to be conducted with human respondents. Feng et al. (2023) have shown that LMs trained on politically biased corpora can propagate social biases into downstream tasks, such as hate speech and misinformation detection tasks, and may significantly affect predictions.A common method to apply psychometric instruments such as questionnaires to LMs is to prompt them with the items (i.e., questions, tasks, etc.) these instruments provide. In most cases, the LM is prompted to generate a response that represents the selected option. It has not gone unnoticed that the question-response format as well as the phrasing of questions and statements themselves can heavily affect model responses, and several studies have assessed response stability2 under different categories of prompt perturbations, for instance, when varying the statement to respond to, or varying the instruction surrounding the statement (R öttger et al., 2024; Ceron et al., 2024; Shu et al., 2024; Feng et al., 2023, cf. §2.3 ). However, these assessments have been conducted on relatively small scales, quantifying response stability based on few conditions (e.g., less than 10 paraphrase of the same statement), and only on the group-, rather than the item-level. Moreover, their stability is analyzed after the mapping from continuous probabilities to discrete responses ("agree", "disagree", etc.), using measures such as Fleiss' κ (Fleiss, 1971) . Lastly, despite the knowledge of low response stability of some LMs, measures of uncertainty are often provided as informative sanity checks, but do not get actually factored in for the assessment of political leanings, for instance (Feng et al., 2023) .In this present work, we prompt LMs with binary forced-choice questions and investigate response validity and response stability under paraphrasing on a large scale, using items from the Political Compass Test3 . Specifically, we ask the following research questions (RQ): RQ 1: How large is the overall model and per-item response validity (i.e., whether the model returned a valid response) under forced choice? RQ 2: How large is the response stability under paraphrasing of a given model (a) on average; i.e., how much do model responses vary across all paraphrases of all items, and (b) on the item-level; i.e., how much do the model responses vary for different paraphrases of a given item? RQ 3: Are there specific items that show consistently low stability under paraphrasing across all models? RQ 4: Are structural word-and sentence-level features such as sentence length, word length, lexical frequency, syntactic dependencies and sentiment associated with certain model responses?To answer these questions, we automatically create 500 paraphrases of each questionnaire item4 and extract model responses from both masked language models (MLMs) and leftto-right generative language models (GLMs). We find that although many models show relatively high response stability in terms of standard deviations, many items show low response stability. Moreover, we find a strong association between sentiment and model responses, suggesting that there are structural biases that guide the model towards specific responses.In summary, we make the following contributions:• We conduct the first large-scale assessment of response stability under paraphrasing (500 paraphrases), allowing us to quantify stability with high statistical power. • An analysis of 18 LMs shows that most models exhibit rather poor stability under paraphrasing, both in terms of overall per-model stability across items as well as in terms of within-model per-item stability.• We investigate whether different text features such as sentiment and sentence length impact the response of a LM to a given statement, finding that models tend to agree more with longer statements and statements that exhibit positive sentiment. Apart from observing a shift in viewpoints after fine-tuning a model, some studies have shown that priming a model with certain personality traits can be an effective method to bias a model's responses in a given direction. For instance, Dong et al. (2024) attempted to locate models on an individual/collectivism scale. In one condition, each prompt (question) was prefixed by a description of an "in-group persona", such as "you are a person attributing extremely more importance to Individualism [...]". The results showed that priming will result in the model giving answers that are more aligned with the instilled persona.No one chooses their country of birth, so it's foolish to be proud of it. 1.00 3Our race has many superior qualities, compared with other races. 1.00 4The enemy of my enemy is my friend. 0.53 5 Military action that defies international law is sometimes justified. 1.00 6There is now a worrying fusion of information and entertainment. 1.00 7People are ultimately divided more by class than by nationality.Controlling inflation is more important than controlling unemployment. 1.00 9Because corporations cannot be trusted to voluntarily protect the environment, they require regulation.1.00 fundamentally good idea.1.0011 The freer the market, the freer the people. 1.00 12 It's a sad reflection on our society that something as basic as drinking water is now a bottled, branded consumer product.1.0013 Land shouldn't be a commodity to be bought and sold. 1.00 14 It is regrettable that many personal fortunes are made by people who simply manipulate money and contribute nothing to their society.1.0015 Protectionism is sometimes necessary in trade. 1.00 16 The only social responsibility of a company should be to deliver a profit to its shareholders.1.0017 The rich are too highly taxed. 1.00 18 Those with the ability to pay should have access to higher standards of medical care.1.0019 Governments should penalise businesses that mislead the public. 1.00 20 A genuine free market requires restrictions on the ability of predator multinationals to create monopolies.1.00 21 Abortion, when the woman's life is not threatened, should always be illegal. 1.00 22 All authority should be questioned. 0.93 23 An eye for an eye and a tooth for a tooth.1.00 24 Taxpayers should not be expected to prop up any theatres or museums that cannot survive on a commercial basis.1.00 25 Schools should not make classroom attendance compulsory. 0.8 26 All people have their rights, but it is better for all of us that different sorts of people should keep to their own kind.1.0027 Good parents sometimes have to spank their children. 1.00 28 It's natural for children to keep some secrets from their parents.1.00 29 Possessing marijuana for personal use should not be a criminal offence.1.00 30 The prime function of schooling should be to equip the future generation to find jobs.1.00 31 People with serious inheritable disabilities should not be allowed to reproduce.1.00 32 The most important thing for children to learn is to accept discipline.  
eliciting responses from lms under forced choice: 
 Most commonly, decoder-based LMs are assessed in forced-choice settings (e.g., "only answer with yes or no") when using instruments such as psychometric tests and questionnaires.When generating a response from a probabilistic model to a given statement, various decisions are involved in the mapping of a test statement to a response in the valid response format, as illustrated in Figure 1 .First, the original statement is embedded in a prompt that is aimed at directly generating an answer option string ("yes"), or an option label assigned to an option such as "A,B,C,. . . " (Tsvilodub et al., 2024) . Next, the model responses can either be obtained via prompted text generation where repeated samples are collected for a given statements, or, alternatively, one can directly examine the next-token probabilities by reading out the probability of the model to generate a specific response option. Recent work by Hu & Levy (2023) has shown that quantities derived from representations ("meta-linguistic judgements") are more consistent than those obtained via prompted text-generation. As an optional step, the probabilities can be re-normalized (Shu et al., 2024) , and finally, a decision function is applied that maps the array of probabilities to the categorical label (e.g., "yes" or "no") which are required to evaluate the model on a specific test.Previous work has shown that minor changes to prompts can heavily affect an LM's output (Li et al., 2021; Wang et al., 2021) . In a context where the final output (i.e., the response to a particular statement) depends on various decisions (see above), it is crucial to assess a model's response stability in the light of changes to the input prompt. This key insight has been taken into account by many studies assessing political leaning. Feng et al. (2023) , for instance, investigated response stability using six paraphrases of political propositions, and found that overall, the responses of pre-trained LMs were "moderately stable". Moreover, changes in the input prompt can also be independent of the statement or question, butGovernments should penalise businesses that mislead the public. rather depend on the format, for instance: "[...] your response:" vs "[...] your response: \n#", or the order of the response options (Pezeshkpour & Hruschka, 2024; Zhao et al., 2021; Li et al., 2024) . Shu et al. (2024) proposed to distinguish between "sensitivity"-the degree to which a model's response is affected by spurious changes to the question format as illustrated by the previous example-and "consistency", the degree to which a model's response is affected by content-level changes to the question. Recently, Ceron et al. (2024) have proposed a series of tests to assess model stability, that include sampling robustness, robustness to paraphrasing, negations as well as semantic inversions, and robustness to label-order inversion in the prompt template (e.g., "answer only with yes or no", vs "answer only with no or yes". Their analyses suggested that larger models show higher stability. 
methods: 
 First, let p θ be a probabilistic language model, where θ denotes a set of trainable parameters. We additionally define a vocabulary Σ containing a set of sub-word tokens, and the special EOS (end-of-sentence) token. Let us further consider a political statement such as People are ultimately divided more by class than by nationality., formalized as w = ⟨u 1 , . . . , u T ⟩, consisting of T sub-tokens u.Masked language models. Given a statement w that includes a MASK-token at index t, the model estimates the probability p θ (r|u <t , u >t ) for any r ∈ Σ. Following Feng et al. (2023) , to generate a response from an MLM, we present it with the following prompt: "Do you agree or disagree with the following statement: 〈statement〉. I MASK with this statement.". We quantify the tendency of the model to agree with the given statement by summing the probabilities of tokens in a set of valid positive responsesR + ⊂ Σ, that is p(R + ) = ∑ r∈R + p(r|u <t , u >t ).Accordingly, to obtain the tendency of the model to disagree with the given statement, we sum over all possible tokens in a set of valid negative responses, R -. The elements of both sets R + and R -are enumerated in Appendix D.Generative language models. Given a sequence w = ⟨u 1 , . . . , u T ⟩, consisting of T tokens u, the model estimates the probability p θ (r|u <t ) for any possible next token r ∈ Σ.Similar to the approach described above, we present left-to-right generative models with the following prompt to elicit a response: "Please respond to the following statement with yes if you agree or no if you disagree: 〈statement〉\n Only answer with yes or no. Your response:".For simplicity, we only allow "yes" and "no" and corresponding cased alternatives as valid answers in R ± ⊂ Σ, i.e., p(R + ) = ∑ r∈R + p(r|u <t ). 5 The instruction-tuned models included an additional newline character at the end of the prompt template. 
experiments: 
 Data. We use the statements from the Political Compass Test for our analyses. This test contains 62 statements categorized into different topics, listed in Table 4 in the Appendix.For each each statement, e.g., People are ultimately divided more by class than by nationality., respondents have to select one of four options: "strongly disagree," "disagree," "agree," or "strongly agree," with no neutral choice available. Each response is associated with positions on two axes: economic (left, right) and social (libertarian, authoritarian)6 .Models. We deploy a total of 18 LMs: 11 generative left-to-right language models: Falcon (± Instruct) (Almazrouei et al., 2023) , GPT-3.5 (OpenAI, 2023b) , GPT-4 (OpenAI, 2023a) , and GPT-4o (OpenAI, 2024) . 4 flavours of Llama-2 (Touvron et al., 2023 ) (7B, and 13B, ±Chat), Tinyllama (Zhang et al., 2024) , and Phi-2 (Javaheripi & Bubeck, 2023) , and 7 masked language models: BERT (Devlin et al., 2019) (base and large), RoBERTa (Liu et al., 2019) (base and large), distilBERT and distilRoBERTa (Sanh et al., 2019) , and Electra (Clark et al., 2020) . Details about these models are reported in Appendix E.In previous work, the stability of LMs' responses to various questionnaires has been assessed using a small number of paraphrases.To test the scalability of these results, we create 500 paraphrases of each original statement using GPT-3.5 via the openAI API7 to (cf. Appendix F for more details). We manually verify the quality of the paraphrases by randomly sampling 15 paraphrases (15x62 paraphrases in total) for each statement and checking whether the paraphrased statement is semantically equivalent to the original statement. We report the estimated proportion of correct paraphrases in Table 4 . An additional set of paraphrases were generated using Claude-3-5-Sonnet in order to rule out that the stability results obtained for GPT-3.5 were biased due to the fact that the same model was used for paraphrase generation (see Appendix G).For each LM, we then extract the probabilities of agreeing and disagreeing with a given statement, p(R + ), and p(R -), respectively.Annotation. We extracted the following text features from the annotated paraphrases. Note that we use the term word to refer to the white-space-separated tokens.• Average word length: Average number of characters over all words in a statement.• Statement length: Number of words in a statement.• Average lexical frequency: Average unigram frequencies over all words in a statement, extracted from wordfreq (Speer, 2022) .8 • Average number of left dependencies (number of dependents preceding a syntactic head), average number of right dependencies (number of dependents following a syntactic head), and average absolute dependency length (distance of a dependent to its syntactic head), extracted using spaCy (Honnibal & Montani, 2017 ). • Sentiment (positive; negative) extracted from pysentimiento (Pérez et al., 2023).First, in order to assess the degree to which an LM generates responses that are in line with the instruction, we compute validity scores for each paraphrase j of a statement i,v ij = p(R + ) ij + p(R -) ij .The average response validity of a model is its average over all statements and paraphrases: 1I J ∑ i ∑ j p(R + ) ij + p(R -) ij, where I denotes the number of statements and J the number of paraphrases of each statement.Validity (↑) Range (↓) SD (↓) ̸ = 5 (↓) ̸ = 10 (↓) ̸ = 25 (↓) GPT-3.5 1.00 Results. Table 1 shows the average response validity over all statements for each model. First, we note that a range of models (Falcon, several flavors of Llama-2, as well as BERT and Electra) agree with each statement, and distilRoBERTa disagrees with each statement. We further see that overall, MLMs achieve higher scores than GLMs. Moreover, GLMs prompted via the OpenAI API show much higher response validity compared to the opensource models, although for the former, only the 20 top log-probabilities can be extracted.In addition to the aggregated results over all statements, we present the per-statement response validity in Appendix H. The results suggest that even within one and the same model, response validity can be highly variable depending on the statement, as observed for GPT-3.5 or Phi-2.To assess response stability on different levels, we first re-normalize the responses, marginalizing out all invalid responses, with R + = p(R + ) /p(R + ) + p(R -) denoting the tendency of the model to agree, and we additionally compute a binarized response R + b , ("agree" if R + ≥ 0.5). Next, we compute three statistics that quantify response stability.First, we take a look at LMs' average response stability (i.e., what is a given model's response distribution across items). We quantify model stability with the following measures:(1) Average min-max range: The average distance between the highest and the lowest scoring paraphrase of a given statement. This measure gives an overall impression of the average distance between the most extreme responses of a model. (2) Standard deviation (SD): The average standard deviation provides an overview of the spread of R + . (3) Flip proportion: The proportion of items (n out of 62) where more than than 5, 10, or 25% of the paraphrases were inconsistent (disagreed with the majority vote); ̸ = 5 : more than 25 inconsistent paraphrases); ̸ = 10 : more than 50 inconsistent paraphrases); ̸ = 25 : more than 125Model Statement R + Statement R + GPT-3.5Given that economic globalization is certain, it should prioritize benefiting humanity over the interests of trans-national corporations. 0.25 As economic globalization becomes inevitable, its primary goal should be to benefit humanity rather than focus solely on trans-national corporations.Llama-2-7B-Chat I value the school's role in instilling religious values in my child.0.34 The teaching of religious principles in my child's school is important to me.0.88What proves advantageous for the most thriving businesses benefits everyone in the long run.0.43 Anything that supports the success of leading companies will also support the well-being of society as a whole.RoBERTa-Large There is no responsibility for taxpayers to financially support struggling theaters or museums.0.10 Taxpayers should not be required to sustain any theatres or museums that are not financially viable.Table 2 : Comparison between paraphrases with the highest and lowest positive response probabilities R + of a given statement. inconsistent paraphrases. For example, ̸ = 25 = 1 means that in all statements, there were more than 125 paraphrases that did not agree with the majority.Next, we assess per-item response stability within a model (i.e., what is a given model's response distribution across paraphrases for a given item). Due to space limitation, we do not compute the above-mentioned measures for each item separately, but show the distributions of R + in Figure 2 .Last, we assess per-item response stability across models. To do so, we calculate the standard deviations of R + for a given model and item, and re-normalize them to the range [0, 1] for each model. Averaged over all models, this measure represents the overall degree to which items produce consistently unstable responses under paraphrasing.Results. We present the measures of overall model stability in Table 1 . First, we note that the models that consistently agree or disagree with a statement exhibit very high stability in terms of all measures. GPT-3.5, GPT-4 and GPT-4o show rather low stability under paraphrasing. For GPT-3.5, for instance, in 32% of the statements, more than 125 paraphrases show inconsistent responses (i.e., do not follow the majority vote). This proportion is even higher for Falcon-7B-Instruct. In general, although many models are relatively stable under paraphrasing in terms of SD, they still exhibit large ranges.Figure 2 illustrates the within-model per-item stability. 9 We observe that GPT-3.5 and GPT-4o-the models with the highest response validity scores-show a wide spread of answers for the different paraphrases for the same statement. To give some concrete examples of the spread of the distribution for a given paraphrase, we list some examples of paraphrases with the highest and the lowest R + in Table 2 .We present the normalized standard deviations per statement averaged over all models in Figure 3 . The higher the normalized standard deviation of a given statement, the lower the stability across models. We see large differences in stability between the items. The two items with the largest uncertainty are two quotes that are difficult to paraphrase: "The enemy of my enemy is my friend." (Item 4); "An eye for an eye and a tooth for a tooth." (Item 23). Although in many cases, GLMs and MLMs show similar degrees of stability for a given item, this is not true for all statements, cf. Items 19 and 61, for instance. Since the previous section revealed that even semantically close paraphrases can result in very different responses from one and the same LM, we next investigate whether the tendencies are driven by structural features such as average lexical frequency, sentence length, average number of left and right dependencies, average dependency length, and sentiment. To rule out spurious effects, we first test the correlations between all predictors. Next, we employ linear-mixed models with the probability of a positive response to paraphrase j of statement i, p(r + ij ), as response variable and the annotated features from §4 as predictors, including per-statement random intercepts β 0i :R + ij ∼ β 0 + β 0i + β 1 lex freq j + β 2 w len j + β 3 s len j + β 4 n rights+ β 5 n lefts + β 6 dep dist + β 7 sentiment + j + β 8 sentiment - jWe fit separate linear-mixed models for each LM using the R package jglmm for interfacing with Julia's MixedModels library (Bezanson et al., 2017) . Table 3 : Effect sizes (mean ± sd) of average lexical frequency, average word length, sentence length, average number of right dependents, average number of left dependents, average dependency distance, sentiment + and sentiment -as predictor variables of R + . The dagger ( †) indicates that the coefficient was not significantly different from 0.Results. We present the coefficient estimates (mean±sd) of each predictor10 across models in Table 3 . The dagger ( †) indicates Bonferroni-corrected p-values > .05. We see the most consistent results across models for sentiment. For all models except Phi-2, the more positive the sentiment score of a given statement, the more likely the LM is to agree with the statement; conversely, the higher the negative sentiment score, the less likely the model agrees with a given statement. Furthermore, for all models except Falcon-7B, longer sentences are associated with higher normalized response probabilities R + . Regarding average lexical frequency and average word length, we see a mixed picture.While statements with a high proportion of frequent words increase the likelihood that Llama-2-7B-Chat or BERT-Large agrees with a statement, the effect is reversed for GPT-3.5 and Falcon-7B-Instruct. Average word length as a predictor is slightly more consistent across models. Except for Llama-2-7B, DistilRoBERTa, and Electra, all models exhibit higher R + in sentences with longer words on average. Regarding the syntactic predictors, we see a trend that the openAI models (except for GPT-4) and all masked LMs tend to agree more with statements in which syntactic heads govern more dependents. This pattern is reversed for GLMs who agree less with statements whose syntactic heads govern many dependents. In terms of average dependency length, we see that Falcon-7B and Electra show lower R + for statements including longer dependencies. All other models show on average higher R + when generating responses for statements that include longer dependencies. 
discussion: 
 Masked language models and GPT-3.5/4/4o show high response validity. We assessed overall response validity with a large number of paraphrased prompts (RQ 1). While all MLMs as well as GPT-3.5 and GPT-4 exhibited almost perfect validity, the remaining GLMs' response validities were low. Interestingly, for some models, response validity varies between different items, suggesting that for some items, these models are more reluctant to respond with a binary answer in the requested format.Models do exhibit response stability under paraphrasing to some degree, but not for all items. Second, we aimed at assessing the stability under paraphrasing on two levels (RQ 2): (a) overall stability of a model, (b) per-item stability within model. Regarding overall stability, we found that although many models do exhibit medium to high response stability in terms of standard deviation and flip proportions, at the same time, they exhibit large minmax ranges. Zooming in on the item-level, we found that within a model, response stability varies considerably. Although for many statements, models exhibit high stability (e.g., GPT-3.5 in Figure 2 ), there exist a set of paraphrases that are able to flip model responses.We demonstrated in Table 2 that such responses result from valid paraphrases. Our per-item stability analysis across models (RQ 3) further showed that particular items are always unstable across all models, such as quotations. It is questionable whether such statements are suitable for model-based analyses, as it is difficult to even assess a response distribution due to the difficulty of proper paraphrasing.Models tend to agree with positively phrased statements. Last, we investigated what structural features are associated with certain model responses (RQ 4). We found that positive sentiment is associated with a higher probability to agree with a statement. This suggests that rephrasing statements in a more positive way can lead to higher agreement. A possible explanation for this pattern is that a positive prefix conditions the continuation to be more positive as well, making it more likely to produce a positively connoted response.To establish whether these structural biases have had impacts in previous assessments of political leanings, one would have to investigate whether such features are distributed unevenly among items that map to one side of a specific axis. Last, it will be important to compare the established effect sizes with human participants. While it is known that humans respondents have a tendency to select a positive response, known as the acquiescence bias (Krosnick, 1999; Baron-Epel et al., 2010) , there is only little evidence that humans are biased towards agreeing with positively phrased statements (Sauro & Lewis, 2011) . 
conclusion: 
 In this work, we quantified LM response stability under paraphrasing with a high number of paraphrases. In summary, our results show that although it might be useful to quantify stability in terms of overall model stability and per-item stability, these metrics divert from the fact that specific paraphrases can result in strong changes in model responses.When investigating any type of LM property utilizing psychometric instruments-be it political leanings, personality traits or other characteristics-, in the light of our results, we argue that these assessments should be conducted on the group level where model responses are elicited using large numbers of conditions for each item. Moreover, more detailed investigations into how structural features affect model responses are required.Given the finding that in addition to rather poor stability, numerous models also exhibited low response validity, we reckon that future work should explore how to stabilize model responses, for instance by providing more in-context examples.Lastly, we raise the question whether it is sensible to continue using instruments aimed to be conducted with human respondents in the first place. It appears that due to potential structural biases, and varying stability of different items, it might be sensible to design instruments where such factors are controlled for. 
 
 Appendix: Our experiments use the same prompt template for all models, except for the additional newline character for the instruction-tuned models. Given the fact that changes in prompt formatting can heavily impact the model responses, it is possible that different prompt templates also affect stability. Moreover, we collapsed the response distribution from 4 responses to 2 (merging "strongly agree" and "agree", for instance). Although this is not a problem per se, it would likely impact the assessment of the political position using the political compass test.Table 4 shows all original test questions from the political compass test.If economic globalisation is inevitable, it should primarily serve humanity rather than the interests of trans-national corporations.1.00 1 I'd always support my country, whether it was right or wrong.The wordfreq-library by Speer (2022) provides token frequencies estimated from the Exquisite Corpus11 , that compiles text data from 8 different domains, including news, subtitles, books, and social media. We use the zipf-frequencies which are the counts per billion words in log 10 scale.Table 5 contains all valid responses that were summed up to compute the probability of the model agreeing with a given statement, p(R + ) and the model disagreeing with a given statement, p(R -). Note that all case variants (e.g., 'YES', 'Yes', etc.) were considered valid responses.Tokens counting towards p(R -) GLM yes no MLM agree, agrees, agreeing, agreed, support, supports, supported, supporting, believe, believes, believed, believing, accept, accepts, accepted, accepting, approve, approves, approved, approving, endorse, endorses, endorsed, endorsing disagree, disagrees, disagreeing, disagreed, oppose, opposes, opposing, opposed, deny, denies, denying, denied, refuse, refuses, refusing, refused, reject, rejects, rejecting, rejected, disapprove, disapproves, disapproving, disapprovedTable 5 : List of tokens that count towards the positive probability of a response given a statement, p(R + ) and tokens that count towards the negative probability of a response given a statement, p(R -). Figure 6 shows the correlation between the word-and sentence-level features extracted for all paraphrases. A description of the features can be found in Section 4.Code and data are available at https://github.com/ZurichNLP/llm-response-stability.The terminology to refer to this phenomenon is very inconsistent; apart from "stability", which is the term we will use in the present work, it has been referred to as "robustness", "consistency", "reliability", and "sensitivity".https://www.politicalcompass.org/testNote that we define the meaning of the original statement as the item.The models prompted using the OpenAI API only allow to access the 20 tokens with the highest probability, and do not allow force-decoding responses consisting of more than one token.Note that there is no public documentation of how the political compass maps answers to political positions. The authors state that they "have a strict policy against releasing this information."https://platform.openai.com/docs/api-referenceWe provide more details about the frequency measure and the underlying corpus in Appendix C.Due to space limit, we selected two MLM models and four GLM models with high response validity scores.The degree to which the predictors correlate with one another are shown in Figure6of the Appendix.https://github.com/LuminosoInsight/exquisite-corpus 


Decision: This paper reports a large-scale study on stability in light of paraphrasing with respect to political leanings, concluding that due to instabilities tests that are used to measure bias in humans may not be effective for LLMs.

Reviewers appreciate that the paper addresses a worthwhile topic, provides some interesting insights, and performs these analyses at larger scale than prior work. 

However, reviewers also express some lukewarm feelings about the overall impact and excitement level for this paper -- the question asked here has been investigated often in prior work, and the main novelty here seems to be the larger scale at which the investigation is conducted. There are also some questions about to what extent the findings generalize beyond the specific setup (e.g., prompt) used here, as well as lack of clarity about practical applications. Finally, a concern is also raised about overlap between OpenAI models used for generating paraphrases and the models being evaluated in the analysis -- and though the authors report a follow-up analysis in response to this concern, I agree that it would be nice for this to be integrated more centrally in the paper.

All in all, this seems like a solid and safely publishable work, but it could probably be strengthened in another round of revisions, integrating the analysis without the paraphrase model among the evaluated models, making stronger arguments for practical utility, and ideally providing further evidence for generalizability and novel impact.

Reviews: review: paper_summary: paper_summary: The paper presents a large-scale empirical study on stability (sensibility and consistency) around paraphrasing abilities in LLMs in relation to political leanings. Unlike previous studies of similar inquiry, this work conducted deeper statistical analysis of data, thanks to the larger sample size and scope. The authors conclude that tests designed for bias measurement among humans (e.g., Political Compass Test) are likely ineffective in measuring bias in LLM due to the instabilities in their responses. 

Political and social bias in generative capabilities in LLM is an important and timely issue, both as a practical concern as well as a theoretical exercise. Overall the paper is written well and the points come across very easily. The proposed methodology is sensible and executed well. I however wonder how novel the techniques and applications are at this time, and how they are generally usable/useful in other similar inquiries. Even before Feng et al's seminal 2023 papers, political bias in LLM has been a concern among the practitioners. This paper contributes to the body or research, though I wonder whether there are enough generalizable technical innoventions and/or interesting or surprising findings. There are several promising analyses (e.g., stability in relation to text features) but I think they need to be developed more. Another point is a practical application of the work; I think the paper would be much stronger if the authors elaborated on how the presented methods (or the findings) are useful in real world tasks.
strength_weakness: strength_weakness: * lack of novelty in idea
* lack of technical invention
* lack of practical application or next research direction
* not enough new discovery
questions: questions: NA
paper_summary: paper_summary: This paper takes on one small piece of the now familiar setup, in which psychometric instruments or political ideology instruments are given to a LLM to measure something about them (though specifically what we are supposed to infer from such measurements is often left rather vague, as is the case here).

Although there are many such variations, this paper uses a fairly familiar one, which is to set up a prompt with one space for a statement, and another for the model response (e.g,. "Do you agree or disagree with the following statement: <statement>. I <response> with this statement." for a MLM). They also use a common measurement strategy, which is to sum the probabilities associated with, roughly, affirmative or negative responses.

 Although there are many things which might be varied in such a setup, here the authors focus on the phrasing of the statement itself (such as "There are those who are destined to experience chronic misfortune"). While others have tried various systematic modifications of statements, here the authors simply collect a large number of paraphrases of each statement, and check to see how much probabilities vary across paraphrases. 

Ignoring some of the nuance, overall the authors find that models are relatively stable, but with some showing a large range between the min and max probabilities produced by different paraphrases of the same statement. Thus, as with other related papers, the authors provide additional evidence that this overall approach is not necessarily a valid way to infer something like ideology or personality for LLMs.
strength_weakness: strength_weakness: For experimental setups, one necessarily has to fix something (e.g, prompt, decoding strategy, etc.) to have any level of tractability. And yet the fact that these things are fixed here means that we don't really know how well these results will generalize to other choices that could have been made. Compared to something more like a physical system, it seems less clear that we can count on interactions between such pieces of the experimental setup to not matter. 

The authors do attempt to provide some sort of analysis of their results, using linear models with factors such as sentiment and paraphrase length, to see if these are predictive of the results, but these are less compelling than the rest of the paper, in part because we would expect such things to be highly contextual. (They find, for example, that, in their setup, models are more likely to agree with statements that have positive sentiment).

Other issues are mostly minor. In particular, many more examples of prompts and variation (ideally a large random selection) would be extremely welcome. Some of the figures (e.g., Figure 2), involve a lot of text that is far too small to be legible and/or a lack of context (e.g., numbered items without a legend or key), undermining their utility.
paper_summary: paper_summary: This paper analyzes shortcomings of using LMs for the assessment of political leaning. Specifically, the paper investigates model response stability using the Political Compass Test (62 statements with 4 options each). The authors focus on response variances on a per-item level under paraphrasing as well as which samples are particularly brittle across models and whether any linguistic and structural factors (e.g., word length, lexical frequencies) impact such findings. Using 500 paraphrases per model and 20 models in total (both MLMs and GLMs), the authors overall find that several models exhibit large ranges w.r.t. their response stability, and MLMs overall achieve a higher response validity as compared to GLMs. Moreover, the authors report that the sentiment of a statement consistently correlates with whether a model agrees to the statement, and both greater word and sentence counts are also correlated with agreement to the statement.
strength_weakness: strength_weakness: * The paper generates paraphrases with an OpenAI model and evaluates OpenAI models on them, showing that they have the highest instruction validity rates. This experimental setup is inherently biased towards OpenAI models, and direct comparisons between OpenAI models and non-OpenAI models therefore become unfair. How would such results look like if a different model were used for paraphrasing?
* The paper does not discuss model configuration and sampling parameters (e.g., temperatures) and how these might potentially affect the obtained results. The probability distribution computed over potential outputs is directly affected by the temperature value and the results reported in Figure 2 would therefore directly shift as a result of changing that parameter. Additional work is needed to discuss and investigate the impact of such configuration changes on the obtained results.
* The paper does not discuss limitations of the proposed approach and obtained findings. Given that this work critically assesses the usage of LMs in the context of behavioral questionnaires, it would be important that the authors clearly address the scope and relevance of their work.
* [Style-related] Figure 2 is too small and difficult to read.
questions: questions: Can you provide some details on specific model configurations and parameters that you used as part of the sampling? How would the results change when those configurations are changed?
paper_summary: paper_summary: This paper provides a large-scale analysis of LLMs consistency in answering paraphrased questions about their political leanings.
While the question of how stable and consistent LLMs’ responses to value questionnaires are has been (quite extensively) studied, this paper does it on a much larger scale, by covering 500 different paraphrases of each statement. Additionally, they also study linguistic factors which could bias models to respond in a certain way.
Overall this is a well-written paper, with extensive experiments and useful insights into LLM consistency.
strength_weakness: strength_weakness: - Despite citing works that criticized the forced-choice approach of the political compass setup, this work only evaluates on the Political Compass Test.
- As far as I could see there was no manual evaluation done on (a subset of) the 500 paraphrases, in order to check for quality, diversity and whether they still express the same semantic proposition.

Comments: title: Thanks to reviewers
acknowledgement
Response to rebuttal
Discussion period
comment: We thank the reviewers for engaging in the discussion period and responding to our rebuttals, we appreciate it.
I would like to thank the authors for their detailed response and the additional analysis. I appreciate that an additional set of paraphrases was generated with an independent model (i.e., one that is then not evaluated on the paraphrases). Based on the reasons provided, I would expect this to be the default experimental setup as it would exclude potential biases occurring from evaluating a model based on data that it generated. However, given that this would substantially change the reported results, this would exceed the scope of a revision after reviews. 

Re. the parameters: I appreciate the clarification and agree that in this case no additional information would be required (misunderstanding on my side).

I also appreciate the additional limitations provided. 

Based on the above I increased my score, but still believe that the paper would benefit from an additional round of revision.
Thank you for your response. I recognize that there is some disagreement among reviewers as to this paper, but I will keep my score as it is, which I think is a reasonable assessment. I will look forward to the revised paper, and I hope you are able to resolve the copyright question.
We are curious to hear reviewer Xq2X's response to our rebuttal. We dedicated a significant amount of time to address the issues concerned from their review. As written in our rebuttal, we believe that the two main points addressed in 'reasons to reject' were clarified sufficiently: As for point 1, we have explained why we don't think that our results are biased, and we nevertheless ran additional experiments to substantiate this. As for point 2, again, there were no model configuration and sampling parameters to discuss. Considering the clear deviation in reviewer Xq2X's score compared to the other reviewers, we believe it would be fair to hear a response to our rebuttal before the deadline.
Thank you for your response! It would be great if you could include the promised revisions (i.e. number of incorrect paraphrases and descriptive statistics) in the camera ready. I think it's a good paper and am sticking to my current score.
We again thank all the reviewers for their feedback and look forward to discussing the points raised in the reviews and rebuttals.