Paper: Do LLMs exhibit human-like response biases? A case study in survey design (ICLR-2024)

Reviewer comment (Experiments):
The paper provides a direct comparison between LLM and human responses in section 3.3 (using Llama-2-70B). However, the section lacks critical details of the experimental setup. Which dataset and human responses were used for the comparison? Were responses collected specifically for that study, or obtained from an existing collection effort? Such details are imperative for the reader to understand the details of the reported results.

Author response:
*“Which dataset and human responses were used for the comparison?”*: The human responses were obtained from a prior collection effort by Pew, as mentioned in our response to the reviewer’s first weakness. This set of human responses was also utilized by Santurkar et al. to evaluate whether LLMs can reflect different human opinions (i.e., “representativeness”). Overall, we agree that the details could have been presented more clearly, even if we largely re-used the analysis protocol from Santurkar et al. to compute representativeness. For completeness, we have updated Section 3.3 and provided more experimental details in Appendix E.

Reviewer comment (Experiments):
The questionnaire also feels limited as has been pointed by authors themselves.

Author response:
*“The questionnaire feels limited”*: We elaborate on our choice of dataset in our general response **point #1**. If the reviewer has more concrete suggestions for us to address, please let us know.

Reviewer comment (Experiments):
It's a bit unfortunate that you decided not to evaluate GPT-4, as that one is the most used as a human proxy. It would be very interesting to me to see the same study done for GPT-4. I would not be surprised if GPT-4 would perform similarly to LLaMa-70B.

Author response:
*“It would be very interesting to me to see the same study done for GPT-4.”*: Thank you for the feedback! We agree that this is an interesting thing to investigate, and so we tried adding GPT-4 (0613) to our study. Unfortunately, we found that it is much more difficult to generate valid responses, most likely due to OpenAI’s generation guardrails. Furthermore, it was nearly impossible to get valid responses to many questions. As an initial experiment, we tried generating 50 responses per question for all $(q,q’)$ in $\mathcal{Q}_\text{bias}$, a total of 1494 questions, and counting the number of valid responses (e.g., one of the letter options) that GPT-3.5-turbo vs GPT-4 generated out of the 50. On average, GPT-4 generated ~21 valid responses per question, with nearly a quarter of the questions having 0 valid responses. For example, GPT-4 tended to generate “As” or “This” (and when asked to generate more tokens, GPT-4 generated “As a language model” or “This is subjective”). This is in stark contrast to GPT-3.5-turbo, which had an average of ~48 valid responses per question, with none of the questions having 0 valid responses. Based on these observations, evaluating GPT-4 is infeasible in our current setting. We have since added these details to Appendix B2.

Reviewer comment (Presentation):
It would be very useful to indicate significance in table 3 instead of separately in a table where you need to go to see which of the effects are significant in table 3, e.g. with asterisk or something.

Author response:
*“It would be very useful to indicate significance in table 3.”*: In Table 2 (formerly Table 3), we use the colors of the cell to indicate significance based on a 0.05 cut-off. We have clarified the caption to more clearly reflect this distinction. For our analysis, we don’t believe it is necessary to indicate more granular levels of significance below 0.05, as is common practice in statistical analyses. We refer readers to the Appendix for the full results.

Reviewer comment (Experiments):
You make several claims about instruction-tuned models without doing a controlled study comparing base models to instruction-tuned counterparts (there are more differences between ChatGPT and LLaMa-2 than instruction-tuning). I am relatively convinced the difference is due to instruction-tuning, also because you do have LLaMa-70B and LLaMa-70b-chat (only difference is instruction tuning), but an interesting addition or direction for future work would be to study the impact of instruction tuning on human-like response biases. For example, could it be that instruction-tuned models are further trained on higher-quality data without typos etc, making them less robust to these kind of examples. Could this also mean that other types of perturbations not involving badly formatted text that humans are not sensitive to would be less of a problem for models?

Author response:
*“You make several claims about instruction-tuned models without doing a controlled study comparing base models to instruction-tuned counterparts.”*: We agree that further results beyond Llama2-70b are of interest! We provided additional evidence in general response **point #2**, namely with the addition of Llama2-7b-chat and Llama2-13b-chat.

Reviewer comment (Experiments):
I would be more compelled by the evidence presented here if the questions matched those presented to humans, but it seems to be the case that the questions are generated by a LM. This feels like a weird way to evaluate whether LMs fall prey to same biases that humans do, and the authors don't justify it. Was it because the original questions from social science/psychology/etc. were not available? Wouldn't you want to replicate the result in the distribution of these better-established question pairs than in question pairs where, as I understand it, half of the question pair is natural and the other is generated by a LM to be inflected by the framing which the bias is sensitive to?

Author response:
*“It seems to be the case that the questions are generated by a LM.”*: We would like to clarify that the question modifications were *not* generated by a language model and apologize for the lack of explicit note of this on our end. The modified forms of the questions for each bias were generated by either modifying them manually ourselves (as was the case for acquiescence and allow/forbid) or by performing systematic modifications such as automatically appending an option, removing an option, or reversing the order of options (for odd/even, opinion float, and response order). We have added a note of this in Section 2.2. These manual and systematic modifications were made to reflect the types of modifications used in prior social science studies, as discussed in A.2.

Reviewer comment (Experiments):
While I can't find overt problems--the methodology seems neat and sound to me--I think the evidence could be heavier. For example, why did you sample only 50 questions? Why only ATP?

Author response:
*“Why did you sample only 50 questions?”*: Due to budget and time constraints, we sampled 50 questions for each perturbation setting. We agree that a larger-scale analysis is beneficial, and have since run experiments across all questions (747 questions in total) for each perturbation modification. Figure 3 and Table 5 have been updated accordingly. After scaling up the number of samples, our conclusion is still that while Llama2-70b is more human-like than other models, the general trend is that LLMs do not respond to instruction changes as humans would.