Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting (2024)

Abstract: Socio-demographic prompting is a commonly employed approach to study cultural biases in LLMs as well as for aligning models to certain cultures. In this paper, we systematically probe four LLMs (Llama 3, Mistral v0.2, GPT-3.5 Turbo and GPT4) with prompts that are conditioned on culturally sensitive and non-sensitive cues, on datasets that are supposed to be culturally sensitive (EtiCor and CALI) or neutral (MMLU and ETHICS). We observe that all models except GPT4 show significant variations in their responses on both kinds of datasets for both kinds of prompts, casting doubt on the robustness of the culturally-conditioned prompting as a method for eliciting cultural bias in models that are not sufficiently stable with respect to arbitrary prompting cues. Further, we also show that some of the supposedly culturally neutral datasets have a non-trivial fraction of culturally sensitive questions/tasks.

## notations, definitions and research questions let us first formally define a typical sociodemographic or culturally-conditioned probing technique. following a recent proposal by @xcite , we shall use the terms semantic and demographic proxies for describing our setup. demographic proxies, such as age, region and gender are used to define the cultural group of interest. since most llm-based studies of culture focus on region as the demographic proxy, we shall restrict this discussion as well as our study to only this proxy. a semantic proxy defines a domain of interest, such as name, food, kinship terms, that can be used to capture the cultural variation across demographic groups. we shall denote such semantic proxies as p 1 , p 2 , . . . , p k , where p i is a set of keywords or cues, {c 1 i , c 2 i , . . . c n i }, that can be used for culturally conditioning a prompt. for instance, if p i is food, the cues could be sushi, hamburger and biryani. suppose that d represent a dataset, {d 1 , d 2 , . . . d m } where each sample d l is a test or question (along with appropriate instructions in natural language) whose answer should vary in a certain way for different cultures (in our case, regions). we define the culturally conditioned prompt under proxy p i as the composition prompt x (c j i , d l ). here, prompt x (c, d) is a lexical template with slots to be filled in with c and d, and the subscript x indicates a particular lexical variant of the prompt. let ŷj i,@xmath0, d l )) represent the response from the model (llm) m for a particular probe prompt x (c j i , d l ). for simplicity of notation, we will omit the subscript i when there is only one proxy. typically, there are two ways to show that a model m is biased towards a certain culture represented by, say, c j , than another represented by c j ′ . first, if accuracy of responses for one culture is higher than another: where, ŷϕ @xmath1, d l )), the unconditioned response. in order to understand why the above formulation could be problematic, it is useful to draw an analogy of these approaches to that of randomized controlled trial for drugs or medical interventions. d is equivalent of a set of participants (we can create as many copy of them). in first case, two different drugs c j and c j ′ are administered on the two groups (who are otherwise identical copies of d), and the outcomes are directly compared. whereas, in the second case, the effect is measured for every participant before and after the drug administration for two different drugs. however, in a typical rct for drugs, one group is provided with the treatment and the other group with a placebo, and the participants are not aware which group they belong to. if a stronger positive outcome is observed in the treatment group than the placebo or control group, then the drug maybe considered effective. use of placebos are conspicuously missing from the culture-bias studies, but on the other hand, as we shall argue and empirically show, they are crucial to understand the effectiveness of any probing technique. in our case, a placebo would be a cue word that is identical to cultural cues in terms of its semantic and syntactic properties, but is not expected to have any impact on the cultural conditioning. we shall call such semantic proxies as culturally non-sensitive (such as house numbers and favorite programming language). note that if d is a culturally non-sensitive dataset, where ground truth y j l does not depend on j, then ideally we expect given this context, here we want to investigate the following research questions: rq1 do we observe a placebo effect in llm probing, where the variation in responses of m is high for cues chosen from p i which is a placebo or culturally non-sensitive proxy? rq2 is the placebo effect as strong as the effect of cultural probing? in other words, are the variations observed for culturally sensitive and non-sensitive proxies at a similar scale? rq3 how does the above effects vary by datasets and models? ## experiments in this section we talk about our experiment design and setup. we evaluate a suite of models on a variety of benchmarks with different degrees of cultural sensitivity, on culturally sensitive and nonsensitive proxies. the setup involves designing the proxies and prompt templates, and extraction of the model responses followed by statistical analysis, which are discussed below. ## qualitative analysis our results highlight the fact that except for gpt-4, for all model-dataset-proxy combinations, we observe a high degree of variation in responses irrespective of the sensitivity of the dataset or the proxy to culture. in this section, we discuss some model and dataset specific observations that came to front while manually inspecting the model responses.