# Going against the tide: experimental design for measuring prejudices in France

Vincent Tiberj

<figure><img src="image_1.png" /></figure>

## To cite this version:

Vincent Tiberj. Going against the tide: experimental design for measuring prejudices in France. APSA Migration and Citizenship Newsletter, 2015, 4 (1), pp.28-33. halshs-01242831

HAL Id: halshs-01242831

https://shs.hal.science/halshs-01242831

Submitted on 21 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Going against the Tide: Experimental Survey Design for Measuring Prejudice in France

Vincent Tiberj

Université de Bordeaux
vincent.tiberj@sciencespo.fr

Social desirability bias is one of the most-documented biases in attitude surveys. Pollsters and academics know quite well that respondents are reluctant to declare certain opinions or behaviors, such as abstention from voting or votes for extreme right parties. This is even more salient when prejudice is concerned and particularly in recent years. As general levels of education increase, so does the capacity of respondents to decode what political scientists are looking for when they administer traditional survey questions. Some critics of mass surveys in France, following the path opened by Bourdieu, argue that if college-educated respondents appear to be more tolerant, it is only because they are better able to provide the acceptable answer (Lehingue 2011).

These concerns are particularly salient in the study of race relations. James Kulinski and his colleagues (1997), for example, have demonstrated how answers about racial equality in the US cannot be taken at face value. Affirmative action creates much more animosity among the general public than what the traditional Q and A design can capture. To prove it, their experimental design is simple and smart: respondents are provided with two identical lists of issues, but one version of this list includes an “affirmative action” item. Respondents are randomly assigned to one or the other version of the list. They are then asked to count the number of issues that anger them. Evaluating the real level of anger produced by affirmative action is only a matter of subtraction.
This experimental design becomes particularly effective in the context of the evolving debates about immigration and integration in contemporary Europe. European polities are creating their own version of symbolic (Kinder and Sears 1981) or subtle racism (Pettigrew 1989), particularly vis-à-vis Muslims. As in the US, the time of biological racism is behind us. Since the beginning of the 2000s, only 8 to 14 percent of French believe that "some races are more gifted than others"; and those that do belong to cohorts born before the Second World War, at a time when such prejudice was considered common sense. But prejudice has not disappeared, it has mutated. First, racial hate has taken a cultural disguise (Taguieff 1987), and is now hidden behind mainstream values. A political observer from the 1980s would be quite surprised to see today extreme right parties struggling for gender equality (rhetorically), the freedom for women to dress as they want, or the freedom of speech of a leftist satirical newspaper such as *Charlie Hebdo*. These mainstream (and in France these Republican) values are frequently used to express and hide xenophobia and prejudice in Europe, in much the same way that opponents of affirmative action argue for "equal chance among candidates" in the US. Consequently, Islam becomes a very convenient source of opposition, much like race four decades ago and immigration more recently.

Overall, the normative mainstreaming of prejudice complicates the task of surveying public opinion. This growing trend, and the increasing capacity of respondents to decode survey questions, requires a more systematic use of experimental design. In order to make my point, I will present the results of two experiments. The first will demonstrate how experiments are superior to the traditional Q and A design in measuring prejudice. The second will address

the interaction between mainstream principles and prejudice.

## Why experiments matter?

In November 2005, France experienced 21 days of riots in deprived suburbs, resulting in the declaration of the state of emergency by President Chirac, a constitutional disposition previously only used once in 1961 during the war in Algeria. A month later, Sylvain Brouard and I had the opportunity to conduct a poll to investigate the perceptions of these events among the general population. These suburbs represented the accumulation of various disadvantages: urban relegation, social inequalities, a high level of immigration (often seen as Muslim immigration though this is more complex). Were these events and the inhabitants of these neighborhoods perceived through a social lens (a revolt of the poor, of the outsiders)? Or were they considered primarily as immigrants, and therefore as non-French, non-Catholic, or non-Like-us, resulting in "ethnic riots"? Some intellectuals such as Alain Finkielkraut, clearly disregarded the social reading of these events: he declared that these riots were a "revolt with an ethnic-religious characteristic" (Ha'aretz, November 17, 2005).

Two types of questions have been developed to sort out this issue. The first was constructed using the traditional Q and A design, the second was an experiment. In the first, respondents were asked successively whom to blame for unemployment: the unemployed themselves who do not put in enough effort to get a job, or society which offers no work opportunity. The same question in the same format was asked regarding more specifically the unemployment problem in the suburbs. In the second type of question, support for three public policies in support of specific groups was tested. For example, respondents were asked whether they will favor or oppose the state giving more mon-
ey to specific schools where some groups are present. These groups were randomly identified as “poor families”, “migrant families” or “suburban families” (the term “suburban” here refers to the “banlieues”, a term that has grown to describe disadvantaged and disenfranchised immigrant or immigrant-heritage communities on the outskirts of urban centers). A similar approach was used to evaluate support for an increase in welfare benefits for poor / migrant / suburban families and the allocation of social housing to poor / migrant / suburban families.

<figure><img src="image_3.png" /><figcaption>**Figure 1. Society is Responsible for Providing Jobs (predicted probabilities)**</figcaption></figure>

Source: Suburb crisis survey, 2005 (co-principal investigators: S. Brouard and V. Tiberj)

The traditional Q and A design points to a false conclusion (see figure 1). The suburban unemployed seem to be treated as if they were “regular” unemployed, no more no less. No one can be accused of harboring a double-standard, since the two answers are strongly correlated (r= 0.56). Additionally, cultural conservative respondents<fnref n="8" /> consistently blame the unemployed for their predicament, while cultural progressive respondents blame society. But I interpret this result as response bias. Even if

question order was randomized, respondents are likely inclined to answer consistently across questions, particularly if they want to hide their prejudicial attitudes against suburbans. This interpretation is validated thanks to the experimental design (see figure 2). In what follows, I present only the case of money for school, but the results for the two other policies are consistent.

<figure><img src="image_4.png" /><figcaption>**Figure 2. Financial Support for School (predicted probability)**</figcaption></figure>

Source: Suburb crisis survey, 2005 (co-principal investigators: S. Brouard and V. Tiberj)

In addition to measuring public opinion toward suburbans, the experiment aims to accomplish several other objectives: how to frame social public policy to increase acceptance from a wider audience? To understand whether the French oppose such policies no matter the target group? To evaluate whether the French are ready to accept a French version of affirmative action? Clearly, respondents favor giving more money to schools in deprived areas (which is not so common in the reality of the French educational system): overall, 86 percent of respondents favor this in the “poor family” condition, 69 percent do in the “suburban family” condition, and 64 percent do in the “migrant family” condition. A strong majority of French support helping out those who try to help themselves.

<footnote n="8">⁸ The level of cultural liberalism is measured with an attitudinal scale based on questions regarding gender roles, death penalty and authority and the number of immigrants. It ranges from 0 to 10; 0 means that the respondents have given a conservative answer to all questions, 10 means they have given systematically liberal ones.</footnote>
But the experiment also provides us other conclusions. First, as is the case for the traditional type of question, support for any of the three policies depends on the respondent's level of cultural liberalism. Cultural progressives are ready to help any disadvantaged group, whereas cultural conservatives are less supportive. Second, the level of support very much depends on the targeted group. Sixty percent of cultural conservatives are supportive when the targeted group is the "poor family". This drops precipitously to 20 percent when the targeted group is the "immigrant family". This clearly demonstrates that support for—or in this case opposition to—social public policy is fueled by racial prejudice. What about suburbs? Are they seen through a social or an ethnic lens?

Among cultural conservatives, variation in support for suburban families mirrors the pattern observed for immigrant families. This experiment reveals how the French perceive suburbs, a fact that would have been obscured by traditional survey question design. For all three policies, only the most cultural progressives treat all three groups equally; cultural conservatives strongly distinguish between social groups (the poor) and ethnic groups (immigrants and suburbs).

## What lies behind the general principles

As I mentioned in the introduction, the debate about immigration and integration in France and Europe, has adopted a new disguise, notably the quasi-systematic reference to general principles supposed to define Western norms shared by entire host societies. The newcomers are either suspect regarding their commitment to these values or regularly asked to prove their commitment to these principles (see among others Joppke 2007). This can be with regard to acceptance of homosexuality, gender equality (for example in the Netherlands), or in France *laïcité* (secularization) or freedom of expression

(though of a very different sort than in the USA) and various other Republican principles (for a presentation of their various implications in this country see Chabal 2015).

Advocating for or referring to general principles (even on pure rhetorical grounds) is in itself perfectly legitimate. But do ordinary citizens and politicians really reason as philosophers? Do they actually judge and opine using general principles or do they rely on these types of concepts to hide other motivations, such as prejudice? Do they apply the same rule of thumb to comparable situations or do they rationalize their positions in order to act in a more socially acceptable way? This is particularly important with regard to xenophobia. It is possible that behind the same denunciation of Islam as a gender conservative religion you can find either a cultural progressive genuinely concerned about gender equality or a cultural conservative who uses gender equality as a guise for their racist attitudes. The work of Sullivan, Marcus and Pierson (1982) on political tolerance has shown that Americans may seem committed to this principle in the abstract, but apply it selectively following their like or dislike of various political groups; they will deny the right to rally for an extreme-right group but will support this right for a group closer to their political leanings for example. Sniderman and Jackman (2002) have found the same in France regarding the right to rally, and I have discovered a comparable mechanism regarding freedom of expression. This experiment was in the field in 2006 (Tiberj 2008) but remains particularly relevant today in France, in the post-Charlie Hebdo world.

It tells the (fictional) story of a high school art teacher. He is presented to the respondent randomly as "French", "Maghrebi" or "Black". This teacher has drawn in class a caricature of Maghrebis or Blacks if he was French, of Blacks or
French if he was Maghrebi, or of Maghrebi or French if he was Black. The school headmaster has suspended the teacher. Respondents are asked if they support the teacher or the headmaster. This experiment is a “test of principles”: attachment to the freedom of expression (or to the neutrality of the school) is supposed to remain constant, whatever the characteristics of the teacher or the targeted group.

This not the case: the origins of the teacher weight a lot on the support he receives (whereas the caricatured group has only a marginal influence on the response). Overall, the teacher is supported by 49 percent of the respondents if he is French, 60 percent if he is Maghrebi and 78 percent if he is Black. Clearly, respondents are quite supportive of freedom of expression, even in schools. But this support is conditional on the ethnic description of the teacher. Additionally, the experiment tells a more interesting story when the level of cultural liberalism is taken into account (see figure 3).

<figure><img src="image_5.png" /><figcaption>**Figure 3. Support of the Teacher (predicted probability)**</figcaption></figure>

Source: BZUS survey, 2006 (co-principal investigators: S. Brouard and V. Tiberj)

A neutral case appears: the Black Teacher is treated equally by cultural progressives and cultural conservatives (which shows, among other things, that the situation of Blacks differs significantly in the United States and France).

The pattern for the two other cases, however, diverges. Support for the French teacher decreases with the level of cultural liberalism (from 61 percent among cultural conservatives to 44 percent among cultural progressives). But when the teacher is Maghrebi, we see the opposite phenomenon: support increases with cultural liberalism (from 47 to 67 percent). Commitment to this general principle of freedom of expression is undoubtedly situational. Clearly xenophobic prejudice fuels rejection of the Maghrebi teacher and support for the French teacher among cultural conservatives. But this double standard is also present among cultural liberals. They probably suspect the French teacher to be racist, and therefore condemn his behavior; on the other hand, they are highly supportive of the Maghrebi (and the Black) teacher, as if he were immune to racist motives.

## Concluding remarks

Experimental designs have opened large avenues of research in mass surveys in the last two decades. Furthermore, experiments provide us with a dynamic explanation of prejudice, which are typically considered to be stable. They provide a better understanding of how frames reveal racial prejudice, and by doing so, they may provide insights into how to fight prejudice. For example, an experiment about the crisis in Syria can test the degree of acceptability of immigrants as compared to refugees. Combined with other forms of experiments, such as the ones proposed by game theory or field experiments, survey experiments give us a better understanding of the conditions under which minorities are accepted by host societies. Nevertheless, as experimental methods have proliferated, so have their fragmented and sometimes-contradictory set of results. Cumulation and comparability may be the next objectives of this research.
References

Chabal, Emile. 2015. *A Divided Republic: nation, state and citizenship in contemporary France*. Cambridge: Cambridge University Press.

Joppke, Christian. 2007. "Beyond national models: Civic integration policies for immigrants in Western Europe." *West European Politics* 30(1): 1-22.

Kinder, Donald R., and David Sears. 1981. "Prejudice and politics: Symbolic racism versus racial threats to the good life." *Journal of personality and social psychology* 40(3): 414-431.

Kuklinski, James, Paul Sinderman, and Kathleen Knight. 1997. "Racial prejudice and attitudes toward affirmative action." *American Journal of Political Science* 41(2): 402-419.

Lehingue, Patrick. 2011. *Le vote: approches sociologiques de l'institution et des comportements électoraux*. Paris: La découverte.

Pettigrew, Thomas. 1989. "The Nature of Modern Racism in the United States." *Revue Internationale de Psychologie Sociale* 2(3): 291-303.

Sniderman, Paul, and Simon Jackman. 2002. "Intolérance et culture politique: la spécificité française." In *La démocratie à l'épreuve*, 171-202. Paris: Presses de Sciences Po.

Sullivan, John, George Marcus, and James Piereson. 1982. *Political Tolerance and American Democracy*. Chicago: University of Chicago Press.

Taguieff, Pierre-André. 1987. *La force du préjugé: essai sur le racisme et ses doubles*. Paris: Editions Gallimard.

Tiberj, Vincent. 2008. *La crispation hexagonale: France fermée contre « France plurielle »*. Paris: Plon.