metadata: authors: 0: author: Koehler,Dominik
1: author: Harley,Georgia
2: author: Menzies,Nicholas
count: Serbia
docna: 0: docna: Discrimination against sexual minorities in
            education and housing : evidence from two field experiments
            in Serbia
docty: Policy Research Working Paper
abstracts: cdata!: Discrimination against lesbian, gay,
            bisexual, transgender, and intersex (LGBTI) people is
            widespread, and LGBTI exclusion from economic markets, vital
            services, and political spaces is entrenched. This is not
            just an individual problem; it is a development challenge;
            not only because discrimination is inherently unjust, but
            also because “there are substantial costs -- social,
            political, and economic -- to not addressing the exclusion
            of entire groups of people.” Understanding the barriers
            LGBTI people face in accessing markets, services, and spaces
            is important for designing more inclusive policies and
            programs. This study documents, for the first time,
            discrimination against LGBTI people in access to education
            and housing in Serbia, using evidence from field
            experiments. In Serbia, "feminine boys," widely
            perceived as being gay, were at least three times more
            likely to be refused enrollment in primary schools (15
            percent) compared to boys not perceived to be feminine (5
            percent). Eighteen percent of same-sex couples were refused
            apartment rentals by private landlords, while no
            heterosexual couples were. The research contributes to the
            growing body of evidence on the economic dimensions of LGBTI discrimination.
geo_reg_key: 517190,517191
repnme: repnme: Discrimination against sexual minorities in
            education and housing : evidence from two field experiments
            in Serbia
display_title: Discrimination against sexual
            minorities in education and housing : evidence from two
            field experiments in Serbia
listing_relative_url: /research/2018/06/30224449/discrimination-against-sexual-minorities-education-housing-evidence-two-field-experiments-serbia
publishtoextweb_dt: 2018-06-29T00:00:00Z
no_of_pages: 29
author_keys: 0: author_key: 1378869
1: author_key: 1153003
2: author_key: 943635
model_extractions: WPS8504 Policy Research Working Paper 8504 Discrimination Against Sexual Minorities in Education and Housing Evidence from Two Field Experiments in Serbia _Dominik Koehler Georgia Harley Nicholas Menzies_ Governance Global Practice June 2018
Policy Research Working Paper 8504

# **Abstract**

Discrimination against lesbian, gay, bisexual, transgender, and intersex (LGBTI) people is widespread, and LGBTI exclusion from economic markets, vital services, and political spaces is entrenched. This is not just an individual problem; it is a development challenge; not only because discrimination is inherently unjust, but also because “there are substantial costs—social, political, and economic—to not addressing the exclusion of entire groups of people.” Understanding the barriers LGBTI people face in accessing markets, services, and spaces is important for designing more inclusive policies and programs. This study documents, for the first time, discrimination against LGBTI people in access to education and housing in Serbia, using evidence from field experiments. In Serbia, “feminine boys,” widely perceived as being gay, were at least three times more likely to be refused enrollment in primary schools (15 percent) compared to boys not perceived to be feminine (5 percent). Eighteen percent of same-sex couples were refused apartment rentals by private landlords, while no heterosexual couples were. The research contributes to the growing body of evidence on the economic dimensions of LGBTI discrimination.

This paper is a product of the Governance Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/research. The authors may be contacted at dkohler@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA**

### **Dominik Koehler, Georgia Harley, Nicholas Menzies**

**JEL classification:** I24 Education and Inequality; O15 Human Development; O18 Housing; O35 Social Innovation

### **Keywords:**

Lesbian, Gay, Bisexual, Transgender, LGBTI, Sexual Orientation, Gender Identity, SOGI, Development Outcomes, Serbia, Education, Housing, Discrimination, and Experiments

## **Acknowledgments**

This study is part of a large World Bank LGBTI data collection initiative in the Western Balkans led by Dominik Koehler, Georgia Harley, Nicholas Menzies, Runyararo Gladys Senderayi and Marko Karadzic. IPSOS Strategic Marketing conducted the field experiments under the leadership of Svetlana Logar. This project could not have been possible without the support from Andrew Park and Andrew Flores from the Williams Institute at UCLA, as well as Dragana Todorovic and Amarildo Fecanji from the LGBTI Equal Rights Association for Western Balkans and Turkey. The authors would like to thank the following for thoughtful guidance and comments: Linda Van Gelder, Lada Strelkova, Adrian Fozzard, Roby Senderowitsch, Clifton Cortez, Mattias K. A. Lundberg and Bojana Naceva.

All errors are the authors’ alone, and the views expressed herein do not represent the position of the World Bank or its Executive Directors.

Funding for this work was generously provided by the Nordic Trust Fund, a knowledge and learning platform promoting human rights‐based approaches to development.
# **Table of Contents**

|Executive Summary .................................................................................................................... 2|
|---|
|Background and Study Objectives .......................................................................................... 1|
|Choosing the Research Areas and Methods .......................................................................... 2|
|1.<br>Access to Primary School Education for Students Perceived to be Gay ............................ 4|
|1.1.<br>Methodology ................................................................................................................... 4|
|Experimental Scenario ........................................................................................................ 4|
|Sampling of Schools ............................................................................................................ 5|
|Measurement ..................................................................................................................... 5|
|1.2.<br>Results ............................................................................................................................. 6|
|Feminine Boys Have More Limited Access to Primary Schools .......................................... 6|
|Reasons for Rejections ....................................................................................................... 8|
|Reasons for Hesitation in Enrollment ................................................................................. 8|
|2.<br>Access to Housing for Gay and Lesbian Couples .............................................................. 10|
|2.1.<br>Methodology ............................................................................................................. 10|
|Experimental scenario ...................................................................................................... 10|
|Sampling of Landlords ...................................................................................................... 10|
|Measurement ................................................................................................................... 11|
|2.2.<br>Results ....................................................................................................................... 11|
|Same‐sex couples face discrimination when renting apartments ................................... 11|
|Gay couples face more discrimination than lesbian couples ........................................... 12|
|Male landlords discriminate more against gay couples than female landlords .............. 13|
|Homophobia was Openly Expressed by Many Landlords ................................................ 13|
|3.<br>Conclusion ........................................................................................................................ 15|
|Avenues for Further Research .............................................................................................. 15|
|Policy Considerations ........................................................................................................... 16|
|ANNEX I: Detailed Methodology .............................................................................................. 18|
|Education Experiment .......................................................................................................... 18|
|Renting Experiment .............................................................................................................. 21|
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **Background and Study Objectives**

Lesbian, gay, bisexual, transgender, and intersex (LGBTI) people are often victims of violence, inequality, and discrimination. Discrimination on the grounds of sexual orientation, gender identity (SOGI), and sex characteristics has wide‐ranging effects on LGBTI people and the societies they live in. When access to markets, services, and spaces becomes more constrained, jobs are harder to find; education and health services are limited; and political and public engagement is riskier. The effects of discrimination push many LGBTI people into poverty, and it is, therefore, likely that they are overrepresented among the poor.<sup>i</sup> Collecting data on the experiences of LGBTI people is challenging, and, as a result, robust data that quantifies and details the effects of discrimination is missing in most countries, including Serbia. This is primarily because many LGBTI people, especially but not only those who live in developing countries or in countries where discrimination is prevalent, choose to stay under the radar. They hide their LGBTI identity out of fear: fear of rejection and exclusion; fear of harassment; fear of discrimination; and fear of physical violence. Therefore, knowledge about the lived experiences of LGBTI people is limited, especially in contexts where discrimination and exclusion are prevalent. In Serbia, like in many other developing countries, reliable and robust data on the social and economic challenges LGBTI people face is not available, making policy interventions difficult. The World Bank is committed to helping to address this evidence gap by undertaking research and collecting rigorous data. Gathering data on the challenges LGBTI people face is an important first step towards developing sustainable solutions that respond to their challenges and needs.

This report is part of a broader World Bank research initiative: “Understanding the Socio‐ Economic Dimensions of LGBTI Exclusion in the Western Balkans.” In addition to this report, two large‐scale surveys (reports forthcoming) were conducted, reaching over 5,500 LGBTI people in the region.

1. The first survey adapts the 2012 European Union Fundamental Rights Agency’s _Survey of LGBT people in the EU and Croatia_<sup>ii</sup> and applies that methodology in Albania, Bosnia & Herzegovina, Croatia, Kosovo, FYR Macedonia, Montenegro, Serbia, and Slovenia. That adaptation enables generalizable comparisons of the experiences of LGBTI people in the EU Member States with those of LGBTI people in the Western Balkans.

2. The second survey adapts Serbia’s general welfare survey, the _Survey on Income and Living Conditions (SILC),_ to LGBTI people in Serbia. The adaptation enables the income distribution and social inclusion of LGBTI people to be compared to that of the general population in Serbia.

The current report complements the SILC survey by providing additional qualitative data about the challenges LGBTI people face in two key areas of life: education and housing. Cumulatively, these studies form one of the largest LGBTI data sets outside of the OECD countries. The multifaceted nature of the research initiative helps to better understand the development outcomes for LGBTI people as individuals, in the economy, and in society.

Experimental studies have provided important insights into discrimination based on race, sex, ethnicity, and religious affiliation. These studies have covered a wide range of areas such as 1
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA access to employment,<sup>iii</sup> housing,<sup>iv</sup> and credit and consumer markets.<sup>v</sup> Experimental studies have also been used to examine discrimination based on sexual orientation, but focusing more narrowly on access to the labor market in developed countries.<sup>vi</sup> Experimental studies offer a simple and effective way of producing reliable data, especially among less conspicuous and hard‐to‐reach populations such as LGBTI people. The two experiments outlined in this report build on the prior body of work by looking specifically at discrimination based on sexual orientation in accessing compulsory primary school education and the private rental housing market. These two areas were chosen as they represent important steps in the life cycle of exclusion faced by LGBTI people, and in Serbia, they were amenable to the experimental methodology. For the first time, these experiments quantify the extent of discrimination LGBTI people face in accessing education and housing in Serbia: to date, data in both areas has been anecdotal only. A better understanding of this discrimination can help to inform policies and programs that address the issue, such as introducing, or adjusting, laws and regulations, launching public awareness campaigns, and providing systematic training.

From a legal and policy perspective, Serbia has made good progress in ensuring the protection of the rights of LGBTI people, but in practice, discrimination against and exclusion of LGBTI people remain a problem.<sup>vii</sup> The European Commission confirmed in its 2016 Progress Report, that “the legal and institutional framework for the respect of fundamental rights is in place,” but goes on highlight the need for sustained efforts to improve the situation of these vulnerable populations.<sup>viii</sup> Many LGBTI people in Serbia report experiencing discrimination in public and private services and economic opportunity, as well as not feeling safe in public spaces.<sup>ix</sup> However, there is a lack of robust, quantitative data that clearly shows differential treatment between sexual and gender minorities, and straight and cisgender<sup>x</sup> citizens.

The two experiments discussed in this report apply experimental methods to collect reliable information that allows comparisons of outcomes among lesbian and gay people and their heterosexual counterparts. This is done in real‐life settings by using the mystery shopper technique.<sup>xi</sup> The mystery shopper technique involves the same actors playing different roles (in this case, mothers of feminine boys and non‐feminine boys, and lesbian, gay, and straight couples) and approaching the same service providers. The differences in the reactions of the service providers (in this case, schools and landlords) are then measured and attributed to the different identities of the actors, thus isolating the discrimination.

# **Choosing the Research Areas and Methods**

The data collection method, the markets, services, and spaces considered, and the scenarios portrayed were guided by the need to address specific challenges in Serbia. Widespread stigma against LGBTI people in Serbia means that the open declaration of SOGI status is quite unusual.<sup>xii</sup> This limits the range of experimental scenarios which would allow collection of data in situations that are natural enough to guarantee data validity, while securing the safety and anonymity of all participants. This study focuses on gay and lesbian people, as they are identities an average Serbian citizen is more familiar with as compared to transgender or intersex people.<sup>xiii</sup> Data were collected via telephone interviews to protect the identity and security of the participants and to eliminate the potential influence of face‐to‐face contact on the research outcome. This significantly limited the scenarios, services, and markets that could be analyzed in this type of study. For example, social welfare centers, financial institutions, and public health institutions generally required face‐to‐face contact and were 2
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA therefore not suitable. Replicating the labor market studies used to show gay and lesbian discrimination in other countries was not viable given the small number of formal job announcements in Serbia. Education and housing were selected as they focus on two key areas that impact the welfare of citizens and data collection under natural conditions was possible without putting participants at risk.

3
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **1. Access to Primary School Education for Students Perceived to be Gay**

Evidence from many countries illustrates that LGBTI students face discrimination and bullying from teachers, school staff, and their peers.<sup>xiv</sup> A 2014 UNICEF study showed that homophobic bullying has negative impacts on the students’ attendance and can increase the likelihood of alcohol consumption, depression, and even attempted suicide.<sup>xv</sup> This in turn leads to lower learning outcomes and higher dropout or expulsion rates. Poor performance in school reduces opportunities for higher education and access to quality employment. Addressing discrimination against LGBTI students is particularly important considering Sustainable Development Goal 4, under which Serbia (and all countries) has committed to ensuring inclusive and equitable quality education for all. In Serbia, discrimination based on sexual orientation, gender identity, or being intersex in schools has not been systematically researched. This experiment, therefore, fills an important data gap for policymakers, school leaders, and the LGBTI community.

## **1.1. Methodology**

### **Experimental Scenario**

The experimental scenario was designed to assess if the presence of perceived femininity among 14‐year‐old boys had an impact on their chances to enroll in the seventh (7th) grade of public primary schools in Serbia.<sup>xvi</sup> In Serbian culture, men and boys who are perceived to be feminine are commonly assumed to be gay. In contrast, masculine girls or women are not always directly perceived to be lesbian. The inclusion of masculine girls, or any other part of the LGBTI population that the Serbian people are less familiar with, would not only have reduced the naturalness of the experimental scenario but also complicated the interpretation of the results. Notably, it is irrelevant in the scenario whether the boy in question is gay— only that he is perceived in the community to be gay, and may therefore be vulnerable to discrimination.

The schools were contacted by six members of the research team acting as mothers looking to enroll their son in a new school because the family was relocating for employment reasons. At the beginning of the interview, before the school administrator has a chance to make any statement about the availability of places, the mother mentions that the boy is “feminine,” and that his femininity is quite obvious, so she wants to draw the school’s attention to the fact. Primary schools are obliged by the law to enroll students. The telephone conversation was designed based on three assumptions:

- i. The femininity of the boy is the only obvious characteristic distinguishing the potential student which could affect the decision about his admission to school;

- ii. The mother’s statement about the femininity of her son is very clear and supported by a convincing argument as to why she is mentioning it at all;

- iii. The conversation should gather information that can be transformed into statistically comparable data.<sup>xvii</sup>

4
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA The scenario of the telephone conversation for the control group was the same, except that the boy was not characterized as feminine.

Sampling of Schools The experiment sampled primary schools located in urban areas of municipalities with at least five primary schools.<sup>xviii</sup> Smaller municipalities and rural areas were excluded because the experimental scenario would become less realistic. Additionally, municipalities with just one or two schools were excluded because the school might have felt pressured to approve the enrollment of the feminine boy, given that primary education is mandatory in Serbia until the completion of eighth grade.

Out of all eligible schools, 184 were selected from 37 municipalities using standard random selection procedures.<sup>xix</sup> Approximately the same number of schools were selected from the three regions: Belgrade, Vojvodina, and Central Serbia. The schools were randomly assigned to treatment (feminine boy) and control group (non‐feminine boy) so that in each municipality there was an equal number of schools contacted on behalf of the feminine boy and non‐feminine boy.

# **Measurement**

Since the data was collected through telephone conversations in natural conditions, the transcripts of the conversations were transformed into quantitative measures. To ensure the robustness of the quantitative data, the evaluation of the outcomes was performed by three independent raters in accordance with the usual standards of evaluations of observations in natural conditions.<sup>xx</sup> Raters were in no way part of the experiment and did not have a position on LGBTI rights that could skew their assessment. Their attitudes towards LGBTI rights were checked in their interviews for the job.

The outcomes were evaluated on a five‐point scale:

1. Accepted without hesitation or any additional conditions.

2. Accepted, but with hesitation, or postponement: the decision was made either with the hesitation of the contacted person or only after consultation with school staff.

3. Unclear outcome: final decision was not achieved, i.e., the decision is postponed until the mother and her son can go to the school and meet the school authorities face‐to‐face.

4. Rejected, but with hesitation: the decision was made either with hesitation of the contacted person or only after consultation with school staff.

5. Rejected without hesitation.

5
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **1.2. Results**

## **Feminine Boys Have More Limited Access to Primary Schools**

<mark>Feminine boys were three times more likely (14 percent) to be refused enrollment in public primary schools than non‐feminine boys (4 percent). Most feminine boys did not face direct refusal; of the refusals, 78.6 percent (11 percent of all boys) came after some delay or hesitation on the part of the school. None of the non‐feminine boys were confronted with a refusal preceded by hesitation; they were refused without hesitation and for strictly technical reasons (schools at full capacity). Amongst the boys who were accepted, feminine boys faced significantly more hesitation in the decision than the non‐feminine boys. Out of the accepted feminine boys, 51 percent were accepted with hesitation compared to only 25 percent of non‐feminine boys. Enrollment of non‐feminine boys was only met with hesitation when the school perceived the boy’s academic performance and discipline to be a potential problem. The difference in the kinds of refusals and acceptances faced by the two different groups of boys reinforces the conclusion that femininity is the reason for discrimination.</mark> <!-- Start of picture text --> Figure 1: Acceptance and refusal of feminine and non‐feminine boys<br>FEMININE BOYS<br>NON‐FEMININE BOYS<br>3% 4%<br>6%<br>11%<br>27%<br>8%<br>25%<br>65%<br>51%<br>Accepted without hesitation<br>Accepted without hesitation<br>Accepted with hesitation<br>Accepted with hesitation<br>Not clear<br>Not clear<br>Refused with hesitation<br>Refused with hesitation<br>Refused without hesitation Refused without hesitation<br>Database: all schools, n=184<br><!-- End of picture text --> Omitting the 7 percent of unclear cases (8 percent for feminine boys and 6 percent for non‐ feminine boys) and combining all acceptances and all refusals clearly shows the discrimination — against feminine boys their chances of being refused enrollment are three times higher (figure 2).<sup>xxi</sup> 6
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA Figure 2: Acceptance and refusal of feminine and non‐feminine boys—simplified <!-- Start of picture text --> NON‐FEMININE BOYS<br>FEMININE BOYS<br>5%<br>15%<br>85% 95%<br>Accepted Refused Accepted Refused<br><!-- End of picture text --> <mark>Database: Cases with clear outcome (unclear cases omitted), n=172, 93% of schools, feminine boys n=85 (92%), non‐feminine boys n=87 (95%)</mark> <mark>The chances of a non‐feminine boy being accepted into a school without hesitation are more than twice as high (72 percent) as the chances for feminine boys (35 percent; figure 3).</mark><sup>xxii</sup> Figure 3: Feminine and non‐feminine boys accepted without hesitation <!-- Start of picture text --> FEMININE BOYS NON‐FEMININE BOYS<br>28%<br>35%<br>65% 72%<br>Accepted without hesitation<br>Accepted without hesitation<br>Accepted with hesitation<br>Accepted with hesitation<br><!-- End of picture text --> Database: Accepted boys only; n=155, 84% of schools; feminine boys n=72 (78%), non‐feminine boys n=83 (90%) 7
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA Reasons for Rejections Non‐feminine boys were only refused enrollment because the schools lacked space for new students. Although the schools used the same argument to refuse the enrollment of feminine boys their responses made it obvious that the reason for the refusal was in fact grounded in the boy’s perceived femininity.

_Our classes have 25‐26 students. I have talked to the class masters and they are not willing to accept new students. There are other schools with a smaller number of students ... So, I can’t make this possible. And another thing, we have aggressive seventh graders and I’m afraid that this wouldn’t be a good environment for your son. You know that they always look for a victim. And the victim is the one slightly different. You will surely do better if you look for a school with a smaller number of students ..._ School administrators stress the homophobic atmosphere among the students and their inability to protect feminine boys from discrimination. See for example, some quotes below.

_Given that the child has such a profile, and the children of that age in our school are cruel, I would kindly ask you to try with another school. And don’t mention that your son is feminine. You will immediately face a strong barrier. That’s what I would suggest. I know what our children are like. They tolerate differences with lots of difficulties. They actually can’t stand any differences._ _We have a lot of seventh graders and they are very problematic ... This characteristic of your boy may be a problem; perhaps children wouldn’t react to it properly. We do have lots of problems with seventh graders ... And our capacities are rather full..._

# **Reasons for Hesitation in Enrollment**

The study reveals that the boys’ perceived femininity is more problematic than low academic performance or behavioral problems. As shown above, considerably more requests for enrollment of feminine boys than non‐feminine boys were accepted with hesitation. Analyses of the interview recordings show that hesitation was in general due to the unwillingness of the schools to enroll students perceived as potential problems. But perceptions of the potential problems were strikingly different in the case of feminine and non‐feminine boys.

While in the case of non‐feminine boys, hesitations were expressed through additional questions about the boys’ school achievement and discipline, in the case of feminine boys they were related to the boys’ femininity. See for example: _I don’t know what to tell you ... Well, I’m not saying that it is a problem, the child is as he is ... But before you make the decision about the school, perhaps it would be advisable to consult professionals, to hear what they would say ... I can’t tell you anything, you can enroll your child wherever you want, in whichever school, that’s your right as a parent. But if I were you, I would first ask around, get informed, check out the classes, how the child would fit in, do_ 8
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA _you understand me? I had to say this, but if you want to enroll your child in our school, you are welcome, and that’s all._ Even when the enrollment of a feminine boy was accepted, all school administrators stressed that they could not guarantee the boy’s safety. See for example: _What you told me is so general ... of course, the child has the right to be enrolled. So, what you have told me about the child being feminine ... If you are asking me for advice ... I don’t know how visible this is on your child ... we can’t protect him from teasing and other similar things... But, of course, you can enroll him here, that’s your civil right._ _Each school can enroll one more child at least, two per class, but I don’t know what to tell you. It is your right to choose where you will enroll your child. I can’t refuse any child... But I can’t promise that other children won’t say anything, you must know how cruel children are, especially at that age, I can’t guarantee anything... the child can be enrolled, of course... But don’t expect the impossible from us, alright? That’s what I can say, and you are entitled to enroll your child wherever you want._ 9
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **2. Access to Housing for Gay and Lesbian Couples**

Access to secure and affordable housing is important for all people but is a particularly pressing issue for LGBTI people. Sexual and gender minorities are often made to feel unwelcome by their families and forced to leave home early, increasing their reliance on the private housing market. In many countries, sexual and gender minorities are overrepresented in homeless populations.<sup>xxiii</sup> Understanding LGBTI discrimination in the housing market becomes especially important considering Sustainable Development Goal 11, which requires Serbia (and all nations) to make cities and human settlements inclusive, safe, resilient, and sustainable. This experiment highlights the extent of discrimination in the private rental housing market and is designed to inform future policies that allow LGBTI people access to adequate, safe, and affordable housing.

## **2.1 Methodology**

### **Experimental scenario**

Couples contacted landlords via telephone to inquire about renting an apartment. All couples were 30 to 34 years old, and both partners were employed with a stable income. They were interested in renting apartments of approximately the same size (one‐bedroom apartments) and price. Couples only differed by their sexual orientation. Each landlord was contacted twice, once by a same‐sex couple (randomly assigned to be a gay or lesbian couple) and once by a heterosexual couple. When a lesbian couple contacted the landlord, the female member of the heterosexual couple made the call as a control for any gender bias. The same applied to the gay couple, where the male member of the heterosexual couple contacted the landlord. The scenario of the telephone conversation has three basic assumptions:

1. Sexual orientation is the only characteristic distinguishing the couples;

2. Statements about the couple’s sexual orientation are very clear, but also conveyed in a relaxed/colloquial manner; 3. The conversation should gather information that can be transformed into statistically comparable data.<sup>xxiv</sup>

### **Sampling of Landlords**

A total of 160 landlords were contacted based on their online advertisement of apartments in four cities (Belgrade, Nis, Novi Sad, and Kragujevac). Since there were two treatment groups (lesbian couple and gay couple) and two corresponding control groups (two heterosexual couples), landlords were randomly assigned to one of the treatment groups and the corresponding control group. Eighty landlords were assigned to the gay couple and the corresponding heterosexual couple, and 80 to the lesbian couple and the corresponding heterosexual couple. Since each landlord was contacted twice, there were a total of 320 observations.

10
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

## **Measurement**

Since the data were collected through telephone conversations in natural conditions, the qualitative data were transformed into quantitative outcomes. To ensure the robustness of the quantitative data, evaluation transcripts were done by three independent raters in accordance with the usual standards of evaluations of observations in natural conditions.<sup>xxv</sup> The raters did not take part in the study, do not identify as LGBTI, and did not have a position on LGBTI rights that could skew their assessment.

To evaluate the outcomes, a five‐level scale was used:

1. Accepted without hesitation or additional conditions.

2. Accepted, but with hesitation or postponement.

3. Unclear outcome (final decision not clear).

4. Rejected, but with hesitation.

5. Rejected without hesitation.

# **2.2 Results**

## **Same‐sex couples face discrimination when renting apartments**

Almost one in five (18 percent) same‐sex couples were refused rental of an apartment by the landlord; none of the heterosexual couples were. Same‐sex couples also faced significantly more hesitation (8 percent) when accepted compared to their heterosexual counterparts (1 percent; figure 4).<sup>xxvi</sup> <!-- Start of picture text --> Figure 4: Acceptance and refusal of same‐sex and heterosexual couples when seeking rental<br>apartments<br><!-- End of picture text --> <!-- Start of picture text --> SAME‐SEX COUPLES HETEROSEXUAL COUPLES<br>1%<br>11%<br>7%<br>8%<br>74%<br>99%<br>Accepted without hesitation Accepted without hesitation<br>Accepted with hesitation Accepted with hesitation<br>Refused with hesitation Refused with hesitation<br>Refused without hesitation Refused without hesitation<br><!-- End of picture text --> <mark>Database: all observations, 160 observations per treatment and control groups, 320 observations in total.</mark> 11
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **Gay couples face more discrimination than lesbian couples**

Gay couples face a much higher likelihood of being refused a rental apartment (29 percent) compared to their lesbian counterparts (8 percent; figure 5).<sup>xxvii</sup> <!-- Start of picture text --> Figure 5: Acceptance and refusal of gay and lesbian couples when seeking rental<br>apartments<br>LESBIAN COUPLE<br>GAY COUPLE<br>8%<br>29%<br>71%<br>92%<br>Accepted Refused Accepted Refused<br>Database: treatment groups only, 80 observations per group, 160 observations in total.<br><!-- End of picture text --> Gay couples also faced more hesitation than heterosexual or lesbian couples. Of the gay couples accepted, 12 percent were accepted with hesitation, while only 1 percent of the corresponding heterosexual couples experienced such hesitation (figure 6).<sup>xxviii</sup> The difference in the case of lesbian couples was smaller, and not statistically significant: 7 percent of lesbian couples were accepted with hesitation and 1 percent of the corresponding heterosexual couples.<sup>xxix</sup> Figure 6: Acceptance with hesitation—gay couples and heterosexual couples <!-- Start of picture text --> HETEROSEXUAL COUPLES<br>GAY COUPLE<br>(CONTACT MALE)<br>1%<br>12%<br>88%<br>99%<br>Accepted without hesitation<br>Accepted without hesitation<br>Accepted with hesitation<br>Accepted with hesitation<br><!-- End of picture text --> <mark>Database: accepted couples only; 100% of heterosexual couples, 71% of gay couples</mark> 12
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **Male landlords discriminate more against gay couples than female landlords**

Male landlords rejected gay couples significantly more often than lesbian couples. As much as 36 percent of male landlords refused gay couples, while only 7 percent refused lesbian couples.<sup>xxx</sup> On the other hand, female landlords rejected gay and lesbian couples to the same extent.<sup>xxxi</sup> Male landlords rejected lesbian couples at similar levels of female landlords (figure 7).

Figure 7: Acceptance and refusal of lesbian and gay couples by female and male landlords <!-- Start of picture text --> 100% 93% 92%<br>90%<br>79%<br>80%<br>70% 64%<br>60%<br>50%<br>36%<br>40%<br>30% 21%<br>20%<br>7% 8%<br>10%<br>0%<br>Gay couples Lesbian couples<br>Accepted (male landlord) Refused (male landlord) Accepted (female landlord) Refused (female landlord)<br><!-- End of picture text --> Database: 86 male landlords—42 observations with gay couples and 44 with lesbian couples; 74 Female landlords—38 observations with gay couples and 36 with lesbian couples.

# **Homophobia was Openly Expressed by Many Landlords**

Analysis of the interview transcripts shows open homophobia by landlords. Once informed that the potential tenants were a gay or a lesbian couple, the landlords reacted in one of two ways.

One group of landlords immediately stated that they were not willing to rent the apartment to a gay or lesbian couple: _Two men, what do you mean a couple, what kind of couple? You mean two men in emotional relationship? Oh no, no! Goodbye._ _Oooooh, well, I have never rented a flat to a male couple, or something like that, they were always a young couple or married couple. I am sorry, but not me, not like that._ _Well I don’t know, I wouldn’t really want to. I have nothing against it, but I don’t want to._ _What can I say, it is a problem. (Could we see the apartment?) No. Bye._ 13
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA The other group stressed the negative reactions of other people—either their families or neighbors in the building in which the apartment was located—as a reason for the refusal.

_My grandmother wouldn’t like it. You know, we have children, so ... you understand._ _My son‐in‐law owns that apartment; I must ask him. Call me in half an hour, please. (Second call) Yes, I have asked him, and he doesn’t approve. (And what are the reasons, what does he say?) He doesn’t want to, as simple as that, he didn’t say why that’s all._ _I don’t know what to say, it’s not a problem for me, but it may be a problem for the neighbours and other tenants, I can’t agree to this._ _Well, I don’t know, I have to check with my mother and then call you back. (Second call) Well, my mother doesn’t really approve. Goodbye._ _Well, I am not sure, I must ask my husband and my children. Please call me in the evening, I must talk with them first ... I have no problem with that, but my husband is a bit conservative. (Second call) The flat has been rented, goodbye._ 14
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **3. Conclusion**

## **Avenues for Further Research**

Collecting data on the experiences of LGBTI people can be challenging. This report contributes to closing the LGBTI data gap by using innovative approaches to measure LGBTI discrimination and exclusion. Although mystery shopping experiments are not new, their use in understanding and revealing LGBTI discrimination has been limited. Previous reports indicated that LGBTI people face barriers in accessing education and housing in Serbia, but there was insufficient robust data to support this. This study is at the forefront of obtaining reliable and quantifiable data on the challenges LGB people face in accessing housing and education. It underlines the applicability and efficacy of mystery shopping experiments in shining a light on LGBTI discrimination and serves as a pilot for future studies.

However, experimental studies have their limitations, and further research is needed to understand the full extent of LGBTI discrimination and exclusion in Serbia and beyond. The difficulties faced by LGBTI people on a daily basis are pervasive, numerous, and cross‐cutting. The World Bank uses a life cycle approach to understand the full extent of discrimination, and in turn, properly address these challenges; additional research that identifies the challenges and their social, economic, and mental effects on LGBTI people is required. Discrimination and exclusion often begins in the family; LGBTI children fear violence by parents, siblings, and extended family members or are even kicked out of their home at an early age. Data on these early experiences of LGBTI children is extremely rare, but in order to fully understand the effects of being LGBTI, more systematic research in this area is needed.

Further, some of the responses by school authorities indicated that barriers in accessing education are not the only ways that LGBTI discrimination manifests in schools. UNICEF has begun to develop methodologies to understand the extent and the consequences of homophobic bullying in schools. Similar research is necessary to understand how LGBTI students can be supported to develop to their full potential in Serbian schools.<sup>xxxii</sup> Following the life cycle approach, finding employment and developing a career is a next critical step that enables the accumulation of income and wealth. Providing policy makers with robust and reliable data on the challenges LGBTI individuals face in finding employment, staying employed, and progressing in their careers will be key to enable LGBTI people to live life to their full potential. Related to this, other research topics should include access to credit and banking facilities, insurance, holding public office, and the accumulation of assets.

Research on discrimination in a range of other services, markets, and spaces is also necessary. Because of discrimination experienced within the family, many LGBTI people don’t have the informal safety net of family ties to rely upon at key stages in the life cycle. Unemployment, illness, and retirement can therefore be particularly challenging for LGBTI people, and more research is needed to ensure they do not fall through the cracks at these stages of life. Data on elderly LGBTI people is essentially nonexistent and highlights just one of the many areas where further research is needed. Other aspects of family life where LGBTI experience institutional and individual discrimination with economic consequences include coverage of 15
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA family health insurance and health care services, so more robust data in this sphere would be welcome.

Such further research should be developed in close consultation with the LGBTI community. This helps to ensure that the findings are applicable and relevant to these communities. It is also more direct and sustainable because it brings LGBTI people on board in a way that allows policymakers, development partners, donors, and advocates to respond directly to their needs and development challenges. For example, existing research reveals that trans and intersex people are the most vulnerable among LGBTI groups. However, they are often overlooked in research projects, and specific efforts should be made to collect data that distinguishes and quantifies the particular challenges they face.

# **Policy Considerations**

<mark>The experiments outlined in this study reveal that more could be done to close the implementation gaps in Serbian anti‐discrimination law and policy, and to support the implementation of the “2013‐2018 Serbian Anti‐Discrimination Strategy.” An antidiscrimination law passed in 2009 prohibits discrimination on the grounds of sexual orientation and gender identity, and the Anti‐Discrimination Strategy makes clear that “the right to education must be effectively enjoyed without discrimination on the grounds of sexual orientation or gender identity.”</mark><sup>xxxiii</sup> <mark>The strategy also creates an obligation for schools to ensure a safe learning environment.</mark><sup>xxxiv</sup> <mark>The following actions would be helpful in this regard.</mark>

1. An effective complaints and reporting mechanism should be put into place to ensure that Serbian antidiscrimination laws are implemented and adhered to. To this end, it is recommended that the Protector of Citizens, Serbia’s Ombudsperson, assesses its reporting mechanisms to identify potential barriers that prevent or discourage LGBTI people from reporting cases of discrimination. Together with the Ombudsperson, the Education Inspectorate in consultation with relevant stakeholders should specifically ensure that reporting mechanisms provide students, parents, and teachers with safe ways to report cases, ensure fair and transparent investigations, and offer meaningful solutions.

2. A comprehensive assessment should be conducted to identify and examine gaps in legislation and regulations and their implementation, as well as grievance mechanisms and resources available to LGBTI people when they experience discrimination. Specifically, a review of the interpretation and application of school regulations is recommended to identify and examine implementation gaps in the education sector.

3. School development plans should include specific actions to create a respectful, safe, and supportive school environment for all students, with special attention to LGBTI students. The Education Inspectorate should be empowered to effectively monitor the implementation of these actions and provide guidance where needed. Comprehensive training of school authorities, teachers, and students should also be carried out to support the establishment of a safe and conducive school environment for LGBTI students.

4. Greater advocacy against homo‐, bi‐, trans‐, and intersex phobia also appears to be necessary. Discrimination based on sexual orientation is widely accepted in Serbia, as 16
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA was evidenced by the open homophobia displayed by landlords. Advocacy should include, but not be limited to, raising awareness among the general public of the adverse effects of, and problems associated with, current discrimination against LGBTI people.

Lastly, it is important to leverage existing positive attitudes towards LGBTI people in awareness‐raising campaigns and in policy considerations. Although evidence from the study makes it clear that discrimination based on sexual orientation occurs in Serbia, and occurs frequently, the variance in the responses also reveals that there is a degree of acceptance of LGBTI people. For example, 35 percent of the feminine‐boys were accepted without hesitation, and 74 percent of same sex couples were also accepted without hesitation. Ideally, there should be no discrimination at all, but the responses suggest that there are those who do not discriminate against LGBTI people. These positive attitudes should be harnessed as an opportunity to influence the attitudes of others. Correctly and consistently implementing policies and laws that forbid discrimination based on sexual orientation can also help to raise awareness and change attitudes.

17
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **ANNEX I: Detailed Methodology**

The Annexes provide further detail (to that provided above) on the methodologies adopted for the experiments.

## **Education Experiment**

### **Independent (treatment) and dependent variables**

The treatment variable (independent variable) in this experiment is the presence or absence of femininity among 14 years old boys, applying to be enrolled in the 7<sup>th</sup> grade of public primary schools in Serbia. The dependent variable of the experiment is the outcome of the request for enrolment in the new school.

Primary schools were selected as the most suitable for two reasons: first, primary education (8 years) is compulsory and free of charge; second, it is common practice in Serbia for parents to contact the schools directly with questions regarding their children, thus limiting the experimental scenario to adults (more details below, in the section on data collection).

Grade seven was chosen because boys are old enough (age 14) for their femininity to be perceived as a distinctive characteristic. The experiment was conducted at the beginning of the second semester of the school year, leaving three more semesters to complete primary school. If we had opted for a boy in grade eight, it would only leave one more semester until the completion of primary school, which might have influenced the enrolment decision.

### **Data collection method**

The schools were contacted by purported mothers on behalf of the boys. In Serbian culture, the mother as caretaker is generally the one who contacts the children’s school. Also, Serbian men are less likely to talk about their son’s femininity with the school authorities.

The role of the mothers was assigned to six members of the Ipsos research team with extensive experience in conducting ‘mystery shopper’ surveys. The ‘mystery shoppers’ attended additional training prior to participating in the experiment.

Data collection was from 10 to 24 January 2017.

### **Scenario of the telephone conversation**

The telephone conversation scenario was as follows:

- The mother mentions the femininity of her son at the beginning of the interview before the school administrator has the chance to make any statement about the availability of places at the school. If the school administrator said that there was a place in the school before he/she was informed that the boy is feminine, it might be harder to change the statement after the mother gives this information.

- As a reason for mentioning this characteristic, the mother states that the femininity of the boy is quite obvious and she wants to draw the school’s attention to this fact immediately.

18
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

- To support the credibility of the boy’s femininity, the mother mentions that in his current school other students occasionally tease him, but she stresses that he never had any real problems. The mother mentions that the boy has no problems in interacting with other students due to his femininity.

- The family is relocating to the town where the school is located because of the father’s new job. This eliminates any doubt that the reason for transferring the son might be related to problems the boy faced in school. It also explains why the mother and the boy cannot have a face‐to‐face meeting with the school authorities.

- When the family moves, they will be living within the school’s territory.

- The student has no learning problems. In terms of academic achievement, the student has “very good” performance which is equivalent to a B. Furthermore, this does not indicate any particularly positive effects, that might be synonymous with an “excellent” grade, or any negative effects associated with anything lower than “very good” performance.

- The student has no problems with discipline.

The scenario of the telephone conversation for the control group was the same except that the boy was not characterized as feminine.

**Sampling of schools, number of observations and assignment of schools and ‘mystery shoppers’ to treatment and control groups**

The sampling frame (i.e. the universe of schools) consisted of all the primary schools located in urban municipalities with at least five primary schools. Cities in municipalities with fewer than 5 primary schools and rural (agricultural) communities were excluded from the sampling frame. Moving to very small cities and/or rural areas in Serbia is quite unusual (due to high unemployment rates, poor infrastructure, and other factors). Such a scenario could have aroused suspicion among school authorities, and shifted the conversation to other topics (Why is the family moving to this small place? What kind of job did the father get? etc.), which might have jeopardized the interview. In addition, if there are just one or two schools in the area (which is common in non‐urban areas), the school authority might feel pressured to accept the enrolment of the feminine boy, given that primary education is mandatory until the completion of eighth grade.

From the sampling frame, 184 schools were selected. Approximately the same number of schools were allocated in the three regions: Belgrade, Vojvodina, and Central Serbia. The schools were selected using standard random selection procedures (with probability proportional to the number of schools in a municipality). The schools were randomly assigned to treatment and control groups so that in each municipality there was an equal number of schools in both groups. The number of schools by region and the number of municipalities in which the schools are located are shown in Table 1.1 below.

19
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA Table 1.1: Sample size of schools per region and number of municipalities per region

|**Region**|**Sample 1: Treatment**<br>**Schools (No. of**<br>**municipalities)**|**Sample 2: Control**<br>**Schools (No. of**<br>**municipalities)**|**Total**|
|---|---|---|---|
|Belgrade|31 (11)|31 (11)|62 (11)|
|Vojvodina|31 (8)|31 (8)|62 (8)|
|Central Serbia|30 (18)|30 (18)|60 (18)|
|**Total**|**92 (37)**|**92 (37)**|**184 (37)**|

In total, 6 ‘mystery shoppers’ were engaged. Mystery shoppers were randomly assigned to an approximately equal number of schools in each region so that each ‘mystery shopper’ contacted an equal number of treatment and control schools.

# **Measurement**

Since data were collected through telephone conversations in natural conditions, the qualitative data had to be transformed to quantitative measures. To guarantee the robustness of the quantitative data, the evaluation of the outcomes was performed by independent raters. Since Serbian law prohibits the recording of telephone conversations, the team hired associates who listened to the telephone conversations and wrote them down verbatim but without recording the schools’ names, or any other information which might reveal the schools’ identity. The transcripts were then analyzed by the independent raters.

The instructions and explanations given to the raters were limited to what was necessary for the completion of their task. The raters conducted the evaluations on Ipsos premises and were seated so that they could not interact among themselves.

# **Rater reliability**

The agreement on the outcomes evaluations among the three raters was found to be quite satisfactory. The inter‐rater reliability expressed by the intraclass correlation coefficient (ICC)<sup>1</sup> was as high as 0.89<sup>2</sup> . Since the inter‐rater reliability was satisfactory, the average ratings of the three raters were used as the final measures of the outcomes.

> 1 Model used for assessing the rater reliability: Shrout, P.E and Fleiss, J.L. Intraclass Correlations: Uses in assessing rater reliability, _Psychological Bulletin_ , 1979, Vol. 86, No. 2, 420‐428.

> 2 ANOVA results based on which ICC was calculated

|**Sources of variance**|**df**|**MS**|
|---|---|---|
|Between raters|2|0.36|
|Between schools|183|2.93|
|Residual+error|366|0.11|

20
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **Renting Experiment**

## **Independent (treatment) and dependent variables**

Sexual orientation was chosen as the treatment variable (independent variable) for the experiment because compared to other parts of the LGBTI community, lesbian women and gay men are the most visible in Serbia. As such, open expression of sexual orientation was not expected to make a landlord suspicious of the genuineness of the rental inquiry. The dependent variable is the readiness of the landlord to rent the apartment to the couple.

## **Experimental design**

The experiment had two treatment groups, a lesbian and a gay couple, and two corresponding control groups of heterosexual couples. Landlords were randomly assigned to either a gay or lesbian couple and a corresponding heterosexual couple. Each landlord was contacted twice – by the same‐sex couple (treatment group) and by a corresponding heterosexual couple (control group). When the lesbian couple contacted the landlord, the female member of the heterosexual couple would be the one contacting the same landlord as a control for any gender bias. The same applies to the gay couple, where the male member of the heterosexual couple contacted the landlord.

## **Data collection method**

Communication with the landlords was conducted by telephone. Members of the Ipsos research team who have vast experience in conducting ‘mystery shopper’ surveys conducted the interviews. The ‘mystery shoppers’ attended an additional training prior to participating in the experiment.

Data collection was conducted in the period from 10 to 22 February 2017.

## **Scenario of the telephone conversation**

The telephone conversation scenario was as follows:

- At the beginning of the conversation, to ensure naturalness and convey a serious interest in renting the apartment, the mystery shopper asks for information that was referred to in the advertisement (e.g. Internet connection, cable television, parking space, how many floors in the building, payment details, etc.).

- Thereafter, the mystery shopper states that the terms are acceptable and says that there will be two tenants in the apartment. In the control group, the two people are a heterosexual couple. In the treatment group, the two people are either two men or two women in a relationship and living together. The mystery shopper then asks when they could come to see the apartment.

- If the answer is positive, the mystery shopper says that he/she will decide with his/her partner and call back to schedule the time to inspect the apartment. If the answer is negative, the mystery shopper kindly asks about the reason for rejection, and without further discussion, he/she thanks the landlord and concludes the conversation.

21
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA If the landlord asked additional questions about the age and/or income of potential tenants the mystery shoppers in both experimental and control groups gave the same answers: the couples are between 30 and 34 years old, employed, and they have a regular income.

**Sampling of landlords, number of observations and assignment of landlords and ‘mystery shoppers’ to treatment and control groups**

The sampling frame included landlords who advertised apartments for rent directly (i.e. not through an agency). As landlords are the final decision makers regarding the rentals, surveying agencies would have required a different, and more complex experimental scenario.

The landlords’ advertisements, were randomly selected from apartment rental websites in four cities (Belgrade, Nis, Novi Sad and Kragujevac). In all these cities, there is a sizable rental market and the practice of renting apartments is widespread enough to fit the purpose of the experiment. The apartment rental websites in each of the cities were visited every other day during the realization of the experiment. Thus, the sample frame was refreshed with new advertisements, while the landlords who were already contacted were excluded.

Since there were two treatment groups (lesbian couples and gay couples) and two corresponding control groups (two heterosexual couples) landlords in each city were randomly assigned to one of the treatment groups and corresponding control group, so that two independent samples of landlords were selected. In total 160 landlords were contacted: 80 were assigned to the group of gay couples (treatment) and corresponding heterosexual couples (control), and 80 to the group of lesbian couples (treatment) and corresponding heterosexual couples (control). Since each landlord was contacted twice (by the experimental and the control group), in total there were 320 observations. The sample size of landlords and the number of observations are presented in Table 2.1.1 below.

Table 2.1.1: SAMPLE SIZE OF LANDLORDS AND NUMBER OF OBSERVATIONS

|**LANDLORDS**|**Sample size=80**|**Sample size =80**|
|---|---|---|
||**No. of observations**|**No. of observations**|
|Treatment – lesbian couple|80|/|
|Control – heterosexual couple, female gives a call|80|/|
|Treatment – gay couple|/|80|
|Control – heterosexual couple, male gives a call|/|80|
|Total No of observation per sample|160|160|
|**Total no. of observations**||**320**|

The sample size disaggregated by the gender of the landlords as well as by city are presented in the Tables 2.1.2 and 2.1.3 below.

22
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA Table 2.1.2: SAMPLE SIZE OF LANDLORDS BY GENDER

|**LANDLORDS**|**Male**|**Female**|**Total**|
|---|---|---|---|
|Lesbian couple and corresponding control|42|38|80|
|Gay couple and corresponding control|44|36|80|
|**Total**|**86**|**74**|**160**|

Table 2.1.3: ALLOCATION OF LANDLORDS AND NUMBER OF OBSERVATIONS PER CITY

|**City**|**Sample size of**<br>**landlords**|**No. of observations**|
|---|---|---|
|Belgrade|70|140|
|Nis|30|60|
|Novi Sad|30|60|
|Kragujevac|30|60|
|**Total**|**160**|**320**|

In total, 8 mystery shoppers participated in the experiment, 4 males, and 4 females. Thus, there were 4 pairs of “mystery shoppers”, two male pairs (who contacted the landlords in the role of gay couples and corresponding heterosexual couples), and two female pairs (who contacted the landlords in the role of lesbian couples and corresponding heterosexual couples). “Mystery shoppers” within the pairs switched roles so that each of them called an equal number of landlords in a role of same‐sex couple representative and heterosexual couple representative.

# **Measurement**

Since data were collected through telephone conversations in natural conditions, the qualitative data had to be transformed to quantitative measures. To guarantee the robustness of the quantitative data, the evaluation of the outcomes was performed by independent raters. Since Serbian law prohibits the recording of telephone conversations, the team hired associates who listened to the telephone conversations and wrote them down verbatim but without recording the landlords’ names, or any other information which might reveal the landlords’ identity. The transcripts were then analyzed by the independent raters.

The instructions and explanations given to the raters were limited to what was necessary for the completion of their task. The raters conducted the evaluations on Ipsos premises and were seated so that they could not interact among themselves.

23
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA

# **Rater reliability**

The agreement of the evaluations of the outcomes among the three raters was found to be quite satisfactory. The inter‐rater reliability expressed by the intraclass correlation coefficient (ICC)<sup>3</sup> was as high as 0.91.<sup>4</sup> Since the inter‐rater reliability was high, the average ratings of the three raters were used as the final measures of the outcomes.

i Lee Badgett, Laura Durso, and Alyssa Schneebaum, “New Patterns of Poverty in the Lesbian, Gay, and Bisexual ‐ ‐ ‐ ‐ Community,” Williams Institute, 2013, https://williamsinstitute.law.ucla.edu/wp <u>content/uploads/LGB Poverty Update Jun‐2013.pdf.</u> Dominik Koehler, “LGBTI People Are (Likely) Over Represented in the Bottom 40%,” World Bank Group SOGI Task Force, 2015, http://blogs.worldbank.org/governance/lgbti‐people‐are‐likely‐over‐represented‐bottom‐40. ii Fundamental Rights Agency, “EU LGBT Survey Technical Report,” European Union Agency for Fundamental Rights, Vienna, 2012, <u>http://fra.europa.eu/sites/default/files/eu‐lgbt‐survey‐technical‐report_en.pdf.</u> iii Philip Oreopoulos, ‘‘Why Do Skilled Immigrants Struggle in the Labor Market? A Field Experiment With Thirteen Thousand Resumes,” _American Economic Journal: Economic Policy_ 3, no. 4 (2011): 148– 171, <u>https://www.aeaweb.org/articles?id=10.1257/pol.3.4.148.</u> iv Katrin Auspurg, Thomas Hinz, and Laura Schmid, “Contexts and Conditions of Ethnic Discrimination: Evidence From a Field Experiment in a German Housing Market,” _Journal of Housing Economics_ 35 (2017): 26‐36, <u>https://www.sciencedirect.com/science/article/pii/S1051137717300037.</u> v Devah Pager and Hana Shepherd, “The Sociology of Discrimination: Racial Discrimination in Employment, Housing, Credit, and Consumer Markets,” _Annual Review of Sociology_ 34 (2008): 181‐209, <u>http://www.annualreviews.org/doi/abs/10.1146/annurev.soc.33.040406.131740.</u> vi András Tilcsik, “Pride and Prejudice: Employment Discrimination against Openly Gay Men in the United States,” _American Journal of Sociology_ 117, no. 2 (2011): 586‐626, https://www.ncbi.nlm.nih.gov/pubmed/22268247. vii United Nations Development Programme (UNDP), “Being LGBTI in Eastern Europe: Serbia Country Report,” UNDP, 2017, <u>http://www.eurasia.undp.org/content/dam/rbec/docs/undp‐rbec‐blee‐SERBIA%20.pdf.</u> viii ‐ European Commission, “Serbia 2016 Report,” Brussels, 2016, 3, https://ec.europa.eu/neighbourhood <u>enlargement/sites/near/files/pdf/key_documents/2016/20161109_report_serbia.pdf.</u> ix UNDP, “Being LGBTI in Eastern Europe.” x Cisgender refers to people whose identity and gender corresponds with their birth sex, i.e., who are not transgender. xi Martijn Hesselink and Ton van der Wiele, “Mystery Shopping: In‐Depth Measurement of Customer Satisfaction,” 2011, <u>http://www.mvmgroup.rs/wp‐content/uploads/2011/04/Importance‐of‐mistery‐shopping.pdf.</u> xii For example, in the SILC survey, (World Bank forthcoming), only 17 percent of LGBTI respondents stated that they were open about their sexual orientation at work in the last five years.

xiii For example, qualitative survey focus groups in Belgrade, Nis, and Novi Sad showed that in the acronym LGBT, most of the participants were familiar only with the meaning of the letters L (lesbian) and G (gay). The result was confirmed in the quantitative survey, showing that majority of the people in Serbia relate LGBT population to gay, lesbian, and bisexual people, while the awareness of transgender and intersex people is quite low. National Democratic Institute (NDI), “LGBTI Public Opinion Poll: Western Balkans,” 2015, https://www.ndi.org/LGBTI_Balkans_poll.

xiv UNESCO, “From Insult to Inclusion: Asia‐Pacific Report on Bullying, Violence and Discrimination on the Basis of Sexual Orientation and Gender Identity,” Bangkok, 2015, http://unesdoc.unesco.org/images/0023/002354/235414e.pdf. xv Mahidol University, Plan International Thailand, and UNESCO Bangkok Office, “Bullying Targeting Secondary School Students Who Are or Are Perceived to Be Transgender or Same‐Sex Attracted: Types, Prevalence, Impact, Motivation and Preventive Measures in 5 Provinces of Thailand,” Bangkok, 2014, <u>http://unesdoc.unesco.org/images/0022/002275/227518e.pdf.</u> 3 Model used for assessing the rater reliability: Shrout, P.E and Fleiss, J.L. Intraclass Correlations: Uses in assessing rater reliability, _Psychological Bulletin_ , 1979, Vol. 86, No. 2, 420‐428.

4 ANOVA results based on which the ICC was calculated

|**Sources of variance**|**df**|**MS**|
|---|---|---|
|Between raters|2|0.35|
|Between<br>landlords|319|1.70|
|(observations)|||
|Residual+error|638|0.04|

24
DISCRIMINATION AGAINST SEXUAL MINORITIES IN EDUCATION AND HOUSING: EVIDENCE FROM TWO FIELD EXPERIMENTS IN SERBIA xvi A detailed description of the methodology can be found in Annex, available upon request. xvii Telephone conversations included in Annex, available upon request xviii Based on the definition by the Serbian Government. For further details see Annex, available upon request. xix <mark>The results did not vary systematically depending on geographic region, municipality size (expressed by the number of primary schools), or school location (on the outskirts or in the town center); hence these variables are not reported.</mark> xx For example: Kilem Gwet, Handbook of Inter‐Rater Reliability _: The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters_ , 4th ed., (Gaithersburg: Advanced Analytics, LLC, 2014), http://www.agreestat.com/book4/. xxi Pearson Chi‐Square = 5.52, df = 1, p < 0.05. xxii Pearson Chi‐Square = 21.97, df = 1, p < 0.001. xxiii Laura Durso and Gary Gates, “Serving Our Youth: Findings from a National Survey of Service Providers Working with Lesbian, Gay, Bisexual and Transgender Youth Who Are Homeless or At Risk of Becoming Homeless,” Los Angeles, The Williams Institute with True Colors Fund and The Palette Fund, 2012, https://williamsinstitute.law.ucla.edu/research/safe‐ schools‐and‐youth/serving‐our‐youth‐july‐2012/. xxiv See the Annex for a fuller description of the scenario. xxv Kilem Gwet, _Handbook of Inter‐Rater Reliability._ xxvi Pearson Chi‐Square = 41.63, df = 3, p < 0.001. xxvii Pearson Chi‐Square = 12.172, df = 1, p < 0.001 . xxviii Gay and corresponding heterosexual group: Pearson Chi‐Square = 7.36, df = 1, p < 0.01. xxix Lesbian and corresponding heterosexual group: Pearson Chi‐Square = 3.11, df = 1, p = 0.08. xxx Pearson Chi‐Square = 2.36, df = 1, p = 0.001.

xxxi In spite of the fact that the difference in the percentage of refusals of gay and lesbian couples might seem to be different, the statistical test shows that difference is not systematic. Pearson Chi‐Square = 10.84, df = 1, p = 0.12. 32 UNESCO, “From Insult to Inclusion.” 33 The Government of the Republic of Serbia, “Anti‐Discrimination Strategy 2013‐2018” (unofficial translation), 44, <u>https://rm.coe.int/16801e8db9.</u>

> 34 “ _Specifically ensure protection of the right of children and youth to education in a safe environment, without violence, harassment, social exclusion or other forms of discriminatory or degrading treatment based on sexual orientation or gender identity._ ” Government of the Republic of Serbia, “Anti‐Discrimination Strategy,” 47.

25