Surveying elites about global governance: challenges and solutions
By: International Affairs
Tags: Surveys, Methodology, Author Views, Research, International Relations

Contemporary society confronts major global transformations and governance challenges, including pandemics, ecological changes, economic restructuring, financial crises, migration, peacebuilding, cybersecurity and so on. Yet, we are commonly told, current prospects for global governance are grim, hampered by institutional shortcomings, dissatisfaction among emerging powers, the rise of economic nationalism and rejection of globalism by populists. But what is the actual situation, if we consider the attitudes of the people who most directly influence whether and how global governance happens, namely elites? Our recent article in International Affairs sought to find out.

We chose to study elites — defined as people in leading positions in key organizations in society that strive to be politically influential — because systematic research on elite views of global governance institutions (GGIs) is lacking. We concluded that it is not a lack of interest in the topic, but a lack of appropriate data that causes this gap in the literature.

We attempted to fill this gap by conducting a large-scale survey exercise with 860 individuals from six elite sectors (researchers, civil society, bureaucrats, media, business elites, and elites in political parties) in six countries (Brazil, Germany, the Philippines, Russia, South Africa and the United States), as well as an international group. To explore our findings, we recommend that you read the article itself.

In this blogpost, we will share the challenges and solutions we discovered in carrying out this survey, in the hope that sharing our experiences will aid future research.

Issues with the target audience

Our central research focus was to understand how elites view global governance. Therefore, the target population included both political and societal elites, situated at any governance level. Indeed, as we wanted to evaluate what governance level they deem most appropriate to deal with issues (international, regional, national etc.), we did not want to limit our study to the people most involved in global governance, as this may lead to a biased picture.

As a consequence, some prospective respondents were hesitant to participate as they felt they weren’t experts on ‘international issues’ (as we described the topic of our study in our invitations). At times, prospective respondents inquired about whether we wanted to talk with their colleague working on internationalization instead. This experience demonstrates the importance of explaining the rationale of the sampling strategy to prospective respondents. It also highlights the risk of composing a biased sample when one is not sufficiently attentive to this issue of self-selection in (non-)response.

Organizational challenges

Our goal was to run the survey in parallel in six countries around the world, and in international organizations, which posed a set of challenges. We organized the research centrally to guarantee comparable data. However, to cater to local customs, language, and time zones we created a large research team with local partners. This was particularly challenging in (semi-) authoritarian states where local partners were harder to find.

Linguistic and cultural challenges

As is the case in most international survey research, much effort went into translating survey questions. More complex issues arose later, such as when searching for the most effective communication methods. Our standard invitation procedure consisted of emails and follow-up calls to prospective respondents’ offices. Yet, our South African interviewers noticed that in certain cases text messages were more effective, while in the Philippines, social media proved useful.

The message in communications also benefited from adaptations in some instances. The standard invitation included information on the study, the research team, the interview and confidentiality. Yet, adaptations were required in the US context, where mentioning the (financial) size of our research project raised the legitimacy of our request for some groups, while in other contexts mentioning this would have been regarded as out of place.

We also observed stark differences in responses to our invitation. In the global sample, Germany, Brazil and South Africa, the response rate (i.e. the proportion of the invited people that actually took the survey) was very similar (ranging between 35.0% and 40.5%). Yet, the refusal rate (i.e. the proportion of the invited people that explicitly refused participation) was much higher in the global and German samples (resp. 32.4% and 31.3%) than in South Africa (22.6%) and Brazil (12.5%). It seems that in some contexts, openly refusing an invitation to a survey is a more obvious choice than continuously delaying the interview.

Finally, we observed a high variation in response rates between elites in different sectors, as shown by the figure below. Researchers, civil society, bureaucrats and permanent officials of GGIs were clearly more willing to take the survey than national representatives at GGIs, media, business elites, and elites in political parties. It is somewhat surprising that the two categories that have most to do with GGIs and thus the topic of our survey (their own staff and national representatives at GGIs) do not rank first.

Challenges when interviewing

A third challenge was to survey people in leading positions in a standardized fashion. Given their expertise, respondents were often more used to in-depth interviews, where they can fully elaborate on their experiences and opinions. Yet, given our goal to map general patterns and compare large data sets we needed to ask standardized questions with standardized response options.

We did two things to attenuate respondent’s frustration. First, as the preferred survey mode was telephone interviews (used in 81.5% of the interviews), interviewers had to strike a balance between showing interest in respondents’ stories and getting through the list of about 50 questions. Second, the survey included an open question, where respondents had the opportunity to explain their view on the legitimacy of GGIs in great detail. Including this open question provided respondents with the space they may have longed for, and provides more detailed information in a standardized survey format.

Conclusion

In the data collection process, we were met by many challenges regarding sampling, response, and survey formulation. Previous research on surveying and elites, networks of helpful colleagues around the world, and generous funding to support this data collection made it possible to sketch a first picture of international elites’ attitudes toward global governance. Our original dataset sheds new light on issues often discussed by academics and practitioners that have so far lacked the data to test commonly held assumptions. Our experiences should help anyone planning to do large scale international research, or surveys with those in high ranked positions. While we are making important steps forward, we hope that sharing our practical experiences will help other researchers to continue where we left off and move beyond our limitations.