Future EarthRisks Perceptions Report 2020
Appendix – Methodology & Demographics

Report TeamDr. Sylvia Wood
Dr. Jennifer Garard
Dr. Amy Luers
Dr. Kalpana Chaudhari
Dr. Matthias Garschagen
Dr. Maria Ivanova
Dr. Markus Reichstein
Dr. Qian Ye

Table of Contents
SURVEY METHODOLOGY .............................................................................................................................. 2

1.

TOP GLOBAL RISKS ............................................................................................................................................ 2
1.1.1
Analysis of likelihood and impact responses ......................................................................................... 3
1.1.2
Analysis of robustness of knowledge and assessment confidence .................................................... 3
1.2 INTERCONNECTED RISKS ......................................................................................................................................... 4
1.3 COMMITTED RISKS ................................................................................................................................................... 41.4 ADDITIONAL AND EMERGING RISKS ........................................................................................................................ 41.1

SURVEY RESPONDENTS ............................................................................................................................... 52.
2.12.23.

RESPONSE RATE AND GEOGRAPHIC DISTRIBUTION ................................................................................................ 5SCIENTIFIC BACKGROUND, EXPERIENCE AND INSTITUTIONAL SETTING ................................................................. 5REFERENCES................................................................................................................................................... 7LogoName11. Survey Methodology
The Global Risks Scientists’ Perception survey was developed by a team at Future Earth under the guidance of an
external Scientific Advisory Committee. The survey instrument was built using the SurveyMonkey platform.
Respondents were asked their perception on four broad topics of risk: i) the top global risks, ii) interconnections
amongst risks, iii) committed risks, and iv) additional or emerging risks. The survey was open from September 18th
2019 through Oct 14th 2019.
The survey was disseminated to the global change scientific community through a combination of direct emails tonetworks within the Future Earth community: Global Research Projects, Knowledge Action Networks, regional andnational committees, and to its Scientific Steering Committee, Academic Council and Governing Council and strategic
partners. Links to the survey were also posted to social media outlets (Future Earth website, Twitter, Facebook,LinkedIn), in scientific community listservs (Ecological Society of America’s listserv ECOLOG-L, International Institutefor Sustainable Development (IISD), Conference of Latin Association of Geography (CLAG)) and to individuals withinorganizations. All recipients were encouraged to share it further within their own networks for broader outreach.
In total 290 people completed at least one question of the survey which could be used in the analysis, but only 250
provided demographic data. To ensure that the perceptions we captured were of scientists, we used only responses
from individuals with at least a Master’s degree and > 1 year of experience. After filtering the data based on thesecriteria, we had n = 222 useable responses from scientists. All analyses were carried out in R-project base software
version 3.1.6.

1.1

Top global risks

We asked responded to evaluate their perception of the likelihood and impact of potential global risk events in thecoming decade. For comparability we used the list of the top 30 global risks and risk categories from the World
Economic Forum’s Global Risks Report 2019 (WEF 2019a) as a starting point for the survey (see Table 1, report).
We used the same definitions of ‘Global Risk’, ‘Likelihood’ and ‘Impact’ as the Global Risks Report 2019 forcomparability. Respondents were asked to assess the likelihood and global impact of each global risk on a Likert
scale for the following questions. Respondents could also choose ‘I don’t know’ if they felt unable to provide an
informed answer or could leave the entire question blank.
‘What is the potential negative impact from adverse consequences of each risk (for several countries or industries) if itwere to occur in the next 10 years?’‘What is the likelihood of each risk occurring globally (with impacts across multiple countries) within the next 10years?”
Table 1. Likert scale provided to respondents and corresponding numeric value for questions on the likelihood andimpact of global risks over the next 10 years.
LikertNegative
Likelihood
scale
impact1
Insignificant
Very unlikely
2
MinorUnlikely
3
Moderate
Somewhat likely
4MajorLikely5SevereVery likely
6
I don’t know
I don’t know2At the outset of the survey, respondents were asked to select the risk category on which they felt mostknowledgeable, i.e. environmental, societal, geopolitical, technological or economic. In addition to the above questions
on likelihood and impact, respondents were also asked to evaluate two additional dimensions of risk for global risks
under their selected risk category using the following Likert scales:i)What is the robustness of the scientific knowledge base round each risk [Very low to Very high] and,ii)

What is your confidence in the evaluation of each risk [Not confident to Extremely confident]

Table 2. Likert scale provided to respondents and corresponding numeric value for questions on the robustness of theknowledge base around risks and personal confidence in evaluation of risks
LikertRobustness of scientificConfidence
scaleknowledge base1
Very low
Not confident
2
Low
Slightly confident
3
Moderate
Somewhat confident
4
HighVery confident
5
Very highExtremely confident
6
I don’t knowI don’t know

1.1.1 Analysis of likelihood and impact responses
Only responses from respondents who evaluated both the impact and likelihood for at least one risk (where theanswer “I don’t know” is considered valid) were retained and used to compute results for this section of the survey (n =221). The retained responses were converted from Likert responses into a 1 to 5 numeric scale (see Tables 1 and 2)
and the arithmetic mean and standard deviation of response were calculated for each global risk following
methodologies used in the WEF Global Risks Report 2019 (WEF 2019b):+,8𝑙𝑖𝑘𝑒𝑙𝑖ℎ𝑜𝑜𝑑( = , ∑79+likelihood(,7 ,-𝑖𝑚𝑝𝑎𝑐𝑡( =+,-,S∑79+impact (,7 ,+B𝜎;(<=;(>??@ = AB ∑D9+
(likelihood(,7 − likelihoodF )H-+B𝜎(TUVWX = A ∑D9+
(impact (,7 − impact F )H
B-

Where likelihoodi and impacti are the averaged numeric scale responses of respondents for a specific risk. Ni is thenumber of respondents for risk i, who evaluated both the likelihood and impact of that specific risk. The likelihoodi,nand impacti,n are respectively the likelihood and impact assigned by respondent n to risk i. Only responses for which
both the likelihood and impact were evaluated were retained for analysis. Partial responses where a respondent
provided an evaluation of either likelihood or impact, but not both, were not considered in these calculations.

1.1.2 Analysis of robustness of knowledge and assessment confidence
We analyzed responses on robustness of the scientific knowledge base around a risk and the respondents’ personal
confidence of their assessment of each risk separately from likelihood and impact, as respondents were only asked toevaluate these for a subset of the 30 global risks based on their expertise in a selected risk category. Of the 221
retained survey responses, 174 elected to evaluate Environmental risks, 24 Societal risks, 10 Geopolitical risks, 8Technological risks and 5 Economic risks. In each case, we converted Likert responses to a numeric scale from 1 to 5

3

for each question and calculated the median response for robustness of knowledge base and respondents’confidence in their assessment separately for each category of risks.

1.2Interconnected risks

Risks rarely occur in isolation, but rather can have interacting and cascading effects. To expand our understanding ofthese interconnections, respondents were asked ‘Based on your knowledge, which subset of the 30 global risks aremost likely to have synergistic effects and to lead to a global systemic crisis?’ Respondents could select a cluster ofbetween 3 and 10 risks from the list of global risks. Of the respondents taking the survey n = 82 provided a responseto this question with at least three identified risks.
We tabulated a pair-wise frequency matrix of the co-occurrences of risk pairs across responses. To accomplish this,all 30 top risks were coded as occurring (1) or non-occurring (0) within each respondent’s response. This matrix wasthen multiplied by its transpose to get the total number of times two risks were listed together across all responses.
This dataset was used to count the number of times risks co-occurred within responses as well as to create a network
diagram in software Gephi (version 0.9.2) where the thickness of the edge connecting two nodes (“risks”)
correspondents to the frequency of co-occurrence across scientists’ responses.

1.3

Committed risks

Survey respondents were asked to look beyond the 10-year time horizon framing of the previous questions to look at
potential longer-term, but delayed impacts that we may already be committing to. Specifically, respondents wereasked “are there risks that we could commit to and/or cross a threshold towards within the next 10 years that could put
us on a path toward irreversible or catastrophic outcomes that manifest after the 10-year timespan?” Respondents
were able to free-list multiple responses. In total, n = 68 respondents provided answers for this question.
Qualitative content analysis was used to categorize free-listed responses into broad themes of risks. This approach
was used rather than topic modelling which relies on algorithms to group words as the latent meaning behind groups
of words can be lost. Furthermore, the limited text (100 character maximum per response) limits the utility of topic
modelling to identify robust themes. We report the top three most commonly identified themes of risk.

1.4

Additional and emerging risks

Respondents were asked to identify up to five additional or emerging risks not included in the top 30 global risks
provided in the first section of the survey. Respondents were also asked to classify their free-listed responses under
existing risk categories or combinations thereof. Of the 222 scientists participating in the survey, n = 173 provided
responses to this question on additional risks.
We used qualitative content analysis (QCA) and manual coding (conducted by three researchers independently toensure inter-rater reliability) to group risks by topic. The goal of QCA is to systematically describe the meaning fromthe response text by identifying themes and patterns. We report the top 7 most commonly identified topics based on
frequency analysis.

42. Survey Respondents
2.1

Response rate and geographic distribution

In total, we received 290 survey responses for which 250 provided demographic data. Respondents providing
demographic data were based in 52 different countries. Summarizing by continent, most retained respondents werefrom Europe (44.4%) or North America (23.6%). A smaller number were from countries in Asia & the Pacific (16.4%),
Africa (7.6%) or Latin America (7.2%). Very few survey respondents were from West Asia (0.2%).

Figure 1. Distribution of respondents by continent
For the purposes of our analysis, ‘scientists’ were defined as respondents having at least a Masters’ degree and morethan 1 year of experience in one of the six scientific domains. Only respondents meeting these criteria were retained
for the analysis, resulting in responses from 222 scientists. The geographic regions of expertise amongst retained
respondents were slightly more evenly distributed. While respondents could identify multiple regions of expertise 21%
claimed expertise in Latin America, 20% in Africa, 29% in North America, 33% in Asia, and 48% in Europe. Only 1.8%of respondents claimed expertise in issues relating to West Asia, while 31% claimed to have global expertise.

2.2

Scientific background, experience and institutional setting

Retained respondents were trained primarily in the natural and social sciences with a smaller number from thephysical, health, technological sciences or humanities. However, 13% of scientists retained in the analysis did notprovide an answer to this question.

5Figure 2. Distribution of scientific background of survey respondents

Of the 222 respondents retained in the analysis, the majority held a PhD degree (78%) and were based at a university(60%) or research institution (28%). At least 75% had 5 years of experience or more working in their respective
scientific domain. Of these respondents, approximately 61% were male and 38% were female, with 1% abstaining
from answering the question.

Figure 3. Distribution of A) institutional origin (orange), B) level of education (green) and C) years of scientificexperience (blue) of the retained respondents providing demographic data.

63. References
WEF [World Economic Forum] (2019a). Global Risk Report 2018-2019. Available at
http://www3.weforum.org/docs/WEF_Global_Risks_Report_2019.pdf
WEF [World Economic Forum] (2019b). Appendix B: Global Risks Perception Survey and Methodology 2018.
Available at: https://reports.weforum.org/global-risks-2018/appendix-b-methodology/

7