T H E FU T U R E O F S K I L L S
EMPLOY MEN T IN 203 0
FUTURESKILLS.PEARSON.COM

Hasan Bakhshi
Jonathan M. Downing
Michael A. Osborne
Philippe Schneider
Research partners

1Creative Commons
This work is licensed under the Creative Commons Attribution-Non CommercialShareAlike 4.0 International License. To view a copy of this license, visit http://
creativecommons.org/licenses/by-nc-sa/4.0/ or send a letter
to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
Questions for the authors
Questions for the authors can be sent to:hasan.bakhshi@nesta.org.uk (Hasan Bakhshi)
jd@robots.ox.ac.uk (Jonathan M. Downing)
mosb@robots.ox.ac.uk (Michael A. Osborne)
philippenschneider@gmail.com (Philippe Schneider)
Suggested reference
Bakhshi, H., Downing, J., Osborne, M. and Schneider, P. (2017).
The Future of Skills: Employment in 2030. London: Pearson and Nesta.
Copyright 2017
The contents of this paper and the opinions expressed herein
are those of the Authors alone.
ISBN: 978-0-992-42595-1

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

ABOUT PEARSON
Pearson is the world’s learning company. We’re experts in educational course ware and assessment, and provide teaching
and learning services powered by technology. We believe that learning opens up opportunities, creating fulfilling careers
and better lives. So it’s our mission to help people make progress in their lives through learning.

ABOUT NESTA
Nesta is a global innovation foundation. We back new ideas to tackle the big challenges of our time, making use of ourknowledge, networks, funding and skills. We work in partnership with others, including governments, businesses andcharities. We are a UK charity that works all over the world, supported by a financial endowment.
To find out more visit www.nesta.org.uk.

ABOUT THE OXFORD MARTIN SCHOOLThe Oxford Martin School at the University of Oxford is a world-leading centre of pioneering research that addresses
global challenges. We invest in research that cuts across disciplines to tackle a wide range of issues such as climate change,
disease and inequality. We support novel, high risk and multidisciplinary projects that may not fit within conventional
funding channels. We do this because breaking boundaries can produce results that could dramatically improve thewellbeing of this and future generations. We seek to make an impact by taking new approaches to global problems,
through scientific and intellectual discovery, by developing policy recommendations and working with a wide range ofstakeholders to translate them into action.
www.oxfordmartin.ox.ac.uk
Twitter: @oxmartinschool
Facebook: /oxfordmartinschool3

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

CONTENTS
7EXECUTIVE SUMMARY

18

1 INTRODUCTION

20

2 LITER ATURE REVIEW

25

2.1 Anticipating occupations and skills
2.2 Changing skills needs
2.3 Labour markets and structural change
Overview of the trends

29

3 APPROACH

20
21
22

30

4 DATA32

4.1 O*NET
4.2 Employment microdata
4.3 Workshop-generated data

36

5 METHODOLOGY

30
32

36
36
37
37
38
38

39
39

5.1 Gaussian processes
5.2 Heteroskedastic ordinal regression
5.3 Active learning
5.4 Assessing feature importance
5.4.1 Pearson correlation	5.4.2 A
 verage derivative andfeature complementarity
5.5 New occupations
5.6 Trend extrapolation

541

6 RESULTS

42

83

6.1 Occupations
6.1.1 US
6.1.2 UK6.2 Sensitivity analysis
6.2.1 US
6.2.2 UK6.3 Skills
6.3.1 US6.3.2 UK6.4 Relative importance of skills, abilities
and knowledge areas
6.5 Skill complementarities
6.5.1 US
6.5.2 UK
6.6 New occupations
6.6.1 US6.6.2 UK87

7 LIMITATIONS OF THE ANALYSIS

89

8 CONCLUSIONS

91

Appendix A
Sensitivity analysis

95

Appendix B
Overall rankings of minor occupation groups, US and UK

99

Appendix C
Skills ranking by average derivative, US and UK

103

Appendix D
UK–US occupation crosswalk

113

REFERENCES

118

TECHNICAL TERMS GLOSSARY

120

ENDNOTES

42
48
52
53
57
61
62
67
72
74
74
77
80
80

ABOUT THE AUTHORS
Hasan Bakhshi leads Nesta's creative economy policy and research and data analytics. His work includes co-authoring
the Next Gen skills review of the video games and visual effects industries, which led to wholesale reforms of the school
ICT and computing curriculum in England, and the Manifesto for the Creative Economy, which sets out ten recommendations by which governments can help the creative economy grow. Bakhshi is a member of the Royal Economic Society
Council, the Senior Management Team at the Economic Statistics Centre of Excellence (ESCoE) supported by the Officefor National Statistics (ONS) and the Expert Peer Review Group for Evaluation at the Department for Business, Energy andIndustrial Strategy.
Jonathan M. Downing is a DPhil (PhD) student in the Machine Learning Research Group at University of Oxford, jointly
supervised by Dr. Michael Osborne and Prof. Stephen Roberts. His academic research focuses on the Bayesian non-parametric, Future of Work, and machine learning interpretability. Prior to starting his DPhil he completed a MEng Engineering
Science at Keble College, University of Oxford.
Michael A Osborne is the Dyson Associate Professor in Machine Learning, a co-director of the Oxford Martin
programme on Technology and Employment, an Official Fellow of Exeter College, and a co-director of the EPSRC Centre
for Doctoral Training in Autonomous Intelligent Machines and Systems, all at the University of Oxford. He works todevelop machine intelligence in sympathy with societal needs. His work in Machine Learning has been successfully
applied in diverse contexts, from aiding the detection of planets in distant solar systems to enabling self-driving cars todetermine when their maps may have changed due to roadworks. Osborne has deep interests in the broader societal
consequences of machine learning and robotics, and has analysed how intelligent algorithms might soon substitute forhuman workers.
Philippe Schneider is an independent researcher who has collaborated extensively with Nesta as a Visiting Research
Fellow and consultant since 2007. He has carried out technical and policy work for a range of private and public sector
organisations, including the World Bank, BlackRock, the African Development Bank, HM Treasury, BP, CISCO, British
Academy and what is now the Department for Business, Energy and Industrial Strategy where he was Advisor to theSecretary of State on innovation policy. He speaks and reads Chinese.

ACKNOWLEDGEMENTS
We thank Logan Graham for his research assistance on coding and preparation of the O*NET and employment data
and to Justin Bewsher for his contributions to the sensitivity analysis. We are indebted to Harry Armstrong and Wendy L.
Schultz, and to those who participated in the two foresight workshops held in Boston, Massachusetts, and in London in
October 2016. We are also grateful to the attendees of a dry run workshop in London who helped us refine the workshop
exercises. Thanks also to Antonio Lima, George Windsor, Juan Mateos-Garcia, Geoff Mulgan and Cath Sleeman at Nesta
and Laurie Forcier, Amar Kumar, Janine Mathó, Tom Steiner, Vikki Weston and other colleagues at Pearson for theirencouragement, support and valuable comments on the findings. Mark Griffiths played an important role in the inceptionof the study and provided valuable support throughout its implementation. Sir Michael Barber, Paolo Falco, Joshua Fleming
and Carl Frey provided invaluable comments on an earlier draft of the report. We are also grateful to Rainy Kang for
helping us track down data and Jessica Bland for her contributions in the early stages of the research.

6

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

EXECUTIVE SUMMARY
Recent debates about the future of jobs have mainly focused onwhether or not they are at risk of automation (Arntz et. al., 2016;
Frey and Osborne, 2017; McKinsey, 2017; PwC, 2017). Studies
have generally minimised the potential effects of automation on
job creation, and have tended to ignore other relevant trends,
including globalisation, population ageing, urbanisation, and therise of the green economy.
In this study we use a novel and comprehensive method
to map out how employment is likely to change, and the implications
for skills. We show both what we can expect, and where we should
be uncertain. We also show likely dynamics in different parts of thelabour market —­ from sectors like food and health to manufacturing.
We find that education, health care, and wider public sector
occupations are likely to grow. We also explain why some low-skilledjobs, in fields like construction and agriculture, are less likely to suffer
poor labour market outcomes than has been assumed in the past.
More generally, we shine a light on the skills that are likely to be ingreater demand, including interpersonal skills, higher-order cognitive
skills, and systems skills. Unlike other recent studies, the method
also makes it possible to predict with some confidence what kinds of
new jobs may come into existence.
The study challenges the false alarmism that contributes to aculture of risk aversion and holds back technology adoption,
innovation, and growth; this matters particularly to countries likethe US and the UK, which already face structural productivity
problems (Atkinson and Wu, 2017; Shiller, 2017).
Crucially, through the report, we point to the actions that educators,
policymakers and individuals can take to better prepare themselves
for the future.

7

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

8T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0 – EXEC U T I V E S U MMA R YOUR CONTRIBUTION
Our research introduces a novel mixed-methods approach toprediction that combines expert human judgement with machinelearning, allowing us to understand more complex dependencies
between job features than previously possible. We exploit thisenhanced capability to assess complementarities between skills
and draw out the implications for new occupations.
In addition, our analysis is grounded in an explicit considerationof the diverse and interacting sources of structural change ­—non-technological as well as technological — all of which are
expected to have major impacts on future skills needs. Althoughsome other studies have also sought to consider a wider rangeof trend influences on the future of work, these have been
largely qualitative in nature.
Finally, our identification of the bundles of skills, abilities
and knowledge areas that are most likely to be important in the
future, as well as the skills investments that will have the greatest
impact on occupational demand, provides information that
educators, businesses and governments can use for strategic
and policy-making purposes.

Job Features
The skills, abilities and knowledge areas that comprise occupations
are collectively called features.
Structural Change
In economics, structural change is a shift or change in the basic
ways a market or economy functions or operates.

9THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

OUR METHODOLOGY
Here’s how our methodology works:
TRENDS ANALYSISMACHINE LEARNING

We start by reviewing the drivers of change and theinteractions that are expected to shape industry structuresand labour markets in 2030. We also assemble detailed
information about occupations (key tasks, relatedindustries and historical growth patterns). This material isused to contextualise and inform discussions at foresightworkshops in the US and UK, our countries of analysis.

We subsequently use this information to train a machinelearning classifier to generate predictions for all
occupations. This relies on a detailed data set of 120
skills, abilities and knowledge features against which
the U.S. Department of Labor’s O*NET service 'scores'
occupations. (We also map this data to the closest
comparable UK occupations using a 'cross-walk'.) Together
with the predictions about changes in occupational
demand, this permits us to estimate the skills that will, byextension, most likely experience growth or decline.

FORESIGHT WORKSHOPSAt the workshops, panels of experts are presented withthree sets of ten individual occupations and invited todebate the future prospects of each in light of the trends.
The first set of ten occupations is chosen randomly.
Participants then assign labels to the occupations
according to their view of its future demand prospects(grow, stay the same, shrink), as well as their level ofconfidence in their responses. To sharpen prediction, an
active learning method is implemented: the subsequent
sets of occupations to be labelled are chosen by thealgorithm. Specifically, the algorithm chooses occupations
in areas of the skills space about which it is least certain,
based on the previously labelled occupations. This process
is repeated twice to generate a training set of
30 occupations.

O*NET
O*NET is the US Department of Labor’s Occupational Information Network(O*NET), a free online database that contains hundreds of occupational
definitions to help students, job seekers, businesses and workforce
development professionals to understand today’s world of work in the
United States. Data from the 2016 O*NET survey was used in this study
to understand the skills, abilities and knowledge areas that make up each
occupation group. onetonline.org

10

ANALYSIS
We interpret the machine learning results with particularattention to the discussions from our foresight workshops,
and highlight findings that are most relevant for employers,
educators and policymakers.

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

MET

HO

INCREASING
INEQUALIT YPOLITICALU N C E R TA I N T Y

U R B A N I S AT I O N

EXTRENDSPER

T INSIGHT

TECHNOLOGIC AL
CHANGEE N V I R O N M E N TA LS U S TA I N A B I L I T Y
DEMOGR APHICCHANGEG L O B A L I S AT I O NLSNE LEARN

IN DEM

O*NET

11

AN
D

S

A

K

IL

HI

I

N

M

C

G

!

DO

LO

GY

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

KEY TRENDS
The future of work isn’t only influenced by automation.
Our model includes an analysis of the following key trends
to determine the bigger picture of work.

TECHNOLOGICAL CHANGEENVIRONMENTAL SUSTAINABILITY­— Estimates of future automation impact range, from 47%
of US employment at risk to only 9%.

­— Climate change consensus largely intact, but withnotable cracks.
­— Structural changes resulting from emerging 'green
economy sector' and 'green jobs', but vulnerable
to political reversals.

­— Perennial fears about impact of automation on
employment.

­— Conversely, technology amplifies human performancein some occupations--and gives rise to entirely new
occupations and sectors.
GLOBALISATIONURBANISATION­— Global labour markets increasingly integrated.

­— More than half of world population lives in cities—70
percent by 2050. Cities attract high-value, knowledgeintensive industries, offer more varied employment
and consumption opportunities.

­— Benefits (e.g.,advanced manufacturing, knowledgeintensive services) and costs (e.g., employment andwage impacts, trade deficits, legacy manufacturing).

­— Uncertainties include fiscal policy, infrastructure
investments, high public debt ratios.

­— Post-financial crisis headwinds (e.g., sluggish world
trade growth, rising protectionism).
DEMOGRAPHIC CHANGE

INCREASING INEQUALITY­­— Rise in income and wealth inequality, middle
class squeeze.
­— Disparities in education, healthcare, social services,
consumption.
POLITICAL UNCERTAINTY­— Indices of geopolitical uncertainty have remained highsince 9/11 spike.
­— Mirrored by political and policy uncertainty—capacity of
institutions and policymakers to act credibly
and consistently.
­— Uncertainty negatively affects economic activity ingovernment-influenced sectors, such as defence,
finance, construction, engineering, and healthcare.

12­— Pressures to control age-related entitlements
vs. investments in education, R&D, infrastructure.
­­— Ripple effects through healthcare, finance, housing,education, recreation.
­— Rising Millennial generation, with divergent
consumption and work behaviours.

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

OUR FINDINGSTHE FUTURE DEMAND FOR OCCUPATIONS
We predict that around one-tenth of the workforce
are in occupations that are likely to grow as apercentage of the workforce. Around one-fifth arein occupations that will likely shrink. This latter
figure is much lower than recent studies ofautomation have suggested.
This means that roughly seven in ten people are
currently in jobs where we simply cannot know
for certain what will happen. However, our findings
about skills suggest that occupation redesign coupledwith workforce retraining could promote growthin these occupations.
A key element of the study is quantifying the extent of
uncertainty about likely future trends. These uncertainties
reflect the challenging task of balancing all the macro
trends that might influence the future of work. Further
uncertainties stem from the distinction between
occupations that are expected to grow in demand(reflecting wider occupation growth) from those that willgrow relative to other occupations. This distinction turns
out to be important because our US and UK expert groups
predict as a whole that the workforce will continue to growthrough 2030.
The uncertainty in our findings also reflects our use of therichest possible data set of occupation-related 'features'—that is, the skills, abilities and knowledge areas required
for each occupation. (This use of all 120 O*NET features
is an important differentiator of our study. For example,
the most recent study by Frey and Osborne (2017) uses
only nine skills categories.) This detailed characterisationof occupations renders them less similar to one another,
thereby limiting the confidence of our model in making
predictions for one occupation based on what has beenlabelled for another. In exchange, however, we are able todevelop a far more nuanced understanding of future skills
demand, as noted below.

Manufacturing Production
Manufacturing production occupations require one to set up, test and adjust
manufacturing machinery or equipment, using any combination of electrical,
electronic, mechanical, hydraulic, pneumatic or computer technologies.
Skilled Trades
In the UK, skilled trades include jobs in agriculture, metalwork, construction,
textiles, food preparation, hospitality and woodworking, among others.
Elementary Services
In the UK, elementary occupations consist of simple and routine tasks whichmainly require the use of hand-held tools and often some physical effort (e.g.,farm workers, street cleaners, shelf fillers).

13

We find that many of the jobs likely to experiencea fall in employment are, unsurprisingly, low- or
medium-skilled in nature. However, in challenge tosome other studies, not all low- and medium-skilledjobs are likely to face the same fate.
Technological change and globalisation may accountfor why many low- or middle-skilled occupations (e.g.,
manufacturing production) are expected to becomeless important in the workforce. The predicted decline inadministrative, secretarial and some sales occupations is
also consistent with these trends. Agriculture, skilled trades
and construction occupations, however, exhibit moreheterogeneous patterns, suggesting that there may bepockets of opportunity throughout the skills ladder.
The results also suggest that non-tradable services, likefood preparation, elementary services and hospitality will
all likely grow in importance. Many of these occupations,again, have lower skills requirements. However, they areassociated with differentiated products, which consumers
increasingly value.
This indicates that these occupations may be ripe for job
redesign and employee skills upgrading to emphasise
further product variety, a development heralded by there-emergence of artisanal employment in occupations likebarbering, brewing and textiles.
In general, public sector occupations —­­ with some
exceptions —­­ feature prominently and are predicted
to see growth.
In the UK, education, healthcare and wider public sector
occupations are, with some confidence, predicted to see
growth.These findings are consistent with population ageing
and a greater appetite for lifelong learning. They are also
consistent with the labour intensive nature of these sectors,
and their traditionally lower potential for productivity growth(Baumol and Bowen, 1966). They are further consistent
with the view that public sector roles are more resistant toautomation (Acemoglu and Restrepo, 2017a).
Similar patterns are evident in the US, though with someinteresting differences. Notably, confidence in the future
growth of healthcare occupations is lower than wemight expect, perhaps reflecting uncertainties related
to healthcare policy and spending. However, consistent
with the UK results, growth is anticipated for occupations
such as sports and fitness, as well as for therapy. These
which are arguably redefining healthcare, a phenomenon
partly attributable to the preferences and consumption
behaviour of Millennials.

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

We also expect buoyant demand for some —­­ butnot all —­­ professional occupations, reflecting thecontinued growth of service industries.

We find a strong emphasis on interpersonal skills,
higher-order cognitive skills and systems skills inboth the US and the UK.

Creative, digital, design and engineering occupations havebright outlooks and are strongly complemented by digitaltechnology. Furthermore, architectural and green occupations
are expected to benefit from greater urbanisation and agreater interest in environmental sustainability.

In the US, there is particularly strong emphasis on
interpersonal skills. These skills include teaching, social
perceptiveness and coordination, as well as related
knowledge, such as psychology and anthropology. This isconsistent with the literature on the growing importanceof social skills in the labour market (Deming, 2015).
There are good reasons to believe that interpersonal
skills will continue to grow in importance —­ not only asorganisations seek to reduce the costs of coordination
but also as they negotiate the cultural context in which
globalisation and the spread of digital technology aretaking place (Tett, 2017).

Interestingly, demand prospects can vary considerably
for some 'white collar' occupations that otherwise appear
very similar. In the US, roles such as management analysts,
training and development specialists and labour relations
specialists —­ occupations which should benefit from thereorganisation of work —­ are projected to grow in theworkforce, whereas financial specialists are expected tofall. The latter is consistent with automation having an
impact on cognitively-advanced occupations as well as
more routine roles.
Additionally, although there is a predicted decline in many
sales occupations, consistent with an expansion in digitalcommerce, niche roles like sales engineers and realestate agents may buck this trend.
THE FUTURE DEMAND FOR SKILLS*
Our results provide broad support for policy andpractitioner interest in so-called 21st century skills in boththe US and the UK.

Our findings also confirm the importance of higher-ordercognitive skills such as originality, fluency of ideas andactive learning.
A similar picture emerges for the UK. The results point toa particularly strong relationship between higher-ordercognitive skills and future occupational demand. Skills
related to system thinking —­ the ability to recognise,
understand and act on interconnections and feedback
loops in sociotechnical systems —­ such as judgementand decision making, systems analysis and systems
evaluation also feature prominently.
We show that the future workforce will need broadbased knowledge in addition to the more specialisedfeatures that will be needed for specific occupations.
Broad-based knowledge areas such as English language,
history, philosophy and administration and management
are all associated strongly with occupations projected tosee a rise in workforce share.

Sales Engineers
A sales engineer is a salesperson with technical knowledgeof the goods and their market.
Active Learning
Understanding the implications of new information for both current andfuture problem-solving and decision-making.
Fluency of Ideas
The ability to come up with a number of ideas about a topic (the number ofideas is important, not their quality, correctness, or creativity).
Sociotechnical Systems
The term sociotechnical system refers to interaction between society's
complex infrastructures and human behaviour. It can also be used to describe
the relationship between humans and technology in the workplace.

14

Systems Analysis
Determining how a system should work and how changes in conditions,operations and the environment will affect outcomes.
Systems Evaluation
Identifying measures or indicators of system performance and the actions
needed to improve or correct performance, relative to the goals of the system.
*A Glossary of Skills laying out the precise definitions of all
120 O*NET skills, knowledge areas and abilities can be accessedat futureskills.pearson.com.

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

Other knowledge features like foreign languages areespecially valuable as complements: that is, they find use
in specialised occupations when other features have alarge value. We find a similar pattern for a number of
STEM-related features like science, technology designand operations analysis. Interestingly these features
are found to be complementary to conventional STEM
occupations, as well as to some non-STEM occupations
such as secretarial and administrative jobs.

In the UK, it turns out that strengthening judgement anddecision-making skills, fluency of ideas and operations
analysis are important demand complements for manyoccupations. The literature underlines the need to matchthese skills with changes in organisational design, such asenhanced delegation, employee involvement in decision
making and other related high-performance work practices
in order to maximise their impact (Ben-Ner and Jones,
1995; Kruse et al., 2004; Lazear and Shaw, 2007).

UNCOVERING SKILL COMPLEMENTARITIESANTICIPATING NEW OCCUPATIONS

Occupations and their skills requirements are not set instone: they are also capable of adjusting to shifts in theeconomic environment (Becker and Muendler, 2015).
Our model identifies how the skill content of occupations
can be varied to improve the odds that they will bein higher demand. As noted above, we define theseas 'complementary skills' in so far as their impact on
demand is conditional on the other skills that make up theoccupation. The notion of complementarity can be used
to determine priorities for skills investment and assist
thinking on how jobs might be redesigned to put theseskills to work.

An attraction of our approach is that it can be used toconsider occupations that do not yet exist, but may
emerge in the future in response to the identified driversof change. The model allows us to identify hypotheticaloccupations, dissimilar to existing occupations, that are'almost certain' to see future growth. In particular, we
can identify the combinations of skills, knowledge areas
and abilities that are most associated with such newoccupations.

Complementary skills that are most frequently
associated with higher demand are customer andpersonal service, judgement and decision making,technology design, fluency of ideas, science andoperations analysis.
In the US, customer and personal service, technology
design and science skills are, according to our analysis,
the job performance requirements seen as most likely toboost an occupation’s demand beyond what is currently
predicted, notwithstanding notable differences acrossoccupation groups.
Take production occupations, for example, which ouranalysis shows are very likely to see a fall in the workforce.
Our model suggests that increasing understanding ofcustomer and personal service, technology design andinstallation will have the greatest positive impact on the
future demand for these occupations, stemming thedecline that they are otherwise projected to experience.

Operations Analysis
Analysing needs and product requirements to create a design.
Production Occupations
The production occupations is used in the U.S. and covers machinists,
operators, assemblers, and the like across a wide variety of industries(e.g., nuclear power, gas and oil, food preparation, textiles).

15

For the US, the model finds four such hypotheticaloccupations, along with their top five ranking features.
We are able to further understand something about
these hypothetical occupations by looking at the existing
occupations that are 'closest to them' and inspecting their
historical growth.
For the UK, two new hypothetical occupations are
discovered by the model, along with their top five ranking
features. We again consider the occupations that areclosest to these and confirm their past growth.

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0 – EXEC U T I V E S U MMA R Y

CONCLUSION
Jobs are the cornerstone of our economic and social lives:
they give people meaning, self-respect, income and thechance to make societal contributions (Banerjee and Duflo,
2008; World Bank, 2013; Taylor, 2017). Today, there are concernsthat this relationship is under strain as structural change once
again disrupts employment levels and occupational patterns.
Our analysis provides grounds for optimism in this respect:
far from being doomed by technology and other trends,
we find that many occupations have bright or open-ended
employment prospects. More importantly, we illustratefor different US and UK occupations, how the skills mix of theworkforce can be upgraded to target such new opportunities.
This, however, requires individuals, educators, businesses andpolicymakers to respond appropriately. History is a reminderthat investments in skills must be at the centre of any long-term
strategy for adjusting to structural change. A precondition forthis is access to good information on skills needs —­ withoutwhich policymakers risk flying blind.
We hope this report is a step towards improving understandingof this vital agenda.

16

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

T H E FU T U R E O F S K I L L S
EMPLOY MEN T IN 203 0

17

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

1. INTRODUCTION
Governing is the art of planning and predicting. Developing a
picture of long-term jobs and skills requirements is critical forpolicymakers as they navigate rapid, complex and uncertain
shifts in the economy and society. A wide range of areas –from curriculum development and careers guidance throughapprenticeships and workplace training to occupational
standards, migration and social insurance – rely on theavailability of accurate labour market information (LMI). It
is a basic precondition for the system resilience of modern
economies – the collective ability of individuals, education andlabour market institutions to adapt to change without breaking
down or requiring excessively costly intervention to remedy.
However, there is also an awareness of the divergence
between the pace of change and the inertia of ourinstitutions. Andreas Schleicher, Director for Education and
Skills at the Organisation for Economic Co-operation andDevelopment (OECD), has pointed out that throughout
history, education has always taken time to catch up withtechnological progress (Schleicher, 2015). In the UK, theEducation Act of 1902, which marked the consolidation ofa national education system and the creation of a publicly
supported secondary school system, arrived a century after
the Industrial Revolution and the growing complementarity
between human and physical capital (Galor and Moav, 2006;
Becker et al., 2009).
Today, educationalists speak about a ‘40-year gap’
between experts who are exploring where the worldof work and the state of learning will need to be in 15
years’ time, practitioners in the trenches and parents,
whose conception of ‘good’ education is framed by theirown earlier experiences. The result is a structure that
resembles sedimentary rock: each layer has its own
assumptions and expectations. But there is little holding
the layers together, and once in place, they can limit
policy change and future choices.
Structural change is affecting labour markets, as it is all
markets, upsetting the balance of supply and demand forskills. While misalignment is normal over the business cycle,
the costs of persistent mismatches can be considerable ifleft unaddressed: they limit the ability of firms to innovate
and adopt new technologies, while impeding the reallocationof labour from less productive activities to more productive
ones (Adalet McGowan and Andrews, 2015). They also lead18

to increased labour costs, lost production associated withvacancies remaining unfilled and all the direct and indirect
costs of higher unemployment (Şahin et al., 2014; OECD,
2016b). Individuals likewise pay a heavy price. They benefit
from economic growth mainly through jobs. Not only arejobs typically the most important determinant of earnings
and living standards. They also critically shape how individuals
view themselves, interact with others and perceive their stakein society, including their sense of control over the future(Banerjee and Duflo, 2008; World Bank, 2013).
The jury is out on the scale of long-term skills shortages in
the labour markets of advanced economies. Much of theevidence comes from employers, typically from surveys.
ManpowerGroup, the human resources company thatpublishes arguably the most authoritative survey on skills
shortages, finds that globally 40% of employers have difficulty
filling jobs. This figure has been largely stable over the past
decade, although considerable differences exist acrosscountries: while shortage levels in the US have tracked theglobal average, they are significantly lower in the UK but
appear to be growing (ManpowerGroup, 2016).
The reliability and validity of employer surveys, however, areopen to question (Cappelli, 2015). The empirical fingerprints
for skills shortages are not where we would expect them
to be – namely in wage inflation not linked to productivity
growth. On the contrary, labour’s share in national income
has trended downwards in most economies since the 1990s(International Monetary Fund (IMF), 2017). Academic studies
examining the issue have also failed to uncover significant
shortages (Weaver and Osterman, 2017). Where they exist,
they are often attributed to the unwillingness of employers
to offer attractive remuneration to workers, suggesting thatinterventions that treat the problem as an educational one
are likely to be poorly targeted (Van Rens, 2015).
Saying that skills shortages are overstated is not the sameas saying that they are unfounded. They are notoriously
difficult to measure, hidden from view as companies workaround problems by increasing the workload of existing
employees, outsourcing work to other organisations or evenadapting their product market strategies so that they areless dependent on a highly skilled workforce.

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

One may also be looking in the wrong places if thechallenge is framed only in terms of workforce skills gaps
and shortages. An equally important problem is if workers
possess skills at a higher level than those required to fill ajob (Sutherland, 2012; Mosca and Wright, 2013; Clark et al.,2014; Montt, 2015). Indeed, for some commentators,
it is skills surpluses and their opportunity costs rather thanshortages that pose the greatest challenge for policymakers(Gambin et al., 2016).
Focussing on gaps and shortages also overlooks the
dynamic context in which individuals increasingly makelabour market decisions: the risk of mismatches arises not
only when they leave education and enter the workforce
but also each and every time they change jobs. For example,
among displaced workers who are re-employed within ayear, between 20% and 70% change occupation or industry(OECD, 2012). Notwithstanding the economic benefits tofirms of this labour supply flexibility, roughly a quarter ofdisplaced workers experience a major change in skills –one that is associated with sizeable adjustment costs andwage losses (Poletaev and Robinson, 2008; Kambourov
and Manovskii, 2009; Gathmann and Schonberg, 2010;
Robinson, 2011). The challenges are even more daunting
upstream where there is a long lead time between
investment in skills and competence in the workplace –where educators are effectively being asked to teachstudents skills to solve problems that no one can foreseeand may not materialise for years.
This paper is motivated by these observations andaddresses the following research question: given thelikely drivers of change in future labour markets, which
occupations will grow or decline in demand by 2030
and what will their skills profile be?
The remainder of this paper is structured as follows. In
Section 2, we review the relevant literature for our study.
Section 3 outlines our approach to the research and sets
out the key structural trends impacting on future labour
markets. Section 4 discusses our data sources and Section
5 sets out our machine-learning methodology. In Section
6 we present our findings and offer interpretation. Section
7 highlights some key limitations in our analysis. Finally,in Section 8, we derive some conclusions and suggest
directions for further research.

19

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

2. LITER ATURE REVIEW
2.1. ANTICIPATING OCCUPATIONS AND SKILLSThis report builds on the growing use of skills assessment
and anticipation exercises (European Centre for theDevelopment of Vocational Training (CEDEFOP), 2008; OECD,
2016a). Such endeavours have a long history, dating backto the 1960s.1 Improvements in the coverage, quality andtimeliness of data and analytical tools have expanded thescope of activity, though there remain bottlenecks to theintegration of public and privately held data (Mitchell andBrynjolfsson, 2017). Today, approaches look 10, 30 andeven 100 years into the future, incorporating elements ofstructural foresight analysis such as the Delphi method
and scenario development (OECD, 2012).2
The ability to link occupation-based information to specificskills through databases such as the US Departmentof Labor’s O*NET, or those aggregating online job
advertisements in real time, has enabled policymakers to
get a direct handle on skills needs. In the past, assessing theseneeds has been more difficult due to problems in definition,
classification and measurement.
The common presumption underlying these efforts isthat predicting occupational and skills demand over longhorizons is feasible. This confidence may seem puzzling at
a time when economic and business forecasting has beenheavily criticised, following a string of errors before andafter the 2008 financial crisis which have contributed to agroundswell of anti-expert opinion.3
The dividing line between what is a ‘good’ and ‘bad’
forecast, however, is not straightforward (Broadbent, 2013;
Chadha, 2017). For some, the value of a forecast lies in thequestions it asks as much as the answers it provides: thestories revealed by errors can sharpen understanding of
uncertain relationships between economic variables andsupport learning. In other cases, the size and direction oferrors may matter more than some exogenous measureof forecast accuracy.
More prosaically, prediction is better suited to someareas of human activity than others. There tends tobe a high degree of persistence, and thus predictability,
in the occupational and skills make-up of the workforce.
This reflects the fact that the labour market is a social
institution embedded in a dense web of rules, habits andconventions and that there are substantial employment
adjustment costs, even in the face of major changes suchas the arrival of new and disruptive technologies (Pierson,
2004; Granovetter, 2017).
This pattern is not unique or even unusual historically. Thus
some observers point to the fitful progress of electrificationin the US, where labour productivity grew slowly between
1890 and 1915, then saw a decade-long acceleration, only toslow down in the mid-1920s, before experiencing a secondboom in the 1930s (Syverson, 2013). The unprofitability of
replacing still viable manufacturing plants adapted to waterand steam power; the slow gestation of complementary

20

innovations such as the electrical grid, unit drive transmission
and hydroelectricity; the challenges of reconfiguringorganisational processes, workforce skills and factory design
to exploit electricity’s potential and political barriers frommunicipal and town governments which restricted the flow ofinvestment capital into utilities – these features all underscorethe lags between the emergence of new technologies andtheir impact on productivity and the structure of employment(David, 1990; David and Wright, 1999).
Parallels between the labour market consequences ofskills-augmenting technological change today and previous
episodes of technological change, however, are limited in
one respect. Despite the perception of rapid technologicalchange associated with Information and Communication
Technologies (ICT), the transition from manufacturing toservice and knowledge-intensive jobs has been remarkably
gradual, stretching over decades (Baraby and Siegel, 2017).
Notably it has been more protracted than the transition
from agricultural to manufacturing employment whichaccompanied electrification and industrialisation (Handel,
2012, 2016; Atkinson and Wu, 2017). This is consistent
with evidence suggesting that the rate of job creation in
new technology industries has in fact slowed over recent
decades (Lin, 2011; Frey and Berger, 2016).
A satisfactory explanation for these developments lies
outside the scope of this report, but at least part of it
can be explained by: a decline in labour market fluidity
associated with the ageing of the population; an increasein the share of the workforce with a college or university
degree in so far as workers with degrees typically havemore stable employment than workers without degrees;
and a shift towards older and larger firms that contractand expand less rapidly than other firms (Hyatt and
Spletzer, 2013; Davis and Haltiwanger, 2014). Theseelements combine to impart an additional degree ofcontinuity on the economic environment, enabling
prediction.
A more refined way of reaching the same conclusion is
to compare the performance of long-term occupational
projections against outcomes. One evaluation of US Bureauof Labor Statistics (BLS) 10-year projections at the onedigit level for the period 1988–2008 finds that they doa good job of anticipating the size of broad occupations(absolute error < 6%), with the exception of service and farmworkers (Handel, 2016). In addition to path dependencein occupational structure, this performance is attributable
to the fact that errors at a more detailed occupational
level cancel out one another. Consistent with this, forecast
errors are found to be inversely related to employment
size: occupations with more than 600,000 workers have an
average absolute error of 14.8% compared with 32.7% for
occupations with between 25,000 and 49,000 workers. Thissuggests that, even in the presence of significant errors withrespect to the size of individual occupations, this should not
be an obstacle to prediction where the goal is to make more

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

general statements about occupational and skills demand.
The BLS’s projections are not exempt from criticism – mostof which is focussed on their tendency to underestimatechanges in the size of occupations (Alpert and Auyer, 2003;
Carnevale et al., 2010; Wyatt, 2010). This largely reflects
the challenges of applying fine-tuned rules and making
point estimates when there are high levels of uncertainty.
By extension, anticipating the direction of change –whether an occupation will grow or decline in relative or
absolute terms – appears to pose fewer issues for thesame reasons. As a coarser assessment, it requires lessinformation about future states of the world and thus is
more robust to ignorance.4
Studies have similarly shown that ignoring some
information can make not only for cheaper but also
for more accurate decisions in such circumstances(Gigerenzer, 2010). We share this perspective andaccordingly produce directional forecasts in this study.
Other sources of error are more problematic. They are
embodied and variously popularised in ‘weak signals’
and ‘black swans’ – shifts that are difficult to observe
amid the noise, yet whose consequences are potentiallytransformative. They resemble George Shackle’s
description of the future “which waits, not for its contents
to be discovered, but for that content to be originated”(Shackle, 1972). Quantitative approaches which assume
that past patterns of behaviour will continue over the
longer term struggle badly with these shifts. Even when
their significance is acknowledged, in many cases they are
ignored on the grounds that they are too unruly to analyse.
Whether this is an adequate defence is debatable.
Policymakers have no alternative but to grapple with all
possible discontinuities and plan accordingly. If planningis silent about discontinuities, its value is reduced. For
this reason, many organisations have found qualitative
foresight processes linked to strategic dialogue a usefullens through which to interrogate these matters. However,they are no panacea to the shortcomings of prediction:subjective judgments often lack external validity andtransparency, meaning that decision-makers are notalways sure if and how they should act on them.
Still, foresight processes have the potential to broaden
thinking about alternatives beyond business-as-usual andtheir implications. By enabling deliberation and challenging
individually held beliefs, these processes can also combat
the types of bias that may creep into long-term, expert-led
planning (Tichy, 2004; Goodwin and Wright, 2010; Kahneman,
2011; Ecken et al., 2011; Nemet et al., 2016): overconfidence(overweighting private information and underweighting public
information); optimism (exaggerating the rate of change,
especially for new technologies); familiarity (relating newexperiences to previously seen ones); and narrative fallacy(creating explanations for phenomena which are essentiallyunconnected). Accordingly, we adopt an integrated approach

21

which combines and builds on both quantitative andqualitative approaches.
2.2. CHANGING SKILLS NEEDS
This report relates to the literature on the changing
demand for skills. Conventional wisdom views such change
as a product of the complementarity between technology
and high-skilled labour. That is, technological progress
raises the demand for skills, and investment in skills, inturn, satiates that demand. This framework has proven aworkhorse for economists and can successfully explain
many salient changes over time in the distribution of
earnings and employment across advanced economies(Goldin and Katz, 2009).
Nonetheless, its implementation rests on a highly
aggregated and conceptually vague measure of skill,typically years of schooling. Recent accounts have sought
to put more meat on its bones by mapping skills to thetasks performed by labour (Acemoglu and Autor, 2011).
Influential work by Autor and Murnane (2003), for example,
distinguishes between cognitive and manual tasks on the
one hand, and routine and non-routine tasks on the other.
Comparing tasks over time, from 1960 to 1998, they findthat routine cognitive and manual tasks declined while
non-routine cognitive and manual tasks grew in importance.
Extending this study, Levy and Murnane (2004) attributethe growth of non-routine cognitive tasks to jobs requiring
skills in expert thinking and complex communication.
Similar frameworks have been used in the trade literature,especially in the context of outsourcing and offshoring andalso to understand the emergence of new occupations
such as green jobs (Consoli et al., 2016).
A growing body of work also underscores the role of ‘noncognitive’ skills, including social skills and leadership skills.
This derives from the pivotal insight of Heckman (1995) thatlabour market outcomes such as earnings are likely shapedby an array of skills insofar as measured cognitive ability
accounts for only a small portion of the variation in suchoutcomes (Heckman and Kautz, 2012). Deming (2015) finds
that, in the US, nearly all job growth since 1980 has beenin occupations that are relatively social-skill intensive.
Strikingly, occupations with high analytical but low social
skill requirements shrank over the same period. One
possible explanation is that social skills provide the tools
for the rich and versatile coordination which underpins a
productive workplace – the subtleties of which computers
have yet to master. This matters for organisations incomplex environments where the classic gains from
specialisation are eclipsed by the need to adapt flexibly tochanging circumstances (Dessein and Santos, 2006).
Measures of social intelligence have also been validated bypsychologists and neuroscientists (Poropat, 2009; Woolley etal., 2010). Indeed, more recent thinking rejects the contrast
between cognitive and non-cognitive skills. Whereas reason
is widely viewed as a path to greater knowledge and better

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

decision-making, some argue that it is much more diverse
and opportunistic – that it has evolved primarily to help
humans justify themselves and influence others, which isindispensable for communication and cooperation. Patternsof thinking that appear ‘irrational’ from a purely cognitive
perspective turn out to be advantageous when seen as
adaptive responses to the dilemmas of social interaction(Mercier and Sperber, 2017).
Unencumbered by analytical tractability, policymakers
have embraced a still wider understanding of skills. Over the
past two decades there has been considerable thinking andadvocacy – both nationally and internationally – focussed
on embedding so-called ‘21st century skills’ into education
systems. The policy literature uses a range of overlapping
concepts, taxonomies, definitions and technical language,
but at their core, skills are viewed as encompassing thefull panoply of cognitive, intrapersonal and interpersonal
competencies (National Research Council, 2012; Reimers
and Chung, 2016).
We are only aware of a handful of academic studies thatview skills in these broad terms: for example, Liu and Grusky(2013) develop an eight-factor representation of workplaceskills, though they focus on returns to skills that reflectchanges in relative supply as much as demand. MacCrory etal. (2014) perform principal component analysis on abilities,
work activities and skills in O*NET to identify five to sevendistinct skills categories that have discriminatory power interms of explaining changes in the skills content
of occupations over the past decade.
We extend this body of work – in part by also drawing
on the knowledge features in O*NET which provide
information on the specific academic subjects and domain
knowledge required by occupations. One advantage of
the O*NET knowledge features is that they are expressedin relatively natural units which make them easier tounderstand and address through policy. They also touchon the knowledge versus skills debate in education
circles between proponents who argue that curriculum
and pedagogy should teach transferable–skills – theability to work in teams, to create and think critically –and those who contend that skills need to be rigorously
grounded in a base of knowledge in order to be mastered(Christodoulou, 2014; Hirsch, 2016).
2.3. LABOUR MARKETS AND STRUCTURAL CHANGE
This report also addresses work on the employment
effects of automation and structural change moregenerally. The rise of robots, artificial intelligence, big data
and the internet of things have raised concerns about thewidespread substitution of machines for labour. Evidence
linking automation of many low-skilled and medium-skilledoccupations to wage inequality, labour market polarisationand the ongoing decline in manufacturing jobs is interpreted
as support for the claim that workers are falling behind in
the race against machines (Autor et al., 2006, 2008; Black

22

and Spitz-Oener, 2010; Dustmann et al., 2009; Goos andManning, 2007; Michaels et al., 2009; Spitz-Oener, 2006).
Technological anxiety is not a new phenomenon (Keynes,
1930; Bix, 2000; Mokyr et al., 2015). Similar fears have been
expressed before: during the Industrial Revolution, thelatter part of the 1930s, and again immediately after WorldWar II. Each time adjustment was disruptively painful forsome workers and industries; but in the long run, such fears
were not realised.5 The employment-to-population ratio
grew during most of the 19th and 20th centuries in the UK
and US, even as the economy experienced the effects ofmechanisation, the taming of electricity, the invention of theautomobile and the spread of mass communication.
History cannot settle whether this time is different: what is
striking about the perspectives of earlier observers is how
narrowly they defined the scope of what technology could
accomplish. Earlier generations of machines were limited
to manual and cognitive routine activities, based on welldefined, repetitive procedures. The newest technology,
by contrast, is mimicing the human body and mind inincreasingly subtle ways, encroaching on many non-routine
activities, from legal writing and truck driving to medical
diagnoses and security guarding.6
The case of driverless cars illustrates the slippery and shifting
definitional boundary around what it means for work to
be ‘routine’. In their seminal 2004 book The New Division ofLabor: How Computers Are Creating the Next Job Market, Levy
and Murnane (2004) argued that driving in traffic, insofar
as it is reliant on human perception, fundamentally resisted
automation: “Executing a left turn against oncoming traffic
involves so many factors that it is hard to imagine discovering
the set of rules that can replicate a driver’s behaviour [. . .]”.
Formidable technical challenges lie ahead: the prospect offleets of cars that can roam across cities or countries in allconditions without human input remains remote (Simonite,
2016; Mims, 2016). Nonetheless, elements of this problem
are now satisfactorily understood and can be specified
in computer code and automated. For example, Google’s
driverless cars have driven over 2 million miles in the past six
years, and have been involved in 16 minor accidents, noneof which caused injury or was the car’s fault (Bank of America
Merrill Lynch, 2017).
A more forward-looking approach can help guard against thesepitfalls. This is exemplified by Frey and Osborne (2017), who
assess the feasibility of automating existing jobs assuming thatnew technologies are implemented across industries on a larger
scale. In this study, a sample of occupations was hand-labelled
by machine-learning experts as strictly automatable or not
automatable. Using a standardised set of nine O*NET featuresof an occupation that measure three bottlenecks to automation– perception and manipulation, creative intelligence, and social
intelligence – they then ran a classifier algorithm to generate a‘probability of computerisation’ across all jobs, estimating thatover the next two decades, 47% of US workers' jobs are ata high risk of automation.

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0This finding has not gone unchallenged. MacCrory et al.
(2014) point out that a handful of variables cannot capture
the diverse economic impact of technological change on
skills, especially across the whole gamut of occupations in
the labour market. Arntz et al. (2016) observe that, within
an occupation, many workers specialise in tasks that cannot
be automated. Using the automation probabilities fromthe Frey and Osborne study and drawing on the Surveyof Adult Skills by the Programme for the International
Assessment of Adult Competencies (PIAAC) that examines
task structures for individuals across more than 20 OECD
countries, they argue that once task variation is taken
into account, a much smaller proportion of jobs (~ 9%)are at risk of being completely displaced. They also findimportant differences across countries that are attributed
to variations in workplace organisation, adoption of newtechnologies and educational levels.7
The McKinsey Global Institute (2017) disaggregatesoccupations into 2,000 constituent activities, rating eachagainst 18 human capabilities and the extent to which they
can be substituted by machines. It estimates that 49% ofwork activities globally have the potential to be automated,though very few occupations – less than 5% – are candidates
for full automation (see also Brandes and Wattenhofer, 2016).
While these studies confirm the importance of considering
the automatability at the task level, this approach raises
its own challenges. In principle, there is nothing in an
occupation-based approach that prevents analysts fromconsidering its constituent tasks when evaluating thepotential for automation. There are also drawbacks with astrictly bottom-up approach in the context of anticipating
occupational and skills demand. In isolation, one mightreasonably infer that similar tasks, such as sales, have
similar levels of demand. But as part of an occupation,
they also belong to different industries with different
growth prospects and require different knowledge
connected to the product, or the buyers of the product(for instance, consider an insurance sales agent vs. a solar
equipment sales representative). By emphasising discretetasks, there is a risk of losing important coordinating
information which gives occupations their coherence –the fabric which distinguishes the whole from the parts.
Practically, unbundling occupations may come at theexpense of quality. Autor (2013) counters the simple view
– popular in some parts of the automation debate – thatjobs can be ‘redefined’ as machines perform routine tasks
and workers perform the rest: “Consider the commonplace
frustration calling a software firm for technical support
only to discover that the support technician knows nothing
more than what is on his or her computer screen – that is,
the technician is a mouthpiece, not a problem solver.

23

This example captures one feasible division of labor:
machines performing routine technical tasks, such as
looking up known issues in a support database, andworkers performing the manual task of making polite
conversation while reading aloud from a script. But thisis not generally a productive form of work organization
because it fails to harness the complementarities between
technical and interpersonal skills”.8
A limitation of all these studies is that they only estimate
which occupations are potentially automatable – not how
many will actually be automated. As discussed earlier,the journey from technical feasibility to full adoption cantake decades involving many steps and missteps. Just
as significantly they do not assess the potential for job
creation in tasks and occupations complemented byautomation or the adjustments that are triggered in otherparts of the economy through relative wage changes andother market forces (Shah et al., 2011; Davenport and
Kirby, 2016; Kasparov, 2017).
The effect of fleshing out these dynamics is to substantiallymuddy and possibly reverse more pessimistic conclusions.
Gregory et al. (2016) develop a task-based framework,
estimating that automation boosted net labour demandacross Europe by up to 11.6 million jobs over the period
1990–2010. They identify a number of channels that
potentially compensate for the job-destroying effects ofautomation, including first, that automation may lead tolower unit costs and prices which stimulate higher demandfor products, and, second, that surplus income frominnovation can be converted into additional spending, so
generating demand for extra jobs in more automationresistant sectors (see also Goos et al., 2015).9
However, a number of strong assumptions are necessary
for this result – notably that additional firm profits are
spent locally in Europe when in fact they may accrue tonon-European shareholders. Relaxing this assumption
results in significantly lower estimates, although they arestill positive (1.9 million jobs). This finding has particular
relevance to debates about ‘who owns the capital’ andthe case for spreading ownership of robot capital throughprofit-sharing programmes and employee stock-ownershipplans (Freeman, 2015).
Acemoglu and Restrepo (2017a) report contrasting results.
They explore the impact of the increase in industrial robot
usage between 1990 and 2007 on US local labour markets.
To identify causality, they use industry-level spread ofrobots in other advanced economies as an instrument
for robot usage in the US. This strategy helps isolate thechange in exposure to robots from other organisational orindustry developments that may also be correlated withrobot usage and influence subsequent labour demand.
They find that each additional robot reduces employment
by about seven workers, with limited evidence of offsetting
employment gains in other industries.10

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0This finding is robust to a range of different specifications,tests and controls, such as demographic and industry
characteristics, share of routine jobs, import competition
from China and overall capital utilisation. Excluding theautomobile industry, the heaviest user of robots, theintroduction of robots does not change results when
the localities most affected by robots had similar
employment and wage levels to other localities –which is to say they were not on a downhill pathbefore robotisation.
Although Acemoglu and Restrepo's study (2017a) is
an important step for understanding the dynamic
employment effects of automation, it is far from the final
word: in the main, it does not address the global effects
of automation which are important given rich patterns oftrade, migration and specialisation across local markets.
Also, with the robot revolution still in its infancy, shortterm consequences may differ from the long-term ones,
once relative prices and investment have had time tofully adjust. Evidence of diminishing marginal returns torobot usage documented by Graetz and Michaels (2015) is
consistent with such a view.
The need to recognise the interactions embedded intrends carries across into other spheres. Parallel toautomation is a set of broader demographic, economic
and geopolitical trends which not only have profound
implications for labour markets, but are raising challenges
for policy in their own right. In some cases, trends are
reinforcing one another; in others, they are producing
second-order effects which may be missed when viewed
in isolation. Consider, for example, the implications of anageing population. While much of the automation debate
has focussed on the potential for mass unemployment,
it overlooks the fact that robots may be required tomaintain economic growth in response to lower labour
force participation. The risk, in other words, is not that
there will be too few jobs but that there will be too few
people to fill them – bidding up wages in the process –which may explain why countries undergoing more rapid
population ageing tend to adopt more robots (Acemoglu
and Restrepo, 2017b; Abeliansky and Prettner, 2017). We
provide an overview of the trends in the following display, anda description of how the trends analysis fits in with our wider
approach in Section 3.

24

The future of work – understood in its widest sense – hasbeen climbing the policy agenda. The topic has featured
on the covers of the popular press; major organisations,
think tanks and consultancies have hosted conferenceson the subject and it has generated a flurry of reports andstudies (PwC, 2016; UK Commission for Employment andSkills, 2014; Beblavý et al., 2015; World Economic Forum(WEF), 2016; Hajkowicz et al., 2016; Canadian Scholarship
Trust (CST), 2017). These efforts mirror our approach
insofar as they assess the multiple trends affecting theemployment and skills landscape. However, there is an
important respect in which their treatment of the trends
differs: in addition to being qualitative in nature, many use
scenario techniques to weave trends into a set of internallycoherent and distinct narratives about the future. Thisapproach brings order and depth to conversations, though
it also has drawbacks. Because scenarios are typicallytaken as given, assumptions about how change takes
place and trends interact are often opaque. This makes
it difficult to explain why one set of scenarios has been
selected from among the infinite number that are possible,
which has the impact of shutting off outcomes whichmight otherwise emerge if these dynamics were explicitly
accounted for (Miller, 2006).

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

OVERVIEW OF THE TRENDSTECHNOLOGICAL CHANGE
Greater connectivity and improvements in computing power and artificial intelligence are enabling ‘intelligence’to be embedded more cheaply and readily in physical systems – from entire cities right down to the individual
human body. With peer-to-peer platforms, activities are amenable to decentralised production, unlocking previously
unused or underused assets – in the process muddying traditional definitions of ownership and employment.
Additive manufacturing and 3D printing could alter the economics of many industries, cutting the costs of
on-demand production.
Material and life sciences have seen major breakthroughs in areas such as graphene and gene-editing with potentiallyradical applications. However, many must contend with a long road to commercialisation and significant barriers to
adoption, especially ethical and safety concerns.
There remains an unresolved tension between the seeming ubiquity of digital technology and a downshift inmeasured productivity growth. Evidence from a wide range of industries, products and firms also suggests that
research effort is rising steeply while research productivity is falling sharply – that is, more and more resources arebeing allocated to R&D in order to maintain constant growth.
History shows that technology optimism can slide into determinism, though there is a mirror image of this logic: people
tend to underestimate the huge effects of technology over the long term. The general pattern of technological progresshas been one of multiple lulls, followed by subsequent surges of creativity. For instance, new materials and processes,
leveraging digital tools that allow improved real-time measurement, experimentation and replication, are inherently
complementary such that advances in one domain may feed back into new technologies in a virtuous cycle.

GLOBALISATIONOver the past three decades, labour markets around the world have become increasingly integrated.
The emergence of countries like China and India, for centuries economic underperformers, has delivered animmense supply shock to traditional patterns of trade.
Globalisation has not only had benefits but also costs. Employment and wages have typically fallen in industries
more exposed to import competition, exacerbated by labour market frictions and social and financial commitments
such as home ownership, which limit workers’ ability to relocate and take advantage of employment opportunities.
US and UK exports to other countries have not grown as much as imports, though large trade deficits have not
reduced jobs so much as redistributed them towards non-tradables, particularly construction.
The manufacturing sector has been a lightning rod for these changes, though the experience has not been
uniform, containing pockets of activity that have thrived – whether because the gains from keeping production athome remain critical or head-to-head competition with emerging economies is limited. Combined with eroding
cost advantages among competitors and new technologies, including breakthroughs in shale oil extraction, these
conditions could support modest forms of reshoring.
As the economic centre of gravity shifts towards the emerging world, supported by a burgeoning middle class,so opportunities may open up in areas such as knowledge-intensive services and advanced manufacturing where UK
and US exporters enjoy a comparative advantage.
However, a number of developments may frustrate this trend. Services are still substantially less likely to be
traded than manufactured products due to the prevalence of non-tariff barriers. Emerging markets face various
obstacles in sustaining their historic growth rates, ranging from the prospect of premature deindustrialisation
to the task of building high-quality institutions.
Sluggish world trade growth since the financial crisis and stiffening protectionist sentiment have challenged
the decades-old rule of thumb that trade grows faster than GDP, raising concerns that globalisation has
structurally ‘peaked’.

25THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

DEMOGR APHIC CHANGE
There is an obvious appeal to basing predictions on population growth and changes to the composition of the
population: given long-term historical trends, it is possible to make grounded assessments about where they arelikely to go, and at what speed.
The global economy has passed an important demographic threshold: dependency ratios – the ratio of non-working
age population to working age population – have begun to rise after nearly half a century of declines. With labour inputs
slowing in advanced economies, the importance of productivity in driving overall growth and policy in boosting labour
force participation has increased. This is especially true in the US, where prime-aged women and particularly men have been
withdrawing from the labour market over a long period.
Countries are coming under fiscal pressure to control age-related entitlements which could draw resources awayfrom education, R&D and infrastructure, especially as older households vote more actively than younger cohorts.
How an ageing population chooses to put its purchasing power to work will have a significant impact on the fortunesof different industries and occupations. This is likely to benefit not only healthcare, finance and housing but also
recreation and education which have traditionally catered to the young.
Millennials – the cohort born between 1980 and 2000 – are poised to grow in influence as they inherit the assets of theirparents. They are the first group to come of age after the arrival of digital technology, bringing with them heightened
expectations of immediacy, participation and transparency. At the same time many became economically active in the
shadow of the Great Recession which may have tempered attitudes to risk and confidence in major institutions.
As a result, this group exhibits quite different consumption and work behaviours compared with previous generations.

ENVIRONMENTAL SUSTAINABILIT Y
A striking development over the past decade has been the growing consensus around man-made global warming.
The scale of the challenge is enormous: to keep the rise in average global temperatures to below 2°C – the de facto
target for global policy – cumulative CO2 emissions need to be capped at one trillion metric tonnes above the levelsof the late 1800s. The global economy has already produced half of that amount.
Climate change has wide-ranging consequences for many industries. Agriculture, tourism, insurance, forestry, water,
infrastructure and energy will all be directly affected, though linkages with socio-economic and technological systems meanthat risks can accumulate, propagate and culminate in even larger impacts. For example, climate change could threaten foodand resource security in parts of the world which may in turn make poverty and conflict more likely.
Meeting emissions reduction targets requires investment in green technologies, including LED lighting, electric vehicles,
solar photovoltaic systems and onshore wind and more sophisticated forms of energy efficiency – which will also create
opportunities for green finance. Despite plummeting costs for ‘clean’ solutions, there are a number of reasons why ‘dirty’technologies are likely to hold sway, including political lock-in and high switching costs. Views differ as to the optimal dynamic
strategy to be followed in such a scenario. Evidence for the broader jobs potential of the green economy is also ambiguous.
Structural changes associated with the green economy are fundamentally dependent on government policy. The number of
supportive regulations has grown across the world. Although initiatives are likely to be set nationally rather than multilaterally,remain tied to specific sectors and technologies and are vulnerable to political reversals.

26

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

URBANISATION
Today, over half of the world’s population live in cities, a number that is expected to grow to 70% by 2050.
This concentration of humanity illustrates the basic unevenness of economic development – the tendency forplaces close to large markets to grow more rapidly than places more distant.
Cities are magnets for high-value, knowledge-intensive industries, where physical proximity enables collaborationand firms and workers benefit from enhanced labour pooling and matching. Urban planners increasingly build
these features into the fabric of cities through the establishment of innovation districts that integrate work,
housing and recreation.
Cities also offer more varied consumption and employment opportunities, though medical conditions such as obesity,
diabetes and depression have been linked to aspects of the urban environment. Pressures on affordable housing may
lock some households out of these opportunities, forcing them to live in older suburbs or low-income areas tethered
to declining industries.
There has been an increasing push for authorities to make cities ‘smarter’ – leveraging the information generated byinfrastructure to optimise performance. This agenda has also come to focus on sustainability, resilience and making
cities more age-friendly. However, in many advanced economies infrastructure investment as a proportion of total
government expenditure has been trending downwards for decades, with serious questions around the quality ofexisting infrastructure. A major uncertainty is how debates about the role of fiscal policy and government activism
will resolve themselves against a backdrop of high public debt ratios.

INCREASING INEQUALIT Y
The sharp rise in income and wealth inequality has been described as the “defining challenge of our times”.
This has been accompanied by a squeeze on the middle class, as the distribution of income has shifted towards
the higher and lower ends of the scale. Countries with higher levels of income inequality tend to have lower levelsof mobility between generations.
Economic distress and the erosion in opportunities for people with low education have, in turn, created a web of social
issues, including rising mortality and morbidity among segments of the US population. They have also fuelled resentmentof elites and the appeal of populist ideas.
The macroeconomic relationship between inequality and growth remains contested – though recent studies havetended to highlight the costs of rising inequality, particularly over longer time horizons. Income inequality andassociated phenomena also have sectoral consequences. They contribute to greater health and social problems,raising the demand for healthcare and social services. Employment in occupations dedicated to protecting property
rights and managing conflict is typically larger in countries with unequal distribution of income. Finally they translate
into disparities in consumption, particularly of non-durable goods and services such as education.
A number of factors have driven higher inequality, including the impact of technology and globalisation, failingsof the educational system, anticompetitive practices, weaknesses in corporate governance, the decline in union
membership and the progressivity of the tax system. Some of these trends may potentially reverse in the future –for instance as ageing reduces labour supply, pushing up wages, or as calls for redistribution grow louder, althoughthese forces are most likely to operate at the margins. Past experience suggests that current levels of inequality
are likely to persist in the medium term, absent an extreme shock of some kind.

27

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

POLITICAL UNCERTAINT Y
The geopolitical landscape is characterised by a greater distribution of power that has challenged the capacity of
the international system to respond effectively to a host of regional and global challenges – from the spread of nuclear
weapons, authoritarianism and terrorism, through historical rivalries in the Middle East and Asia-Pacific to a growing
rejection of free trade and immigration. Indices of geopolitical uncertainty have persisted at higher levels after they
spiked on 9/11. This has been mirrored by elevated policy uncertainty – a weakening of the institutional structures
which enable policymakers to act credibly and consistently.
Increases in uncertainty are found to have significant negative impacts on economic activity, raising the user costof capital, increasing the option value of deferring investments where there are sunk costs, and hindering the efficient
reallocation of resources from low to high productivity firms. The impacts are felt most strongly in sectors like defence,
finance, construction, engineering and healthcare which require extensive investment commitments and/or are
exposed to uncertain government programmes.
The trend towards policy uncertainty is a function of structural changes in political systems – above all, the risein partisanship which has impeded compromise and effective negotiation, reinforced by the growth in the scaleand complexity of government regulation. Even in systems designed to produce moderation, institutions may have
had the effect of marginalising important conflicts over policy rather than resolving them, increasing public apathy
and dissatisfaction.

28

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

3. APPROACH
The dynamic interdependencies of the trends have
implications for our research design. In an earlier piece ofanalysis, we reviewed the drivers of change that are expected
to shape industry and occupational structures in the US
and UK workforces in 2030 (Schneider et al, 2017). Drivers
were selected on the basis that they are relatively stablewith clear possible directions. Where the available evidence
offered contrasting views of a trend and its implications or
identified possible disruptions, the analysis aimed to capture
this uncertainty – rather than reach a verdict on how things
would play out.
This analysis was used to contextualise and guidediscussions at two foresight workshops that we convened
between small groups of thought leaders with domain
expertise in at least one of the seven trends identified.
These foresight workshops were held in Boston on 20
October 2016 with 12 participants and in London on 28
October 2016 with 13 participants.
In the second part of the US and UK workshops, thedomain experts were presented with three sets of 10
individual occupations at the six-digit and four-digit
Standard Occupation Classification (SOC) levels, and wereinvited to debate their future prospects. They then assigned
labels to the occupations (individually) according to whether
they thought they would experience rising, unchanged or
declining demand by 2030.11 The experts were also asked
to record how certain/uncertain they were in making theirpredictions. Factsheets presenting information on each of
the 30 occupations (containing a list of related job titles,
related industries, key skills and tasks) and their historical
growth patterns were made available to the experts when
making their predictions.
We used these labels to train a machine-learning classifier
to generate predictions for all occupations, making useof a detailed data set of 120 skills, abilities and knowledge
features against which the US Department of Labor’s
O*NET service 'scores' all four-digit occupations in the
US SOC on a consistent basis (we used a cross-walk toalso apply this data set to the UK SOC). To maximise
the performance of the algorithm we used an active
learning method whereby the second and third sets of 10
occupations to be labelled were selected by the algorithm
itself (intuitively, these occupations were selected to coverthat part of the skills/abilities/knowledge space where thealgorithm exhibited highest levels of uncertainty based on
the previously labelled occupations).12 From the model wedetermined which skills, abilities and knowledge features
were most associated (on their own and together) withrising or declining occupations.
Our mixed methodology approach – making use ofstructured foresight and supervised machine-learningtechniques – was crafted to tackle the limitations in
traditional qualitative and quantitative exercises. In
particular, qualitative approaches based purely on eliciting
the judgements of experts are likely to be subject to29human biases, while quantitative approaches based purely
on trend extrapolation are likely to miss structural breaks
in past trends and behaviours. By combining a machinelearning algorithm with structured expert judgment wehope to have the best of both worlds.
Our research design is elaborate, matching the ambitious
nature of our research question, but it is important tonote that, as a consequence, our findings could reflect
any number of assumptions. For example, the subjective
judgments of one group of domain experts could be very
different to another, or some parts of the O*NET data set
could be more accurate characterisations of occupations
than others. The provision of historical data on occupations
and the main trends designed to establish a common frameof reference among experts mitigates some of these risks.
However, it remains the case that predictions generated
might have differed if a different group of experts had
participated in the workshops or if we had used different
selections of O*NET features in our model.
A separate, though related, challenge is how to evaluate
our findings. As a forward-looking exercise, we might simply
compare our predictions with labour market outcomes in
2030. Notwithstanding the fact that this is 13 years away,
a concern is that because our predictions are conditional(see above), we cannot in any straightforward way identify
the source of prediction errors.
We try and partly tackle this by investigating the sensitivityof our findings to key features of our research design.
In particular, we present predictions that used (nonparametric) trend extrapolation of employment inan occupation to label the 30 occupations in place ofthe experts’ judgments. These data-driven labels givea baseline against which those built on our foresight
exercises can be compared.
It is also important to note that our UK and US findings are
not directly comparable. While the common use of O*NETdata means that it is tempting to compare the O*NETfeatures that are most and least associated with predicted
higher demand occupations in the two countries, wewould actually have made significant changes to theresearch design if our objective had been to undertake across-country study. For example, we might have asked
one common group of domain experts to label occupation
prospects for both the US and UK using a set of the mostsimilar occupation groups (crosswalked) across the twocountries. Differences in the SOC structures in the twocountries also complicate comparison of the occupation
predictions. As such, while our use of standardised
reporting in the US and UK results might invitecomparison and contrast, any such inference would not bevalid. Our focus on obtaining the best possible results foreach country compromises our ability to compare the two.

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

4. DATA
4.1. O*NETTo derive the demand for skills, abilities and knowledge
from our occupational projections, we rely on data fromthe (O*NET), a survey produced for the US Departmentof Labor (Occupational Information Network (O*NET),2017). The O*NET survey contains information on morethan 1,000 detailed occupations, using a modified formof the (SOC) system.13 It began in 1998 and is updated
on a rolling basis by surveys of each occupation’s worker
population as well as job analysts' assessments.
The scope and sampling of O*NET are viewed as an
improvement on its predecessor, the Dictionary of
Occupational Titles (DOT) and also standard household
surveys where self-reporting can result in substantial
measurement errors. Reported response rates are high –at around 65% – and have been rising over time (Handel,2016). We take advantage of the 2016 O*NET to reflect
most accurately the current make-up of occupations,though results from previous versions of O*NET are
broadly similar.
A major strength of O*NET is that it asks many different
questions about the skills, abilities, knowledge and work
activities that make up occupations. Respondents/analysts
are asked about the importance of a particular feature fora job (for example, critical thinking, persuasion, manual
dexterity and so on) and the level or amount of the feature
required to perform it. The questions are rated on an ordinalscale which are standardised to a scale ranging from 0 to100. We use all 120 features from the skills, abilities andknowledge categories in O*NET, designed to provide asrich a picture of occupations as possible.14 These features
are detailed in Table 1.
Our implementation strategy departs from Frey andOsborne's (2017) study of automation in that it relies
on O*NET’s ‘importance’ rating. Analyses of O*NET datasuggest that there is substantial overlap between theimportance and level ratings, so this modelling choice does
not lead to vastly different predictions in practice (results
are available on request from the authors). Critically, theimportance rating is available for all combinations offeatures and occupations. This is in marked contrast to thelevel rating for which O*NET recommends suppressing
a large number of estimates on account of their low
precision. This problem is most serious for knowledge
features: to implement O*NET’s recommendations in fullwould entail removing occupations equivalent to 89% of
total US employment. Similarly, the scales and anchor
points used to construct the level ratings have been
criticised for their complexity which may affect the reliabilityof some ratings (Handel, 2016).

305.4). Finally, we use the model to predict future occupati
hotspots of high demand that are not associated with an exis
(Section 5.5).
To benchmark the efficacy of our foresight exercise, we alprocesses to perform extrapolation out to the year 2030 of ptrends (Section 5.6). These extrapolations provide alternabels produced in the workshops, and provide results that do
subjective judgments of the experts.
5.1. Gaussian processes

In this section, we give a brief review of Gaussian processes.
(Rasmussen and Williams, 2006) is a probability distribution
f : X → R, such that the marginal distribution over the fun
any finite
subsetof X (suchas X) is multivariate Gaussian.
In the remainder of this report,
we willuse
to represent
f (x), the prior distribution over its values f on a subset x ⊂ X
the vector of length 120 containing
thesevariables.
specified by a mean vector m and covariance matrix K

In our workshop factsheets, we also include the11
exp − f  K
p(f | K) := N (f ; m, K) := √2
occupation description and five common job title
det 2πK
15 by a covariance function
Thealsocovariancegenerated
examples for the occupation,
takenmatrix
from isO*NET.

that is, K = κ(X, X).
Given training data D, we use the gp to make predict
function values f∗ at input x∗ . With this information, we hav
equations

where



p(f∗ | x∗ , D) = N f∗ ; m(f∗ | x∗ , D), V (f∗ | x∗ , Dm(f∗ | x∗ , D) = K(x∗ , X)K(X, X)−1 y
V (f∗ | x∗ , D) = K(x∗ , x∗ ) − K(x∗ , X)K(X, X)−1 K(

Inferring the label posterior p(y∗ | x∗ , D) is complicated by thform of the ordinal likelihood to be introduced below.
5.2. Heteroskedastic ordinal regression

Gaussian process ordinal regression consists of combining a
is, ordered numerical scores) observations with a Gaussian36

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 1: List of all O*NET features used in this study
T YPE
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Skill
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge

FEATURE
Reading Comprehension
Active Listening
Writing
Speaking
Mathematics
Science
Critical Thinking
Active Learning
Learning Strategies
Monitoring
Social Perceptiveness
Coordination
Persuasion
Negotiation
Instructing
Service Orientation
Complex Problem Solving
Operations Analysis
Technology Design
Equipment Selection
Installation
Programming
Operation Monitoring
Operation and Control
Equipment MaintenanceTroubleshooting
Repairing
Quality Control Analysis
Judgment and decision-making
Systems Analysis
Systems Evaluation
Time Management
Management of Financial Resources
Management of Material Resources
Management of Personnel Resources
Administration and Management
ClericalEconomics and Accounting
Sales and Marketing
Customer and Personal Service
Personnel and Human Resources
Production and Processing
Food Production
Computers and Electronics
Engineering and Technology
DesignBuilding and Construction
MechanicalMathematics
Physics
Chemistry
Biology
Psychology
Sociology and Anthropology
Geography
Medicine and Dentistry
Therapy and Counseling
Education and Training
English Language
Foreign Language

31

T YPE
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability
Ability

FEATURE
Fine Arts
History and Archeology
Philosophy and Theology
Public Safety and Security
Law and Government
Telecommunications
Communications and Media
Transportation
Oral Comprehension
Written Comprehension
Oral Expression
Written Expression
Fluency of Ideas
Originality
Problem Sensitivity
Deductive Reasoning
Inductive Reasoning
Information Ordering
Category Flexibility
Mathematical Reasoning
Number Facility
Memorization
Speed of Closure
Flexibility of Closure
Perceptual Speed
Spatial Orientation
Visualization
Selective Attention
Time Sharing
Arm–hand Steadiness
Manual Dexterity
Finger Dexterity
Control Precision
Multilimb Coordination
Response Orientation
Rate Control
Reaction Time
Ability Wrist-Finger Speed
Speed of Limb Movement
Static Strength
Explosive Strength
Dynamic Strength
Trunk Strength
Stamina
Extent Flexibility
Dynamic Flexibility
Gross Body Coordination
Gross Body Equilibrium
Near Vision
Far Vision
Visual Color Discrimination
Night Vision
Peripheral Vision
Depth Perception
Glare Sensitivity
Hearing Sensitivity
Auditory Attention
Sound Localization
Speech Recognition
Speech Clarity

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 04.2. EMPLOYMENT MICRODATA
To form yearly estimates of employment by occupationand industry for our workshops, we used US and UK
employment microdata. For the US, we used 1983 –2015 data from the Current Population Survey’s (CPS)Annual Social and Economic Supplement (ASEC) fromthe Integrated Public Use Microdata Series (IPUMS) by
the Minnesota Population Center (King et al., 2010). We
used an IPUMS-provided best-guess harmonisation ofoccupation codes over time to 1990 Census occupation
codes. We then crosswalked these codes to six-digit US
SOC 2010 codes.
For industry, we presented CPS-derived estimates ofoccupation employment by industry in 2015, harmonisedin the same way. We used the most granular common
level of North American Industry Classification System(NAICS) 2012 code available for that occupation (either the
four-, three-, or two-digit level).
The comparable high-resolution microdata is only readily
available in the UK over a shorter time period due tochallenges in matching the SOC codes across changes in
the classification over time. We used yearly occupational
employment estimates based on the Labour Force
Survey and provided by the Office for National Statistics(ONS) for 2001 and 2016 inclusive (ONS, 2017a). These
were provided at the four-digit ONS SOC 2010 level, theequivalent level of granularity as the six-digit US SOC
2010 occupation code. We further generated estimatesof occupation employment by UK Standard Industrial
Classification (SIC) 2007 industry class and subclass in2015 using the Labour Force Survey provided by ONS(2017b).
The US employment results from our machine-learningclassifier were weighted using the May 2015 Occupational
Employment Statistics from the Bureau of Labor Statistics(US Bureau of Labor Statistics, 2015). The corresponding
UK employment results are weighted using the August
2016 Labour Force Survey from the Office for National
Statistics (ONS, 2017a).

Over the course of the workshop, the group participated inthree prediction sessions. In each session, the participants
viewed the information described above for 10 occupations,displayed on two slides ('factsheets') per occupation. (SeeFigures 2a and 2b for examples of these factsheets.)
Note that the factsheets presented time-series plots of theoccupation to participants, such that they could form theirpredictions with proper historical context. After viewing theoccupation descriptions the group was directed to an online
form to answer two questions:
1. What will happen to the share of total employment
held by this occupation?
{Higher share, Same share, Lower share}
2. What will happen to the number of people
employed in this occupation?
{Grow, No change, Decline}
With only a three-point scale, it was important to consider
both employment share and absolute employment levels
so as to allow a fuller expression of judgments of future
demand. For example, only knowing that an occupation
will grow slower than the workforce as a whole says
nothing about whether it will add or shed jobs.16 In theevent, both US and UK workshops were of the view – albeit
with significant differences in opinion across individuals– that total employment would grow over the prediction
horizon, consistent with the historical pattern.
Given the inherent difficulties in making long-term predictions,
our workshop participants were also asked to provide a
0-9 ranking of how certain they were in their answer, with 0
representing not certain at all, and 9 representing completely
certain. They were also given a space to provide freeformthoughts they felt were necessary to qualify their answers.

4.3. WORKSHOP-GENERATED DATA

After the group submitted their answers using the online
form, an experienced foresight workshop facilitator
reviewed the responses with the group. After the groupdebated their perspectives during a half-hour session,
the group was then allowed to change their answers,
after which the workshop moved onto the next setof 10 occupations.

To collect labels to train our machine-learning classifierof future demand for occupations, we held two expert
foresight workshops in Boston and London in October
2016. Each workshop brought together a diverse groupof 12 to 13 experts from industry, government, academia,
and the social sector. Our experts were instructed toconsider the net impact on the workforce occupation
composition of all the trends discussed above, guided byour trends analysis. Figure 1 features a sample page ofthe trends analysis, which was shared with our experts inadvance, and presented at the workshops.

The first 10 occupations presented to participants wereselected randomly. For the second and third rounds, therespective batches of 10 occupations were selected so
as to be maximally informative for the machine-learningmodel in light of the answers previously gathered fromparticipants. In a way, the participants could be seen as
teaching the machine-learning algorithm throughout thecourse of the day, with the algorithm able to respondto the information from participants by proposing aprioritised list of further questions. This process is
described formally in Section 5.3.

32

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Figure 1: Sample page from trends analysis presented at the workshops

Future of Skills

Technological change

Peak globalisation?

Globalisation
World trade (percentage of GDP)

Unwinding trade imbalances• Leveling off of offshoring?

Urbanisation• Stabilisation of China’s manufacturing share

Increasing inequality• Stronger domestic production base inemerging economies

Political uncertainty
References

40

30

20

• Weaker (trade-intensive) business fixed investment
as percentage of GDP in advanced economies
Going forward
If trade slowdown is structural, impacts of tradeon labour market will in future be very different
from what they have been in past.

Source – World Bank (2016)20
15

Environmental
sustainability

50

20
05

Demographic change

60

19
95

Growing global
middle class70

Evidence that trade has become less responsive
to global GDP growth - suggesting that trade
slowdown is not just a temporary phenomenon
reflecting the crisis (Constantinescu, Mattoo
and Ruta, 2014).

19
85

Specific trade opportunities

World trade (percentage of GDP)

19
75

The importance of place

Rapid expansion of global trade may haverun its course

19
65

Peak globalisation?

Source: World bank

Four year rolling sensitivity (elasticity) of global
real-trade growth to global real-GDP growth4

End of Soviet Union(1991)

Oil Shock(1979)US recession(1960-61)
2

GATT(1947-94)

WTO(1995-Present)

0
1955

1971

1987

2003

Source – Goldman Sachs (2016)

46

33

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Figure 2A: Factsheet for UK occupation farm workers

Farm workers (9111)
DESCRIPTION

Farm workers perform a variety of tasks, by hand and machine, to produce and harvest crops and to breed andrear cattle, sheep, pigs and poultry.

SAMPLE JOB TITLES

Agricultural Worker; Agricultural Labourer; Estate Labourer; Gang Man (Agriculture); Agricultural Craftsman

TOP JOB TASKS

Operates farm machinery to prepare soil, fertilise and treat crops.
Cultivates growing crops by hoeing, spraying and thinning as necessary.
Weighs and measures foodstuffs, feeds animals and checks them for any signs of disease.
Cleans barns, sheds, pens, yards, incubators and breeding units and sterilises milking and other equipment as
necessary.
Treats minor ailments and assists veterinary surgeon as required.

TOP JOB SKILLS

Critical Thinking
Active Listening
Monitoring
Operation and Control
Operation Monitoring

Skills based on U.S. "Farmworkers, Farm, Ranch, And Aquacultural Animals" (45-2093.00)

Figure 2B: Statistics factsheet for UK occupation farm workers

Farm workers (9111)
EMPLOYMENT SHARESTOP EMPLOYING INDUSTRIES34

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 035

ing completely
certain.foresight
They werealso givena spaceto provide
freeformrienced foresightworkshopfacilitator
reviewed
the After
responses
withthegroup.
riencedworkshopfacilitator
reviewed
thewith thetheirgroup.
the responses
groupsubmitted
answers using the online form, an expeith 0 representing
notcertainat debatedall, and 9theirrepresentthoughts
they
felt
were
to qualify
answers.
After
the
group
perspectives
duringa during
half-hoursession, theAfter
the necessarygroup debatedtheirtheirperspectives
a half-hoursession,
thereviewed the responses with the group.
riencedforesight
workshopfacilitator
They were After
also
given
a
space
to
providefreeform
thewasgroup
theiranswersusing
theonlineform,
anexpegroup
thensubmitted
allowedchangeanswers,after
theworkshopgroup
wasthen toallowedtotheirchangetheiranswers,
after
which
the
workshopAfterthewhichgroup
debatedtheirperspectives during a half-hour session, thenecessaryrienced
to qualify
theiranswers.
foresight
workshopfacilitator
reviewed
the responses
withtheallowed
group. to change their answers, after which the workshop
moved
onto
the onto
next thesetoftensetoccupations.
moved
next
of ten occupations.
group wasthenmitted theiranswersusing
theonlineform,
anexpeAfter theThe
group
their
duringhalf-hoursession,THE
F UTpresented
URE OF
SKIparticipants
La
L
S:
OYMEN
Tthe
I N ran2 0ten
3 0 occupations.
firstdebatedten first
occupations
toThe
ten perspectives
occupations
presented
toEMPLparticipants
selected
ranmoved
ontowere
the selected
nextweresetof
hop facilitator
reviewed
theallowed
responses
withthegroup.
group
was thenchangetheiranswers,
after
which
the
workshop
domly.
For
the second
and
third
rounds,respective
batchesofbatchesten oc-of presented
domly.
Fortothesecondandthird therounds,the first
respective
ten oc- to participants were selected ranThetenoccupations
ed their perspectives
duringahalf-hoursession,the
movedcupations
onto thecupations
next selected
set wereof tenso
occupations.
were
as to bemaximally
informative
for second
the machine
selectedso as
to be maximally
informative
forandthethird
machinedomly.
For
therounds, the respective batches of ten ocd to change theiranswers,
after
workshopThe
first tenoccupations
topreviously
participants
were wereselectedranlearningmodel
in whichlight ofthe
theanswers
gathered
fromselectedparticipants.
learning
model
inpresented
lightof theanswers
previously
gathered
fromsoparticipants.
cupationsas to be maximally informative for the machinet of ten occupations.
domly.
theInthesecondandrounds,the respective
ofintenocIn Fora way,
participants
could
be could
seen
as
the
machinelearning
a way,
the third
participants
be teachingseen
asbatchesteachingthemachinelearningpreviously gathered from participants.
learning
model
light
of the answers
pations presented
to wereparticipants
were
ran-the day,cupationsselectedsoastocourse
be maximally
informativeforthemachinealgorithm
throughout
theselected
algorithm
able to realgorithm
throughout
theof courseof with
the
with
the
algorithm
able tobere-seen as teaching the machine learning
In day,
athe
way,
the
participants
could
and thirdlearning
rounds,therespective
batchesoftenocmodel
in
light
of
the
answers
previously
gatheredfromparticipants.
spond to spondthe informationfrom participantsby algorithm
proposinga prioritised
list courseof listto the informationfrom participants
by proposing
a prioritised
throughout
the
of of
the day, with the algorithm able to red so as to In
be amaximally
informative
for
thebe
machineway,thequestions.
participants
could
seen
as describedteachingthemachinelearning
furtherThis
processbeformallyinSectionfurtherquestions.
Thiswill
process
will besponddescribed
formallyin5.3.
Section
5.3.
to theinformationfrom participants
by proposing a prioritised list ofof the answers
previously
gatheredparticipants.
algorithm
throughout
thefromcourseof the day, with thealgorithm
able
torefurther questions. This process will be described formally in Section 5.3.
ants could spondbe seenthe machinelearning
to as
theteaching
informationfromparticipants
byproposingaprioritised
listof
to manage participant labels that are both ordinal and of
Our methodology uses the foresight exercises described5.day,METHODOLOGY
5. theMETHODOLOGY
the course furtherof thewithalgorithm
ablebetodescribed
requestions.
This
process will
formally in Section 5.3.
varying uncertainty. Our model is put to work on a variety of
in Section
data for a machine learningon from participants by proposing a prioritised
list4.3of as training
5. METHODOLOGY
methodology
uses
the usesforesight
exercises exercises
describeddescribedin Section
as 4.3 as
Our
methodology
the foresightin 4.3
Sectionprocess will beOur
described
formally
inmodel.
Section
5.3.
tasks.
In particular, it is the basis of our means of selecting
The
primary
goal
of
the
model
is
to
learn
a
function
5. METHODOLOGY
training dataforamachinelearning
model.
The
primary
goal
of
the
model
training data for a machine learning model.
The primaryuses
goaltheof themodel exercises described
Our methodology
foresight
in Section 4.3 asis to learnis atofunction(x) thatfrom
120 training
O*NETvariables
xmachine(captur-xlearninglearn a ffunction
(x)
that
maps
from
120
O*NET
(captur-model. The primarydata
forvariables
avariables
goal of the model
thatfmaps
maps
from
120
O*NET
(capturing
occupations
to be labelled through active learning (Section
Our methodology
uses theforesight
exercises
described
inoccupational
Section
4.3 demand.
as
GY
ing skills,ing
knowledge
andabilities)
to futureoccupational
skills,knowledge
andabilities)
tothisfrom 120 O*NET variables x (capturisfuture
to learna demand.
functionfIn(x)thisthat Inmaps(i) of toskills,knowledge
andabilities)
futureoccupational
5.3).
Interpretingthe patterns discovered by the model istraining
data foramachinelearning
model.
The
primary
goal
the
model
(i)
framework,theithoccupationisconsideredapoint,x,in
120-dimensionalframework, the ith occupation is considereda point,
x , in and120-dimensionaling skills,
knowledge
abilities) to future occupational demand. In thisthe foresight
exercises
described
Sectionas 120(i)
is to learn
a function
f (x)inthat
maps4.3
fromO*NETvariablesx(capturskills-/knowledge-/ability-space,whose
associateddemandisf (xoccupation).is Ourskills-/knowledge-/ability-space,whose
associateddemandf (x(i) ).
Ourframework,theith
is considereda point, x(i), in basis
120-dimensional
demand.
In
thisframework,
the
the
of
our
assessmentof the importance of O*NET
occupationischine learningmodel.
Theisprimary
goalof thetomodel
ing skills,
knowledge
demand.
In thisapproachbuiltandonanexpectation
thatoccupational
demandshouldvarysmoothlyasapproach
is abilities)
built
on an future
expectation
thatdemandshouldvary smoothly
as associated demand is f (x(i) ). Our
skills-/knowledge-/ability-space,
whose(x) that maps
from 120
variables(captur-a point,in 120-dimensionala is120-dimensional
skills-/featurestofuturedemand(Section
5.4). Finally, we use theconsideredframework,
theO*NET
ith occupation
is xconsidereda point, approachx(i) ,, inbuilt on an expectation that demand should vary smoothly as
nd abilities)
to future occupational demand.
In thisskills-/knowledge-/ability-space,whose
associated demand is f (x(i) ). Our
17knowledge-/ability-space,whose
associateddemandmodel
topredict
futureoccupations,defined as hotspots
17(i)
Considerengineersandmetal
workers
andplastic
workers–two
occupationgroupsConsiderengineersandmetal
workers
andplastic
workers–two
occupationgroupsupation is approachconsideredisa built
point,onx an, in
120-dimensional
expectationthat demand should vary smoothly as
the which
BLS projects
in share
Over2024.
thisperiod,17 and 2024.
the BLSwill
will
declinebetween
in share2014
between
2014
and
Over
period,and plastic workers – two occupation groups
Considerengineersandmetalthisworkers
ity-space, whosewhichassociateddemandisprojects
fdecline(x(i) ).
Our
is
.
Our
approach
is
built
on
an
expectation
that
of
high
demandthatarenotassociatedwith an existing
metal workers
andplastic
workers
areprojected
to
lose
99,000jobs
whereasengineersaremetal workers and plastic workers are projectedto thelose BLS
99,000jobs whereas
engineers
are between 2014 and 2024. Over this period,
whichprojects
will decline
in sharen expectationthat demandshouldvarysmoothlyasadd
65,000jobs.65,000
17 projected toprojected
to addjobs.

5. METHODOLOGY

Consider engineers and metal
workers andplastic workers– two
occupation
workers
andworkers arewhereas engineers
aredemandshouldvarymetalsmoothlyasplasticagroupsfunction
ofprojectedskills, to lose 99,000 jobs
occupation(Section
5.5).
which the BLS projects will decline in share between 2014 and2024. toOver
period,projected
addthis65,000jobs.
a function
ofareskills,knowledge
and abilities:
thatis,
if aretwo
occupations i andknowledge
andabilities:
that
is,
if
two
occupations
metalandworkers
andplastic
projected
to
lose
99,000jobs
whereasengineers
d metal workers
plastic
workers–workers
twooccupationgroupsa functionof skills,
knowledge
andabilities:
that is,that
if twoia functionof skills,
knowledge
andis, abilities:
ifoccupations
two occupations
i Tooccupations(i) of(j) abilities:
benchmarka function
andi the efficacy of our foresight exercise, weto add2014
65,000andj 2024.
have Oversimilarxskills,, weknowledge
expect theirassociated that is, if two
34variables,ll decline inprojectedshare between
andjobs.
thisO*NET
period,O*NET34 x (i) , we (j)and j have
have
variables,
x(i)  x(j)expecttheirsimilarvariables,
weexpect
expecttheirand jsimilar
havesimilar
O*NETvariables,
 xvariables,
, we
their
(i)associated(j)associated(i)O*NET
and
j (j)
have
O*NET
x

xoccupations, we expect
their use
associatedworkers are projected to lose 99,000
jobs whereas
demands
to bea engineerssimilar,fare(x)
f(i)(x).(i) similar(j) and(j)34functionofskills,knowledge
abilities:
that
is,
if
two
i
also
Gaussian
processes to perform extrapolation out
demands
to be similar,f (x ) f(xf (x) ).f (x ).
demands
to be similar,(i)bs.
besimilar,
similar,)  f (x)..
(j)
We chooseto model fdemands
with ademands
Gaussian
process,to (i)
bef (xdescribed
in(j)Section
associated
to to
be

5.4). Finally, we use the model to predandWej havesimilar
O*NETvariables,
x process,toxbe, describedweexpectassociatedWe chooseto modelf witha withGaussian
process,
intheir
Sectionchooseto modela toGaussianto bedescribed
inSectionto theinyear
2030 of past employment trends(Section
5.6).
34
Weffchoosewith a(Ghahramani,
Gaussian
process,tobe described
Section(i)5.1. The Gaussian
process
is asimilar,
Bayesian
non-parametrichotspotsof highdemand that are not asso
demands
toprocessbe(x
non-parametricfmodel(x(j) non-parametric). fmodel
5.1. The
Gaussian
is a Bayesianmodel model(Ghahramani,
5.1.
The Gaussian
processisGaussiana )Bayesian(Ghahramani,
5.1.
The
processisaBayesian
non-parametricmodel(Ghahramani,2013), meaningthat
its expressivity
will
naturally
adaptprocess,to process,thattoinherent
in
5.4).
Finally,
we
use
the
model
to
predict
futureoccupations,defined(Section
5.5).
5.4).
Finally,
we
use
the
model
to
predict
futureoccupations,definedasas defined asWechoose
to
model
f
with
a
Gaussianbedescribed
inSectionwithaGaussiantobeWechoose
to
model
These
extrapolations
providealternatives
tothelabels2013), 2013),
meaningthat itsthat
expressivity
will naturally
adapt tothattoinherentinmeaning
its expressivity
will
naturally
adapt
that inherent
into that inherent in 5.4). Finally, we use the model to predict future occupations,342013),
meaningthat
its
expressivity
will
naturally
adapt
the data. This5.1.
gives
us Gaussian
an in-builtprocessresistanceto over-fitting(learning patterns
hotspotsofof
highdemandthatarenotassociatedwithToanan
existing
occupation
benchmarktheefficacy occupationof our fores
hotspotshighdemandthatarenotassociatedexisting
occupationThe
isa Bayesiannon-parametricmodel(Ghahramani,
the data.
gives
us
an in-built
resistanceto over-fitting(learning
patterns
hotspotsof highdemandthatare results
notwithassociatedwith
an existing
theThis
data.
This
gives
usdata.
an
in-built
resistanceto over-fitting(learning
patterns (learning
described
inSection
5.1.
The
Gaussian
process
isa
producedintheworkshops,
andprovide
that
do
the
This
gives
us
an
in-built
resistancetoover-fitting
patterns
that do not generalise
to unseendata):
the model 5.4).
will
notinducetheuseflexibil5.4).
Finally,
weto
use
the
model
to predict
futureoccupations,defined(Section
5.5).
Finally,
we
the
model
to predict
futureoccupations,definedas asprocesses to perform extrapolation out to(Section
5.5).
2013),
meaningits
expressivity
will
naturally
adapt
that
inherent
inthat dothat
not dogeneralise
tothatunseendata): data):
the model
will notinducetheflexibil5.5). future
5.4).
Finally,
we
usewe
the
model
to (Section
predict
futureoccupations,defineddefinedas
not generalise
tounseen
the
model
will
nottheinducetheflexibil5.4).
Finally,
use
the
model
to
predict
occupations,
as
that
do
not
generalise
tounseendata):
model
will
not
inducetheflexibility requiredBayesian
tothe
give
a near-perfect
fit anonin-built
training
data
unlesstheandhotspotsofqualityhighdemandthatareassociatednot5.4).
associatedwithansubjective
existing
occupation
ToTobenchmark
theefficacy
foresight
exercise,
weGaussian
hotspotsof over-fitting
highdemandthat and
are
not
with
anbenchmark
existing
occupation
trends(Section
5.6).
extrapolatiotheefficacyofour
ourforesight
exercise,
wealsoalsouseuse
Gaussiandata.
Thisgivesaus
resistanceto(learning
patterns
non-parametric(Ghahramani,
not
rely
the
judgments
ofoftheexperts.
Finally,
we
the
model
toof
predict
occupations,definedas
ity required
to givea near-perfect
fit model
on training
data
unlessthe2013),qualityTobenchmarktheefficacy
ourforesight
exercise,
we These
also use
Gaussian
hotspotsof highthatonarenotuseassociatedwith
anexisting
occupation
ity requiredtogivenear-perfect
fit
ontraining
datafitunlessthe quality
andofdemand
hotspots
high
that
are
not
associatedwith
an future
existing
occupation
ity required
to data):give
a near-perfecton training
data
unlessthe demand
qualityandquantity of datasuggeststhat thisfit will
extendequallywell
unobserved(Sectionprocesses
extrapolationout
2030
of pastemployment(Section5.5).to5.5).
bels
producedinthe ofworkshops,
and prov
processes
toperform
perform
extrapolation
outtotothetheyear
year
2030
past
employment
that
do notgeneralise
unseen
the
model
will
inducetheflexibilhotspotsof hightodemandthat
not associatedwithanquantityof datasuggests
thattothis
fitthiswill fit
extendequallywellnottounobserved
processes
toare
perform
extrapolation
outexistingtotheofoccupation
year
2030
past employment(Section
5.5).
quantityofdatasuggeststhatwillextendequallywell
toto
unobserved(Section
5.5).
meaningthat
its
expressivenesswill
naturally
adapt
that
quantityofdatasuggests
that
this
fit
will
extendequallywelltounobserved
data. This desirable
propertyis ainducedby the‘Occam’s
implicit
To Razor’
benchmarktheefficacyour(Section
foresight
exercise,
we usealso
useGaussiantrends(Section
5.6).
These
extrapolations
providealternatives
totothela-la-experts.
Tobenchmarktheefficacyof ourofforesight
exercise,we
also
Gaussiansubjective
judgmentsofthetrends(Section
5.6).
These
extrapolations
providealternatives
the
ity requiredtogivenear-perfect
fit
ontraining
data
unlessthequalityand
5.5).
data. data.
This
desirable
propertyis
inducedbythe‘Occam’s
Razor’
implicit
trends(Section
5.6).
These
extrapolations
provideoccupations,alternativesdefinedto the asla5.1.
GAUSSIAN
PROCESSES
Tothe
benchmarktheefficacyof our5.4).
foresight
exercise,
we alsouse
GaussianThis desirable
property
is inducedby theis‘Occam’s
Razor’
implicitTobenchmarktheefficacyof
ourforesight
exercise,
we
also
use
Gaussian
Finally,
we
use
the
model
to
predict
futuredata.
Thisdesirable
property
inducedby‘Occam’s
Razor’
implicit
within Bayesian
reasoning(MacKay,
2003),
and fitissuggested
empiricallybyextrapolation
processes
to perform
the
year
2030
ofin
past
employment
bels
producedintheandprovideresults
thatdodonot
theprocesses
to perform
extrapolationout toout
theto
year
2030
of pastemployment
bels
producedtheworkshops,
workshops,
andprovideresultsthat
notrely
relyononquantityofdatasuggests
that
this
will
well
to unobserved
inherent
in
the
data.
This
gives
us
an
in-built
resistanceTobenchmarktheefficacyof
ourforesight
weprovidealso
use
Gaussian
withinBayesianreasoning(MacKay,
2003),
andisextendsuggested
empirically
byto
bels
producedinyeartheworkshops,
andresultsthatdo thenotoccupation
rely on theprocesses
perform
extrapolation
out
to
the
year
2030
of2030past
employment
withinBayesianreasoning(MacKay,
2003),
andisequallysuggested
empiricallybyto perform
processes
extrapolation
out
to
the
ofexercise,past
employment
5.4).
Finally,
wereasoning
use
the
model
to
predict
futureoccupations,definedashotspotsofhighdemandthatare not
associatedwith
anexisting
within(MacKay,
2003),
andis implicit
suggested
empiricallybywe
the cross-validation
tests desirable
presented
inBayesian
Section
6.2.
The
Gaussian
process
5.4).
Finally,
we
use
the
model
to
predict
futureoccupations,definedastrends(Section
5.6).
These
extrapolations
providealternatives
tothelasubjective
judgmentsoftheexperts.
trends(Section
5.6).
These
extrapolations
providealternatives
tothela5.4).
Finally,
use
the
model
to
predict
futureoccupations,definedassubjective
judgmentsoftheexperts.
data.
This
propertyis
inducedbythe‘Occam’s
Razor’
5.1.
Gaussian
processes
processes
toperform
extrapolation
out
to
the
year
2030
of
past
employment
the cross-validation
tests
presented
inSection
6.2.
The
Gaussian
process
5.4).
Finally,
we
use
the
model
to
predict
futureoccupations,definedassubjective
the ofexperts.
5.4).give
Finally,
weuseofalternatives
the
model
occupations, defined atrends with
(Section
5.6).
These
extrapolations
providealternatives
toGaussian
theto
la-predict
cross-validation
presented
intests
Section
6.2.
The
Gaussian
process
trends(Section
5.6).section,
These
extrapolations
providetothela- future
In
this
we
ajudgments
brief
review
hotspots
ofpatterns
highdemandthatarenottheassociatedanexisting
occupation(Section
5.5).
tothe
over-fitting(learning
that
do
not
generalise
thetests
cross-validation
presented
inSection
6.2.
The
process
gives a flexible,
non-linear,functionclass
suitableforthecomplexinterachotspotsofhighdemandthatarenotassociatedwith
anexisting
occupation
bels
producedinworkshops,
the interacworkshops,
andprovideresults
that
do
not
rely
on
the
produced
andprovideresults
do
not
rely
on
the
hotspots
ofthathighdemandthatarenotassociatedwith
anexisting
occupation
withinBayesianreasoning(MacKay,
2003),
andisinforsuggested
empiricallybyGaussian
trends(Section
5.6).
These
extrapolations
providealternatives
tothelagives agives
flexible,
non-linear,functionclass bels
suitableforthecomplexhotspotsofhighdemandthatarenotassociatedwithexisting
hotspotsofhighdemandthatarenotassociatedwith
anexisting
occupation
bels
producedintheworkshops,
andprovideresults
that
do
not
rely
on
the
a flexible,
non-linear,functionclass
suitablethecomplexinteracbels
producedintheworkshops,
andprovideresults
that
do
rely
on theIn
thissection,weangivea brief
reviewof G(Section
5.5).
To
benchmarktheefficacyofnot
ourforesight
exercise,we
also
useoccupation
Gaussiangives
a flexible,
non-linear,functionclass
suitableforthecomplexinteractions (for instance,complementarities)
thatwewillexpectbetweenvariables(Section
5.5).
subjective
judgments
ofexperts.
theexperts.
subjective
judgments
ofand
the(Section
5.5).
5.1.
Gaussian
processes
thecross-validation
tests
presented
inSection
6.2.
The
Gaussian
process
gp(Rasmussen
andWilliams,
2006)
processes.
Formally,a
5.1.
Gaussian
processes
unseendata):
the
model
not
inducetheflexibility
bels
producedintheworkshops,
andprovideresults
that
do
not
rely
on
the
tionsto(forinstance,complementarities)that
weexpectbetweenvariables
and(Section
5.5).
5.4).
Finally,
weexperts.
use
the
model
to
predict
future occupations,definedas(Section
5.5).
subjective
judgmentsof theexperts.
tions (for instance,tions
complementarities)
that
we expectbetweenvariablesandsubjective
judgments
of
the
5.1.
Gaussian
processes(Rasmussen
andWilliams,
is a proTo (forbenchmarktheefficacyof our foresight
exercise,
we alsouse
Gaussian
processes
toperform
extrapolation
out
to
the
year
2030
of
past
employment
instance,complementarities)that
weexpectbetweenvariablesanddemand.
Tobenchmarktheefficacyof
ourforesight
exercise,
we
also
use
Gaussian 2006)
Tobenchmarktheefficacyof
ourforesight
exercise,
we
also
use
Gaussiangivesaflexible,
non-linear,functionclass
suitableforthecomplexinteracsubjective
judgmentsoftheexperts.
demand.
Tobenchmarktheefficacyof
ourforesight
exercise,
we
also
use
Gaussian
hotspotsofhighdemandthat
notassociatedwith
anexistingTobenchmarktheThese
efficacyof ourforesight
exercise,also
usedemand. to giveprocesses
InInthis
we
give
a are
review
ofthe
Formally,athe
f processes.
: of
X
→past
R,employment
suchthatwemarginal
isaprobability
distribution
over,occupationsuch
to performoutdatatotheyear 2030 of past
employment
this
section,we
giveabriefbrief
reviewofGaussian
Gaussian
processes.
Formally,
agp
gp
trends(Section
5.6).
extrapolations
providealternatives
to Gaussian
thedistrirequiredacomplementarities)
near-perfectfitextrapolationon
training
unlessdemand.
This modelis trainedon a dataset
containinglabelsforeach
occupation
processes
toperform
extrapolation
out
to
year
2030
of
employment
processes
to section,
perform
extrapolationout
tofunctions
year
2030
past
5.1.
Gaussian
processes
In
this
section,
we
give
athe
brief
review
Gaussian
processes.
Formally,a lagp5.1.
Gaussian
processes
(for
that
we
expectbetweenvariablesand (Section
Thistionsmodel
isinstance,
trainedon a dataset
containinglabelsforeach
occupation
processes
perform
out
to
the
year
2030
ofdo
past
employment
5.5).
processes
toperform
out
to
the
year
2030
of
past
employmen
This
model
is trends
trainedona datasetcontaininglabelsforeach
occupation
5.1.
Gaussian
processes(Rasmussen
andWilliams,
2006)
isextrapolation
distributionoverfunctions
5.1.
Gaussian
processes
anyfinitesubset
of
X
(such
as
X)
mu
(Section
5.6).
These
providealternatives
to(Rasmussen
theandWilliams,
isaextrapolation
aprobability
probability
distributionoverfunctionsbels
producedin2006)
theworkshops,
andprovideresults
that
not
rely
onisthe5.4).
Finally,
weusela-themodel
totopredict
futureoccupations,defined
as
This
model
is
trainedon aextrapolations
datasetcontaininglabelsforeach
occupationfrom each theindividual
expert,ratherthan
on datathegroupconsensus.
This
aptrends(Section
5.6).
These
extrapolations
providealternatives
tothelatrends(Section
5.6).
These
extrapolations
providealternativestothela(Rasmussen
andWilliams,
2006)
is
a
probability
distributionoverfunctionsdemand.
thatthemarginal
distribution
overthefunctionvaluesonqualityandquantityofsuggests
that
this
fit
will
from eachindividual
expert,rather
than
on
the
group
consensus.
This
aptrends(Section
5.6).
These
extrapolations
providealternatives
to
the
5.1.
Gaussian
processes
Tobenchmarktheefficacy
ofmarginal
ourforesight
exercise,
we
also
use
Gaussiantrends(Section
5.6).
These
extrapolations
providealternatives
to values
thela-lafrom each individual
expert,ratherthan
onsection,therathergroup
consensus.
This
apIn
this
section,we aongive
a brief
review
ofbelsGaussian
processes.
Formally,
a
gp
f
:
X
→
R,
such
that
the
distribution
overthefunctionvaluesonIn
this
we thangive
brief
review
ofdoGaussian
processes.
Formally,
a
gp
f
(x),
the
prior
distribution
over
its
bels
producedin theworkshops,
andprovideresults
that
not
rely
on
the
f
:
X
→
R,
such
that
the
marginal
distribution
overthefunctionvaluesonsubjective
judgmentsoftheexperts.
hotspotsofhighdemandthatarenotassociatedwith
anexisting
occupation
from
each
individual
expert,thegroupconsensus.
Thisapproach permits thediversityofviews
within
the
group
to
be
captured
withintheworkshops,
andprovideresults
that
not
rely
on
bels
produced
in intheworkshops,
provideresults
do
not
rely
on
thethefreview: extrapolation
XGaussian→ofR,Gaussian
such
that
the
marginal
distribution
overthefunctionvaluesonIn
thissection,weprocesses
givewea produced
brief
reviewofprocesses.
Formally,
athat
gp
This
model
is trained
onofawithin
datasetcontaininglabelsforeach
occupationIn
this
section,give
aperform
brief
processes.
ado
gpresults
proachproachpermits
thediversityof views
the
group
to
be
captured
withinbels
produced
inand
theworkshops,
andprovide
that
do
not
on
the
to
out
tois
the
yearFormally,2030
ofprovidepast
employment
bels
producedintheandresults
that
do
not
rely
on
the
permits
thediversity
views
withinthetobecaptured
withinandWilliams,
2006)
isbeaIn
probability
distributionoverfunctions5.4).
Finally,
we
use
the
model
to
predict
futureoccupations,definedasrelyany
finitesubset
of
X
(such
as
X)
Gaussian.
andWilliams,
2006)
isatoprobability
distributionoverfunctionsspecified
by
aFor
mean
vectorm
andcovariaany
finitesubsetof(suchas)workshops,
ismultivariate
multivariate
Gaussian.
subjective
judgmentsof (Rasmussentheexperts.
any
finitesubset
X
(such
as
X)
multivariate
Gaussian.
For
afunctionfunction(Section
5.5).
equallywell
tomodelled
unobserved
data.
This
desirable
proachpermits
thediversityofgroupviews
withinthegroup
captured
withinthe model.extend
Theparticipantlabelsareas(Rasmussen
conditionallyindependent
subjective
judgmentsoftheexperts.
this
section,
we
give
aaof
brief
review
Gaussian
processes.
Formally,
aa gp
subjective
judgmentsoftheexperts.
any
finitesubset
ofof
Xof
(such
as
X)alternatives
isfunctions
multivariate
Gaussian.
For
a function(Rasmussen
andWilliams,
2006)
isasubjective
probability
distributionoverfunctionsfrom
eachindividual
expert,ratheron
thegroupconsensus.
This
ap(Rasmussen
andWilliams,
2006)
isprobability
distribution
overthe model.
The
participantlabelsaremodelled
independentjudgments
theexperts.
trends(Section
5.6).
These
extrapolations
providetothelasubjective
judgmentsoftheexperts.
themodel.
The participantlabelsarefthanasconditionallyindependentf→: conditionally
XR,
→
R,
such
thatmarginal
the
marginal
distribution
overthefunctionvalueson
hotspotsofhighdemandthatarenotassociatedwith
anexisting
occupation
f
(x),
prior
distribution
overits
valuesfonasubsetx⊂
X
arecompletely:modelled
Xas
such
that
the
distribution
overthefunctionvalues
on
f
(x),
the
prior
distribution
overits
valuesfonasubsetx⊂
X
arecompletelyTobenchmarktheefficacyof
ourforesight
exercise,
we
also
use
Gaussianthemodel.
The
participantlabelsaremodelledasconditionallyindependent
5.1.
Gaussian
processes
given the latentfunctionf(x).
The
dependence
amongstthegroup’slabels(Rasmussen
andmarginal
Williams,
2006)
isdistribution
athe
probability
distributionoverfunctions
1
f
(x),
the
prior
over
its
valuesfasubsetx⊂
X
arecompletely
f
:
X
→
R,
such
that
the
marginal
distributionoverfunctionvalueson
proachpermits
thediversityofviews
within
the
group
to
be
captured
within
f
:
X
→
R,
such
that
the
distribution
overthefunctionvaluesonForafunction
,
the
prior
distribution
overits
valuesisinducedbythe‘Occam’s
Razor’
implicit
withingivenproperty
thelatentfunctionf(x).
The
dependence
amongstthegroup’slabelsa function of skills, knowledge
andabilities:
that
is,
if
two
occupationsibels
producedintheworkshops,
andprovideresults
that
do
not
rely
on
the
given the latent function
f (x).
The
dependence
amongstlabelsany
finitesubsetofXgroup’s(suchas
isf :multivariate
Gaussian.
a function(Section
5.5).
| K) :=onN (f ; m, K) := √specifiedbyaprocesses
vectormm
andcovariancematrix
KK p(f values
anyfinitesubsetofdependence
X the
(such
asamongst
X)processes
isX)
multivariate
Gaussian.
For
aFor
function
specified
by
amean
mean
vectorandcovariancematrixtoperform
extrapolation
out
the
year
2030
of
past
employment
given
the
latentfunctionf (x).
The
thegroup’slabels5.1.
Gaussian
induced by discussionis (i)
modelled
through
thisprocesses
sharedlatentfunction.
X (such
→
R,
such
that
the
marginal
distribution
thefunction
5.1.
Gaussian
specifiedbyatoGaussian.
mean
vectorandcovariancematrixK(j)any
finitesubsetofsubjectiveXasX)processes
is5.1.
multivariate
FormaoverfunctiontheThelabelsaremodelledasconditionallyindependent
5.1.
Gaussiananyfinitesubset
ofon
X
(such
as
X)
isthe
multivariate
Gaussian.
For
a offunctiondet
inducedby model.
discussion
modelled
throughsharedlatentfunction.
and j have similar O*NETvariables,
x discussion
 isxparticipant, we
expecttheir
associatedIn
this
section,weprocesses
give
a brief
review
Gaussian
processes.
Formally, a gp
judgmentsofexperts.
inducedbyis(MacKay,
modelled
throughthissharedlatentfunction.
Gaussian
processes
5.1.
Gaussian
fthe
(x),
the
prior
distribution
over
its valuesfasubsetx⊂
X
arecompletelyTobenchmarktheefficacyof
ourforesight
exercise,
we
also
use
Gaussian
fthis
(x),
prior
distribution
overits
valuesfonasubsetx⊂
X
arecompletelyonasubsetarecompletely
specifiedbyamean
trends(Section
5.6).
These
extrapolations
providealternatives
to
the
la

inducedbydiscussionismodelled
throughthissharedlatentfunction.
Bayesianreasoning2003),
andissuggested
Resiliencetouncertainty
iscrucialtoourexercise.
Not
only
did
our
workanydistribution
finiteover
subsetof Xits(suchX)
issubset
Gaussian.
function
1 completely
1For
1X are1 a −1(i)given the
four
(x),
the
prior
itsovervaluesf on aasfsubset
xmultivariate⊂ X xarelatentfunctionf (x).
dependence
amongstthegroup’slabelsfour(x),thedistribution
priorvaluesonreview
aWilliams,⊂
completely−1Resilience
to(j)uncertainty
is crucial
toTheourspecified
exercise.
Not
only
did
workInthis
section,we
give
aspecified
brief
reviewofGaussian
processes.
Formally,aKgp
demands to be similar, f (x) Resiliencef (x).
(Rasmussen
2006)
isexp
aThe
probability
distributionoverfunctions
1− Formally,
to uncertainty
is crucial
to ourexercise.
Not
only
did
work√on
:=
:=
In
this
section,
we
give
aover
brief
of
Gaussiana.1 gp
p(f
K)
N
(f
;and
m,
K)
f Formally,−covarianceK
fmatrix. gp
byarepresenmean
vectormbels
andcovariancematrixK|to√processes
extrapolation
out
to
the
year
2030
past
:=
exp
generatedacp(f| perform
K)
N
(f
;K)
m,
K)
by
acrucial
mean
vectorm
andcovariancematrix
fof
K
f−
(1)
 (1)
−1
In
this
section,
we
give
a:=
brief
reviewofprocesses.
aemployment
producedintheworkshops,
andprovideresults
that
do
not
on
the
Resilience
to uncertainty
is sharedtoour
exercise.
only
did
our
workshops gather observations
from individual
participants
withexplicit
fvectortheprior
distribution
valuesfGaussian
a2πK
subset
x√of
⊂rely
X
are
completely:=:=processes.
exp
fis
p(f
|its
N
(f
K)
K
f values. abygp(1)
In
this
section,
we
give
a det
brief
review
Gaussian
processes.
Formally,Insection,we
givea;marginal
brief
reviewofprocesses.
Formally,aon
gp
specifiedby Nota meanvectorm
andcovariancematrixK
inducedby bydiscussion
isindividualmodelled
throughthislatentfunction.
2Gaussianspecified
by
a(x),
mean
vectorm
andcovariancematrixK
2 overandmatrix:m,det
2πK
empiricallythecross-validation
tests
presented
in
shops
gatherobservations
fromparticipants
withrepresen(Rasmussen
andWilliams,
2006)
is explicit
awith
probability
distributionoverfunctions
We choose to model
f witha Gaussian
process,tobedescribed
in Section
fcovariance:Williams,
Xthis
→
R,
such
that
the
distribution
overthefunctionshops
gatherobservations
fromindividual
participants
explicit
represen5.1.
Gaussian
processes(Rasmussen
and
2006)
is
a
probability
distributionfunctions2trends(Section
5.6).
These
extrapolations
providealternatives
tothelathat
is,
K
=
κ(X,
X).
det
2πK
(Rasmussen
andWilliams,
2006)
is
a
probability
distributionoverfunctionssubjective
judgmentsoftheexperts.

shops
gatherobservations
fromindividual
participants
withexplicit
representations of their uncertainty,
the
model
must
also
try
to
fuse
observations
fromspecifiedbyamean
vectorm
andcovariancematrixK
1 values on1(Rasmussen
1
1 (Rasmussen
andWilliams,
2006)
is is
a probability
distributionoverfunctionsandWilliams,
2006)aprobability
distributionoverfunction
Resiliencetouncertainty
iscrucialtoourexercise.
Not
only
did
our
work
−1
−1



tations
their
uncertainty,
the
model
must
also
try
to
fuse
observations
from
f
:
X
→
R,
such
that
the
marginal
distribution
overthefunction
5.1. The Gaussian process
is tations
aofBayesiannon-parametricmodel(Ghahramani,anyfinitesubset
of
X
(such
as
X)
is
multivariate
Gaussian.
Forafunctionof
their
uncertainty,
the
model
mustgives
also
try
to:=
observations
from√f :fX: The(1)
1matrix
1 distribution:=exp
fmatrix
p(f
|fuse
K)
N
; m,:=
K)
f1marginal
. (1)
(1)
√:= observations
1a−1provide
X→
→
R,
such
over
the
functionvaluesonon
exp
f−thatp(f|amodelK)N(f ;nonm,(fK)
−
Kthat
fKthe
. marginal
covarianceis
generatedcovariancefunction
κ
:
X
×
X
→

R;
bels
producedinworkshops,
andresults
that
do
not
rely
the
Given
training
data
D,
we
use
the
g
The
covarianceis
generatedbyacovariancefunction
κ
:
X
×
X
→

R;
R,
such
the
distribution
overthefunctionvalueson
by
−1
tationsof
their
uncertainty,
the
must
also
try
to
fuse
from
6.2.
The
Gaussian
process
flexible,the diverseSection
range
of
opinions
producedbyourdomainexperts.
Our
model
√
:=
ffThe
:fX
R,
such
marginalovermatrixisthe
generatedbya fcovariancefunction
κare: X values
×valuesX →on
R;
exp
p(f
| K)
N
(f
;section,
m,
−that
K
fK
. its
√brief
In
this
weK)
a→
review
off1theGaussian
processes.
Formally,athe:= values
2give: Xcovariance
→
R,
such
that
over
the
on
exp
p(f:=
|For
K)
N K)
(f ;2m,fmarginal−
fdistribution.distribution(1)a subset
shops
gatherobservations
fromindividual
participants
explicit
represendet
2πK
det
2πK
1(1)
theexpressivity
diverserange
opinions
producedbydomainexperts.
Our
model
any
finitesubsetof ourX (suchasdomainX)
is withmultivariate
Gaussian.
a:=function2013), meaning that its
willofnaturally
adapt
to
that
(x),
the
prior
distribution
xgp⊂function
Xfunction
thediverserange
of opinions
producedbyourinexperts.
model
 on
any
finitesubsetofX(such
(such
as
X)
is
multivariate
Gaussian.
affunction
that
is,
K
κ(X,
X).
subjective
judgmentsoftheexperts.
2X
functionvaluesx∗completely.aWith
this
that
is,
K=:=
=
κ(X,
2 overany
finitesubsetof
X finiteasX)
multivariate
Gaussian.
For
a.functions
function
det
2πK
∗ at
det
2πK
the
diverserange
ofinherent
opinions
producedbyour Ourdomainexperts.
Our
model
√
:=
choice is informed
byoftheprobabilistic
foundations
ofthealsoGaussian
process,
exp
fmultivariate
p(f
|and
K)
N
(f
; X).
m,
K)
−is
K −1
fFor
(1)inputFor
5.1.
Gaussian
processes
anysubsetasand
X)multivariate
Gaussian.
function
that
is,
K
=
κ(X,
(Rasmussen
Williams,
2006)
is of
aisX).
probability
distribution
overany
finitesubsetofX(such
(such
as
X)
is
Gaussian.
For
a function
theiruncertainty,
the
model
must
try
to thefuseobservations
fromistationsinformedby theprobabilistic
foundationsoftheGaussian
process,functionsuitableforthecomplexinteractions
fclass
(x),
the
prior
distribution
overits
valuesfonasubsetx⊂
X
arecompletely
the data. This gives uschoice
an linear,in-built
resistancetoover-fitting(learning
patterns
specifiedbyamean
vectorm
covariancematrixK
2choiceis informedbytheprobabilistic
foundationsofGaussian
process,
det
2πK
TheAs
covariancematrixis generatedbyf (x),af (x),
covariancefunction
:data
×
→
R;
the
prior
distribution
overits
valuesf gp
ongp
atosubset
x predictions⊂predictions
X arecompletely
Given
training
data
D,X
use
the
make
about
the
The
covariancematrixis generatedbycovariancefunctionκ : X κ×XX→R;weequationsGiven
training
D,
we
use
to→
make
about
the completely
the
prior
distribution
overits
valuesfκtheonsubsetxthe⊂
completely
isinformedby thefoundations
ofcovariancetheamatrix
Gaussian
process,
which give it athecoherent
way to choicereasonabout
uncertainty.
such,
we expectf (x),
thedistributionfX
onsubsetxpredictions⊂
X XareGiven
training
data
D,
we
gp
to
the
covarianceis→
generatedbycovariancefunction: overXaover×
X:useR;
f : Xmatrix
R,
that
the
marginal
distribution
overthevaluesonfa(x),
prior
distribution
its
values
fare
on
amake
subsetx⊂
areaboutcompletelyof will
opinions
by probabilistic
ourdomainexperts.
Our
model
The
issuchgeneratedbythe
apriorcovariancefunctionκits
Xvalues×
Xfunction
→
R;
give
itdiverse
a coherent
way
toway
reasonabout
uncertainty.
As
such,
we
expectspecifiedbytoinduce
aproduced
meanthat
vectormis,
andcovariancematrixK
that do not generalise which
to unseendata):
thenotthe
flexibilIn
thisAssection,
we
give
amean
brief
of
Gaussian
processes.
Formally,
aahave
gp
which
give
itrangea modelcoherent
about
uncertainty.
AsThe
such,
we expect
the

that
K
=
κ(X,
X).
specifiedbyamatrix
vectorm
matrixK wefunctionvaluesf∗fisreview.covariance
With
this
information,
thepredictive
at
input
xand
is,
K
=
κ(X,
X).
functionvalues. covariance
With
this
information,
predictive
at
input
specifiedbyaGaussian
mean
m
K
5.1.
processes
1R; have
∗x
which
give
itofareason
coherentwaytoreasonabout
uncertainty.
such,
we
expect∗ generated∗isinstance,complementarities)that
weexpectbetweenour model (forto givean ishonest
representationthetrendsthatcanbeinferredThe
covarianceavectorcovariancefunction
κwe
:have
X
×
X
→

−1the predictive
specified
by
a|and
mean
m
and
covariancematrixK
functionvaluesfby. Withthis1functions
information,we
input
xmatrix
thatis,Kinferred=is,κ(X,X).
anyfinitesubsetofXvector(suchasX)
multivariate
Gaussian.
Forap(ffunction
specifiedbyamean
vectorandcovariancematrixK
choiceinformedbytheprobabilistic
foundationsof cantheGaussian
process,
that
K
=
κ(X,
X).
∗ at
∗m
|
x
,
D)
=
N
√
:=
:=
our model
tofit
give
an
honestrepresentationof thetrendsthatbeinferred
exp
p(f
K)
N
(f
;
m,
K)
f
−
K
f .f∗ ; m(f(1)
ity required to give a near-perfectontraining
data
unlessthequalityand∗∗ |(Rasmussen
andWilliams,
2006)
is
a
probability
distribution
over
∗
our
model
to
give
an
honestrepresentationofthetrendsthatcanbe we the
training
data
D,trends
use
the
to
make
predictions
equations
GivenGiven
training
data
D,
we
use
gp
togp
make
predictions
aboutabout
the the 1
equations


1
1
our
model
to
give
an
honestrepresentation
of
the
that
can
be
inferredfrom noisy participantlabels.
2
that
is,
K
=
κ(X,
X).
1

−1
2πK
equationsGiven
training
D,
we
use
the
gp
to
make
predictions
the1fpredictions
fdata(x),fsuchthe
prior
distribution
overvalueson thea about
subsetx⊂ valuesX
arecompletely
1det
which
give
it aextend
coherentwaytoreasonuncertainty.
As such,weGivenexpecttraining
D,
we
use
the
gp
to
make
about
−1
In
this
we
give
aits
brief
reviewofGaussiana gp 
:=function−1
p(f well| K)
Nabout(ffunction; m,valuesK) :=K. datafromthatnoisy
participantlabels.
variablesanddemand.
quantity of data suggests
this
fit
will
equallytounobserved1(1)f.Formally,
Xf →
R,
that
the
marginal
distributionoverfunction
on
from
noisy
participantlabels.
1the
:=
:=
1(1)
√
exp
p(f
|section,
K)
(f
;we
m,
K)
fK
− f1processes.
K.  thevaluesfinputx∗f−. :Withthis
information,
wehave
predictive
√
:=
:=
f∗√at
x∗exp
. Withthis
information,
we
have
theN
predictive  −1
exp
p(f
|(1)
K)
N
(f
;the
m,
K)
−
f
∗ at input
−1
from
noisy
participantlabels.

Beyond thestate-of-the-art
inGaussian
processes,
weintroduce
a
novel
2
where
The
covariancematrixis
generatedbyacovariancefunction:
Given
training
data
D,
use
the
gp
to
make
predictions
about
√
:=
:=
det
2πK
exp
p(f
|
K)
N
(f
;
m,
K)
f
−
K
f
.
(1)
:=
K)
N
(f
;|matrixm,f (2)− functions
f X. → R;(1functionf∗ specified
at subset
input
xby
this
we
the
2acovariance
aWith
mean
vectorm
andcovarianceK
ourBeyondtoan honest
representation
ofthewetrends
canbevaluesfunctionvaluesf∗ at(Rasmussen
inputx∗ .p(fWith
information,
we
have
the
predictive
2πK|information,
, ∗D)
=|=
N
f:=
;have
x| ∗predictive
,K)
D),
VFor
(f
| functionx∗x, ∗D), function∗. X
andWilliams,
2006)
is
2πK
p(f
|this
,p(f
D)
N
;det
m(f
V2√
, D)exp,over(2)
Beyondthestate-of-the-art
in Gaussian
processes,
introduce
a novel∗ascovariance∗a∗distributionThematrixis
generated
by
a(f
κK
:X×
data. This desirable property
ismodelinducedbythe‘Occam’s
implicit
∗x
∗x
∗m(f
any
finite(such
X)
ismultivariate
Gaussian.
thegivestate-of-the-art
inRazor’Gaussian
processes,
wethatintroduce
ainferrednovel
equations
equationsp(f| ∗xfdet
D)=∗probability
Nx∗f, ∗D),
; m(f
|∗2πK
x|2πK
, D),
V (f
(2)
∗ this∗det∗ |2x2
Beyondthestate-of-the-art
in Gaussian
processes,
weintroduce
aofnovel∗ , information,∗the∗ , D) ,heteroskedastic(thatis, with
location-varying
noise
variance)
ordinalregresfunction fvalues
at suchinputthatx∗ . With
wedet
have
predictive∗R,
equations
from
noisy
participantlabels.
equationsX |×xX, D)
X R;→;fmatrix
distribution
thevalueson, X)K(X, X
heteroskedastic
(that
is,
with
location-varying
noise
variance)
regresThecovariancematrix
iscontaining
generated
byordinala covariance:The
X covariance×distribution
X : →
thatis,Kvalues
= the
κ(X,
X).

generated
within Bayesian reasoning(MacKay,
2003),
andisis,
suggested
empiricallybythatfvariance)(x),theκThe
prior
over
its
fmarginalona.asubset
x ⊂ X
arecompletely
 function
 is,
1covarianceheteroskedastic
(that
with
location-varying
noise
variance)
ordinalregres1 overm(f=covariancematrixis
generatedbya covariancefunctionκX: function
R;K(x
model
is trainedon adevelopment
datasetlabelsfor∗ X ∗→→isbyfunction
:
×
R;
 κ−1
∗
heteroskedastic
(that
is,
with
location-varying
noise
ordinalregreswheresion model,This
described
inSection
5.2.
This
is
necessary
toman
equations
where


√
:=
:=
The
covariancematrixis
generatedbyacovariancefunctionκ:
X×X X→→R;R
expuse−the
p(f
|x
K)
N
(fcovariance
;of
m,
K)
K
fmake
. For
(1)
The
matrixisisgenerated
byf agp
covariancefunction
κ:X×theinstate-of-the-art
inGaussian
processes,
wenovel|introduce
x∗to
=fto
f∗ ∗; m(f| ax∗mean
D),
(f
|
x
,
D)
,
(2)
p(f
xis
, necessary
D)
=, D)
N
;aspecifiedm(f| x∗by
,that
D),
V,is,
(fK
|finite
,
D)
,
(2)
any
subset
X
(such
as
X)
multivariate
Gaussian.
afunctionwhere∗∗sion model,
described
Section5.2.
Thisdevelopment
is necessarymanthatis, SectionK= κ(X,
X).
∗
Given
training
data
D,
we
to
predictions
about
the
∗ |p(f∗N∗V
the cross-validation tests
presented
in Section
6.2.
The
Gaussian
process∗∗∗
vectorm
andcovariancematrixKsion Beyond
model,
described
in
5.2.
This
developmentmanκ(X,
X).
2
=
κ(X,
X).
p(f
| that
xp(f
=K
f=
x∗is,
,K
D),
, D)
V (f∗ | x∗ , D) = K(x∗ , x∗ ) − K(x
V
(2)
|x
,N
D)
=
Nthat
fthat
|=x
D),
Vxdet
| x,∗ , D)
, (2)
sion
model,
described
in Section
5.2.
This
development
is∗necessarytoman∗ ; m(f∗∗ ;| m(f∗ |X).
∗ , D)∗(fage participantlabelsthat are(thatbothordinalandof varying
uncertainty.
Our
∗ is,
∗K
∗ 2πK
∗ ,(f
is,
κ(X,
=
κ(X,
X).
each
occupation
from
each
individual
expert,ratherthanheteroskedastic
is,withlocation-varying
noise
variance)
ordinal−1−1f∗(x),
thevaluesf(fonathis
Xaboutarewecompletely
age participantlabelssuitable
that
are
both
ordinalandinteracof
varying
uncertainty.
Our regresGiven
training
data
D,and
we
use
the
gp
to
make
predictions
about
the
values.toWith
information,have
the predictive
at
input
x
gives a flexible, non-linear,function
class
for
the
complexage
participantlabels
that
are
both
ordinalofvarying
uncertainty.
Our uncertainty.
m(f
, ∗D)
K(x
,over
X)
ymake
Given
training
D,
the
gp
to
the(3)(3)p(f|priorx|∗data
,x|D)
N
fwe
;fm(f
|its
x=
D),
xsubset, D),xy⊂about(2)
xdistribution
,=
D)
=
K(x
,the
X)K(X,
X)
−1∗function
,=
we
use
make
predictions
Given
training
data
∗data
∗∗use
∗m(f
∗use
∗the
∗y | predictions
Given
training
D,
we
gp
to∗we
make
the
∗ ,
∗predictions∗X)K(X,
ageofparticipantlabelsthat
bothordinalvarying
Our
where
1∗V
where
model is put towork
on a describedvarietytasks.
In particular,
it areis thebasis
ourand of to
m(f
D)
K(x
, X)K(X,
X)
(3)
Given
training
D,
use
the
gp
make
predictionsabout
function−1∗ 1| xm
The
covariancematrixis
generatedbyadata
covarianceκgp:totheXto
×label
X →
R;
Given
training
D,
we
use
the
make
predictions
aboutthethe∗ ,datasionmodel,
in
Section5.2.
Thisdevelopment
isofbasis
necessarymanspecified
by
a
mean
vectorandcovariancematrix
K
model
is
put
to
work
on
a
varietyoftasks.
In
particular,
it
is
the
of
our
√
where
functionvalues
f
at
input
x
.
With
this
information,
we
have
the
predictive:=:=−1
equations
Inferringposterior
p(y
exp
p(f
|
K)
N
(f
;
m,
K)
f
−
K
f
.
(1)
tions (for instance, complementarities)that
weexpectbetweenvariables
and
where
−1
model
is
put
to
work
on
a
varietyoftasks.
In
particular,
it
is
the
basis
of
our
∗
∗ | x∗ , D) i∗
functionvalues
fat
at
input
x
. Withthis
information,
we
have
the
predictiveon theoccupationsgroup consensus.
Thisapproachpermitsthe∗∗ xfunctionvalues
f
input
x
.
this
information,
we
have
the
predictive
V
(f
|
,
D)
=
K(x
,
x
)
−
K(x
,
X)K(X,
X)
K(X,
x
)
.
(4)
∗With
V
(f
|
x
,
D)
=
K(x
,
x
)
−
K(x
,
X)K(X,
X)
K(X,
x
)
.
(4)
−1
model
is
put
to
work
on
a
varietyoftasks.
In
particular,
it
is
the
basis
of
our
∗
∗
∗
∗
∗
∗
∗
∗
means of selecting
tobelabeledthrough
activelearning(Section
∗
∗
∗
∗
∗
2
functionvalues
f
at
input
x
.
With
this
information,
we
have
the
predictive
det
2πK
that
= κ(X,
X).
functionvaluesatinput
input
With
this
information,
we
have
the
predictive
∗f
V
(f
|
x
,
D)
=
K(x
K(x
,
X)K(X,
X)
K(X,
)
.
(4)
age
participantlabelsthatarelabeledbothordinalandofvarying
uncertainty.
OurX)−1 is,−1K
about
thefunctionvalues
at
this
where
∗x∗
 x
∗
∗,. xWith
∗(3)
∗) −
∗
∗
∗
means meansofselectingoccupations
to bethrough
activelearning(Section
equations
form
of
the
ordinallikelihood
tobeintroddemand.


m(f
|
x
,
D)
=
K(x
,
X)K(X,X)y(3)of selecting
occupations
tobelabeledthrough
activelearning(Section
m(f
|
x
,
D)
=
K(x
,
X)K(X,yequations∗ our∗active learning−1∗to be11∗ ∗labeled∗equations
−1
means
of
selectingoccupations
through(Section−15.3), and interpretingthepatterns
discovered
by
the
model
is
the
basis
of
m(f
|
x
,
D)
=
K(x
,
X)K(X,
X)
y
(3)
p(f
|
x
,
D)
=
N
f
;
m(f
|
x
,
D),
V
(f
|
x
,
D)
,
(2)
equations
m(f
|
x
,
D)
=
K(x
,
X)K(X,
X)y(3)
training
data
D,
we(f
use
gp
tois makeabout
theequations∗m,the∗∗∗ fdiversity
views
withinthegroup
within
∗ 
∗ −1
isof
put
tothe
work
on apatterns
varietyoftasks.
Inbe
particular,
it isisthetheofbasis
basis
of∗ourour∗ Given∗by∗.
 captured√:=by∗ Inferring∗p(f| K)posterior
N
;p(y
K)
fR;
−by
Knon-Gaussian(1)−1
Inferringthelabel|∗∗ x| ∗:=D)
complicatedthenon-Gaussian5.3),onandinterpreting
patterns
discovered
by to
the
model
is ,theThis model is trained
amodeldatasetcontaininglabelsfor eachoccupation
the
label
p(y
x, ∗−1,functionD)iscomplicatedThe
covariancematrixisX)
generatedcovarianceκ predictions
:exp
×isX
→
∗(4)
5.3),
and
interpretingthediscovered
by
model
of
a(4)
(f
| x| ∗basis=∗VK(x,| xxK(x−is∗information,K(x, X)K(X,
K(X,
x
we
have
predictive
VN
(fthe
D)
=, D)
K(x
,our
x(f
−
, the
X)K(X,X)x∗ )K(x.posterior
Inferringthelabelposterior,X
D)complicated∗ (Section∗ ), D)
2 ,theand interpreting
the=patterns
discovered
by
the
model
basis
of,∗x
our
m(f
| )x
D)
=
, .X)K(X,X)
yxequations:(3)by(2)the∗∗)−1 non-Gaussian
∗ )∗∗
det
∗activeassessment of means
the importanceof5.3),
O*NETvariables
futuredemand∗ | xwe−1p(f∗variables| xto
D)f∗ ∗V|; xm(f,(SectionD),, =∗form(2)∗∗, ,at
fK(X,
xthe
With
information,
thepredictive∗ x∗learningof
selectingoccupations
belabeledthrough(Section∗ , ThisV(f| is,,function
D)X)K(X,K(X,).V2πK
p(f
=
fm(f
;this
m(f
|∗introduced
,∗p(y
D),
Vx∗below.
(f
, D)
∗
Vx∗(f
,K(x
D)values
=
K(x
)D)
K(x
X)K(X,.x|∗have
∗ ∗
∗ the
∗K(x
∗to
∗(4)
p(f
x|∗∗input,x=likelihood
N.∗ ,N
fp(f
|X)
xD)
,xD),
|∗below.
,x| D),, D),
oftheordinallikelihood
assessmentof theimportanceof O*NET
toapfuture
demand∗ of∗,, D)∗−∗K(X,∗Heteroskedastic∗ |=xfrom each individual expert,
rather
than
on
group
consensus.
form
ordinal
that
K
X).
∗κ(X,
∗|where∗ )5.2.
∗−∗ ;X)assessmentofthetheimportance
of to
O*NET
variablestofuturedemand(Sectionordinal
p(f
xto
D)
=
N
f∗(f
m(f
,(4)
D),
V
(f
x(2)
(2)(2
the
model.
The
participantlabelsaremodelled−1|∗bex∗, be,∗introduced
=
Nto
f;X)
V (f
|∗x, D), D), , regression∗−|∗K(x∗K(X,∗ |∗(4)formoftheordinallikelihood
beintroduced
below.
assessmentoftheimportanceofO*NETvariablestofuturedemand(Section∗ ; m(f∗ x|∗x
∗
∗
∗
V
(f
|
x
,
D)
=
K(x
,
x
)
,
X)K(X,x).
equations∗covariance5.3),
andwithininterpretingthe topatterns
discoveredby theisthep(y
basis
our
(2)
∗the
∗ by predictions∗ the κ : X ×
Thedatamatrixis∗ gpgenerateda covarianceabout
functionX → R;
Inferringthemodellabel
posterior| x∗is, D)
is complicatedbywe
thenon-GaussianInferringthelabelposterior
xof, Given
D)
complicated
by∗D,
the
non-Gaussian
proach permits the diversity of
views
the
group
be captured
withintrainingto∗ |p(ywhere
Inferringthe∗∗label
posteriorp(y∗ | xp(yis∗usebymakethe bynon-Gaussian
where
Inferringthelabelposterior|x,complicatedD)is complicatedthenon-Gaussian
 process ordinal regression cons
−1
where
35
∗ , D)∗ κ(X,
Gaussianassessmentof theindependent
importanceof O*NET
variables
to ordinal
futuredemandaslabelsconditionally
given
the
function
that
is,
K
=
X).
m(f
|
x
,
D)
=
K(x
,
X)K(X,
X)
y
(3)
where
ofordinal
thelikelihood
tobeintroduced
below.
where
form
of
thelatent
likelihoodto (Section
beintroduced
below.
∗
the model. The participant
are modelled
as conditionallyindependent35
functionvalues
f
.
With
this
information,
we
have
the
predictive
at
input
x
∗
∗
5.2.
Heteroskedastic
ordinalregression
5.2.
Heteroskedastic
ordinalregression∗
35 form
theto
label
posteriorD)complicated
by
the
p(fbe
x
D)Heteroskedastic= p(yNbelow.
V regression(f∗ |is,xD) non-Gaussian, numerical(2) scores) observation∗f|∗ ;xm(fof the ofordinallikelihood∗ |introduced∗ |isxordinal∗ ,below.
∗∗,be∗ , D),∗ ,ordered
theInferring
ordinallikelihood
tointroduced
35 form
−1
5.2.
−1 gp
−1
m(f∗the| xgroup’s, D) = K(xy formequations(3)|∗x| ,xtraining
Given
data
D,
use
the
to K(xmake
predictions
about
∗the
∗ , X)K(X,
given the latent function f (x).. The
The dependence
amongstlabelsm(fX)K(X,X)y, x∗ ) −(3) the
VK(x
(f
x
, we
D) =
K(x
X)K(X,X)−1
K(X,x∗ ) .
(4)
m(f
==
K(x
,∗X)K(X,X)=
y ∗below.
(3)∗ , D)∗|,be−1∗ ,introduced∗ , −1
theordinallikelihood
dependence
amonggroup’slabelsX)
inducedby −1 form of
∗
∗ D)
∗to
m(f
|
x
D)
=
K(x
,
X)K(X,
X)
y
(3)
m(f
|
x
,
D)
K(x
,
X)K(X,
X)y(3
Gaussian
process
ordinalregression
consistsofcombininganordinal(that∗∗∗
Gaussian
process
ordinalregression
consistsofcombininganordinal(that∗ x ∗. With this ∗information,
35=5.2.
f∗ at process −1valuesinput
have
predictive−1 weofwhere
induced by discussion is modelled through this shared Vlatent
ordinal∗ordinal
regression
consists(f∗ | xfunction.
K(x∗Heteroskedastic, x∗ ) − K(x∗ordinal, X)K(X,
X)regression
K(X,x∗ ) . function(4)
5.2.
Heteroskedastic
regression∗ , D)V(fD)
xGaussian
, D)
=
K(x
x∗−,)observations
−
K(x
X)
K(X,
x.∗combining) . the−1(4)an ordinal (that∗x|numerical
V
(f
|
,
D)
=
K(x
,
x
K(x
,
X)K(X,
X)
K(X,
x
)
(4)
∗N
∗|,∗ )∗ , X)K(X,
−1
∗
5.2.
Heteroskedastic
ordinalregression
∗
∗
∗
∗
is,
orderedscores)withaGaussian
processprior.
5.2.
Heteroskedastic
ordinalregression
p(f
|
x
,
=
f
;
m(f
x
D),
V
(f
|
x
,
D)
,
(2)
is,
orderednumerical
scores)observations
withaGaussian
processprior.
Inferringthelabelposterior
p(y
|
x
,
D)
is
complicatedbythenon-Gaussian=scores)
K(x
) ∗−
K(x
X)
x∗x) ∗. ) . prior.
(4)(4
36
∗
∗ V (f
∗K(x
V (f
=
)∗ −
K(x
, X)K(X,
X) K(X,
K(X,
discussion
throughshared latent function.
∗observations∗∗numerical∗xequations∗ , ∗x∗ , X)K(X,∗ |∗xD)∗ , D)∗∗
Resilience to uncertainty
is crucial to is
ourmodelled
exercise. Not
only did thisour workis,
orderedwithaGaussian
process
−1
where
5.2.
ordinalregressionm(f| xlabelD)posterior=,ordinal
X)K(X,
ycomplicatedGaussian
regression
consistsof thecombininganordinal(that
Gaussian
process
regression
consistsofHeteroskedastic
combininganordinalInferringwith
the labelposteriorp(yx∗ , ordinalD) isordinal
complicatedbythenon-Gaussian
 X)
 (3)
ofK(x
the
to be(thatintroduced
below.
∗ ,form∗(that∗ | process
Inferringp(yisordinal
the
non-GaussianInferringthe∗ label
posterior|∗ofx|∗posterior,xlikelihood
D)is complicatedbyby
non-Gaussian
shops gather observations from individual participants
explicit
represen∗ , D)
Gaussian
process
ordinalregression
consistsofp(ycombining∗ labelGaussian
process
ordinalconsists
ordinal(that
where
Inferringlabel
p(yp(yD)
isthecomplicatedbyby
thenon-GaussianInferringtheposterior|∗x, −1
thenon-Gaussian∗x∗|∗,xD),p(f|processxthe=prior.
Ncombiningf∗ ;an|an(fxx∗ ,)D), (4)(2)
is, orderedscores)observations
witharegression
Gaussian∗ ,VD)∗K(x∗is| complicatedis,orderednumerical
scores)observations
with
athe
Gaussian
process∗∗,prior.
form
likelihood
tobe numericalintroduced
below.
36
Vprocess
(f
x
D)
=
,D)
xa∗with
)be
−be
K(x
,m(f
X)K(X,
X)
K(X,
form
the
likelihood
tointroduced
below.
∗ |ordinal
Gaussian
regression
consistsof∗ below.
combininganordinal(that∗ ,ordinal
∗36
∗ .
form
of of
ordinallikelihood
tointroduced
to uncertainty
isobservations
crucialtofromour
exercise.
tations of their uncertainty,Resiliencethe model mustalso
try ofto thefuseordinalis, orderednumerical
scores)observations
with
Gaussian
processprior.
36
is, orderednumerical
scores)observationsaGaussian
processprior.
form
of of
the
ordinallikelihood
toto
bebe
introduced
below.
form
theordinallikelihood
introduced
below. (3)5.2.∗ ,scores)
Heteroskedastic= K(xX)K(X,X)−1 y ordinal(3) prior.
is,m(forderednumerical
observations
with regression
a Gaussian process
the diverse range of opinions produced by our domain experts. Our model∗ | x∗ , D)
wherethe label posterior p(y∗ | x∗ , D) is complicatedby the non-GaussianNot only did our5.2.
workshops
gatherobservations
365.2. Heteroskedastic
36 Inferring
Heteroskedastic
ordinalregression from
regression
choice is informed by the probabilistic foundationsof the Gaussian process,
V (f∗of| xthe= K(x)ordinal−process
K(x
X)−1
K(X,
x∗ ) .
5.2.
Heteroskedastic
ordinal36∗ 36∗ , D)∗, x∗ , X)K(X,
Gaussianordinalregression
consistsof(4)combining
(that
form
ordinallikelihood
toregression
beintroduced
below.
−1
5.2.
Heteroskedastic
ordinalregression
5.2.
Heteroskedastic
ordinalregression(4)an ordinalm(f(3)∗ | x∗ , D) = K(x36∗ , X)K(X, X) y
which give it a coherent way
to reason about
uncertainty.
As
such,
we
expectindividual
participants
withexplicit
representationsof
theirGaussian process ordinal regression consists of combining
anordinal(thatis,orderednumerical
scores)observations
witha Gaussian
process prior.
Gaussian
process
ordinalregression
consistsofcombininganordinal(that−1Gaussian
process
ordinalregression
consistsofcombininganordinal(that
Inferringthelabelposterior
p(y
|
x
,
D)
is
complicatedbythenon-Gaussian∗x∗ , D)∗process
V
(f
=
K(x
,
x
)
−
K(x
,
X)K(X,
X)
K(X,
x
)
.
(4)
|
Gaussianordinalregression
consistsofcombininganordinal(that
our model to give an honest representation of is,
the orderedtrends thatcanbeinferred
Gaussian
process
ordinalregression
consistsofcombininganordinal
(tha
∗
∗
∗
∗
∗
numerical
observations
with a Gaussianprior.
Heteroskedastic
ordinalregression
is,ordered
orderednumerical
scores)observations
witha aGaussian
Gaussianprocessprocessprior.
prior.
uncertainty, the model mustalso scores)
try to fuseobservations
is,process
numerical
scores)observations
withform of 5.2.
theordinallikelihood
tobeintroduced
below.
is,is,orderednumerical
scores)observations
withfrom noisy participant labels.
orderednumerical
scores)observations
witha aGaussian
Gaussianprocessprocessprior.
prior
36
Inferring
the labelposterior p(y∗ | x∗ , D) iscomplicatedby thenon-GaussianInferringthelabelposterioriscomplicatedbyBeyond the state-of-the-art
Gaussian
processes,
novel
Gaussian process ordinal regression consists of combining an ordinal (thatfrominthe
diverserange weof introduce
opinionsa produced
36by ourform ofordinal
the ordinal
likelihood36
to36be introduced below.
5.2. Heteroskedastic
regression
heteroskedastic (that is, with location-varying noise variance) ordinal regresis, orderednumerical scores)
observations
withlikelihood
a Gaussian
process
3636 tothenon-Gaussian
form ofthe ordinalbe prior.
domainexperts.
Our
model
choiceisinformedbythesion model, described in Section 5.2. This development is necessary to manGaussian
process
ordinalregression consists
ofregression
combining an ordinal (that
5.2.
Heteroskedastic
ordinalintroduced
below.
age participant labels thatprobabilistic
are both ordinaland of varyingof
uncertainty.
Our process, which is, ordered numerical scores) observations withfoundations
the Gaussian
36 a Gaussian process prior.
model is put to work on a variety of tasks. In particular, it is the basis of ourGaussian process ordinal regression consists of combining an ordinal (that
give
it
a
coherentwaytoreasonabout
uncertainty.
5.2. HETEROSKEDASTIC
ORDINALREGRESSION
means of selecting occupations to be labeled through active learning (Sectionis, ordered numerical36scores)
observations
with a Gaussian process prior.
5.3), and interpreting the patterns
discoveredby theourmodel
is the tobasis
of ourAs such,we expect
model
give
an honest
assessment of the importance of O*NET variables to future demand (Section
Gaussian process ordinal regression36consists of combining

representation of the trends that can be inferred from35 or noisy, participant labels.
uncertain,

Beyond the state-of-the-art Gaussian processes, we
introduce a novel heteroskedastic ordinal regression model(that is, a model with location-varying noise variance),described in Section 5.2. This development is necessary

36

ordinal (that is, ordered numerical scores) observations
with a Gaussian process prior. Our approach differsfrom the state-of-the-art (Chu and Ghahramani, 2005), in
Our approach differs
fromthe state-of-the-art
incorporating a heteroskedastic
noise
model.
Recall that (Chu and Ghahramani, 2
firstly, in incorporating a heteroskedastic noise model. Recall that pa
participants describehowconfident
they
are
using
choiceipants describe how confident they areausing
a choice from an ordina
We assume
thatthat
the noisestandard deviation associated withassumethe noisefrom an ordinal list {0, . . . , 9}.. We

observation is an affine transformation of the chosen value: σnoise = α
for i ∈ {0, . . . , 9} with hyperparameters α ∈ R+ and β ∈ R+ . Since
value of noise is an ordinal variable as well, we build a secondary or
regression model to predict the ordinal value of noise at different poinfeature space.
As there are two questions being asked, one relating to share change
the other absolute change, two different types of observations are madebuild a model capable of acknowledging and fusing these two differen
servation types by employing and extending a Python library called GP(Matthews et al. (2016)). We use a Gaussian process to model a latenttion f (x), representing the change in demand in absolute employment.

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Our approach differs from the state-of-the-art (Chu and Ghahramani, 2005),
Our approachdiffersdiffersfrom the(Chu and Ghahramani,2005),
approachfromstate-of-the-art
the state-of-the-art
andthatGhahramani,firstly, in Our
incorporatinga heteroskedastic
noise model.(Chu
Recallpartic- 2005),
firstly,firstly,
in incorporatinga heteroskedastic
noise noisemodel.model.
RecallRecall
that particin incorporatinga heteroskedastic
that participants describehowconfident they
are using a choicefrom an ordinal
list
ipantsipants
describe
how confident
they areaeachchoicefrom froman ordinallist list 5.4. ASSESSING FEATURE IMPORTANCE
describehowtheyusingareusing
a observation
choicedeviation
associatedwithisordinal
an{0, . . . ,standard9}. We assume
thattheconfident
noise standard
deviation
associatedwithaneach{0, . . .{0,, 9}.
assumethat thenoise
deviation
associatedwith each each
.We
, 9}.
We transformation
assumethat
theofstandard
noise
deviationobservation
is. . anaffinethe standard
chosenvalue:
value:
σnoiseassociated= α + βiwith
foraffinetransformationofthechosenobservation
is an isaffinetransformationof the chosen
value: σnoise σ=noise
α + βiobservationanaffinetransformation
the chosen
for i ∈ {0,. . . , 9} withhyperparametersα ∈ R+of and
β +∈ R+value:. Sincethe = α + βi One of the core goals of this work is to assess thefor i ∈
. . .{0,, 9}. . with
α ∈ R
βand
∈ R
. Sincethe thefor{0,
i is∈. , 9} hyperparameters
withhyperparameters
α+∈andβ+∈
R+. Since
Since
with
hyperparameters
value of noise
an ordinalvariable
as well, we build
aRsecondary
ordinalsignificance of the 120 O*NET features to future demand,value valueof noise
is an isordinalvariable
as well,wea secondary
ordinal
of to
noise
anthe
ordinalvariablewell,build
wedifferentbuild
a points
secondaryregression
model
predict
ordinalvalue variableofasnoiseatin aordinal
the
value
of
noise
is theanordinalasatwell,
we build
regression
model
topredict
ordinalvalue valueof noisedifferent
pointspoints
in
regressionmodel
to predict
the ordinalof noiseat different
in and thereby inform skills policy decisions.
feature space.
featurespace.space.
featuresecondaryordinal regression
model
to to
predictthe ordinal
As there
are two questions
being asked, one
relating
share change,andAs there
are twobeing being
asked,asked,
one relating
to sharechange,and and
As therearequestions
twoquestions
one relating
sharechange,theOurother
absolutechange,twodifferent
types ofin
observations
areto
made.
We
First, however, our research question needs further
value
of
noise
at
different
pointsfeaturespace.
approachdiffersfromthestate-of-the-art(Chuand
Ghahramani,2005),
the otherabsolutechange,two different
types typesof observations
are made.
Wethe otherabsolutechange,two differentof observations
are
build
a modelcapableof acknowledging
and noisefusingmodel.
these
two
different
ob-made. We
firstly,inincorporatingaheteroskedastic
Recallthatparticbuild builda modelcapableof acknowledging
and fusing
these thesetwo different
ob- ob- clarification: what exactly does it mean for a feature to beabymodelcapableofextending
acknowledging
andlibraryfusingtwo
different
servation
types
employingandaPythoncalledGPflow
ipantsdescribehowconfident
theyextending
arebeing
using
choicefrom libraryancalledordinallistGPflow
Asservation
there
two
questionsonerelating
servation
typesareby employing
andaaasked,Pythonlibrary
GPflowtypes
byemployingandextending
Pythoncalled(Matthews
et al.
(2016)).
We
anoiseGaussian
process
toa model
a latentwith
funcOur{0,approachdiffers
fromthat
the usestate-of-the-art(Chuand
Ghahramani,2005),
important to demand? We propose two primary criteria. . . ,(Matthews
9}.
We
assumetheusestandard
deviationeach(Matthews
et al.
(2016)).
We
Gaussian
process
toassociatedmodel
a latent
func- funcet
al.
(2016)).
Weademanduse
a Gaussian
process
to modela latenttosharechange,
and
the
other
absolutechange,
two
tion
f
(x),
representing
thechangeininabsolute
employment.
The
firstly,inincorporatingaheteroskedastic
noise
model.
Recallthatparticobservationisanrepresenting
affine transformation
ofinthechosenvalue:
σnoise= α +Theβi The for a scheme to measure importance:tion ftion(x),(Chu
the change
in2005),demandin+absolute
employment.
frepresenting(x),
the changedemandinabsolute
employment.
OurOurapproachdiffersfromthethestate-of-the-art
andGhahramani,2005),+ changeapproachdiffersfromstate-of-the-art(Chuand
Ghahramani,
relativechange
in
share
can
be
represented
bydividingtheabsolute
ipantsdescribehowconfident
they
are
using
a
choicefromanordinallist
for
idifferent∈relative{0,. . . ,change
9}
hyperparameters
α are
∈
and
βthe
∈ absolute
Rbuild. Sincethechange
types
observations
made.
We
achangerelativechangein with
share
can
becan
represented
byRdividing
inof
share
beparticrepresented
bydividingtheabsolute
firstly,
in incorporatinga heteroskedastic
noise
model.
Recallthatparticfirstly,
in incorporatinga heteroskedastic
noise
model.
Recall
that
in
by
the
total
size
T of
the
workforce
inwe
2030.
That
is, thesecond{0,demand. . . , 9}.
We
assume
that
the
noise
standard
deviation
associated
with
each
value
of
noise
isthe
an
ordinalvariableaswell,a2030.
secondaryordinalin demand
by
total
size
Tsize
of
the
workforce
inbuild2030.
That
is, thesecondinusingdemandbythefromtotal
T listoftheworkforce
in
That
is,
the second
1. An important feature must be clearly predictive
ipantsdescribe
howhowconfident
they
are
using
a
choicefromanordinalmodel
capableofacknowledging
andfusingthese
twoipantsdescribe
confident
they
are
a
choiceanordinal
list
of
the
two
workshopquestionsprovidesanobservation
f
/T
,
where
T
is
an
observationisanaffinetransformationofthechosenvalue:
σ
=
α
+
βi
noise
regression
model
topredict
the provides
ordinalvalue
ofannoise
atfdifferentof thetwoworkshopquestionsan observation/T , fwhere
T is in
an
of
the
two
workshopquestionsprovidesobservation/T ,points
where
T is an{0,{0,. . . ., .9}.
WeWe
assumethatthethe
noise
with
each
. , 9}.
assumethat
noisedeviation
associated
with
each
unknown
positivevalue
tobe inferred
for
each
for
istandard∈standard{0,. .deviation. , 9}
withassociatedhyperparametersα ∈ participant.
R+ and β ∈ R+ . Since thefeaturespace.
of demand.
different
observation
types
byemployingandextendingunknown
positive
value
to
be
inferredfor
eachparticipant.
unknown
positive
value
to
be
inferred
for each participant.
observation
is anaffinetransformation
ofeachthechosenvalue:
σvariable=α+asked,βi
observation
is anaffinetransformationof thechosenvalue:
σnoise=asα+βi oneForofquestion,themodelingests
ternary-valued
from
one ofand
noise
value
noise
isare
antwo
ordinalwell,
we builda labelssecondaryordinalAs
therequestionsbeing
relating
to sharechange,
For
each
the
ingests
ternary-valued
labelslabelsfrom
one ofone of
For
each
the
model
ingests
ternary-valued
from+question,+model+ question,+Sincefor fori ∈i {0,. . . ., .9}
withhyperparameters
α
∈
R
and
β
∈
R
.
the
∈ {0,
. , 9}
withhyperparameters
α
∈
R
and
β
∈
R
.
Since
the
GPflow,apackageforbuilding
Gaussian
processmodelsthesets{Highershare,
Sameshare,
Lower
share}
and{Grow,
No
change,regression
model
topredict
theordinal
value
of
noise
at
different
pointsintheother
absolutechange,two Same
different
types
of observations
made.
Wechange,thesets{Highershare,
Sameshare,
Lower
share}and {Grow,
Nochange,thesets{Highershare,
share,
Lower
share}
andare{Grow,
No
2. An increase in an important feature must leadvalue
of noise
is anordinalvariableas space.
well,
we
build
a secondary
ordinalvalue
of noise
is anordinalvariable
as
well,
we
build
aof
secondaryordinalDecline},forrelative
shareof
employment
andabsolutechangein
employfeaturebuild
aDecline},model
capableacknowledging
andandfusingtwo
differentinPythonusing
TensorFlow(Matthewsetal.,2016).
Decline},fornoiserelative
shareof points
employment
absolutechangein employfor
relative
shareof employment
andthese
absolutechangeinobemployregressionmodel
to predict
thetheordinalvalue
ofatdifferentin
regressionmodel
to predictordinal
value
of
noise
at
different
pointsinment,respectively.
From
these
ternary-valued
observations
the
model
proAs
there
are
two
questions
being
asked,
onerelating
tosharechange,andto a strong increase in demand.
servation
types
by employing
andthese
extendinga Pythonlibrary thecalledGPflow
ment,
respectively.
From
these
ternary-valued
observations
model
proment,
respectively.
Fromternary-valued
observations
the
model
profeature
space.
featurespace.
duces
aWeposterioroverbinary-valued
labels,ato
namely,use
adistribution
Gaussian
process
to modellatentfunctiontheotherabsolutechange,twodifferent
types
ofprocessobservations
are{Increasing
made.
We(Matthews
eta al.
(2016)).
Weuse
a Gaussianmodel
anamely,latentfuncducesaone
posterior
distribution
overbinary-valued
labels,labels,
namely,{Increasing
duces
posterior
distribution
overbinary-valued{Increasing
As As
there
arearetwotwoquestions
being
asked,
relating
tosharechange,andthere
questions
being
asked,
onerelating
tosharechange,anddemand,Decreasing
demand},
which
are
used
as
the
foundationoftheanalbuildamodelcapableofacknowledging
andfusingthese
twodifferent
obtionf (x),
representing
the demand},changein whichdemandinas
absolute
employment.
The
We also propose two secondary criteria for a scheme to, Decreasing
representing
thechangeinare
demandinwithabsolute
demand,demand},
which
are
used
theasfoundationof theofanaldemand,usedthefoundation
thetheother
absolutechange,twotwodifferent
types
of by
observations
are
made.
We
other
absolutechange,different
types
of Decreasing
observations
are
made.
We
ysis.
The
derivative
ofshare‘Increasing
demandofaan
respect
to the analservation
types
employingandextending
Pythonlibrary
calledGPflow
relativechangeincanberepresented
byoccupation’dividingtheabsolutechangeysis. andThe
derivativeof two‘Increasing
demandofanoccupation’with respect
to
ysis.
The
derivative
ofdifferent‘Increasing
demandofan occupation’withrespect
to measure importance:build
a modelcapableof acknowledging
fusingthese
obbuilda modelcapableof acknowledging
andfusingthese
twodifferent
obtheinoccupation
featuresismadepossible
by
the
use
of
automatic
differentiaemployment.
The
relativechange
in
share
can
be
(Matthews
etal.
(2016)).
We
use
aGaussian
processtomodelalatentfuncdemandby the
total
size
T is
of madethe workforce
inuse
2030.
That
is, thesecondtheoccupation
featuresiscalledmadepossibleby thebyofuse
automatic
differentiathea occupation
featurespossibletheof automaticdifferentiaservation
types
by by
employing
andandextending
Pythonlibrary
GPflow
servation
types
employingextendingaofPythonlibrary
calledGPflowtion,
featureGoogle’sTensorflow
Python(Abadial. (2015)),
tion
fathe
(x),
representing
thechangeindemandinabsolute
employment.
The
oftion,twoworkshopquestionsprovidesanpackageobservation
f /Tet, (Abadi
whereisrepresented
bydividingtheabsolutechangeination,
featureoftoGoogle’s
Tensorflow
Pythonpackage(Abadietdemandal. Tet(2015)),
aprocessfeatureof
Google’sTensorflow
Pythonpackageal.an(2015)),(Matthews
et al.
(2016)).
WeWe
useusea Gaussian
processmodelalatentfunc(Matthews
et al.
(2016)).
aGaussiantomodelalatentfuncwhichprovidestheframeworkto
GPflow.
relativechange
in
share
can
be
represented
bydividingtheabsolutechange1. It must be able to uncover non-linear interactions
unknown
positive
value
toframework
be inferredeach participant.
which
providestheframework
to GPflow.
whichprovidesthe
tofor
GPflow.
tiontionf (x),
representing
thethechangedemandinabsolute
employment.
Thef (x),
representingchangeinFor
demandinabsolute
employment.
The
in in
demand
by
the
total
size
T
of
the
workforce
in
2030.
That
is,
the
second
by
the
total
size
of
the
workforce
in
2030.
That
is,
each question, the model ingests ternary-valued labels from one of
between features.
relativechangein sharecancanbe berepresented
dividing
thetheabsolutechangerelativechangein sharerepresented
bydividing
absolutechangeofthetwoworkshopquestionsprovidesan observationf /T{Grow,, whereNo
T is
an
5.3.
learning
theActive
sets by{Highershare,
Sameshare,
Lower
share}
andchange,thesecondofthetwoworkshopquestionsprovidesan
5.3.
Activelearning
5.3.
Activelearning
in demandby bythethetotal
sizesize
T oftheworkforce
in
2030.
That
is,
the
secondin demandtotal
T unknownof Decline},theworkforce
in
2030.
That
is,
the
secondpositive
value
to
be
inferredfor
eachparticipant.
for relative share of employment and absolute change in employAsprovides
described
inobservation
Sectionthe
ourforesight
workshops
requiredchoosing
which
2. It must be able to capture complementaritiesof thetwotwoworkshopquestionsprovidesanAs
observation
f4.3,
/T
,4.3,
T
isT anof theworkshopquestionsanfmodel
/T
, our
where
is
an
For
each
question,ingests
labelsfromone
ofpro-to, wherewhere
isternary-valued
anunknown
positive
value
ment,
respectively.
From
these
ternary-valued
observations
the
model
As observation
described
in Section
foresight
workshops
requiredchoosing
which
described
Section
4.3,
foresight
workshops
requiredchoosing
which
occupations
were
to beinSame
presented
to ourparticipants.
We {Grow,
introducedtheuse
unknown
positive
value
to be
inferredfor
eachparticipant.
unknown
positive
value
to be
inferredfor
eachparticipant.
thesets{Highershare,
share,
Lower
share}
andNochange,duces
aoccupations
posterioroverbinary-valued
labels,{Increasing
occupations
weredistribution
tobepresented
to participants.
Wenamely,
introduced
the usewere
tobepresented
tochoice.
participants.
We introduced
the usebetween features: we wish to discover featuresbeinferredfor
eachparticipant.
ofamachinelearning
model
toautomate
thisThe
machinelearning
ForForeach
question,thethemodel
ingests
ternary-valued
labels
from
one
of
each
question,model
ingests
labelsfromofDecline},foraternary-valued
relative
shareof employment
andabsolutechangein of
employdemand,
Decreasing
demand},areusedaschoice.
the choice.
foundation
theanalofa machinelearning
model
towhichautomate
this
The machinelearningofmachinelearning
toone
automate
thisThe
learning
model
used
was
a reduced
form
ofmodelthemodeldescribed
in Section
5.2
tomachine
ensurewhose importance is contingent on the values
thethesetssets{Highershare,
Sameshare,
Lower
share}
and{Grow,
No
change,{Highershare,
Sameshare,
Lower
share}
and{Grow,
No
change,ment,
respectively.
From
these
ternary-valued
observations
the
model
proysis.
The
derivativeofa‘Increasing
demandan occupation’respect
to ensure
model
used
was
reducedform ofthemodel
described
in Section
5.2
to 5.2ensuremodel
useda wasreducedformofthe ofmodel
described
inwithSectiontoFor
eachquestion,theingests
ternary-valued
Decline},for forrelative
shareof employment
andabsolutechangein binary-valued
employDecline},
relative
share
ofduces
employment
andabsolute
inmodel
employposterior
distribution
over{Increasing
the aoccupation
featuresischangemadepossibleby thelabels,
use ofnamely,
automatic
differentiaof other features.
ment,respectively.
From
these
ternary-valued
observations
the
model
proment,
respectively.
From
these
ternary-valued
observations
the
model
pro- as package
demand,demand},
which
are
used
theshare,foundation
ofettheanaltion,labelsaDecreasing
featureof
Google’sTensorflow
Python(Abadial.
(2015)),
fromone ofthesets{HigherSameshare,
duces
a posterior
distribution
overbinary-valued
labels,namely,{Increasingduces
a posterior
distribution
overbinary-valued
labels,namely,{Increasing
ysis.
The provides
derivativeof framework‘Increasing
demandof an occupation’ with respect to
which
the
to
GPflow.
One approach to assessing feature importance is feature
share}{Grow,
No
change,Decline},differentiafor
demand,Decreasing
demand},which
areLowerusedasfeatures
thefoundation
ofpossible
theanaldemand,
Decreasing
demand},
which
are
used
as thefoundation
of 37
the
analtheoccupation
isand
made
by
the use of automatic
37
37
ysis.
TheThederivativeof ‘Increasing
demandof anoccupation’withrespect
to topackage
ysis.
derivativeof ‘Increasing
demandofanoccupation’withrespecttion,
afeatureof
Google’sTensorflow
Python(Abadietal.
(2015)),
selection (Guyon and Elisseeff, 2003). Feature selection isshare of employment and absolute change in
5.3. relative
Active learning
thetheoccupation
featuresis madepossibleby bythetheuseofautomatic
occupation
featuresis madepossibleof automaticdifferentiawhich
providestheuseframework
todifferentiaGPflow.
a broad and well-studied topic, and aims to choose those
employment,
respectively.
Fromtheserequired
ternary-valuedtion,
a featureof Google’s
Tensorflow
package(Abadial.
(2015)),
AsPythondescribed
in Section
4.3,etour
workshops
choosing whichtion,
a featureof Google’s
Tensorflow
Pythonpackage(Abadiet foresightal.
(2015)),
which
providesthetheframework
to 5.3.
GPflow.
weretobepresented
toparticipants.
We
introduced
the
use
which
providesframework
tooccupations
GPflow.
features that are most informative of the function. In our
Activelearning
observations the model produces a posterior distributionof a machine learning model to automate this choice. The machine learning
context, it might be thought that the ranking of O*NETAs
described
inSection4.3,
ourforesight
workshops
requiredchoosingwhich
overlabels,namely,{Increasing
model
usedbinary-valued
was a reduced formof the model
described
in Sectiondemand,
5.2 to ensure
5.3.5.3.
Activelearning
Activelearning
occupations were to be presented to participants. We introduced the use
features selected through such a scheme is a means of
Decreasing
demand},which
arechoice.
usedTheas thefoundation
As As
described
in Section
4.3,4.3,
ourourforesight
workshops
requiredwhich
described
in Section
workshops
requiredwhich
of foresighta machinelearning
modelchoosing
tochoosing
automate
thismachinelearning
occupations
were
to to
be be
presented
to toparticipants.
We
introducedtheuseranking their importance to demand. We suggest thatoccupations
were
presented
participants.
We
introduced
the
use
model
used
was
a
reduced
form
of
the
model
described
inSection5.2
toensureof the analysis. The derivative of ‘Increasing demand ofof aofmachinelearningmodel
to automate
thisthischoice.
TheThemachinelearninga machinelearningmodel
to automate
choice.
machinelearning
the bulk of methods of feature selection give, at best, an
37 to the occupation features isan describedoccupation’
with5.2respect
model
used
waswasa reduced
formof themodel
in Section
to ensure
model
used
a reduced
formof themodel
described
in Section
5.2
to ensure
OneOneapproach
One approachto assessing
to assessing
featureimportancefeatureimportanceis featurefeatureselection(Guyon(Guyonapproachtoguideassessingimportanceisfeatureselection(GuyonOne
Oneapproach
approachtotoassessing
assessingfeaturefeatureimportanceimportanceisfeaturefeatureselese
One approachtoassessingfeatureimportanceis featureisselection(G
insufficient
tofeatureimportance,and,atisselection
worst,
actively
made possible by the use of automatic differentiation, a andand
Elisseeff,andElisseeff,2003).
Feature2003).
Featureselection
selectionis aisbroad
is
aand
broad
well-studied
andwell-studied
topic,
topic,
Elisseeff,2003).
selectiona2003).
broad
andwell-studied
topic,
andand
Elisseeff,
Elisseeff,2003).
FeatureFeatureselection
selectionisis
aand
a broadbroad
andand wellwellt
andFeature
Elisseeff,2003).
Featureselectionisa broadwell-studied
One approachtoassessingfeatureimportanceisfeatureselection(Guyon37
misleading:featureselectiondoes
notaddressthesecond
and
aims
and
to
choose
aims
to
those
choosefeaturesthose
features
that
are
that
most
are
informative
most
informativeofthefunction.
ofthefunction.
and
aims
to
choosethose
features
that
are
most
informativeofthefunction.
feature of Google’s
Tensorflow
packageandaims
aimstotochoosechoosethose
thosethatfeaturesfeaturesthat
thatarearemostmostinformative
informative
ofoand aimsand
tochoosethose
featuresare most
informativeof the funcOne
to assessing
featureimportanceisfeatureselection(Guyonand Elisseeff,2003). Python
Feature selectionis (Abadi
a approach
broad etandwell-studied
topic,
One
approachfeatureimportanceisrankingfeatureselection(GuyonIn ourcontext,
In
our context,
it to
might
itbemight
thoughtbethought
thatthethatrankingtherankingof O*NETof O*NETfeaturesfeaturesIn
ourcontext,
itassessing
mightbethought
that
offeature
O*NETfeaturesOne
approachtoassessing
isO*NETfeature
InIn
our
our
context,
itbe
itmight
might
bethought
thought
that
the
theof
rankingranking
ofofsele
O*
O
In
context,
itcontext,
might
thoughtthattheimportancerankingfea
primaryThatis,thedetermining
that
athat
featureand2003). Feature
selectionis oura criteria.
broad
and
well-studied
topic,beandprovides
aims to choose
those features tothatareElisseeff,most informative
ofof
theourfunction.
al, 2015), which
the framework
GPflow.
37 37
andselected
Elisseeff,2003).
selection
is
a
broad
and
well-studied
topic,
selected
selectedthroughthroughsuchsuchaFeatureschemesuch
aisscheme
a
means
is
a
of
means
rankingofranking
theirimportance
theirimportancetotothroughascheme
is
a
means
of
ranking
theirimportancetoElisseeff,
selection
is a ofbroad
andwell-sselected
selectedthroughthroughsuch
suchFeature
aaisscheme
schemeisisaofameansmeans
oftheir
rankingranking
their
theiriselectedthroughsuch
atheschemea means
ranking
importanaims
choose of
thosefeaturesthat aremost andinformativeof 2003).
function.
In our context, it might be thoughtand
that
thetoranking
O*NETfeaturesandaims
tohighlychoosethose
features
that
are
most
ofsignthefunction.
isinformative
gives
no
sense
of
the
of
the
demand.
demand.
WesuggestWe
thatsuggesttheimportance
that
bulk
the
of methods
bulk
offeaturemethods
ofinformative
featureof
selection
featureselection
give,
at
give,
at
One approach to assessing
featureisfeaturefeatureselection(Guyondemand.
Wesuggest
that
the
bulk
ofis
methods
ofselection
featureselection
give,
at informative
andaims
choosethose
features
that
most
of
One
approachtoassessingimportanceis(GuyonOne
toimportanceassessingfeaturefeatureselection(Guyondemand.
demand.
Wesuggestsuggest
that
that
the
the
bulk
bulk
ofofaremethods
methods
ofoffeaturefeaturesele
sedemand.
Wesuggest
that
the
bulk
of
methodsoffeatureselectiongiv
our context,
itapproachmight
bethoughttheranking
oftoWeO*NETfeaturesOne
approachto assessing
featureimportanceis feature
selection(Guyonselected through such a scheme is a In
means
of ranking
theirimportancetothatOne
approachtoassessingfeatureimportanceisfeatureselection(Guyon5.3. ACTIVE LEARNING
In
ourcontext,
it
might
be
thoughtthattherankingofO*NETfeatures
best,
an
best,
insufficientaninsufficient
guide
to
guide
importance,toimportance,
and,atworst,
and,atactively
worst,
actively
misleading:misleading:and Elisseeff, 2003).
Featureselection
is
a
broad
and
well-studied
topic,
best,aninsufficient
guidetoimportance,
and,atworst,
actively
misleading:One
approachtoassessingfeatureimportanceisfeatureselection(GuyonIncontext,
itto
might
be
that
the atranking
O*N
andElisseeff,2003).
selectionisafeatureabroadbroad
andwell-studied
topic,
andFeatureselection
isour
andwell-studied
topic,
best,
best,
an
an
insufficient
guide
guide
totothought
importance,
importance,
and,
and,
atworst,worst,ofactiveactiv
best,
insufficient
guideimportance,
and,topic,
atfeature
worst,
actively
misleaselectedthroughsucha2003).
schemeisFeaturea Feature
means
ofan
ranking
theirimportancetoandElisseeff,2003).
selectionisainsufficient
broad
and
well-studied
relationshipbetweenanddemand.
Mostdemand. We suggest that the bulk of
methods
of Elisseeff,featureselection
give,
at
and
Elisseeff,2003).
Featureselection
is
a of
broad
and
well-studied
topic,
selectedthroughsuchascheme
is
a
means
of
ranking
theirimportancetofeatureselection
featureselection
does
not
does
addressnottheaddress
secondtheofsecondour
primary
ourprimary
desiderata.
desiderata.
That
That
and aims to chooseandthosefeatures
that
are
most
informativeofthefunction.
featureselectiondoes
notaddress
secondofourprimary
desiderata.
That
andElisseeff,2003).
Featureselection
is
a
broad
and
well-studied
topic,
selectedthroughsuch
anot
schemeisfunction.
aofmeansof
theirdesimaimsaims
tochoosechoosethose
features
that
are
most
informative
ofthe
the
function.
aims
to
those
features
that
are
most
informativeoffunction.
featurefeatureselection
selection
does
does
not
address
addressthethesecondsecondofofranking
ourourdesiderata.
primary
primarydefeatureselectiondoes
notaddressthesecondour
primary
demand.
suggest
that
the
bulk
of
methodsoffeatureselection
give,
at
and
to choosethosefeatures
that
are
most
informativeof
thebest, an insufficient guide to importance,
and, We
atand
worst,
actively
misleading:
and
aims
to
choosethose
features
that
are
most
informativeofthefunction.
demand.
Wesuggest
that
the
bulk
of
methods
ofinformative
featureselection
give,
atselection
adoptsan
information-theoretic
approachthatis,determiningis,determining
that
athought
featurethat
afeatureisfeaturehighlyisWe
informative
highly
gives
no
gives
sense
no
of
sense
the
ofoftheIn our
it to
might
be
thoughtthattheranking
O*NETfeaturesis,determining
afeaturefeatureishighlyinformative
gives
no
sense
of
the
and
aims
to
choosethose
features
that
are
most
informative
thefunction.
Onecontext,
Oneapproachapproachassessingtoassessingfeaturefeatureimportanceimportanceisis of
selection
selection(Guyon(GuyonAs described in
Section
4.3,does
ournotforesight
workshops
demand.
that
the
bulk
of
methods
featuresele
In
ourcontext,
might
bethoughtthattheranking
ofO*NETO*NETfeaturesIn
ouritbethatthattherankingoffeaturesis,is,determining
determining
that
that
aofaisfeaturefeatureisinformative
isof
highlyhighlyinformative
informative
gives
no
is,
determining
that
asuggest
featurehighlygives
no gives
sense
on
best,
anofinsufficient
guide
toitmight
importance,atworst,
actively
misleading:
In context,
ourcontext,
it might
beand,
thoughtthattheranking
O*NETfeaturesfeatureselection
address thesecondour
primary
desiderata.
That
In
our
context,
it
might
be
thoughtthattherankingofO*NETfeaturesbest,
aninsufficient
guidetoimportance,
and,atworst,
actively
misleading:
sign
of
the
sign
relationshipoftherelationshipbetweenbetweenfeatureandfeaturedemand.
anddemand.
Most
featureMost
selection
featureselection
selectedthroughsuchascheme
is
a
means
of
ranking
theirimportance
to
sign
of
the
relationshipbetweenfeatureanddemand.
Most
featureselectionIn
ourcontext,
it
might
be
thoughtthattherankingofO*NETfeaturesand and
Elisseeff,
Elisseeff,2003).
2003).
FeatureFeatureselection
selection
is
a
is
broad
a
broad
and
and
well-studied
well-studied
topic,
topic,
aninsufficient
guide
toimportanceimportance,
and,at worst,
activelselectedthroughsuchsuchanot
schememeansranking
theirimportanceto toselectedthroughsuch
aof
schemeis isofaofaourmeansofranking
theirimportancetodemand.
sign
sign
of
ofof
the
the
relationshiprelationshipbetweenbetweenfeaturefeatureandanddemand.
demand.
Most
Most
fea
fe
sign
the
relationshipbetweenfeatureand
Most
featuresele
featureselectiondoes
notaddresstheseconddesiderata.
That
selectedthrougha distinguishschemeisbest,aprimarymeansofranking
their
would
betweenathat
featureis, determiningthat a feature wereis highly
informative
gives
no
sense
the
required choosing
which occupations
tochoosebeselectedthroughsuch
a approachschemea meansofranking
theirimportancetofeatureselectiondoes
notaddressthesecondofourprimary
desiderata.
Thatofadoptsanadoptsinformation-theoretic
aninformation-theoretic
approach
that
would
not
would
distinguishnotdistinguishbetweenbetweendemand.
Wesuggest
that
the
bulk
of
methodsoffeatureselection
give,
atadoptsanthatinformation-theoretic
that
would
not
distinguishbetweenselectedthroughsuchaofscheme
is
aisof
means
of
ranking
theirimportancetoand aims
and aims
toto choose
those
those
featuresfeatures
that
are
most
are
most
informative
informative
ofapproach
theofanfunction.
thefunction.
selectiondoes
notaddressthesecondprimarydesi
demand.
Wesuggestthebulk
offeaturemethodsoffeatureselection
give,
atdemand.
Wesuggest
that
the
bulk
methods
featureselection
give,
atadoptsadoptsan
information-theoretic
information-theoretic
approachapproach
that
would
would
not
notdistin
disti
adoptsan
information-theoretic
approachthat
wouldnotourdistinguishbet
is, determining
that
a featurefeatureisthathighlyinformative
gives
no
of
the
demand.
Wesuggest
that
the
bulk
of methods
ofsensefeatureselection
give,
atthatsign of the relationship between featureand demand.
Most
selection
demand.
Wesuggest
that
the
bulk
of
methodsoffeatureselection
give,
at
is,
determining
thatafeatureishighlyinformative
gives
no
sense
of
the
a
featureax
feature
x
for
which
,
for
f
(x)
which

α−x
f
(x)
,

and
α−x
a
feature,andax
feature
,
for
which
x
,
for
f
(x)
which

β+x
f
(x)

β+x
best,
an
insufficient
guidetoimportance,
and,atworst,
actively
misleading:
,
for
which
,
and
a
featureforwhichafeaturex
demand.
Wesuggest
that
the
bulk
of
methodsoffeatureselection
give,
at
,
for
which
f
(x)

α−x
and
a
feature
x
,
for
which
f
(x)

β+x
In
our
In
context,
ourcontext,
it
might
it
might
be
thoughtbethought
that
that
the
rankingtheranking
of
O*NET
of
O*NET
featuresfeatures
1
1
1
1
2
2
2
2
1
1
2
2
presented to participants.
We introduced
theof
a best,is,thataactively
isa feature
highlyinformative
gives
noβ
best,
aninsufficientinsufficient
guide
and,
at
actively
best,
an
guide
totoimportance,
and,atactively
misleading:axa1featurefeaturexworst,x1feature,1worst,
,for
which
which
ffeature
f(x)
(x)

α−x
α−x
andax
a2featurexx2 ,2 ,for
which
whic
aimportance,,determining
for
which
ffor
(x)

α−x
and
,feature
for which
ffor
(x)

sign useof therelationshipbetweenfeatureanddemand.
Most
selection
an insufficientguide
tofeatureimportance,
and,atworst,
misleading:
adopts an information-theoretic
approach
that
would
not
distinguishbetween
1 ,1 ,and
1 ,misleading:
best,
an
guide
to
importance,atactively
misleading:
sign
ofand
betweenfeatureanddemand.
Most
featureselection
(α
and
βthe
(α
some
βinsufficientbeingparameters).
someparameters).
Both
x
Both
and
xand,
x2 1arehighlyxworst,areinformative
highlyinformativeofdemand.
of Most fea
featureselectiondoes
notaddressthesecondofourprimary
desiderata.
Thataninsufficient
guidetoimportance,atworst,
actively
misleading:(αbest,βandbeingparameters).
Bothxparameters).
and
xand
are
informativeof
selected
selectedthroughthroughsuchsuch
a scheme
aselection
schemeisabeingmeans
isrelationship
a not
means
ofsomerankingrankingtheir
their
importanceimportancetoto1relationship2highly
1and,2desiderata.
sign
of
the
betweenfeatureandfeaturedoes
addressthesecondour
primary
featureselectiondoes
notaddressthesecondofof
ourprimary
desiderata.
That
(α
(α
and
and
β
βsome
being
being
some
some
parameters).
parameters).
Both
Both
and
and
xx2 2arearehighlyhighly
(α
and
βthe
being
some
Both
xThat
and
x2xx1arehighlyinformati
adoptsanTheinformation-theoretic
approach
that
would
not
distinguishbetweenselectiondoes
notsecondofourprimary
desiderata.
Thata feature
x1 , forf (x)  α−xanda featurexfeature
which
f (x)

β+x
(ofdoesbeing
parameters).
1and
1Both
1 ,choice.
2 , for2 address
machine learningmodel
towhichautomate
thisfeatureselectionnotaddressthesecondofatnot
ourprimary
desiderata.
That
adoptsan
information-theoretic
approachthat
would
distinguishbetweendemand.
demand.
However,
However,askillspolicyaskillsthatpolicyresultinresultbroad
inincreases
broad
increases
in
x
in
x
would
would
is,
determining
thatafeatureishighlyinformative
gives
no
sense
of
the
featureselectiondoes
notaddressthesecondofourprimary
desiderata.
That
demand.
However,askillspolicythat
resultinbroadincreases
in
x
would
demand.
demand.
We suggest
Wesuggest
that
that
the
bulk
the
bulk
of
methodsofmethodsoffeatureoffeatureselection
selection
give,
give,
at
1
1
1
information-theoreticthat inwould
notincrease
distingthatishighlyhighly
gives
no
sense
ofthe
the
is,1is,determining
thata afeature
informative
nosenseofapproachdemand.
demand.
However,
However,
agivesaskills
skillspolicythat
that
resultresultinbroadbroad
increasdemand.
However,a skills
policythat
resultbroad
increasesin
x1 w
a feature
x
,xfor
which
f (x)

α−x
, aandaisfeature
x
foran
which
fgives
(x)

β+x
is,
determiningthatis adopts
highlyinformative
nosense
ofinthe(α and β being some parameters). Bothx1 andarehighlyinformative
offeature
1feature
2 ,informative
2 policy
2 determiningis,thataoccupations;feature
is
gives
no
sense
of that,theathefeaturexdetermining
, to
for
which
fworst,
(x)

α−x
and
ahighly
featurexconverse.
,2f,feature
for
fNote
(x)

β+x
lead
to
harmfulleadharmfuloutcomes
outcomesforoccupations;x2featurethethewhichconverse.
Note
Note
that,
also
sign
of
relationshipbetweenfeatureanddemand.
featureselectionis,determining
that
aMost
featurehighlynoofarehighlyinformativeofdemand.
However,
a1xalsopolicyleadtooutcomesforoccupations;xwhich,informative
theconverse.
also
that,
best,
best,
anthe
insufficientaninsufficient
guide
guide
to
importance,
importance,
and,
and,
at
atfor
worst,
actively
actively
misleading:misleading:
1harmful
1 , is2 , informative
2outcomes
machine learningmodel
useda wasreducedformofaxleadx,x
for(x)
givesα−x, skills
a featurex2converse.
,converse.
for whichsignofrelationshipbetweenanddemand.
Most
selection
sign
of
the
betweenanddemand.
Most
selection
lead
to
to
harmfulharmfuloutcomesforfor
occupations;occupations;x2x2 ,2the
,the
the
lead
to
harmfuloutcomesforoccupations;,sensetheconverse.
Note
alsoN
1demand.
(αresult
and
βin
being
some
parameters).
Bothxfeatureandarehighlyinformativeofsignofrelationshiptherelationshipbetweenfeatureand
Most
featureselection
demand.
However,skills apolicy
that
broad
increases
in
would
2and
1 feature2featuresign
ofx1therelationshipbetweenfeatureanddemand.
Most
featureselection
(α
and
β
being
some
parameters).
Both
x
and
x
are
highlyinformative
of
for
our
complex,for
ourcomplex,
non-linear,
non-linear,functionfunctionf,relationships
f,relationships
with
x
with
x
are
unlikelyareunlikelytotoadoptsan
information-theoretic
approach
that
would
not
distinguishbetweensignofrelationshipbetweenfeatureanddemand.
Most
featureselectionforourcomplex,
non-linear,functionf,relationships
with
x
are
unlikely
tofeaturefeatureselection
selection
does
does
not
addressnotaddressthesecondthesecondof
ourofprimary
ourprimary
desiderata.
desiderata.
That
That
1
2
i
i
i
(α
and
βthat
being
some
parameters).
Bothx2 xarehighlyadoptsapproach
that
would
not
distinguishanan
information-theoretic
approach
that
would
not
between
for
for
our
our
complex,complex,
non-linear,
non-linear,function,harmful,xrelationships
relationships
with
with
xxi iiafor
ourcomplex,
non-linear,
ffunction,between
relationships
withunlike1 anddemand.
However,a information-theoretic
skillspolicythat
resultinapproach
broadincreases
indistinguishx
would
adoptsan Noteinformation-theoretic
would
not
distinguishbetweenlead to harmful
outcomes
occupations;x2 ,adopts
theconverse.
also
that,
that
resultinbroadincreases
in
would
lead
toffwould
i are
1function
the model describedin Section
5.2forensure‘real-time’
adoptsinformation-theoretic
approachthat
distinguish
However,
ainformativeskillspolicythat
resultbroad
increasesin
xfeature,beasassimpledescribed
asdescribed
above
for
above
xof
for
xthe
and
xawould. would
andxnot
Anotherfeature,feature,x3 ,between
x3 , increaseato
x1 , for
which
f (x)

α−x
, whichandafan
feature
xα−x
, for
which
fapproach(x)aninformation-theoretic
that
betweenbeassimpleasthat
described
above
for
xas
and
x
Another
xand
is,feature
determining
is, determining
that
afeaturefeatureademand.
issimplehighlyisashighlyinformative
gives
gives
no
sense
no
sense
the
of
1 result
1β+x
21skills
2 .distinguish
1be
2that
2 .(x)
3 ,anddemand.
However,policy
that
broad
athat
fthat
(x)

,simpleandaasfeature
,in
for
which
fnot
(x)

β+x
a featurexfeature, 1adopts
for
(x)
α−x
aconverse.
featurex,describedwhich
fAnotherβ+xas
simpleasdescribed
described
above
above
for
xx2 .2 . Anothe
Anoth
be
as
that
above
for
x12β+xand.1infeatureto harmful
outcomes
occupations;
xas
,and
the
Note
also
that,
a feature
xare,forforunlikely
which
fto
(x)

,be
and
asimplefeaturex1 22that
,that
for
which
f (x)

2for
2 for
for our complex, non-linear, functionlead
f , relationships
withx, ifor1x
1α−x
2x
1 Another2x2, 1
1which
1be
2 xx
17
a
feature
x
,
for
which
f
(x)

α−x
,
and
a
feature
x
,
for
which
f
(x)

β+x
lead
to
harmfuloutcomesforoccupations;,theconverse.
Note
also
that,
may
give
may
f
(x)
give

cos(x
f
(x)

cos(x
):
while,):
again,
while,
xagain,
ishighlyx
isinformative
highlyinformativeofdemand,ofdemand,
(α
and
β
being
some
parameters).
Both
x
and
x
are
highlyinformative
of
a
x
1
1
2
2
may
give
f
(x)

cos(x
,
for
which
f
(x)

α−x
,
and
a
feature
x
,
for
which
f
(x)

β+x
):
while,
again,xis
highlyinformativeofdemand,
sign
sign
of
the
of
relationshiptherelationshipbetweenbetweenfeaturefeatureanddemand.
anddemand.
Most
Most
featurefeatureselection
selection
outcomesforoccupations;theconverse.
Note
2
3
3
3
3
1
2
1
1
2
2
3
3
given
that above
ourforgoal
to(αandperformance. be In
leadoutcomesforoccupations;, highlytheconverse.
N
and
β
being
some
parameters).
Both
and
are
highlyinformative
(α
being
some
parameters).
Both
x(x)
and
x
informative
ofof
may
give
give
fx
f(x)


cos(x
cos(x
):
while,
while,
again,
xx3 3xisinformative
highlyinformativ
informat
give
fmay

cos(x
while,
again,
x3again,ishighlyof dem2is
ourxisnon-linear,function
relationships
areunlikelyto(αandβ beingsomeparameters).
Bothxharmful
and
x3 ):
are
highlyinformative
of
1with
2(x)
as particular,
simple as that
described
forandx
Anotherfeature,xf3 ,, may
1xto
2x
3 ):
iare
2highly
1complex,
2 .β
and
βthat
being
parameters).
Bothxoccupations
and
x3are
are
highlyinformativeof
ourcomplex,
non-linear,function
fcomplex,,1Bothwithxhighlyareunlikely
tox3ofitfor
isit
unclearit
isaβresult
whether
unclearwhether
itsomeisitthat
important:itisimportant:forsomeoccupations(valuesxof
demand.
ademand.
skillspolicy
that
in
broad
increasesinxrelationships
would
1 someis(αunclearwhetherisimportant:forsomeoccupations(values
(α
and
being
some
parameters).
xwhetherand
xit
informative),ofxoccupations
adoptsadoptsanHowever,information-theoretic
an information-theoretic
approachapproach
that
would
not
distinguishnotdistinguishbetweenbetweeni(values
3 ),3 ), with x a
1broad
1for
2it
for
our
fof
relationships
However,skillspolicythat
resultinincreasesinwoulddemand.
However,skillspolicyresultinbroadincreases
xfunction
would
itit
unclearunclearis2function
important:
for
some
some
occupations
it wouldis
itbroadisnon-linear,
important:foroccupations(valuesi of(betoassimpleasthat
described
above
for
x1unclearandxis.whetherAnotherfeature,xin, important:demand.
However,a skills
policythat
resultin whether
increases
x1somewould
1
may andgive fthat(x) ourcos(xagain,
x3provideis highly
informativeof ademand,
1xin
also
that,
for
our
complex,
non-linear,,, for2is
3is3 ): while,
predict demand,
model
is to
demand.
However,
asignificanceskillspolicythat
resultinbroadincreasesinxwouldwouldbeas
simpleasthat
described
above
for
x
and
x
.
Anotherfeature,
x
x
x
will
have
will
very
have
different
very
different
significance
from
that
from
for
that
other
for
occupations.
otheroccupations.
To
To
lead
harmfulforoccupations;x,theconverse.
Note
also
that,
1, To
demand.
However,askillspolicythat
resultinbroadincreases
in
x
x
will
have
very
different
significance
from
that
for
other
occupations.
a feature
aablefeaturex1 , forx1which,outcomes
for
which
f
(x)
f

(x)
α−x

α−x
,
and
,
a
and
featurea
feature
x
,
for
x
which
,
for
which
f
(x)
f

(x)
β+x

β+x
1
2
3
3
3
2
1
3
1
1
2
2
2
2
be
simpleasverythat
described
above
for
x1 otherandxoccupations
Anotheharmfuloutcomesforoccupations;,have
converse.
Note
also
that,
lead
to
outcomesx, 2the
Note
also
that,
xx3 3as
will
will
have
very
different
different
significancesignificance
from
from
that
thatfor
for
other
oc
o
xfor
have
very
different
significance
from
that
for
2 . otherflead(x)toharmfulcos(xwhile,
again,
isoccupations;highlyinformativeofdemand,lead
to harmful
outcomes
xthe,converse.
theconverse.
Note
also
that,
it is unclear whether it is important:may
forgivesomeoccupationsofxfor),occupations;
2x
3 will3 ):(values
leadtooccupations;xthe,informative
theconverse.
Note
also
that,
may
give
ffnon-linear,
(x)

cos(x
):
again,
xf23Determination
is highlyofdemand,be
explicit,beexplicit,thethe
Automatic
Relevance
Relevance
Determination
Determination
approachapproach(Rasmussen(Rasmussenfor
our
non-linear,function,harmful
relationships
with
x
are
unlikelyto2oftooutcomes
occupations;xbeleadexplicit,
theRelevance
approach,toconverse.
Notealso(α and(αincomplex,
βand
being
βoccupations,
being
some
some
parameters).
parameters).
Both
Both
xharmfulandx3Automatic
xandarexwhile,arehighlyinformative
informative
of
with
are
unlikelybeassimpleasifor
2
1Automatic
1function
23outcomes
2 highlyestimates of theuncertainty
of demand
all
give
(x)

cos(x
while,
again,xis that,highlyinformativ
ourcomplex,
complex,functionfoccupations,berelationships
with
xare
unlikelytoforforour
non-linear,
f may
,for
relationshipswithxto(Rasmussenbeexplicit,thethe
Automatic
Automatic
Relevance
Relevance
Determination
Determination
approacapproabe
explicit,
theAutomatic
Relevance
Determination
approach(Rasmu3i ):it isunclearwhetheritoccupations.
isrelationships
important:forsome(values
xare
for
our
complex,
non-linear,function
fexplicit,, relationships
with
x3 ),
are
unlikely
to3 thatx3 will
have very different
significance
from
that
for
other
To
iof
iunlikelyforourcomplex,
non-linear,functionfrelationships,feature
relationships
with
xare
unlikelytoitisunclearwhetheritisimportant:forsomeoccupations(valuesofx),andWilliams
andWilliams(2006))
often(2006))
used
often
for
used
embeddedforembedded
selection
featureselectionfor
Gaussianfor
Gaussianbeasdemand.
simpleasthat
described
above
for
x
and
x
.
Anotherfeature,
x
,
i are
and
Williams(2006))
often
used
for
embeddedfeatureselectionfor
Gaussianfor
ourcomplex,
non-linear,functionf,withxunlikelytodemand.
However,
However,askillsaskillspolicypolicythat
that
resultresult
in
broad
in
broad
increases
increases
in
x
in
x
would
would
3
1
2
3
i
1
1
it
isxunclear
itoftenisTo
important:foroccupations(v
assimplethat
described
above
x
and
x
..Another
Anotherfeature,,feature
bebeasasasdescribed
that
above
for
xand
.(2006))
feature,x,embedded
andandWilliams
Williams
often
used
used
for
for
featurefeatureselection
selectio
andWilliams(2006))
often
forembedded
selectionforGau
x3Determination
will havevery
different
significance
from
that
for
other
besimpleassimpleas describedthat
describedabove
x1 occupations.
xAnotherAnotherfeature,x3 , some1and2(2006))
3embedded
be explicit,
the Automatic
Relevance
approach(Rasmussen
1for
2whether
3x
above
for
and
feature,may
2 .usedwe took the natural
optionof uncertainty
sampling.
beassimpleasdescribed
above
for
xfor
xdescription. description
Anotherfeature,x3 processes
will
have
very
different
significance
from
that
other
To
processes
processes
inappropriate
for
our
ends.
for
our
It
ends.
providesprovidesonly
a description
of
the
the
may
give
f (x)
outcomes
cos(x):givewhile,
again,xis
highlyinformativeofdemand,
1 and
2only
3,
is
for
our
ends.
It
providesaoccupations.
the
be
as
simplethat
described
above
for
xIt
and
xa2ends.
Anotherfeature,xof3a,xfor
lead leadto harmfultoharmfuloutcomesforfor
occupations;
xis
x2processes, inappropriate
the
,that
converse.
theconverse.
Note
Note
also
also
that,
that,
3give
3inappropriate
1different
2as
xprocesses
will
have
very
from
that
other
occ
may
foccupations;
(x)
is
cos(x
again,
isinformativeofdemand,
may
fgive
(x)
cos(x
again,
x
is approach
informativeof.significancedemand,processes
inappropriate
inappropriate
for
for
our
our
ends.
ends.
ItItofprovides
providesonly
only
aadescdesinappropriate
forouronly
It demand,
providesonly
descriptiono
3is
be explicit,
theAutomatic
Relevance
Determination(Rasmussen
may
f(x)
Gaussiancos(x):
while,
again,
xhighly
isisishighlyinformativeof
3and Williams (2006)) often used for embeddedfeatureselectionfor3 ):3 ):while,
3x3while,3highly
may
give
f
(x)

cos(x
):
while,
again,xis
highlyinformativeofdemand,be
explicit,
theAutomatic
Relevance
Determination
approach(Rasmussen
informativenessinformativenessoffeatures,offeatures,ratherthanrathertheirthanimportance.
their
importance.
itisunclearwhetheritisimportant:forsomeoccupations(valuesofx),
3
3
informativenessoffeatures,ratherthan
theirimportance.
may
give
f
(x)

cos(x
):
while,
again,xis
highlyinformativeofdemand,for
ourforcomplex,ourcomplex,
non-linear,
non-linear,functionfunctionf,relationships
f,relationships
with
with
x
x
are
unlikelyareunlikely
to
to
give
:
while,
again,
ishighlyinformative
3
3
3
i
i
Uncertainty sampling
the setfor
ofour
all ends.
occupations
beexplicit,
theoccupations
Automatic
Relevance
Determination
approach
unclearwhetheritforsome
occupations(values
ofx3x),),
it itis is(2006))
unclearwhetheritfor
is isimportant:forsomeoccupations(valuesoftheir
informativenessinformativeness
ofofGaussianfeatures,features,ratherratherthan
their
theirimportance.
importance.
ofsomefeatures,ratherthan
importance.
andItWilliams
often
used
embeddedfeatureselectionforitonlyis unclear
whether
itimportant:isinformativeness
important:for(values
of
x3 ),
3than
processes isranks
inappropriate
providesa descriptionofthe
it unclear
isunclearwhether
it2managing
isto
important:forsomeoccupations(values
of
x3 ),
and
Williams(2006))
often
used
for
embeddedfeatureselectionfor
GaussianA
secondA
approachsecondapproachtomanaging
managingfeaturesfeaturesisdimensionality
isdimensionality
reduction,
reduction,
xbe
have
very
different
significance
from
that
for
other
occupations.
ToAsecondapproachtofeaturesisdimensionality
reduction,itiswhetheritisimportant:forsomeoccupations(valuesofx),asbesimple
as simple
as thatasthat
described
described
above
above
for
x
for
x
and
and
x
.
Anotherx.
Anotherfeature,feature,x,
x,
3 will
3
1
1
2
3
3
and
Williams(2006))
often
forToembeddedfeatureselectionwill
havehavevery
different
significance
from
that
for
other
occupations.
Toforxis3xinappropriate
very
different
significance
from
that
for
other
occupations.
AA
secondsecondapproachapproachtotousedmanaging
managingfeaturesfeaturesisis dimensional
dimensiona
A unclearsecondapproachtomanagingfeaturesisdimensionality
reduc
processesforourdifferent
ends.
Ititsignificanceprovidesonly
a descriptionofthex3 have
will
very
from
that
for
other
occupations.
To
3will
informativenessof features,
rather thantheir
importance.
ofdemand,iswhetheritisimportant:from high- to lowuncertainty,
and bechooseshighestxwill
will
very
different
significance
from
that
other
occupations.
Toprocesses
isagain,
inappropriate
for
our
ends.
It
providesonly
afor
description
of
the
would
which
involvewould
projecting
involveprojecting
the
data
the
into
data
ademand,lower-dimensional
aforlower-dimensional
space.
To
space.
To
explicit,
thef(x)Automatic
Relevance
Determination
approach(Rasmussen
3 would
projecting
the
into
ainto
lower-dimensional
xwhile,have
different
significance
from
that
other
occupations.
may
maygive give
f the
(x)
cos(x
 explicit,
cos(x): whichwhile,
again,
xhave
isveryxhighlyishighlyinformative
informative
ofdatademand,of3the3explicit,
3 ):which
3 involve
3Relevanceprocesses
isinappropriate
for
our
ends.
Itintoprovidesonly a space
desc
Automatic
Relevance
Determination
approach(Rasmussen
theAutomatic
Determination
approach(Rasmussen
which
which
would
would
involveinvolveprojecting
projecting
the
the
data
data
intoToaToalower-dimensio
lower-dimensi
which
would
involveprojecting
the
data
into
aspace.
lower-dimensionalfeatures,ratherthan
theirimportance.
beofexplicit,
theAutomatic
Relevance
Determination
approach(Rasmussen
A second approach to managinginformativenessfeaturesbebe
is dimensionality
reduction,beexplicit,
theAutomatic
Relevance
Determination
approach(Rasmussen
informativenessoffeatures,ratherthan
theirimportance.
give
anexample,give
anexample,dimensionality
dimensionality
reduction
reductioncanbeachievedcanbeachievedbytheubiquitousbytheubiquitousandWilliams(2006))
often
used
for
embeddedfeatureselectionfor
Gaussiangive
anexample,dimensionality
reductioncanbeachievedbytheubiquitousbeexplicit,
theAutomatic
Relevance
Determination
approach(Rasmussenit
isit
unclearisunclearwhetherwhetheritisit
important:isimportant:
for
some
for
some
occupations
occupations(values(values
of
x
of
x
),
),
some
occupations(values
of
will
have
very
3
3
ranked for labellingby participants.
Thethemotivation
forWilliams
ofselection
features,ratherthan
importance.
andWilliams(2006))
often
used
for
embeddedfeatureGaussian(2006))
often
used
for
embedded
selection
forforGaussian
give
give
an
anfeature
example,example,dimensionality
dimensionality
reduction
can
can
be
beachievedachieved
by
byt
give
an
example,dimensionality
reductioncanbetheir
achievedbythe ubiqusecondapproachtomanagingfeaturesisinformativenessdimensionality
reduction,
andWilliams(2006))
often
used
for
embeddedfeatureselection
for reduction
Gaussian
which would
involve projecting
dataAinto
aand
lower-dimensional
space.
To
andWilliams(2006))
often
used
for
embeddedfeatureselectionfor
GaussianA
secondapproachtomanagingfeaturesisdimensionality
reduction,techniquetechniqueofPrincipalofPrincipal
Component
Component
AnalysisAnalysis(Pearson,(Pearson,1901).
Dimensionality
1901).
Dimensionality
processes
is inappropriate
for
our
ends.
It
providesonlyadescriptionofthetechniqueofPrincipal
Component
Analysis(Pearson,1901).
Dimensionality
andWilliams(2006))
often
used
for
embeddedfeatureselectionfor
Gaussianx3 willx3 havewill
havevery very
different
different
significancesignificance
from
from
that
that
for
other
for
other
occupations.
occupations.
To
To
A
secondapproachtomanaging
is dimensionali
for
our
ends.
providesonly
adescription
description
theprocesses
is isinappropriate
our
ends.
Ittechniqueprovidesaonly
ofofthetechniqueofofonlyPrincipal
Principal
Component
AnalysisAnalysis(Pearson,1901).
1901).D
techniqueofIt
Principal
Component
Analysis(Pearson,1901).
Dimension
which
would
involveprojecting
the fordata
intoa lower-dimensional
space.
To
processes
isthe
inappropriate
for
our
ends.
It provides
aComponent
descriptionof featuresthe (Pearson,
an example,
dimensionality
reduction
canbeprocesses
achieved
byinappropriate
ubiquitous
different
significance
from
that
for
other
occupations.
the approach giveis theexpectation
theselabelsshouldbeprocesses
isoninappropriate
for
our
ends.
It
providesonly
a description
of
the
which
would
involveprojecting
the
data
into
a
lower-dimensional
space.
To
reduction
reductiononxcancertainlyxcancertainly
be
used
to
be
discoverusedtodiscoverthatcertainthat
featurescertainfeaturesco-vary.
co-vary.
informativenessfeatures,ratherthan
theirimportance.
reductiononxcancertainly
be
used
to
discoverthatcertainfeaturesco-vary.
processes
isinappropriate
for
our
ends.
It
providesonlyadescriptionofthebe explicit,
be explicit,
the ofAutomatic
theAutomatic
Relevance
Relevance
Determination
Determination
approachapproach(Rasmussen(Rasmussen
involveprojecting
thedata
intocertain
a lower-dimension
informativenessfeatures,ratherthan
importance.
informativeness
ofoffeatures,
ratherthan
theirimportance.
reduction
reduction
on
on
xxcan
can
certainly
certainly
be
used
used
totothat
discoverdiscoverthat
that
certaincertaincofea
fe
reduction
ontheir
x would
cancertainlybeused tobe
discoverfeaturesgive an (Pearson,
example,dimensionalitycanbewhichachievedbytheubiquitous
informativeness
ofreduction
features,ratherthan
theirimportance.
technique of Principal Component Analysis
1901). Dimensionality
informativenessoffeatures,ratherthan
theirimportance.
give
anexample,dimensionality
reductioncanbeachieved
by
the
This,
of
course,This,ofisexplicit,
course,nottheisfeature
same
not
the
as
same
discovering
asdiscovering
that
the
that
tworeduction,arethesimilarly
twoubiquitous
are similarly
im- imimsecond
approachmanagingfeaturesismanaging
dimensionality
reduction,This,ofcourse,nottheasdiscovering
that
the
two
are
similarly
informativenessoffeatures,rathertheir
importance.
andAWilliams
andWilliams(2006))(2006))
often
often
used
used
for
embeddedforembeddedfeatureselection
selectionfor
Gaussianfor
GaussianTo
bethe
Automatic
Relevance
Determination
most informativeabout
overall:theirobservation
give
anexample,reductionachieved
byare
th
Ato
secondapproachtoismanaging
featuresisdimensionality
reduction,A
secondapproach
toAnalysis
featuresisdimensionality
This,
This,
of
of
course,course,isdimensionality
isnotnotthethediscovering
same
same
asasdiscovering
discovering
that
that
the
thesimilarl
two
two
ar
ofsamecourse,isthan
not
the
same
as
thatcanthe betwo
aretechniqueofPrincipal
Component(Pearson,1901).
Dimensionality
A second
approachtoThis,managingfeaturesisdimensionality
reduction,
reduction
on xdemandcan certainly
be usedtodiscoverthatcertainfeaturesco-vary.
Asecondapproachtomanagingfeaturesisdimensionality
reduction,techniqueofPrincipal
Component
Analysis(Pearson,1901).
Dimensionality
portant
portant
todemand.
todemand.
MoresophisticatedMoresophisticatedusesofdimensionality
usesofdimensionality
reduction,
reduction,
which
would
involveprojecting
the
data
into
a
lower-dimensional
space.
To
portant
todemand.
Moresophisticatedusesofdimensionality
reduction,A
secondapproachtomanagingfeaturesisdimensionality
reduction,
processes
processes
is inappropriate
is inappropriate
for
our
for
ends.
our
ends.
It
providesIt
provides
only
only
a
descriptionadescription
of
the
of
the
techniquePrincipal
Component
Analysis(Pearson,1901).
D
which
would
involveprojecting
thedatadata
intoalower-dimensional
lower-dimensional
space.
To ofwould
involve
the
into
ainto
space.
To
portant
portanttoof
todemand.
demand.
MoreMoresophisticatedsophisticatedusesusesforofofdimensional
dimensiona
portant
todemand.
Moresophisticatedusesdimensionality
reduc
reductionwhichonx whichcancertainly
beprojectingusedto (Rasmussen
discoverthatcertainfeaturesco-vary.
would
involveprojectingthedata
aWilliams
lower-dimensional
space.
Toof course,
is not theasuncertainty.
discovering
that
the
two
are
similarly
imapproachand(2006))
often
usedshould lead toThis,a largereduction
insame
total
The
which
would
involveprojecting
the
data
into
a discover
lower-dimensional
space.
To
reductiononxcancertainly
be
used
to
discoverthatcertainfeaturesco-vary.
that
includethat
theincludevaluestheofvaluesf(x)
itself,
of
f
(x)
can
itself,
be
used
can
be
to
used
to
discoverrelationships
relationships
give
anexample,ofdimensionality
reductioncanbeachievedbytheubiquitousthat
includethevaluesoff(x)
itself,
can
be
used
to
discoverrelationships
which
would
involveprojecting
the
data
into
a
lower-dimensional
space.
To
informativenessinformativenessfeatures,offeatures,ratherrather
than
than
their
their
importance.
importance.
reduction
on
xthe
can
certainlybeuseddiscoverthatcertainfea
give
anexample,dimensionality
reductioncanbeachievedbythetheubiquitousgive
andimensionality
reductioncanbeachieved
by
that
that
includeincludetheofofubiquitous
fthe
f(x)
(x)
itself,itself,
can
canto
be
bediscoverusedusedtotorelation
discoverdiscovethat
includethevaluesof values
fvalues(x)
itself,canbetousedThis, usesof course,
isexample,
notthe same
as discovering
that
the
two
are
similarly
imgivean example,
dimensionality
reductioncanbeachievedbyubiquitous
portant to demand. More sophisticatedofdimensionality
reduction,
give
example,reductioncanbeachievedbyubiquitousThis,ofcourse,isfeatures
notfeatures
theasdiscovering
that
the
two
areas
similarly
imbetween
O*NET
O*NET
featuresfeaturesanddemand.
anddemand.
However,
However,dimensionality
dimensionality
reduction,
reduction,techniquePrincipal
Component
Analysis(Pearson,1901).
Dimensionality
betweenO*NETanddemand.
However,dimensionality
reduction,
give
an an
example,reductioncanbeachievedbythetheubiquitous
A second
A ofsecond
approachapproachtobetween
managing
tomanagingfeaturesisdimensionality
dimensionality
issamedimensionality
reduction,
reduction,
featureselectionfor
Gaussianisthat dimensional
active learningthat
approach
is values
one thataimstointeractivelyThis,ofcourse,
is
not
the
same
discovering
thetworeducaretechniqueofPrincipal
Component
Analysis(Pearson,1901).
Dimensionality
techniqueofPrincipal
Component
Analysis(Pearson,1901).
Dimensionality
betweenbetween
O*NET
O*NETfeaturesfeaturesandandprocesses
demand.
demand.
However,
However,dimensiona
betweenO*NETfeaturesanddemand.
However,dimensionality
portant
demand.
Moresophisticatedusesofdimensionality
reduction,techniqueofembedded
Principal
Component
Analysis(Pearson,1901).
Dimensionality
include theof f (x)itself,
can
beto
used
to discover
relationships
techniqueofPrincipal
Component
Analysis(Pearson,1901).
Dimensionality
portant
todemand.
Moresophisticatedusesofdimensionality
reduction,inconsideringinconsidering
combinations
combinationsoffeatures,offeatures,
will
fail
to
will
satisfyfailtoour
satisfy
first
ourdesiderafirstdesiderareduction
onwould
xinvolve
caninvolvecertainly
be
used
to
discoverthatcertainfeaturesco-vary.
inconsidering
combinationsoffeatures,
will
fail
to
satisfy
our
firstdesideratechniqueof
Principal
Component
Analysis(Pearson,1901).
Dimensionality
which
which
would
projecting
projecting
the
data
the
data
into
into
a
lower-dimensionalalower-dimensional
space.
space.
ToTo
portant
todemand.
Moresophisticateduses
oftoourdimensionali
reductioncancertainly
be
used
to
discoverthatcertainfeaturesreduction
ononx xof
can
be
used
to
discoverthat
featuresco-vary.
inin
considering
combinations
combinationsofofco-vary.
features,features,
will
will
fail
fail
to
satisfysatisfy
our
ourinconsidering
combinationsoffeatures,will
fail tosatisfy
first desi
thatHowever,includethevaluesfxcertainly(x)itself,canusedtoconsidering
discoverrelationships
reduction
oninappropriate
cancertainly
befor
used
to
discoverthatcertainfeaturesco-vary.
betweenfeaturesand demand.
dimensionality
reduction,
our
ends.
Itcertain
providesonly
afeatures
description
acquire data so
as toO*NETprovidethe greatest
confidence
insamereductionon
xmixingcancertainly
be
used
to
discoverthatcertainfeaturesco-vary.
that
includethevaluesoffachieved(x)thatitself,
can
be
used
to
discoverrelationships
tum.
That
tum.
is,
That
in
mixingis,infeaturesmixingfeaturestogether,
together,dimensionality
dimensionality
reduction
will
fail
will
This,
of example,
course,isdimensionality
not
the
as
discovering
that
the
two
are
similarly
imtum.
Thatis,canfeaturestogether,dimensionality
reductionwill
fail
reduction
on
xin
can
certainly
be
used
to
discoverthatcertainco-vary.
give angive
an example,
dimensionality
reduction
reductionbecanachieved
beby
byubiquitoustheubiquitous
includethevaluesoffreduction(x)
itself,canusedfail
to discoverThis,
course,
isnot
not
the
same
asdiscovering
discovering
that
the
two
are
similarly
imThis,ofof
course,
is
the
same
as
that
the
two
are
similarly
imtum.
That
That
is,
is,
in
in
mixingmixingfeaturesfeaturestogether,
together,dimensionality
dimensionality
redu
red
tum.
That
is,theinmixingfeaturestogether,dimensionality
reductionwi
betweenO*NETfeaturesanddemand.
However,dimensionality
reduction,
ofour
course,
is
not
the
same
as tum.
discovering
that
the
two
are
similarly
im- bein considering combinations of features,will failto This,satisfy
first
desideraThis,ofcourse,isdimensionality
not
the
same
as
discovering
that
the
two
are
similarly
imbetween
O*NETfeaturesanddemand.
However,dimensionality
reduction,touncovertouncover
clear
and
clear
interpretableandinterpretable
relationships
relationships
toincreasing
toincreasing
demand.
demand.
portant
todemand.
Moresophisticatedusesof
reduction,touncoverclear
andinterpretable
relationships
toincreasing
demand.
This,ofcourse,
is
not
the
same
as
discovering
that
the
two
are
similarly
imtechnique
techniqueofPrincipalofPrincipal
Component
Component
AnalysisAnalysis(Pearson,(Pearson,1901).
1901).
Dimensionality
Dimensionalityoftheinformativenessoffeatures,ratherthan
their
betweenO*NETfeaturesanddemand.
However,dimensionali
resulting predictions.
portant
demand.
Moresophisticatedusesandofdimensionality
dimensionality
reduction,
portant
totodemand.
Moresophisticatedusesof
reduction,
totosatisfy
uncover
uncover
clear
clear
and
and
interpretableinterpretable
relationships
relationships
totoincreasing
increasingtouncover
clear
interpretable
relationships
to increasing
demand. din considering
combinationsof features,willsophisticatedfail toour
firstdesideraportant
to demand.
More
uses
of
dimensionality
reduction,
tum. That is, in mixing features together,dimensionality
reductionwill
failportant
todemand.
Moresophisticatedusesofdimensionality
reduction,inconsidering
combinationsoffeatures,
will
fail
to
satisfy
our
firstdesideraWe
make
We
two
make
complementary
twocomplementary
proposals
proposalsforassessingforassessingfeatureimportance,featureimportance,that
includevaluesoff(x)
itself,
can
be
used
to
discoverrelationships
We
make
two
complementary
proposalsforassessingfeatureimportance,
portant
todemand.
Moresophisticatedusesofdimensionality
reduction,
reduction
reductionon the
x on
can
xthat
certainlycancertainly
be
used
be
used
to
discoverto
discoverthat
that
certaincertainfeaturesfeaturesco-vary.
co-vary.
inconsidering
combinationsoffeatures,
will
fail
to
satisfy
ourthevaluesfof(x)fitself,
itself,canbe
beusedusedtodiscoverrelationships
includethevaluesofoff (x)cantotwo
discoverrelationships
We
We
make
make
two
complementarycomplementary
proposals
proposalsforforassessing
assessingfeaturfeatufi
We itself,
maketwocomplementary
proposals
for assessing
featureimporttum. That thatis, increasing
ininclude
mixingfeaturestogether,dimensionality
reduction
will
fail
that
includethevalues
(x)
can
be
used
to
discoverrelationships
to uncover clear and interpretable relationships
todemand.
importance.
that
includethevaluesoff (x)itself,canbetodiscoverrelationships
tum.
That
is,
in
mixingfeaturestogether,dimensionality
reductionwill
failultimately
andultimately
present
resultspresent
from
each.
from
each.
betweenO*NETdemand.
However,dimensionality
reduction,
andultimately
present
results
from
each.
that
includethevaluesoffresults(x)
itself,canbeusedtoreduction,
discoverrelationships
This,This,of course,of course,
isfeatures
notis thenotand
same
the
same
as
discovering
asfeatures
discovering
that
that
the
two
the
are
two
similarlyaresimilarly
imim-used
tum.
That
is,
inresults
mixingfeaturestogether,
betweenO*NETanddemand.
However,dimensionality
reduction,
O*NETfeaturesanddemand.
However,dimensionality
andandultimately
ultimately
present
present
results
results
from
from
each.
each. dimensionality redu
andultimately
presentfromeach.
to uncover
clear
and
interpretable
relationships
toincreasing
demand.
betweenO*NETfeaturesanddemand.
However,dimensionality
reduction,
We make two complementary proposals
forbetween
assessingfeatureimportance,
betweenO*NETfeaturesanddemand.
However,dimensionality
reduction,
toof
uncoverclear
andinterpretable
relationships
to increasing
demand.
inconsideringfeatures,will
failoftosatisfy
our
firstdesiderabetween
O*NETfeaturesanddemand.
However,dimensionality
reduction,
portant
portant
to demand.
tocombinations
demand.
MoreMoresophisticatedsophisticated
uses
uses
of
dimensionalityof dimensionality
reduction,
reduction,touncoverclear
andinterpretable
relationships
to increasing dconsidering
combinations
offeatures,features,will
failtosatisfy
satisfy
our
firstdesideradesideraininconsidering
combinations
will
fail
tofail
our
first
two
complementary
proposals
forfeatures,assessingfeaturein considering
combinationsofwill
toimportance,
satisfy
our
firstdesideraand ultimately present results from each.We makeinconsidering
combinationsoffeatures,will
failsatisfy
our
firstdesideraWe
maketwodimensionality
complementaryforassessingfeatureimportance,tum.
Thatinclude
is,thein mixing
featurestogether,willinconsidering
combinations
ofproposals
features,
will
fail
to to
satisfy
our
firstdesiderathat that
includevaluesthevalues
of
f (x)
ofis,
fis,
itself,(x)
itself,
can
be
can
used
be
used
to
discovertoreduction
discoverrelationships
relationships
39
39fail
39
We
make
two
complementary
proposals
tum.
That
in
mixingfeaturestogether,dimensionality
reduction
will
fail
tum.
That
in
mixingfeaturestogether,dimensionality
reductionwill
failand ultimately
present
results
each.
tum.
That
is,
infrommixingfeaturestogether,dimensionality
reduction
will
fail for assessing feature
39
39
39
tum.
That
in
mixingfeaturestogether,dimensionality
reduction
will failand demand.
ultimately
present
results
from
each.
to
uncoverclear
andinterpretable
relationships
increasing
demand.
tum.
Thatis, is,in tomixingfeaturestogether,dimensionality
reduction
betweenbetween
O*NET
O*NETfeaturesfeaturesandanddemand.
However,
However,dimensionality
dimensionality
reduction,
reduction,
andultimately
results
fromeach. will failuncoverclear
andinterpretableinterpretable
relationships
increasing
demand.
toto
uncoverclear
andrelationships
totoincreasing
demand.
to uncover
clear
and
interpretable
relationships
topresent
increasing
demand.
39
touncoverclear
andinterpretable
relationships
increasing
demand.
We
twocomplementary
proposalsforassessingfeatureimportance,
uncoverclear
andinterpretable
relationships
to toincreasing
demand.
in consideringin makeconsidering
combinations
combinationsof tofeatures,offeatures,
will
fail
will
to
fail
satisfytosatisfy
our for
first
our
first
desideradesideraWemakemaketwocomplementary
proposalsforassessingfeatureimportance,We
twocomplementary
proposals
assessingfeatureimportance,
We
make
two
complementary
proposals
for assessing
featureimportance,
39
37
We
make
complementary
proposals
assessingfeatureimportance,andpresent
results
from
each.
We
make
twotwocomplementary
proposals
forforassessingfeatureimportance,
39
tum.ultimately
tum.
ThatThatis, inis,mixing
inand
mixingfeaturesfeaturestogether,
together,dimensionality
dimensionality
reduction
reduction
will failwill
failultimately
present
results
fromfrom
each.
39
and
ultimately
present
results
from
each.
and
ultimately
present
resultseach.
ultimately
present
resultsfromeach.
andandultimately
presentfromeach.
to uncover
to uncoverclearclear
and interpretable
and interpretable
relationships
relationships
toincreasing
toresults
increasing
demand.
demand.
We make
We maketwo complementary
two complementary
proposals
for assessing
for assessing
featurefeatureimportance,
importance,
39proposals
3939 39and ultimately
and ultimately
present
present
results
results
fromfrom
each.each.
39 39

a feature x1 , for which f (x)  α−x1 , and a feature x2 , for which f (x)  β+x2
feature selection does not address the second of our primary desiderata. That
(α and β being some parameters). Both x1 and x2 are highly informative ofis, determining that a feature is highly informative gives no sense of thedemand. However, a skills policy that result in broad increases in x1 would
sign of the relationship between feature and demand. Most feature selection
lead to harmful outcomes for occupations; x2 , the converse. Note also that,
adopts an information-theoretic approach that would not distinguish between
THE F UT URE OF SKI L L S: EMPL OYMEN for
T IN
30
our2 0complex,
non-linear, function f , relationships with xi are unlikely toa feature x1 , for which f (x)  α−x1 , and a feature x2 , for which f (x)  β+x2
be as simple as that described above for x1 and x2 . Another feature, x3 ,(α and β being some parameters). Both x1 and x2 are highly informative of
may give f (x)  cos(x3 ): while, again, x3 is highly informative of demand,
demand. However, a skills policy that result in broad increases in x1 would
it is unclear whether it is important: for some occupations (values of x3 ),lead to harmful outcomes for occupations; x2 , the converse. Note also that,x will have very different significance from that for other occupations. To
for our complex, non-linear, function f , relationships with xi are unlikely3 to
be explicit, the Automatic Relevance Determination approach (Rasmussen
5.4.2. AVERAGE DERIVATIVE AND FEATURE
A second
managingfeaturesis dimensionality
be as simple as that describedabove approachfor x1 and x2to
. Anotherfeature,x,
and3 Williams (2006)) often used for embedded feature selection for Gaussian
may give f (x)  cos(x3 ): while, again, x3 is highly informative of demand,
COMPLEMENTARITY
reduction,
which
would
involveprojecting
data
processesthe
is inappropriate
for our ends.
It provides only a description of the
it is unclear whether it is important: for some occupations (values of x3 ),informativeness of features, rather than their importance.
x3 will have very different into
significancefrom that for otherspace.
occupations.
To an example,
a lower-dimensional
To give
A second approach to managing
is dimensionality
reduction,
Ourfeatures
secondproposal givesa means of satisfying ourbe explicit, the Automatic Relevance Determination approach (Rasmussen
which would
involve 5.4.2.
projecting
the data
into a lower-dimensionalTo
dimensionality reduction can be achievedby theAveragederivativeand featurespace.
complementarity
and Williams (2006)) often used for embedded feature selection for Gaussian
secondary
criteria,
while
perhapsgive an example, dimensionality reduction can be achieved by the ubiquitous weakening the case forprocesses is inappropriate forour ends. Ittechniqueprovides only
descriptionComponentof theubiquitous
ofaPrincipal
AnalysisOursecondproposal
gives
a
means
of
satisfying
oursecondarydesiderata,
Averagederivativeand featurecomplementarity
technique of Principal Component 5.4.2.
Analysis(Pearson,1901). Dimensionality
the firstof theprimary
averagederivative, as
informativeness of features, rather than their importance.
while perhaps
for features
the criteria.
firstco-vary.
of theThe
primary
desiderata.
reductionon
on x cancertainly
be usedweakening
to discoverthe
thatcasecertain(Pearson,1901).
Dimensionality
reductioncanA second approach to managing features is dimensionality reduction,
Our
secondproposalgives
a means
ofal.satisfying
oursecondarydesiderata,
Theaveragederivative,asdescribed
in
Baehrenset(2010),
isforthenth featuredescribed
in
Baehrensetal.
(2010),
isfor
theThis, of course, is not the same as discovering that the two are similarly imwhich would involve projecting
the data
a lower-dimensional
space.
To features co-vary.simplywhile perhaps weakeningthe casefor the first of the primary desiderata.
certainly
beintoused
to discover thatcertain
portant to demand. feature
More sophisticateduses of dimensionality reduction,
simplygive an example, dimensionality reduction can be achieved by the ubiquitous
The average derivative, as describedin Baehrens et al. (2010), is for the nth∂m(x)that include thevaluesf (x) itself, can be usedto :=discoverrelationships
This, of course, is not the same as discoveringthatthe of
two
AG(n)
E
,
(8)
technique of Principal Component Analysis (Pearson, 1901). Dimensionality
feature simply∂xn reduction,

between O*NET features and demand. However, dimensionality∂m(x)reduction on x can certainlyare
be usedto discoverthat certain
features co-vary.
similarly
importantto demand.
Moresophisticated:= E in (6). ,(8)
in considering combinations
features, will fail to satisfy
ourAG(n)firstdefineddesidera(8)using theof employment-weighted
expectation
This, of course, is not the same as discovering that the two are similarly im∂xtum.
That is,thein mixing
featuresdimensionality
reduction
will failthe nexpected increase
By
way oftogether,interpretation,
the derivative
measuresof dimensionality
reduction,that
includevaluesofportant to demand. Moreuses
sophisticateduses of dimensionality
reduction,
using
the
employment-weighted
expectation
definedin(6).
to uncover clear and ininterpretable
toinincreasing
demand.
demand for relationships
a unit increasea particularfeature (for instance, as a resultthat include the values of f (x) itself,
used
to discover
relationships
itself,cancanbebeused
to discover
relationships
betweenBy wayinterpretation,measuresexpected increase
We make two complementary
proposals
forof
assessingfeatureimportance,of a policy intervention).
By
averagingovertheall derivative
occupations,
we getthea senseusing
theemployment-weighted
expectation
definedin (6).
between O*NET features and demand. However, dimensionality reduction,
indemandforaunit
increaseina particularfeatureinstance, as a result
and ultimately
present
results
from each.
of theaggregateincrease in demand as a resultof this increasein(fora feature.
O*NETfeaturesanddemand.
However,dimensionality
in considering combinations of features, will fail to satisfy our first desideraofapolicyintervention).
By
averagingoveralloccupations,weget
The average derivative
gives
an interpretable thenotionof sign: itmeasures
can clearly the a sense
By wayofinterpretation,
derivative
tum. That is, in mixing features
together,
will fail of features, will
reduction,in dimensionality
considering reduction
combinationsof theaggregateincreasein demandas demand.
a result of this increase in a feature.
distinguish positive
negative
relationships
with39 fromto uncover clear and interpretable relationships to increasing demand.
The
averagederivative
an tointerpretable
notionof sign:is it
expected
increasein demanda unitincreaseincan
a clearly
The first advantageof thismetric gives
relative
the formarginal
correlation
fail toproposals
satisfy our
first criterion.
That is, in mixing features
We make two complementary
for assessing
feature importance,
distinguishpositive fromwithfirst
demand.
that it is sensitive
to non-linearities
in negative
the data,relationships
addressing theof ourparticularfeature(for
instance,asaresultofapolicyand ultimately present results
from
each.
together, dimensionality reduction will fail to uncoverclear desiderata.
The first
advantageof this gives
metrica relative
to the marginal
secondary
While
the derivative
linear approximation
to correlation isthatis sensitive
tonon-linearities
in considering
theaddressing
theintervention).
averaging overalldata,occupations,wefirstgetof ourdemand, it is only
aitlocallylinearBy
approximation.
By
the approxand interpretable
relationships
toincreasing
demand.
39
desiderata. inWhile
the derivative gives a linear approximationtoimation at allsecondary
points (occupations)skills-/knowledge-/abilities-space,
we
a
sense
of
the
aggregateincreaseindemandasaresultdemand,it isrelationships
only a locallythat
linearapproximation.
Byatconsidering
are able to bettermeasurehave
different slopes
different the approxWe make two complementary proposals for assessing
all pointsin(occupations)skills-/knowledge-/abilities-space,ofthisatincreasea feature. inTheaverage derivative gives weregions of the imation
space.
feature importance, and ultimately present results
aretoable
to bettermeasure relationships
have different
slopes at different
This ability
managenon-linearity
also enables thatthe averagederivativeaninterpretable
notionofsign:
it
can clearly
distinguishof the ofspace.
to capture theregions
importancefeatures whose significance is conditional on
from each.
Thisability
managenon-linearity
enables
the averagederivative
positive
fromtonegative
relationships
withdemand.
the values of otherfeatures.
For
instance,
Fine Arts also
is veryimportanttoto important
capture thefeatureswhose
significanceis conditional on
Artists, but lessforimportanceoccupationsofwith
differingskillsprofiles. This
5.4.1. PEARSON CORRELATION
the
of
other
For
instance,
Fine
Artstoverymarginalimportant tois achieved throughreportingthefeatures.
derivative
averaged
oversubsets
ofistheoccuThevalues
firstadvantageof thismetricrelative
20 profiles. This
Artists, those
but less
importantforaoccupations
with differingskillspations: for instance,
thatfall within
major occupationalgrouping.
is averagethat
it derivative
is sensitive
non-linearities
in theWe first consider a direct means of achieving our primaryiscorrelationachieved
reportingthe derivative
averagedoversubsubsetsof occuFor a non-linear
function,throughthefor atofeaturemay be20
pations:forinstance,
those
that
fall
agrouping)major occupational
stantively different
one
region ofspace(occupational
thanfor grouping
data,overaddressing
the
first
of within
our secondary
criteria.
While .
two criteria, while ignoring the secondary criteria. ThisFor a non-linear function, the average derivative for a feature may be subanother.
the derivative
a linear
tobothdemand,metric of the importance of a feature, which we abbreviate
stantively
differentgives
overregionofapproximation
space
grouping)
than forFor each subset
of occupations,
we onewill
highlight
those(occupationalfeatures with
Pearsoncorrelation
5.4.1.5.4.1.
Pearsoncorrelation
large positive another.
andaveragederivatives.
Those withposit islarge
onlynegative
a locally
linearapproximation.
Bylargeconsidering
5.4.1. Pearson
correlation
as Pearson
correlation,is the employment-weighted
For sayeacharesubsetof occupations,
we willhighlight thosewith bothitive derivatives wecomplementary
to theoccupationalgroupfeatures(inWe consider
first consider
a directmeans
of achieving
our primary
two desiderata,
We first
a directmeans
of achieving
primary
twoofdesiderata,theapproximation
at all points
in withskills/We first
consider our
a 18
directmeans
our primary large
two desiderata,
betweenourachieving
model’s
Pearsoncorrelation
coefficient
positive and large negative
average(occupations)derivatives. Thoselarge pos5.4.1.
correlation
ignoring
the secondary
desiderata.
This
metricofimportancethe importanceofwhilewhile
ignoring
thePearson
secondarydesiderata.
This
metricoftheofwhile correlation
ignoring the secondary desiderata. This metricof theitive
importanceof we say are complementary to the occupational group (inderivatives
5.4.1.
Pearson20
knowledge/abilities/space,
we areabove
ableastoemploymentbetter measurea feature,which
weabbreviate
as Pearson
correlation,
the employmentpredictive
meanfordemandandtheisourfeature.
Morea feature,which
weconsider
abbreviate
as Pearson
correlation,
theisemploymentTo be precise,
this entails only redefining the expectations
We
firsta adirect
meansof achieving
primary
two desiderata,
feature,whichis the employment19 we abbreviate as Pearson correlation,
19
weightedsums over that subset of occupations, rather than over all occupations.
weighted
Pearsoncorrelation
coefficient
betweenourmodel’s
predictivemean
5.4.1.
Pearsoncorrelation
weighted
Pearsoncorrelation
coefficient
betweenourmodel’s
predictive
mean
We
first
consideradirectmeansofachievingourprimary
twodesiderata,
19
while
ignoringthesecondarydesiderata.
Thisthemetricof theimportanceofrelationships
weighted
Pearsoncorrelation
coefficient
betweenour model’s
predictive
mean that have different slopes at different regions
precisely,lettheposteriormean
forlatentdemand20
for demand
andthefeature.
More
precisely,
letposterior
the posteriorfortheprecise, this entails only redefining the expectations above as employmentfor demand
and thefeature.
More
precisely,
let
the
mean
for ofthewhile
ignoringthesecondarydesiderata.
This
metrictheimportanceofTo be fora feature,which
weabbreviate
asofPearson
correlation,
ismeanthedemandandthefeature.
precisely,
let theposteriortheWefirst consider
a for
directmeans
achievingourprimary
twoemploymentdesiderata,(i) More(i)weightedover that subset of occupations, rather than over all occupations.
ofmeanthesums
space.
demandfeaturefor
theoccupationith
occupationbem(x)..canWe
candefine19
featurefor
occupationbeWe
canthenthenlatentlatentdemandfeatureforthetheithbe m(x
).the
Weour
then
definea Pearson
feature,which
weabbreviateas
Pearsoncorrelation,isbe
theemployment(i)weightedcorrelation
coefficient
betweenmodel’s
predictivemeanlatentdemandfeaturefor
ith
occupation
m(x
).
We
can
then
define5.4.1.
Pearsoncorrelation
while
ignoringthesecondarydesiderata.
This
metricoftheimportanceof
5.4.1.
Pearsoncorrelation
41
19as
the Pearson
correlation
value
for
the
nth
O*NET
featureas
5.4.1.
Pearsoncorrelation
the Pearson
correlation
value
for
the
nth
O*NET
featureweighted
Pearsoncorrelation
coefficient
betweenourmodel’s
predictive
mean
demandandthefeature.
More
precisely,
letfortheposteriormean forthe asPearsoncorrelation
value
forthe
the
nth
O*NET
featuredefinethePearsoncorrelation
value
O*NET
afor
feature,which
we theabbreviateas
Pearsoncorrelation,istheemploymentposterior



(i)
We
first
consideradirectmeansofachievingourprimary
twodesiderata,




We
first
consideradirectmeansofachievingourprimary
twodesiderata,
fordemandandthefeature.
More
precisely,
let
the
mean
for
the
This
abilitytomanagealso enables the average




19
latent demandfeaturefor(i)thecoefficientith
occupation
be m(x
). We
can
then
define direct
I 
We
first
considerachieving
two desiderata, non-linearity
weighted
Pearsonbetweenour
predictive
mean ourIcorrelation
(i)
(i)
 of

 primary41(i) xa
(i)

w(i)
m(x
) m(x)
−the
E ith
m(x)
−nmodel’s
E(i)ofxnmeansw(i)m(x) − for
E
xThis
−E
x
while
ignoringthesecondarydesiderata.
This
metricof canthexthenimportancefeatureas:
IO*NET
while
ignoringthesecondarydesiderata.
metricthe(i)of define ofn featurei=1
latentdemandfeaturefor
occupationbem(x).importanceWen w(i)
i=1
the
Pearsoncorrelation
value
theprecisely,
nth
as
m(x
)−
E
m(x)
−E
xn of the importancewhile
ignoringthesecondarydesiderata.
This
metricoffordemandandthefeature.
More
let
the
posterior
mean
for
the
n
i=1
derivative
tocapturetheimportanceof features whose




:=
PC(n)
, (5)
 Pearson
 the:=the awhichPC(n), isfeaturefeature,which weabbreviate
ascorrelation,
Pearsonthe employment-,a feature,
wecorrelationabbreviateasthePearson
value
for
nth
asis(5)
employmentPearson
define(i) correlation,
PC(n)O*NETm(x)xn beaσ feature,
which
we
as
correlation, is the (5)employmentlatent demand featurefor σthem(x)ithoccupationm(x). Wecanthenσ(i):=x σ19
19abbreviateI(i)weighted
Pearsoncorrelation
coefficient
betweenourmodel’s
predictive
mean
σ−
m(x)
σ
x
n

weighted Pearson
correlation
coefficient
betweenourmodel’s
predictivemeannsignificanceisconditionalonthevaluesof other features.
19
w(i)
m(x
)
−
E
m(x)
x
E
x



n
n
i=1valueIforweightedcorrelation
between our model’s predictive mean
the Pearson correlation
the nthPearson
O*NETfeatureascoefficient(i)(i) posterior
foranddemandand thefeature.
More
precisely,
let xthemean for the
w(i)
−let
Ethe
m(x)
E
for
feature.
More
precisely,
posterior
mean
the
 m(x
) 
:=the
, xfor
(5)
n (5)
n −
(fordemand
anyPC(n)functiong(x))
we definetheemployment-weighted
expectation
i=1feature.
 precisely,for
demand
and
the
More
let
the
posterior
mean
for
the
wherewhere(for anyfunctiong(x))
wedefinetheemployment-weighted
expectation




For
instance,
fine
arts
is
very
importanttoartists, but less
(i)
(i)
σ the
m(x)
σoccupationxnwe
 define We:=feature
where
(for
any
functiontheemployment-weighted
expectation
PC(n), then (5)latentfeature
demandithg(x))m(x).
Wedefine
candefinelatent demand
the m(xithforoccupation
be
m(x
then
I for
(i)
(i) ).be(i)and variancew(i))demand− E m(x)
xfor
E ith
xcan
latentfeaturetheand variancen occupation be m(x ). We can then definen x−
i=1 variance
σ the
m(x)
σ
n as feature
andthePearsoncorrelation
value
for
nth
O*NET
as
the Pearson
correlation
value
for
the
nth
O*NET
featureimportantforoccupationswithdifferingskillsprofiles. This




:=
PC(n)
(5)
Pearsoncorrelation value 
for the, nth O*NETfeature as
Ithewhere (forany functiong(x))
wedefinetheI employment-weighted
σ m(x)
 expectation  
σ xn
I
(i)



I (i)
we
definetheemployment-weighted
expectation
where
g(x))
(i)
(i)
:=




g(x)
w(i)g(x
)
and
(6)
Eany
where
(for
any
function)wedefinetheemployment(i)(i)
isachievedthroughreportingthederivative
averaged
over:=Ifunction
g(x)w(i)g(x)and(6)
E (forandvariancem(xEI xm(x)x)−E
E xnx(i)n(i))− E
i=1 w(i):=g(x)) −w(i)g(xE m(x)n −i=1 w(i) m(x) −andE nm(x)
x(i)
xn(6)
i=1
and variance
i=1 :=n  i=1
 w(i) m(xexpectation
the employment-weighted (5)− E of:=,(5) , subsets
i=1
2 PC(n)



where
(for PC(n)anyfunctiong(x))
weexpectation
I2
weighted
variance:
occupations:for
instance,
those
that
fall
within2  and  define2:=2σ m(x)
PC(n)
,
(5)
g(x)
2 ,

σ xn  (7)  (7) 2
2g(x)
(i)
:=
− E
E :=
σ g(x):=g(x)g(x),σ σ)xnm(x)σ g(x)and varianceandE Eg(x) − Ew(i)g(xσ g(x)
m(x)(6):= E g(x)2 −E,σ xn a major occupational(7)
σ I g(x)grouping.19 For a non-linear function,
w(i)g(x(i) ) and(6)E g(x)
i=1 :=
g(x))
2we define the employment-weighted
 theemployment-weighted
Ig(x))
where
(for
any
functiondefineexpectation2
where(fornumber
any
expectation

(6)
i=1
I istotalthetotal
of occupations,
and
w(i)
is
the
fractionoftotal
em2anyI is thenumberof function
occupations,
and
w(i)
iswe
the
fraction
of
total
em(i)
where
(for
functiong(x))
wedefinetheemployment-weighted
expectation
:=
g(x)
g(x)
−
E
g(x)
,
(7)
E
σ






the
averagederivative
for a feature may be substantively
I
is
the
total
numberofoccupations,
and
w(i)
is
the
fraction
of
total
em:=
E
g(x)
w(i)g(x
)
and
(6)
2
2
varianceand variance2
ployment
withintheoccupation.
ith occupation.
ployment
withinthe andith:= ithg(x)− E g(x) ,(7)
E g(x)
σwithinandvariancei=1
ploymenttheoccupation.
the linear
the




different
overone
regionof space (occupational grouping)Pearsoncorrelation
measureslinearrelationshipbetweendemandand2
2I
Pearsoncorrelation
measuresrelationshipbetweendemandand(7)between demand andI2 measures occupations, correlation
 is theI is the total number
of
and
w(i)
fractionofI total emPearson
thelinearrelationship
:=
σthe
g(x)
E )w(i)g(xg(x)butE, (i) and
:=− (i)g(x)(6)
a feature.
AsIsuch,
gives
sign
of
clear
relationships,
satisfies
neither of(7):=Eg(x)w(i)g(x(6)Eg(x)
a feature.
As such,
itis
gives
the
of
clear
relationships,
butandsatisfies
neither
theit total
numberofoccupations,andw(i)is)clearthefraction
total
em(i)
ployment
withintheasign
ithoccupation.
:=
feature.
As
such,
it
gives
the
sign
of
relationships,
butsatisfies
neitherthanforanother.
g(x)w(i)g(x
)
and
(6)
E
i=1
i=1 linearity
our secondary
desiderata.
One
consequenceof linearity
thatthe Pearsonof ourofsecondary
desiderata.
One
consequencethattheployment
withintheoccupation.
 of

 is
is
Pearson
ofi=1
correlationthelinearbetweendemandand is that the Pearsonmeasures
ith
is

of oursecondarydesiderata.
One
consequencelinearity
2 relationship2 of2
I is Pearson
themaytotal
numberofoccupations,
and
the
total
2  2fractionem-  and2(7)2:=w(i)correlation
place
low
weighton
featuresthat
linkedhighdemand2
correlation
placeAs
lowweightfeaturesthatarelinkedtohighdemand
g(x)
−,to
E
g(x)
, neitherEare
σ E
Pearsoncorrelation
measuresthelinearrelationshipbetween
:=
g(x)
g(x)
−
Eg(x)
g(x)
(7)
σ on
2demanda may
feature.
such,
it
gives
the
sign
ofplaceclear
relationships,
butsatisfies
correlationmaylowweighton
featuresthatarelinkedtohighdemandployment
withintheithoccupation.
:=
g(x)
−
E g(x),each
Ethat
σtheg(x)Forsubset of(7)occupations, we will highlight those
for
a
small
numberofoccupations.
Nonetheless,featuresit
only only
for aofsmall
numberofoccupations.
Nonetheless,thefeaturesthatita feature.desiderata.
Asonlysuch,forit One
thenumbersign of clear
relationships,
butsatisfies theneitheroursecondaryconsequenceof linearity
is thatNonetheless,the Pearson
agivessmallofoccupations.
features that it
Pearsoncorrelation
measuresthelinear relationshipbetweendemandandhighlight
willunquestionablybeimportant:ifastrongpositivelinear
I
is
the
total
numberofoccupations,
and
w(i)
is
the
fractionof
total
emdoes does
highlight
willunquestionablybeimportant:ifastrongpositivelinear
I
is
the
total
numberofoccupations,
and
w(i)
is
the
fractionoftotalemis thetotal
numberofsignoccupations,issatisfies
of oursecondaryOne
consequenceoflinearity
is thatPearsonwithbothemlarge positive and large negative
correlation
may
place
lowdesiderata.
weightfeaturesthatandare oflinkedtothe
highdemanddoes
highlight
willunquestionablybeimportant:
if thea w(i)
strongpositivelinearI isonofthe
total
numberoccupations,andisfeatures
thefractionof totalafeature.
As
such,
it
gives
the
clear
relationships,butneitherinteraction
exists,it should
certainly
influence
our resulting
skillspolicy.
ployment
withinithweight
occupation.
interaction
exists,shouldcertainly
influenceourresultingskillspolicy.
ployment
theithplace
occupation.
correlation
may
low
on
featuresthatarelinked
toour
high
only
for
aitwithinsmall
numberof theoccupations.
Nonetheless,thefeaturesthat
itdemandinteraction
exists,itshouldcertainly
influence
resultingskills policy.
ployment
withinthe
ith occupation.
ofourwe
secondarydesiderata.
One
consequencelinearity
is thatthe Pearson
fractionof
total
employment
withintheoccupation.
averagederivatives. Those with large positive derivatives
As such,would
considerPearsoncorrelation
toof
sufficientbutnotPearsoncorrelation
measuresthelinearrelationshipbetweendemandAs such,we
would
considercorrelation
to
give
agive
sufficient
butpositivenot
Pearsoncorrelation
measuresthelinearrelationshipbetweendemandandonly
fora Pearson
small
numberofoccupations.
Nonetheless,thefeaturesitand but notdoeshighlight
willunquestionablybeimportant:ifaalinked
strongAslow
such,
we wouldconsiderPearsoncorrelation
tolinear
give
athat
sufficient
Pearsoncorrelation
measuresthe linear
relationshipbetween demand andcorrelation
may
place
weighton
features
that
are
to
high
demandnecessaryconditionforimportance.
afeature.
As
such,
it
gives
the
sign
of
clear
relationships,
butsatisfies
neithernecessaryconditionforimportance.
afeature.
As
such,
it
gives
the
sign
of
clear
relationships,
butsatisfies
neitherdoeshighlight
willunquestionablybeimportant:ifastrongpositivelinearwe sayarecomplementary
to the occupational
interaction
exists,number
itnecessary
shouldcertainly
influenceourresultingskillspolicy.
conditionforimportance.
afeature.
As
such,
it
gives
the
sign
of
clear
relationships,
butsatisfies
neither
only
for
a
small
of
occupations.
Nonetheless,thefeaturesthat
it
Pearsonmeasuresthe consequencelinearrelationshipofcorrelation
our secondaryOne
ofissufficientlinearity
that
Pearsonof oursecondarydesiderata.
One
consequenceoflinearity
thatthe isPearsoninteraction
exists,shouldcertainly
influence
resultingskillspolicy.
As
such,
we
would
consideritdesiderata.
Pearsoncorrelation
to
aour
but
nottheof ourbesecondary
desiderata.
One
consequenceof linearity
is(increasingthat the Pearson
does
highlight
will unquestionably
important:if givea strong
positivelineargroup
suchafeatureincreases demand),correlation
may
place
low
weighton
featuresthatarelinkedtohighdemandcorrelation
may
place
low
weighton
featuresthatarelinkedtohighdemand
19
As
such,
we
would
considerPearsoncorrelation
to
give
a
sufficient
but
not
19
necessaryimportance.
Note
thatPearson
thecondition
Pearsoncorrelation
coefficientis influence
often
used
for feature
selection(aspolicy.
abetweendemandandacorrelation
feature.
As
such,itselection
gives
the
Note thatthecorrelation
coefficientis oftenusedforfeature(asa features
may
place
low
weightonthat are selection
linked to(ashighinteraction
exists,itfor
shouldcertainly
ourresultingNoteoffor
that
the Pearson
correlation
coefficientisskills
often
for
a demanda 19
small
numberof occupations.
Nonetheless,theusedfeatures
that
it
only
for
a onlysmall
numberoccupations.
Nonetheless,featuresthat
it featurenecessarycondition
importance.
an exception
totheforinappropriate
information-theoretic
methods
ofthefeatureselection
whereas
those
with
large
negative
derivatives we say
filter),filter),an exception
totheinappropriate
information-theoretic
methods
ofgivefeatureselection
only
for
a
small
numberofoccupations.
Nonetheless,thefeatures
that
it
As
such,
we
would
considerPearsoncorrelationtoasufficientbut
notfilter), an
exception
to theinappropriate
methods
feature selection
signofdescribe
clear
relationships,
butsatisfies
neither
ofifpositive
our
we genericallydoesabove.
highlight
willunquestionably
be important:a stronglinear
positiveoflinear
does
highlight
willunquestionablybeimportant:if ainformation-theoretic
strongwe genericallydescribe
above.
19
we genericallydescribe
above.
does
highlight
will usedunquestionably
be important:
if a strong
positive linear
necessaryconditionforimportance.
Note that
the Pearson
correlation
coefficientis oftenfor feature
selection(as skillsaareanti-complementary(increasing such a featureinteraction
exists,itshouldcertainly
influenceourresultingpolicy.
interaction
exists,itshouldcertainly
influenceourresultingskillspolicy.
19
secondary
One interaction
consequence
isfeature
Notecriteria.
thattheinappropriate
Pearsoncorrelation
coefficient
often
used
selection (asexists, ofitis linearity
shouldcertainly
influenceoura resulting skills policy.
filter),an exception
to theinformation-theoretic
methods
offorfeatureselection
Asdescribesuch,
we wouldconsidercorrelation
Pearsoncorrelation
togive aofbut
sufficient
but not
Asgenerically
such,
we
would
considerto give a sufficient
notselection
demand).
filter),an
exception
to thePearson
inappropriate
methods
featureweabove.
Ascoefficientsuch,
weinformation-theoretic
wouldconsiderPearsontodecreases
give a sufficient
but not It is also of interest to speak of
19
that
Pearsoncorrelationmay
lowforweightoncorrelation
Notethethat
the Pearson
correlation
isplaceoften usedfeature selection(as anecessaryconditionfor importance.
necessaryconditionforimportance.
wegenericallydescribe
above.
necessarycondition formethods
importance.
complementarities between features, rather than simply
filter), an exception to the inappropriate
information-theoreticof feature selection
featuresarelinkedwegenerically that
describe
above.
40high demand only for a small
40 to
40
19
of(as
the
Note
that thePearson coefficientcorrelationiscoefficient
often
used selection
for feature
a complementarity of a feature to an occupational
Note that 19
thePearsoncorrelation
often usedisforfeature(asselectiona19
numberof occupations.
the correlation
featuresmethods
thatselection
Note that theinformation-theoretic
Pearsoncoefficientoften selectionused for feature selection (as afilter),an
to theNonetheless,inappropriate
ofisfeaturefilter), an exception
toexception
the inappropriate
information-theoretic methods of feature
to the inappropriate information-theoretic
methodsToof feature
selectiongroup.
do so,
we must find some way of singling out the
40exceptionbe
we describe
genericallydescribefilter),above.an
wegenericallyabove.
it does highlight will unquestionably
important:ifa
we generically describe
above.
40
features associated with the occupational group. Thosestrong positive linear interaction
exists, it should certainly
40
features that are large on average for an occupational
influence our resulting skills policy. As such, we would
group are, speaking roughly, those that are mostconsider Pearson correlation40
to give a40sufficient but not
40
exceptionally significant, and hence most characteristic,necessary condition for importance.

38

by the5.5.
averagederivative:
the recommendations
ofmodelling,
this metricbeother targeted
facet of ouras cannot
in producing
occupationoccupations
5.5.NewNew
occupations
occupationalgroups.
However,
there is a
derivative varying over x, thereby
increasing
the standard deviation.
Notetrusted for these features. The excluded
featuresareusedofasexclusion.
normal in Any
everynon-linearity in d
predictions
demand.
the ofthresholdthat features excluded underWe
thisschemeareexcluded
only
forofconsideration
defineapotential
newoccupation
asacombinationofskills,knowledgeWedefineapotential
newoccupation
asacombinationofskills,knowledgeother facet of our modelling, as in producing
occupation-level
andderivative
varying
overx, aggregate
thereby increasing the st
by the average derivative: andtherecommendationsof
thiscannotbedemand,that
totometricseeseehighfutureandabilities
thatis islikely
likelyhighfuturedemand,butbutis isnot
notassociassocipredictions
ofabilities
demand.
that features
excluded under this scheme are exclud
5.5.
New
occupations
trusted for these features. The
excludedfeaturesareused
as normal
in every
ated
with
an
existing
occupation.
To
forecast
where
newoccupations
might
ated
with
an
existing
occupation.
To
forecast
where
newoccupations
might
T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0bytheaveragederivative:therecommendationsoother facet of our modelling,emerge,asin producing
occupation-level
andwewesimply
optimisetheposteriormeanlatentemerge,simplyoptimisetheposteriormeanfornew
forthethelatentdemand
demandvariWedefineaaggregate
potential
occupation
as a varicombinationotrustedforthesefeatures.
The
excludedfeatures are
5.5.
New
occupations
predictions of demand.
able,
m(x)
asas
a function
of of
x.and
precisely,
start
byby
able,
m(x)
a function
x.MoreMoreprecisely,
start
randomly
selectingabilities
thatweiswelikelytorandomly
see highselecting
futuredemand,other facet of our modelling, as in producing occupa
5050current
occupations
asasourdemandoccupations.
occupations
ourstarting
starting
pointsof ofhighdemandoccupations.
We defineacurrent
potential
new occupationasa combination
ofhighskills,knowledge
atedwithanpoints
existing
occupation.
To
forecast where newpredictions of demand.
We
local
BFGS,
observingbox-constraints,
Werun
run
local
optimisers
BFGS,
observingbox-constraints,
and abilities
that
isoptimisers
likelyto (limited-memorysee(limited-memory
high future
demand,
but
isthe
not
associ- mean for the la
emerge,we simply
optimiseposterior
5.5. New occupations
creasing such a feature increases demand), whereas those with large negative
asas
per
Byrd
etet
al.
(1995))
initialised
atat
each
of of
the
50
occupations.
This
will
per
Byrd
al.
(1995))
initialised
each
the
50
occupations.
Thiswillstart byated withanexisting
occupation.
To
forecast
where
newoccupations
able,
m(x)
as
a functionofx.
More might
precisely,
weWe
new occupation
as aoptimisers
combinationof skills,
knowledgeNew† occupations
derivatives weofsay
anti-complementary(increasing
such awefeaturede- a potentialemerge,† skills, knowledge and abilities
theareoccupationalgrouping.
As such,willdefine
definereturn
5050
local
of
m(x):
pointsxfor
returnlocal
optimisersofm(x):pointsxininknowledge
andabilities
welocalsimplyoptimisetheofposteriormean
thelatentdemandvari:5.5.
pointsinskills,
theabilities
50
optimisers
50
current
occupations
as skills,
ourstarting
pointsof high decreasing
a feature
increases
demand),whereas
those
with
large
negativecreasing
suchsucha feature
increases
demand),whereas
those
with
large
negative
abilities thatis likely
to
see
high
futuredemand,but
isnotassocicreases demand). It is also of interest to speak of complementaritiesand
betweenspace
areassociated
with
high
demand.
Many
these
optimisers
space
are
associated
with
high
demand.
Many
of
these
optimisers
will
m(x)
asthatathat
functionMoreprecisely,
we optimisers
start
by of
randomly
selectingwill
We
run
local
(limited-memory
BFGS,
observi
derivatives
we say(increasingsuch
ax.feature
decreasing suchcreasinga featuresuch
increases
demand),whereasthose
withlarge
large
negativeWe
defineaispotential
newoccupationasbeabecombina
derivatives
wesay
are areanti-complementary(increasing
such
a of
featuredefeatureswithpositiveaveragederivatives
toanti-complementary
beexistingable,
andknowledge
featuresspace
thatassociated
with
high
featureincreases
demand),whereasthosewith large
large
negativewithanoccupation.
To
forecast
where
newoccupations
mightcreasing suchaa feature
increases
demand),whereaswithnegativefeatures,ratherthansimplyof thethosecomplementarityof a feature
toatedan occu(near-)identical,
andotherswillbe(near-)identicaltoexisting(near-)identical,
otherswillbe(near-)identical
existing
occupations.
50
occupations
asand
our
starting
pointsofdemandoccupations.
as
per
Byrd
et highal. (1995))
initialisedateach
the 50 dem
occ
creasesdemand).
is alsoof interest
tocurrent
speakofcomplementarities
betweenderivatives wesay are anti-complementary(increasingsuch
a feature
de-increases
andabilitiesthat
istolikelytooccupations.
seehighof futurecreasesdemand).
It
isItalsoofinterest
to
speak
ofthe
complementarities
betweencreasingsuchafeaturedemand),whereas
those
with
large
negative
derivatives
we saysay
are anti-complementaryanti-complementary(increasingsuchafeaturedeemerge,
we
simplyoptimiseposterior
mean
for
the
latentdemandvariderivatives
weare(increasingsuchafeaturedepationalgroup.
To
do
so,
we
must
find
some
way
of
singlingout
thefeatures†,0†,0†complementary
to thosecharacteristic(large-valued)
Beginningwitha singlesuch
optimiser,x x ,optimisers
we
add
each
successive
optimiser
Beginning
witha(limited-memory
singlesuch
optimiser,
,will
we
add
each
successive
optimiserWe rundemand.
local
BFGS,
observingbox-constraints,
return50
localofoccupation.
m(x):pointsx forecastin skills,wher
know
Many
ofantothese
optimisers
be(near-)identical,
features,ratherthansimplyof thecomplementarity
a featureanoccucreases demand).
It isalso of interest
to of
speak
of complementarities
between
ated
with
an
existingTofeatures,ratherthansimplyof thecomplementarity
ofoptimisers
a offeaturetooccuwesay
areanti-complementary(increasinga feature
decreases
demand).
It isis also
alsointerest
toderivatives
speak ofof complementarities
complementarities
betweenm(x)a function
of x.
More
precisely,
we
start
by
randomly
selectingcreasesdemand).
It
ofwithinterest
speak
betweenassociatedthe tooccupationalgroup.
Thosefeaturesthat areable,
large
on as such
unlessininitialised
sense,
than
asimplypresetthreshold(will==0.1)
toto
unlessitissingling
iscloser,closer,inthe
the2-norm
2-normsense,
than
aoccupations.
presetthreshold(
0.1)
as perByrd
etit(1995))
at each
ofare
the
50
This
space
that
associated
with
high
demand.
Manyof thespationalgroup.
Toso,
do weso, must
we must
some
way
ofal.
the
featuresfeatures, rather
than simplyofthesimply
complementarityof20a. feature
toofanoccuemerge,weoptimiseposteriormean
forpationalgroup.
To50
dofindfind
some
way
of
singlingout
thefeaturescreasesdemand).
Itis alsointerest
to thatspeak
ofmostcomplementarities
betweenfeatures,rather than
than
offeatures
the complementarity
complementarity
a featurefeaturetoofananoccuandotherswilloutbem(x):(near-)identical
toand
each
other
tothe
existing
current
occupationsasourstarting
pointsofhighdemandoccupations.
features,rathersimplyoftheof aspeaking
tooccuaverage
for anoccupational
group
are,
roughly,those
areeither:an
existing
occupation,or;axpreviously
includedThis
proan
existinga† inpreviously
optimiser.
This
pro- wereturn50either:local
optimisers
ofoccupation,
pointsskills,asknowledge
and(near-)identical,
others optimiser.
will(near-)identicalto staex
associatedtheoccupationalgroup.
Thosefeatures
that
are
large
onor;
pational group.
To dogroup.
so, we mustsome
wayfindof singling
outofthefeaturesable,m(x)aincluded
functionof bex.abilities
More
precisely,
associatedwithwiththeWeoccupationalgroup.
features
that
are
large
on
features,thansimplyoftheofcomplementarityof alarge
featureto ThoseanoccupationalTodofindso, we
we
must
some
way
singlingout
thefeaturesrun with
local
optimisers(limited-memory
BFGS,
observingbox-constraints,
pationalgroup. To
do
so,
must
find somewayof singlingout
thefeaturescreasingsuch
a ratherfeatureincreases
demand),whereasthose
negative
exceptionallysignificant,
and
hence
most
characteristic,theoccupational†,i †,0†,icedurewill
returnawith
list
ofmosthotspotsdemand(localoptimisers){x
ofadd
cedurewill
return
athat
list
of
hotspotsofdemand(localoptimisers){x;, ∀i},
of each
space
that
are
associatedhighdemand.
Many
these
optimisersbeBeginningwithaofoccupations
singlesuch
optimiser,x; ∀i},
wepoints
occupations.
Beginningwithsinglesuch
optimiser,
averageforoccupationalan occupational
group
are,
speakingroughly,
those
are
most
associated withthe occupationalgroup. Thosegroup.
featuresthatfeatures
are large
50ofacurrent
as ourwill
startingof hisaverage
foron
an
group
are,
speakingroughly,
those
that
are
Tothe
doon
so,
welargemust
findsomeaswayofsingling
out
thedefeaturesassociatedwith thethe occupational
occupational
Those
that
are
large
per
Byrd
eta al.
(1995))
initialised
at
each
of
the
50
occupations.
This
will
associatedwithgroup.
Thosefeatures
that
are
on
derivatives
we group.
say
areanti-complementary(increasing
such
featuregrouping.
As Thesuch,
we pationalwilldefinefeatureswithlarge
positiveaveragederivakey
drawbackofaveragederivative
approachlengththatbedifferent
fordifferentmeanfunctions.
Eachhotspot
lengththatmaybedifferent
fordifferentmeanfunctions.
Eachhotspot(near-)identical,
andotherswillbeunless(near-)identical
toin
existing
occupations.
it isruncloser,the2-normsense,
thancanacanpresetexceptionallysignificant,
and
hence
most characteristic,of
occupational
average for anaverageoccupationalgroup are, speaking
roughly,
thoseroughly,
that
are
most
We
local
optimisers
BFGS,thr
ob
exceptionallysignificant,
and
hence
most
ofmay
the
occupational† the21
associatedwiththeoccupationalgroup.
Thosethat
arecharacteristic,
large
on
for anan occupational
occupationalgroupcreases
are,speaking
those
that
are most
most
returnfeatures
50. local
optimisers
of,be
m(x):
pointsxthroughinsuccessive
skills,knowledge
abilities
we
add
each
optimiser
unlessitknowledge
is(limited-memory
closer,average forare,speakingroughly,those
aredemand).
Itcharacteristic
is alsoofthatinterest
tospeak
offeaturescomplementarities
betweentives to be groupcomplementary
to those(large-valued)†,0 andinterpreted
returning
vectorofeachskills,knowledge
abilitiesbeinterpreted
through
returning
its
vectorofskills,andabilities
Beginningwithapositive
singlesuch
optimiser,
xits
,per
we
add
successive
optimiser
either:existing
occupation,
or;
a and
previously
includedgrouping.
As accuratesuch,
we willdefinefeatureswithlarge
averagederivaexceptionallyexceptionallysignificant, andhence mostof the occupational
is thattheaveragingitselfmay
obstruct
asan
Byrd
et
al.
(1995))
initialisedat eachof the o5grouping.
As such,we willdefinefeatureswithlarge
positiveaveragederivaaveragean occupational
group
are,
that
are
most
significant,andcharacteristic,
hence
most
characteristic,of thetheofoccupational
occupational
space
arethose
associated
with
high
demand.
Many
of
these
optimisers
willbeexceptionally significant,
hence
most
of
features,ratherthansimplytheapproachcomplementarity
ofroughly,
athat
featureto anoccu†,i †,i
The andkey
drawback
ofcharacteristic,
the foraveragederivative
isspeaking
that
the
av21
21
valuesx(large-valued)
well
as
the
list
of of
which
are
closest
to
it.
x, as
,inas
well
as
the
list
occupations
closestit.
unlessit values
isthecloser,the2-normsense,
than
areturnpresetthreshold(
0.1)
topoints(local
will
awhichlist
ofare
hotspots
oftoin
2-normsense,
than
areturnpresetthreshold( of=m(x):)demandtotives
tocomplementary
be complementary
to thosecharacteristicfeatures.occupationsgrouping. Asgrouping.
such, we willdefinewefeatureswith featureslarge exceptionally
positiveaveragederiva50
local
optimisersx† in optim
tives
to
be
to thosecharacteristic(large-valued)
features.cedureskills10
significant,andfindhence
characteristic,theoccupational
As such,such,
will
definewithlarge
positiveaveragederiva(near-)identical,
andothers will be (near-)identical to existing
occupations.
grouping. As
we willdefinefeatureswithlarge
averagederivapationalgroup.
To positive
do
so,
we
must
of cos(xsingling
outofthefeatureseraging
itselfmayoccludesignal.
As
anexample,fsome(x)mostway
10),, thereinterpretation.
As
anexample,if
21
either:an existing
occupation,or;thealengthpreviously
included
optimiser.
This
pro-demand.
that thatmay
be associateddifferent
forwithdifferentmean functions
drawbackof theaveragederivative
approachis thatav- spacetives to be complementary
to those characteristic(large-valued)
features
TheThekey
drawbackof ntheaveragederivative
approach
the
avare
high
Many o21 key†,0is that21
grouping.
As
such,we.derivativewill
definefeatureslarge
positiveaveragederivatives toto be
be complementarycomplementary
to thosethosecharacteristic(large-valued)
features
Beginningwitha singlesuch
optimiser,
xexisting, we add
eachsuccessive
optimiser
either:anoccupation,or;apreviously
included
tives
tocharacteristic(large-valued)
features.. Those
associatedwith thetheoccupationalgroup.
features
are
large
on
there are many
pointsfor whichaveragewould
bewithverylarge:that†,i
101010 cos(xcedurewill
demand(localoptimisers){x its; ∀i},of of skills, know
through
returning
vectoreraging
itselfmayoccludesignal.
As example,
an example,
ifaflist(x)
hotspots), (near-)identical,
The key drawbackof drawback
the averagederivative
approachis thataveragingitselfmayoccludesignal.
As an
if f (x) of10cos(xthreshold),of ninterpreted
manypointsfor thewhichtheaveragederivativewould
21 returnnbe
tives
complementary
toare,those. sense,The keykeyof thethe areaveragederivative
approachthattheavunlessitthosein features
the2-normthan a preset( = 0.1) to and others will be (near-)identicalThe
drawbackofaverageapproachisisgrouptheavaverage
for toan beoccupationalroughly,that aremostfor instance,allthosederivative
pointsimmediately
tothattheleftspeaking
ofcharacteristica peak,{xn(large-valued)=is closer,
†,i
10
†,0
lengththatmaywould
be woulddifferent
different
hotspot
can whichvaluesxmean
asfunctions.
well astheEachlist hotspotsofoccupations
are cloare many
pointsfor whichaveragederivative
be forvery
large:will
optimiser.
This
procedurea listoferaging itselferaging
may occlude
signal.
As ansignal.
example,if fThe(x) key 10drawback
cos(x
are
many
pointsfor approach
which
theistheaveragederivative
be verylarge:
Beginning
10theren ), there
10
43,return43
oftheaveragederivativethat theavitself may
may
occludeAs ananexample,
(x)
This
10
cos(x
),characteristic,
either:an
existing
occupation,
or; a previously
included
optimiser.
This withpro- a single such optimiser, x , we add
eraging itself
As
example,ifif ff(x)10
exceptionallysignificant,
and
hence
most
of theoccupational−occlude+ 2πn; ∀signal.
small
positive
 andintegern}.
xcos(xisann),featurethat
is unbeverylarge:
forinstance,all
thosepointsimmediatelyn
be interpreted
through
returningitsof †,iskills,
knowledge
abilities
forinstance,
all thosepointsimmediately
to theleft
a peak,= unless
there are many
pointsfor whichtheforaveragederivative
would
be verylarge:beforvery
instance,
all thosepointsto theof aofpeak,{xoptimisers)=nvectorit; ∀i},is closer,
in theand2-normsense, than a presn {xeraging
itselfmay
signal.
As
example,if returnfimmediately(x) a deriva1010ofcos(x), leftthere
are many
many
pointswhich
theaveragederivative
wouldlarge:
cedurewill
list
hotspotsof demand(local{xof length
there
arepointswhich
thederivative
would
be verylarge:
of(localoptimisers)
{
},
of
that
may
grouping.
As
wewillocclude
definefeatureswithanlarge
positiveaveragelikely for
to beusefulinaverage
policy:itsuch,has anequal
numberofpoints(occupations)ndemand†,ivaluesx This, aswell
asa thelist ofwhich
are closest
to it.
−2πn;+ 2πn;∀ smallpositive
 andinteger
xna isfeaturefeaturethat
is un- either:for instance, for
all instance,
those points
immediately
tothe
theleft
left of
ofare
peak,
{x
=for
−which+∀ average
small
positive
 andintegern}.
Thisisoccupations
unan existing
occupation,
or; a previously inclu
toimmediately
peak,
small
positiveandn is21 n}.
there
many
pointsderivative
would
very
allwiththose
pointstoaathetheleftofnaatopeak,{xnnthe
=
that
may
bebe
different
forxdifferent
meanthat
functions.
Each hotspot
canfor instance,
all
those
points
to
left
of
peak,
=
tives tobecomplementarythosecharacteristic(large-valued)
features. large:verylargeimmediately
negative
derivative.
Increasing
xn {x
would
be verylength
harmfullikelytouseful
be usefulin policy:
it hasan equal
numberof points(occupations)
varyfordifferentmean functions.
Eachhotspot
can43beof demand (loca− + 2πn; ∀ small
positive andpositive
integer n}.
This
xnfor
isn}.
ainstance,featurethatisun- points
likelytobein
it hasanequal
numberof points(occupations)cedure will
returna list of hotspots
all
those
immediately
topolicy:
the
left
of
a peak,
{x
−+
+2πn;
2πn;∀∀ small
small
and
integerThis
isisaaifa
featurethatthatis
unbeinterpreted
through
returning−
integern}.
Thisxxnn isfeature
that
is
unThe
key
drawbackofthechosen
averagederivative
approachisusefulthatthe avtopositive
all such andoccupations.
Nonetheless,thesamples(occupations)n = its vector of skills, knowledge and abilities
integer}.
Thisfeatureisunlikelytobewithnegative
Increasing
xn would
be very harmful length that may be different for different mean func
likely to be usefulit hasan equal
numberof +points(occupations)withveryverylargelarge
negative
derivative.
Increasing
x would
be very†,i isderivative.
−
2πn;∀ smallpositive
 and
n}.
This
featurethat
isinterpreted
unlikely into policy:
be usefulusefulin policy:
policy:has
an equal
equal
numberofpositive
points(occupations)valuesasa wellas thelistof noccupations
whichharmful
arereturning
closest to it.its vector of skills, abilities
likely
to
be
in
has
an
numberofpoints(occupations)eraging
itselfoccludesignal.
As integeranexample,if fx(x)1010 cos(xthrough
containevenititone
more
point
ofmay
large
derivative
than
large
negativen , n ),to suchallofsuchoccupations.
Nonetheless,if thechosensamples(occupations)with very large
derivative.
xnIncreasing
would
bebe
very
harmfulto alloccupations.
Nonetheless,if thechosensamples(occupations)in therepolicy:itmanyhas
equal
numberpoints(occupations)
43 be interpreted through returning its vector of skills
likelytoaverage
policy:
it
has
an
equal
number(occupations)
withnegative
very large
large
negative Increasing
derivative.
xan
would
be
very
harmfulwithvery
negative
derivative.
Increasing
xpointswouldvery
harmfulareforinbewhich
theaveragederivative
be verylarge:
derivative,
xn mayhavehighderivative.
This
drawback
leads
towould
usof points
nnusefulcontain
even
one
more
point
of large
positive
derivative
than
large
negativeto all such occupations.
Nonetheless,ifthechosensamples(occupations)andknowledge
featuresvalueswellasasofcontain
even
one
more
point
of
large
positive
derivative
than
large
negativevalues x†,i, as
as wellthethelist listof occupations
which a
with
very
large
negative
derivative.
Increasing
be very
to all
all suchsuch occupations.
occupations.
Nonetheless,if thethechosensamples(occupations)toifchosen(occupations)for very
instance,
allnegativethose
pointsimmediately
to theleftxnofwoulda peak,{xn harmful=
regardingNonetheless,the with
averagederivativeasasamplesnecessary
but
not sufficientcondition
large
derivative.
Increasing
would
derivative,
xn mayhave
highaveragederivative.
This
drawback
leads
to us
contain even containone more
point
of
large
positive
derivative
than
large
negative
derivative,
x
may
have
high
averagederivative.
This
drawback
leads
to
us
n
to allsuch
occupations.
Nonetheless,if thexnchosen
sampleseven one
oneformore
point ofof large
large
positive
derivative
thantolarge
negativecontain even
more
point
derivative
than
large
negative− +positive
2πn;
small
positiveandintegern}. correlation.
This
is a featurethat(occupations)is un- occupations
importance;
renderingit∀ complementarythemarginal
whichareclosesttoit.
regardingtheaveragederivative
a necessary
notsufficientcondition
derivative, xnderivative,
may have highaveragederivative.
This
drawback
leads
to uspoint
regardingthepositive
averagederivative
as aasnecessary
but but
not43
sufficientcondition
be
very
to
all
such
occupations.
Nonetheless,contain
even
more
of to
large
derivative
than
large
negative
may
have
high
averagederivative.
This
drawback
leads
to
us
derivative, xxnn may
have
high
averagederivative.
This
drawback
leads
uslikelytoharmfulbeinonepolicy:it hasanequal
numberof points
To slightly
ameliorate
theusefulproblemsofaveraging,
wecalculatetheem- (occupations)for importance;
renderingit complementary
to themarginal
correlation.
regarding theregarding
average derivative
as derivative
a necessary
not sufficientconditionforimportance;
renderingit complementary
correlation.
derivative,
xnnot
may
have
high
averagederivative.
This
drawbackleadsto5.6.
tothe
us marginal43
the average
averageas butnecessarybutnotsufficientcondition
regarding thederivativeasaachosennecessarybut
sufficientcondition
with
very
large
negative
derivative.
Increasing
xn wouldbevery harmfulifthesamples(occupations)contain
even
one
TREND
EXTRAPOLATION
To slightly
ameliorate
the problemsof averaging,
we calculate
for importance;
renderingit complementary
to the marginalcorrelation.
To slightly
ameliorate
averaging,
we calculate
the theem- emthe
averagederivativeasa necessary
butthenotproblems
sufficientofcondition
for importance;
importance;
rendering
complementarytothemarginal
correlation.
forrenderingitit complementarythemarginal
correlation.
to all regarding
such tooccupations.
Nonetheless,if thechosen samples(occupations)21
Our definitionof complementarity
is loosely
relatedtoderivativethat used by economists:
two negative
To slightly ameliorate
theproblemsofaveraging,
wecalculatetheemmore
point
of
large
positive
than
large
for
importance;
rendering
itofthecomplementarythelevelmarginal
To slightly
slightly ameliorate
ameliorate
theproblems
averaging,
wecalculatetheemTo
problemsofaveraging,
wecalculateemcontaineven
one more
pointof large
derivativethanlargecorrelation.
negativefeatures arethe
complementaryifof
themarginal
value
product
onepositiveisincreasing
intotheAsantoalternative
to thelabels from our foresight workshop
21
21
Our
definition
complementarity
is loosely
thatby economists:
ameliorate
theproblemsofaveraging,
we iscalculate
therelated
emOurdefinitionof complementarity
loosely
to thatusedusedby economists:
two twoderivative,
xnslightly
may
have
high
averagederivative.
This
drawback
leads
torelatedusof the second. derivative,This
definitionTo
fails
to meetourneeds.
The
first
problem
isofthat
the
may
have
high
averagederivative.
This
21
featuresarecomplementaryifthemarginal
productof isone
is increasing
in levelthe levelfeaturesarecomplementaryifthemarginal
valuevalue
productof oneincreasing
inthedefinition
makes
no regarding
accommodationforlocationderivativein feature
The economics
definition
Our definition
is loosely
relatedto that usedbyeconomists:
two space.
participants,
we
use
non-parametric
extrapolation oftheaverageasanecessarybutnotsufficientcondition
21of complementarity
21
Our definition
definition ofof complementarity
complementarity
loosely related
related toto thatthattoused byby economists:
economists:
two
Our
isis loosely
used
twoof second.
thesecond.
definition
to meetour needs.
problemis thatoffunction
theThisThisdefinition
fails fails
to meetour needs.
The Thefirst firstproblemis thatthe thedrawbackustheaveragederivativeasis a statement
derivativeofwithrespect
tomarginal
thetwofeatures are complementary if the marginalvalue about
productof second-orderone isleads
increasingin theregard
level
fortheimportance;
renderingit thecomplementary
tothefeatures areare complementarycomplementary ifif thethe marginal
marginal
value21productproductof oneone isof
is increasing
increasing
indefinition
the level
level
features
value
of
in
the
Our
definition
complementarity
is loosely
relatedtononthat correlation.
usedby location
economists:
two
makes
no
accommodationfor
inhistoricalfeaturespace.
The economics
definition
definition
makes
nofor locationin feature
space.
Theemployment
economics
definition
trendstogive
a more data-driven
featuresbeing
arbitraryfunction,
asthe
may
be
learnedbyouraccommodation
flexibleof the second. This definition fails
to meetourpositive;needs. for
Theanfirstproblemis thatof thethe second.
second. This
This definition
definition fails
failsato
to
meet
our
needs.
The
first
problemisstatement
that
the
Toour
slightlyameliorate
theproblemsof valueaveraging,
wecalculatethe emof
meetneeds.
The
problemthatthefeaturesarecomplementary
marginal
product
ofsecond-orderone
is increasinginnecessary
notfirst
sufficientforimportance;
isconditiona different
statement
about
thederivative
oflevelthe function
respect
to twothe twoaisvery
about
the ofsecond-orderderivativeof thethefunction
withwith
respect
to thethesecond-orderderivatives
may
beifisinthe
regions
space.
definition makes no accommodationparametricfor locationmodel,in feature
space.
The but
economics
definition

definitionmakes
makesnonoaccommodationaccommodationfor
forlocation
location
inthefeaturespace.
Thedefinition
economics
definition
definition
featurespace.
The
definition
ofin
second.
This
fails
to
meet
our
needs.
The
problemisasthatalternative
predicting
demand in the year 2030.
featuresbeing
positive;for
an first
arbitraryfunction,as themay
be learnedbyour flexible
featuresbeing
positive;
forfeature
anarbitraryfunction,may
be learned
byto
our
flexible
non-nonThe derivative
second, related,withthedefinitioniseconomics
that
may
lead
to
highlighting
is a statement about the second-orderof theproblem
functionwithrespect
to thetwo itstatement about
about thethe second-ordersecond-orderderivative
ofitthethefunctionwithrespect to
to
the
two
rendering
complementarythemarginal
isis aa statement
derivativeoffunction
respect
tothetwodefinition
nowithaccommodationforlocationinspace. The
economics
definition
parametricmodel,
thecorrelation.
second-orderderivatives
mayin verydifferent
regionsof space.
model,
thefeaturesecond-orderderivatives
may
be inbevery
different
regionsof space.
combinationseven
thesecond-orderderivatives
arepositive
andconstant
acrossfeatures being positive; for an arbitrary
function,which,as maylearnedbymakes
our flexible
non- parametric
5.6.
Trend extrapolation
21be if
Ouris definitionof
isTheloosely
related
to
that
used
by
economists:
twofeatures being
being positive;
positive; forfor anan arbitrary
arbitrary function,function,as maymay bebecomplementarity
learnedthe
bysecond-orderourflexible
nonfeaturesaslearnedbyour
flexible
nonSpecifically,
weuseaGaussian
process to regress UK andastatement
about
derivativeofthefunctionwithrespecttothetwosecond,related,problemwiththedefinition
is
that
it
may
lead
to
highlighting
featureThe
second,related,problemwiththedefinition
is
that
it
may
lead
to
highlighting
featureare actively
As a simple
example,for a bivariate quadratic function
parametric model, the second-orderspace,
derivatives
may beharmful.
in very different
regionsof space.extrapolation
5.6.
Trend
featuresfeatures
areif themarginal
value
product
one
is
the
level
parametric model,
model, thethe second-ordersecond-order derivatives
derivatives
maycomplementary
bebeing
in veryvery
different
regionsof space.
space.
parametric
may
be
in
different
regionsofpositive;anarbitraryfunction,
asof
may
be
learnedbyinour
flexiblenoncombinations
which,
even
ifincreasing
the second-orderderivatives
are positiveand constantacrosscombinations
which,
even
ifofthesecond-orderderivatives
are positiveand constantacrossAs
analternativetothe labels
our foresightworkshopwithdefinition
positive-definitebowl),
anfor
occupationon the
wrong
side
the
The second, related, problem with theis thatitHessian
may lead(atoconvex
highlighting
feature5.6.
Trend
extrapolation
Todefinition
ameliorate
theproblemsofaveraging,weUS
employment
(for
the
years
we
have)
as a fromfunctionofofslightly
the parametricsecond.
This
fails
to
meet
our
needs.
The
first
problemisthat
theThesecond,second,related,
related,problem
problemwith
withthethedefinition
isthatthat
itmay
maydefinition
lead
tohighlighting
feature
The
is
it
lead
to
highlighting
featuremodel,
thesecond-orderderivatives
may
be
in
very
different
regionsofspace.
space,
areactively
harmful.
Asasimpleexample,forabivariate
quadratic
functionspace,
actively
harmful.
a simpleworkshop
example, participants,
for a bivariate wequadratic function
critical point(the minimiser,or theofacrossthe bottom
thebowl) fromwould oursee Asits
combinations which, even if the second-orderderivatives
are positive
andlocation
constantAs
an alternative
toofaretheforesight
use extrapolation of historical employment trends to give a mdefinition
makesnoaccommodationforlocation
inlabelsfeaturespace.
The
economics
definition
combinations which,which, even
even ifif thethe second-ordersecond-orderderivatives
arepositiveand constantconstant
acrosscombinations
derivatives
arepositiveandacrossThe
second,related,problemwiththedefinition
is
that
it
may
lead
to
highlighting
featurewith
positive-definite
Hessian(aconvexbowl),
anoccupation
on
the
wrong
side
of
the
with
positive-definite
Hessian(aconvexbowl),
anoccupation
on
the
wrong
side
of
the
As
an
alternative
tothelabelsfrom
ourforesight
workshopparticipants,
wewithincreasesineither
or
both
of
the
features.
Our
means
of
assessingspace, are actively harmful. As ademand
simpledecreased
example,forabivariate
quadratic
function
calculatetheforempirical
distributionofthederivative
years,
forof bothabsolute
employment
valuesandofhistoricalemployment
trendstogiveaofmoredata-driven
alternative
to predicting
demandin sharethe year 2030. Specifi
is a statement
about
theextrapolation
second-orderderivativeofthefunctionwithrespect
to constantthespace, areare actively
actively harmful.
harmful. As
As aa simple
simpleexample,for aausebivariate
quadratic
functionspace,
example,bivariate
quadratic
function
combinations
which,
even
ifthe
the
second-orderderivatives
areacrosscriticalpoint(theminimiser,or location
theandlocation
the tobottomthe bowl)
would
see its
criticalpoint(theminimiser,
orpositive
theof twothebottomof thebowl)
would
see its
complementarity,
however,will
identify
differing
importancesoffeatureswith positive-definite Hessian (a convexbowl), an occupation
onmorethe correctly
wrong
side
of theextrapolation
ofbehistorical
trendsgive
a moredata-driven
featuresbeing
positive;foranfunction,
asexample,
maybyemployment
ourflexiblenonwith positive-definite
positive-definite Hessian
Hessian (a(a convex
convexbowl),
an occupation
occupation
onharmful.
thearbitrary
wrong
side
of
the
alternative
touse
predicting
demandinlearned
theyear
Specifically,
we Ouruse
a values,
withbowl),
an
on
the
wrong
side
of
the
process
to regress
UK and US (for the years wespace,
activelyAs
aits
simplefora bivariate
quadratic
demanddecreasedwithincreases
in2030.
eitheror
bothoffeatures.
the features.
Our
means
ofGaussian
assessingdemanddecreasedwithincreasesineitheror bothoffunction
the
means
of assessingforeach–wouldparticularly,mean
andstandard
combinations
anybottom
pointinfeaturespace.
critical point (the minimiser, or thelocation ofatthe
offeature
the arebowl)
see its alternative
employment
projecting
forwardto2030.
topredicting
demand
in
the
year
2030.
Specifically,weuseaparametricmodel,
thesecond-orderderivatives
may
be
in
very
different
regionsofspace.
critical point
point (the(the minimiser,minimiser, oror thethe location
locationwithof thethebottomof thetheHessian
bowl)
would
see
its
criticalofbottom
of
bowl)
would
its
positive-definite(a complementarity,
convexbowl),however,an however,occupation
on
the
wrong
side
ofhave)
the
will
more
correctly
identify
the differing
importancesof features
Gaussian
process
tosee
regressUK
andwill
US
(for
the
years
wethe
employment
as a function of years, for both absolute employment v
complementarity,
more
correctly
identify
differing
importancesof features
demand decreased with increases in either or both of the features.
Our
means
of assessing
Theor
related,problemwiththedefinition
isat
that
mayregress
lead
tohighlighting
featureGaussian
process
to
UK
and
US
(forvalues
theitsyears
we have) employdemand decreased
decreased with
with increases
increases inindeviation.
eitherorsecond,
both
of
the
features.
Our
means
ofcombinations
assessingdemandeitherbothofthefeatures.
Ourofassessingcriticalpoint(theofminimiser,
or
the
of
bottom
the
bowl)
would
see
We
then
excludefeatureswhose
derivativehas
atitthe
any
point
inof
featurespace.
combinations
any
point
in
featurespace.
ment
asa meansfunctionof location
years,
forboth
absolute
employment
and share
employment values, projecting forward to 2030.
complementarity, however, will more correctly identify
the differing
importancesfeaturescombinations
which,
even withifimportances
theincreases
second-orderderivatives
arepositiveandbothconstant
acrosscomplementarity,however,
however,will
willmore
morecorrectly
correctly
identifythediffering
importancesoffeaturescomplementarity,
identifythedifferingoffeaturesment
asafunctionofyears,
forabsolute
employment
valuesandsharedemanddecreasedineither
or
both
of
the
features.
Our
means
of
assessingEachoccupationismodelled
separately
using separately
a Gaussian
combinations at any point in feature space.
employment
projecting
to 2030.
Each occupation
is modelledusing a Gaussia
space, complementarity,
are actively
harmful.
As willavalues,
simpleexample,forforward
a bivariate
quadratic
combinations at
at anyany point
point inin featurefeaturespace.
aspace.
standard
deviation
thatisemployment
in
the
top
97th
percentileof function
combinations
however,morecorrectly
identifythediffering
importancesof features
42
values,projecting
forwardto 2030.
Eachoccupationismodelled
separately
using
a GaussianProcess (gp).
A Matérn
Covariancewithparameter ν =3/2
= 3/2 is used fo
with positive-definite(a convexbowl),
an occupationon the wrong
sideof the Process).
A
MatérnCovariancewithparameter(
combinations atHessian
any point
in feature
space.
Each
occupationismodelled
separately
usingaGaussian
Process(gp).
thecritical
distribution.
rationaleis thattheinfluencepoint (the minimiser,orCovariancethe locationof thebottomof thewould
see
42 itsfor each gp; thisAThe
Matérnwithparameter
ν bowl)= of
3/2these
is42usedensures sufficiently smooth extrapolation without losing the
5.6.
Matérnwithparameter
ν extrapolation=
is usedfor
each gp;; this
this ensures sufficiently smooth
is
used
each
demand decreased withincreases
in A
eitheror bothCovarianceofextrapolation
the features.
OurmeansTrendofassessing42
ensuressufficiently
smooth
without
losingthe3/2structureinforthetrends.
To control the characteristic scale on which the G
features‘noisy’examplewithina particular
42 is veryhowever,42
complementarity,will(formoreensurescorrectlysufficiently
identify
the differing
importancesof featuressmoothextrapolation
without
losingthestructure
in
the
As
an
alternative
tothelabelsfrom
ourforesight
participants,
trends.
To
controlthecharacteristic
scale
on
which
the
Gaussian
Processvaries(aworkshop
lengthwe assigna priorthat centres the val
extrapolation
without
losingthe scale),structurein wethetrends.
42
combinations at any point in featuretrends.
space. To control
the characteristic scale on which the Gaussian Process

occupational grouping,demandstrongly
usecentres
extrapolation
of ofhistoricalemploymentscale
trendsto give
morepoints
data-driven
varies (a lengthscale),might
we assigna prior increasethatthe value
the length
suchthat adatafrom the last year in our employ
varieslengthfrom
scale),
assigna priorthat
centresthedemandvalue
the
To control
theofincharacteristic
scale
on numbers
which
Gaussianalternative
to predicting
thelengthyear 2030.employment
Specifically,
we usethescale suchthatdata(apoints
the welastyear
in
our employment
serieswill
influence
ina 2030.
The gp is modelled
for some
occupations
andstrongly
decrease
for
other
scale
such
that
data
points
from
the
last
year
in
our
employment
serieswill
pirical distributionof thederivative foreach feature
its mean
Gaussian
process
to regress
UK
US(foremployment
the years
wehave) employinfluence
employment
numbersin 2030.– particularly,The
gp is modelled
overthe logof andthenumbers.
Processvaries(alengthscale),
weassign
a
prior
that
42
influence
employment
numbers
in
2030.
The
gp
is
modelled
over
the
log
of
and standard
deviation.
We thenexcludewhose
derivative
has a of years, for both absoluteAbsolute
occupations)
and
hence
unlikely
to be afeatures
reliablebasis
ment
as a function
employment
values andshare are modelled directly fromtheemployment
numbers.
employment
numbers
thenumbers.
centresthevalue toofusing
the alength
scale
suchThethat
data
pointsstandard deviationthat is intheemployment
top 97th
percentileoftheemployment
distribution.
The
projecting
forward2030.
Absolute
employment
numbers
modelled
directly values,
fromhistorical
data
Gaussian
Process.
sharevaluesare slightly more
uponrationale
which istothat
designpolicy.
Here
wehaveisare
implicitly
taken
Absolute
employment
numbersaremodelled
directly
fromhistoricaldata
theinfluenceofthesefeaturesvery‘noisy’(e.g.
withinaoccupation
is modelled
separately
using
a the
Gaussian
Process(gp).
using a Gaussian Process. The share values areEachslightly
more frominvolved.
First,
wemodel
total workforce,
Twill, asinfluencea function of time and
the
last
year
in
our
employment
seriesusingaGaussian
Process.
The
sharevaluesareslightly
more
involved.
First,
particularoccupationalgrouping,
demandmightstrongly
increaseforsome
a conservative viewof theof skills
AtoMatérn
Covariancewithmodel
parameter ν to
= extrapolate
3/2 is usedthat
for forward
each gp;tothis
we modelthepotential
total workforce,
T , aspolicy
a functionof timeand use our2030. We then divide the exwemodel
workforce,
T ,divide
as sufficiently
a functionoftime andour model
employment
numbersin 2030.
The
gp ismodelled
occupations andstrongly
decreasefor theothertotal
occupations)
and
hence
unlikely
ensuressmoothextrapolation
without
losingthein thevaluesover
extrapolatethatforward
to
2030.
We
then
the
extrapolated
totaluse
workforcevaluesbystructuretheabsolute
to givethe
the share val
precisely
affect totargeted
occupationalgroups.
However,toextrapolatethatforward
to
2030.
We
then
dividetheextrapolated
total
to be a reliable
basis
upon
which
to
designpolicy.
Here
we
have
implicitly
trends.
To control
thecharacteristic
scale onwhich
the GaussianProcess to compute the probabi
workforce values by the absolute values to givethe share
value
predictions.
We
use these
trend extrapolations
log
of
the
employment
numbers.
workforcevaluesbytheabsolute
values
to
give
the
share
value
predictions.
takenaconservative
view
of
the
potentialofskillspolicytoprecisely
affectthere is a tradeoff We
in theselectionof the thresholdof thevaries(a length
scale),ofwe
centresvalue ofthe
thesamelengthuse these
trend extrapolations
to computeprobability
theassigntrend a prior thatbeing
higherthedemand,demand or lower demand be
We
use
trend
to selection
computeprobabilityof lastthe trendtargeted occupationalgroups.
However,there
is aextrapolations
tradeoffin suchthescale
thatbetween
datathepointsyear
our probability
employmentofseries
being higher
demand,thethesesamedemandor inlower
demand
2015from
and the2030.inThebeingwillhigh is taken as the total po
exclusion.
Any non-linearity
indemandwill
resulttheAbsolute
employment
numbers
are modelled
directly
being
higherdemand,thesamedemandorlower
demandbetween2015
andof the thresholdof
exclusion.
Anynon-linearity
indemandwill
resultintheinfluence
employment
gp 2isstandard
modelleddeviations,over the loglow
of is taken as the total neg
2030. The probability of being high is taken
as the total
positivenumbers
differencein 2030. The
above
2030.
The
probability
of
being
is
taken
as
the its
total
positivedifference
piricaldistribution
ofincreasing
the derivative
forstandard
each
feature– Noteparticularly,itsmean data
piricaldistribution
ofdeviations,thederivativeeach
feature–totalparticularly,
meanderivative
varying
overx,
therebyincreasing
thefordeviation.
, therebythederivative
varying
overtheemployment
numbers.
fromhistoricalusing2 astandard
Gaussian
Process.
The
share
above
2 standard
low
isstandard
taken
ashighthenegativedifference
below
deviations
and thesame
otherwise.
above
2 standard
deviations,lowistakenwhoseasthederivative
total
negativeand2 standard
deviation.
Wethenexcludefeaturesstandard
deviation.
We arethenexcludefeatureswhose
derivative
has
ahas aaredifference
that featuresandexcludedunder
thisschemeexcluded
onlyforconsideration
Absolute
employment
numbers
modelled
historical
data
below
standard
deviations
and
the
same
otherwise.
To directly
generatefrom
a dataset equivalent
to our workshop resp
deviation.
thatfeaturesexcludedunder
thisschemevaluesareslightly
more
involved.
First,
we
model
the
total
below
2
standard
deviations
and
the
same
otherwise.
standard
deviation
that
is
in
the
top
97th
percentileof
thedistribution.
The
standard
deviation
that
is
in
the
top
97th
percentileof
thedistribution.
The
by the Noteaveragederivative:therecommendationsof
thismetriccannotbeusing
a GaussianProcess.
The share valuesfrom
are theseslightly
more involved.
First,
To generate a data set equivalent to ourworkshop
response
we sample
extrapolated
probabilities.
A three-parameter mu
Toinfluence
generate
athesedata
set
equivalent
workshop
response
rationaleis thethat
theinfluence
ofaveragethesefeaturesisto
very‘noisy’(e.g.
withinaweaasample
rationaleisconsideration
that
ofthe
featuresvery‘noisy’(e.g.workforce,
withina T , astrusted for thesefeatures.
The
excludedfeaturesareusedasisnormal
inour
every
are excluded
only
for
by
we
model
the
total
as
functiontime
andusing
use ourmodel
function
of
time
and
use
our model
workforce,
from
these
extrapolated
probabilities.
A
three-parameter
multinomial
distribution ofis sampledtheprobabilities
of higher, same and
from
these
extrapolated
probabilities.
A
three-parameter
multinomial
distriparticularoccupationalgrouping,
demandmightstrongly
increaseforsomeparticularoccupationalgrouping,
demandmightstrongly
increaseforsomeother facet ofourmodelling,asinproducingoccupation-level
andaggregate
extrapolatethatforwardto 2030. We thendivide wethesample
extrapolated
total
bution is sampled using the probabilities of tohigher,same andlower.
For eachoccupation
12 values
and use these as the partici
derivative:
theof
recommendationsof
thismetriccannottoextrapolatethatforward
to
2030.
We
then
dividethebutionissampledusing
theprobabilities
ofhigher,same
andlower.
For
each
occupations
andstrongly
decreasefor
otheroccupations)
and
hence
unlikelyoccupations
andstrongly
decreasefor
otheroccupations)
and
hence
unlikelypredictions
demand.
workforceby theresponses.
absolute values toAgive
thevalue
shareofvalue
predictions.
occupation we sample 12 values and use theseas the values
participant
noise
0 is assigned
for every data point.
occupation
wesample12tovalues
and
use thesethese
as theparticipant
responses.
toa bea reliable
upon
which
designpolicy.
Here
we
have
implicitly
reliable
basis
upon
which
toevery
designpolicy.
Herewe have
implicitly
be trusted fortothesefeatures.
excludedfeatures
are
We
use
trend
extrapolations
to compute
thevalues
probability
theAbenoise
value
of
0 The
isbasisassignedfor
data
point.
extrapolated
total
workforcevaluesbyof theabsolute
valuesThese
are thenfedtrend
into the Ordinal
Regression gp
A noisevalue
of into0potentialisassigned
for
every
data
point.
avaluesconservative
view
of
the
potentialofskillspolicytoprecisely
affecttakentaken
a conservative
offed
the
of beingskillspolicytoprecisely
affecthigherdemand,theandsame
demand orcorresponding
lower demandprobabilities
between 2015
New occupations
These
areviewthentheOrdinalRegression
gp
model
the
areandcomputed as described in Se
used5.5.
as normal
intargeted
every
other
facet
of ourmodelling,
as
in
to
give
the
share
value
predictions.
Thesevalues
are
then
fed
into
the
OrdinalRegression
gp
model
and
the
occupationalgroups.
However,there
isatradeoffintheselection
targeted
occupationalgroups.
However,there
isatradeoffintheselection
The probabilityof being high is taken as the total positive difference
corresponding probabilities are computed as2030.
described
in Section 5.2.
We define a ofpotential
occupation
as aAny
combinationof computed
skills,knowledge
corresponding
probabilities
areaswilldescribed
in theSectionthenewthresholdofaggregate
exclusion.
Anynon-linearity
demandwill result
intheis5.2.
theofthresholdofand
exclusion.
non-linearity
in demand
resultinabove
2 instandard
deviations,lowtaken as the total negative difference
producingoccupation-level
predictionsofand abilitiesderivative
thatderivative
is likely
to oversee high
futuredemand,but
isthe
notstandard
associ6.otherwise.
RESULTS to compute thevarying
overx, thereby
increasing
deviation.
Note
We
use
these
trend
varyingx,
therebyincreasing
thestandard
deviation.
Note
below 2 standard deviations
andthesame extrapolations
demand.
6. featuresRESULTS
ated with anthat
existing
occupation.
To
forecast
where
newoccupations
might
that
featuresexcludedunder
thisschemeareToexcluded
onlyforconsideration
excludedunder
thisschemeareexcluded
onlyforconsideration
generate aprobability
data set equivalent
totrendourworkshop
response
we sample
6.
RESULTSofthebeingbelow
higherWe
present
our demand,
main
results,theandsametheir interpretation,
emerge, we simply
the posterior
the latent demandvari-metric
byoptimise
the averagederivative:
theforrecommendationsof
thiscannotby theaveragederivative:
themeanrecommendationsof thismetriccannotbe be A three-parameter multinomial distrifrom
these
extrapolatedpresentof
below
ourmain results,
and their
interpretation,forbothprobabilities.
theinUSand the UK economies. In presenting results for both count
m(x)
as
aWe
functionx.
Moreprecisely,
we
start
by
randomly
selectingtrustedforthesefeatures.
The
excludedfeatures
are
used
as
normalevery
5.5. able,
NEW
OCCUPATIONS
trustedfor thesefeatures.
The
excludedfeatures
are
used
as
normalin
every
demandorlower
demandbetween
2015
and
2030.
The
We
present
below
our
main
results,andtheirinterpretation,
for
both
the
US
butionis sampledusing
probabilities of higher, same and lower. For eachand the UK
economies.
In presenting
results
foroccupation-level
bothcountries,wethe
caution
50 current occupations
asour
our
startingof inhighoccupations.
ofmodelling,our
producingandaggregateotherotherfacet facetof
as in asproducingoccupation-level
andbothaggregateandthemodelling,
UK points
economies.
In demand
presenting
resultsforweuse
cautionoccupation
we sample
12countries,
valuesandtheseasistheparticipant
responses.
probability
of
being
high
taken
as
the
total
positive
We run localpredictions
optimisers
observing box-constraints,
44
predictionsof demand. BFGS,
of(limited-memory
demand.
We definea potential
newoccupation
as a combinationof of 0 is assigned for every data point.
A noise value
as per Byrd et al. (1995)) initialised at each of the 50
44 occupations. This will
difference
above
two standard
values arethen fed into
the Ordinal
Regressiondeviations.
gp model andLow
the is taken
44†that is likely toTheseskills,return
abilities
andknowledge
featuressee
50 localof occupations
m(x): points x in skills, knowledge and abilities
5.5.
5.5.optimisers
New New
occupations
corresponding probabilities
are computed
described inbelow
Section two
5.2. standard
as the total
negativeasdifference
that are associated with high demand. Many of these optimisers will behigh space
futuredemand,but
isanotassociatedwith
anexistingWedefinepotential
newoccupation
asacombinationofskills,knowledgeWedefineapotential
newoccupation
asacombinationofskills,knowledge(near-)identical, and others will be (near-)identical to existing occupations.
deviations.
†,0 seeandthat
is likely
seeeachhigh
futuredemand,not associandthatis new
likelytohighfuturedemand,but isbut
notis associ6.
RESULTSoccupation.
forecast
where
might
BeginningTo
withaabilities
singleabilities
such
optimiser,
xoccupations, wetoadd
successive
optimiser
ated withan existing
occupation.
To forecast
new occupations
ated withan existing
occupation.
To forecast
wherewhere
new occupations
mightmight

unless it is closer, in the 2-norm sense, than a preset threshold ( = 0.1) toemerge,we optimisetheweposteriormean
forposterior
themeanlatentToa datasetinterpretation,equivalent to
Wemeanpresent
below
ourgenerate
mainresults,
and theirfor ourboth workshop
the US
emerge,simplyoptimise
thefortheprolatentdemandvariemerge,
we simply
optimise
the posterior
forthelatentdemandvarieither: an existing
occupation,or;a previously
included
optimiser.
This†,i randomly
andtheeconomies.
In selecting
presenting
results fromfor bothcountries,
we caution
asfunction
a functionofx.. (local
Moreprecisely,
we{xUKstart
by ofrandomly
able, able,
m(x)
as
x.of
More
precisely,
we startbyselectingasafunction
More
precisely,
demandcedurevariable,
will returna listm(x)of ahotspots
ofofdemand
optimisers); ∀i},
response
we sampled
theseextrapolated
50 different
current
occupations
our functions.
startingofhighdemandoccupations.
50 current
occupations
as ourasstarting
pointspointsof highdemandlength that may
befor different
mean
Each
hotspot
canoccupations.
we start
by randomly
selecting
50 current
occupations
probabilities.
A three-parameter
multinomial distribution is
Welocalrun
local
optimisers
BFGS,
box-constraints,
Wethroughrun
optimisers
BFGS,
observingbox-constraints,
44
be interpreted
returningits (limited-memory
vector(limited-memoryof skills, knowledge
andobserving
abilitiesetal. (1995))
initialised
atclosesteachof
50 occupations.
will the probabilities of higher, same and lower.
aspoints
perasByrd
ethighal.of(1995))
initialised
at are
each
of
the
50the
occupations.
This Thiswill
as ourstartingofByrddemandoccupations.
Wevaluesx†,i , as
well
asper
the
list
occupations
which
to
it.
sampledusing†
50 local
optimisersof m(x):in skills,
knowledge
and abilities
returnreturn
50 local
optimisersof m(x):pointspointsx† in xskills,knowledge
and abilities
run local optimisation
algorithms(limited-memory
BFGS,
For
each
occupation
we sample 12 values and use thesethatassociatedare associatedwith demand.
high demand.
of these
optimisers
will bespacespacethat arewith high
ManyManyof these
optimisers
will be(near-)identical,
andByrdotherswill
be1995)(near-)identicalto existing(near-)identical,
andperotherswill etbe(near-)identicalto existing
occupations.
observing box-constraints,
as
al.,
as
theoccupations.
participant responses. A noise value of 0 is assigned
43†,0Beginninga singlesuch optimiser,
x add, weeach
add successive
each successive
optimiser
Beginningwith awith
singlesuch optimiser,x†,0 , we
optimiser
initialised at each
occupations.
Thissense,
will returnfor
data
unlessit50
is closer,
the 2-norm
thana preset
threshold( = to0.1) topoint.
unlessofittheis closer,
in thein2-norm
sense,
thana preset
threshold( every= 0.1)
either:an existing
occupation,
or; a previously
included
optimiser.
This proeither:an existing
occupation,
or; a previously
included
optimiser.
This pro†,i
Thesevaluesare then fed into the ordinal regression gp
cedurewill returnof hotspotsof demand
optimisers);of∀i}, ofcedurewill returna list aoflist
hotspotsof demand(local(local
optimisers){x†,i ; {x∀i},that bemay
be different
for different
functions.
Eachhotspot
canlengthlengththat maydifferentfor different
meanmeanfunctions.
Eachhotspot
can themodel
andcorresponding probabilities are computed
be interpreted
through
returningits vectorof skills,
knowledge
and abilities
be interpreted
through
returningits vectorof skills,
knowledge
and abilitiesx†,iwell, as aswell
theoflistof occupations
are closest
to it.
asit.describedin Section 5.2.
valuesvaluesx†,i , astheaslist
occupations
whichwhich
are closest
to

39

43

43

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 040

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

6. RESULTS
Here we present our main results and analysis for both the
US and the UK economies. In presenting results for both countries,
we caution that, as discussed in Section 3, cross-country comparison
is difficult, and is not the focus of this research. The first analysis we
present relates to the share of employment in 2030 by occupation,and our model’s outputs are the probabilities of that share
being greater than at present. (Findings for the absolute level of
employment are available on request from the authors.) Below, we
informally use ‘increased demand’, ‘future demand’ (and, at times,
simply ‘demand’) as a shorthand for ‘increased share of employment
in 2030’.

41

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 06.1. OCCUPATIONS
The primary outputs of our model are the probabilities of each occupation experiencing a rise in workforce share (that is,increased demand). These occupation-level results can then be aggregated to give the figures below. We distinguish thepercentage of the workforce in occupations predicted to see a rise in workforce share in 2030 with a ‘low probability’ (lessthan 30%), ‘medium probability’ (> 50%) and ‘high probability’ (> 70%). (These probability thresholds are the ones used inFrey and Osborne (2017)). That is, we calculate total employment that has probability of future increasing demand lying
above and below these three thresholds.
6.1.1. US
As per Section 4.2, our calculations are made at the finest level available for US occupations; that is, the six-digit US SOC
2010. The percentage of the US workforce as partitioned by the thresholds above is provided in Table 2.
Table 2: The fraction of the us workforce above and below varying thresholds for the probabilityof increasing demand
Number of Occupations Employment772
135 million

below 0.3
18.7%

above 0.5
43.2%

above 0.7
9.6%

Note: employment does not equal total US employment as it excludes any 6-digit SOC occupations for which O*NET data is missing. This consists of “All Other”
titles in the BLS data, representing occupations with a wide range of characteristics which do not fit into one of the detailed O*NET-SOC occupations. It also
consists of occupations for which O*NET is either in the process collecting data e.g. underwriters or has decided not to collect data e.g. legislators. Hence ouranalysis excludes occupations equivalent to roughly 2 percent of total US employment.

42

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

In Figure 3, we plot (following Frey and Osborne, 2017) the distribution of current US employment over its probabilityof future increased demand. We additionally distinguish this employment by an intermediate aggregation of the Major
Groups, as specified by the BLS 2010 SOC user guide (US Bureau of Labor Statistics, 2010).

Figure 3: The distribution of US employment according to its probability of future increased demand.
Note that the total area under all curves is equal to total US employment.

Management, Business and Financial Occupations

Office and Administrative Support Occupations

Computer, Engineering and Science Occupations

Farming, Fishing and Forestry Occupations

Education, Legal, Community Service, Arts,
and Media Occupations

Construction and Extraction Occupations

Healthcare Practitioners and TechnicalOccupations

Installation, Maintenance and Repair Occupations

Service Occupations
Sales and Related Occupations

Production Occupations
Transportation and Material Moving Occupations

Figure 3 reveals a large mass of the workforce in employment with highly uncertain demand prospects (that is, aprobability of experiencing higher workforce share of around 0.5). Note that this contrasts sharply with the U-shaped
distribution by probability of automation in Frey and Osborne (2017), where the workforce is overwhelmingly inoccupations at either a very high probability or a very low probability of automation. That our predictions are more
uncertain is a direct result of the distinctions of our methodology from previous work. Firstly, the expert labels we gather
in our foresight exercises (see Section 4.3) are explicitly clothed in uncertainty, whereas Frey and Osborne (2017) assume
that participants are certain about their labels. This humility is partially motivated by the difficulty of the task assigned
to our experts: balancing all the macro trends that might influence the future of work. Our allowance for our experts’
self-assessed degrees of confidence also recognises that many of the macro trends act at cross-purposes, leading to
uncertainty about which will dominate in the case of any one occupation. Secondly, we use 120 O*NET features, against
the nine used in Frey and Osborne (2017). This more detailed characterisation of occupations renders occupations lesssimilar to one another, and hence limits the confidence of our model in predicting for one occupation based on what has
been labelled for another.
Table 3 lists the minor occupation groups about which our model is most optimistic.

43

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 3: For the US, the minor occupation groups with the greatest probabilities of future
increased demand
For these occupations, we characterise the fraction of their current employment that has a probability ofincreased demand above two thresholds.
TITLE
Preschool, Primary, Secondary and Special
Education School Teachers
Animal Care and Service Workers
Lawyers, Judges and Related Workers
Post-secondary Teachers
Engineers
Personal Appearance Workers
Social Scientists and Related Workers
Counselors, Social Workers and Other
Community and Social Service Specialists
Librarians, Curators and Archivists
Entertainers and Performers, Sports
and Related Workers
Other Management Occupations
Media and Communication Workers
Operations Specialties Managers
Religious Workers
Other Teachers and Instructors
Other Personal Care and Service Workers
Construction Trades Workers
Business Operations Specialists
Physical Scientists
Other Sales And Related Workers
Architects, Surveyors and Cartographers
Other Education, Training,
And Library Occupations
Other Healthcare Support Occupations
Occupational Therapy And Physical Therapist
Assistants And Aides
Health Diagnosing And Treating Practitioners

44

EMPLOYMENT

>0.7

>0.5

4,050,880

97.8

100

185,780
672,580
1,328,890
1,610,470
504,640
239,170

93.7
90.7
83.0
70.0
69.0
65.6

100
98.1
100
100
100
92

1,715,190

54.0

100

253,800

51.8

62.9

483,450

46.4

96.1

2,185,950
542,570
1,663,790
68,530282,640
2,619,120
4,076,790
4,424,800
266,050
585,030
168,650

42.9
40.3
29.8
29.6
23.0
21.9
21.8
19.6
13.8
12.3
11.8

100
89.4
46.5
100
100
100
64.7
77.4
100
14.4
67.3

1,386,830

10.1

100

1,451,710

6.3

54.3

174,800

4.3

100

4,944,470

4.0

100

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

We derive a number of insights from Table 3, informed in part by our workshop discussions.
— Education and personal care occupations feature prominently in the rankings; however, healthcare occupations arelower than expected by trends such as ageing, potentially reflecting uncertainty over the trajectory of healthcare
policy and spending in the US or technical issues related to the composition of the training set (which, in practice,underrepresented healthcare occupations).
— Construction trade work, as a larger employer, is another beneficiary. It is supported by a number of trends, including
urbanisation, ageing and globalisation and is expected to be an important source of medium-skilled jobs in the future.
— Demand prospects can vary considerably for occupations that are otherwise very similar. For example, business
operations specialists – which typically need information management expertise – are set to grow as a share of theworkforce while neighbouring minor occupation groups in the SOC such as financial specialists (see Table 4) are
predicted to fall in share. Looking at the detailed occupation level, the results for business operations specialists
are driven by management analysts, training and development specialists, labour relations specialists, logisticians
and meeting, convention and event planners in particular – occupations that will conceivably benefit from thereorganisation of work and the workplace.
— Another niche anticipated to grow in workforce share is other sales and related workers and, within that in particular,sales engineers and real estate agents, notwithstanding the predicted decline in general sales occupations.

45THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 4: For the US, the minor occupation groups with the lowest probabilities of future
increased demand
We characterise for these occupations the fraction of their current employment that has a probability of increased
demand below two thresholds.
TITLE
Woodworkers
Printing Workers
Metal Workers and Plastic Workers
Financial Clerks
Other Production Occupations
Plant and System Operators
Assemblers and Fabricators
Communications Equipment Operators
Food Processing Workers
Forest, Conservation and Logging Workers,
Extraction Workers
Financial Specialists
Rail Transportation Workers
Cooks and Food Preparation Workers
Sales Representatives, Services
Retail Sales Workers
Other Construction and Related Workers
Water Transportation Workers
Vehicle and Mobile Equipment Mechanics,
Installers and Repairers
Librarians, Curators and Archivists
Material Recording, Scheduling, Dispatching,and Distributing Workers
Other Installation, Maintenance,
and Repair Occupations
Entertainment Attendants and Related Workers
Motor Vehicle Operators
Material Moving Workers
Other Office and Administrative Support Workers
Agricultural Workers
Construction Trades Workers
Other Healthcare Support Occupations
Health Technologists and Technicians
Information and Record Clerks
Secretaries and Administrative Assistants
Legal Support Workers
Electrical and Electronic Equipment Mechanics,
Installers and Repairers
Business Operations Specialists
Other Protective Service Workers
Grounds Maintenance Workers
Drafters, Engineering Technicians,
and Mapping Technicians
Life, Physical and Social Science Technicians
Other Transportation Workers

46

EMPLOYMENT
236,460
256,040
1,923,050
3,144,540
2,552,400
311,060
1,571,480
110,250
738,030
42,740561,550
253,530
2,607,770
117,460
3,132,040
8,799,240
393,710
77,270

<0. 3
100%
100%
98.7%
97.7%
96.9%
94.1%
92.2%
91.2%
89.1%
83.9%
81.5%
66.7%
66.3%
53.2%
49.0%
44.9%
39.8%
39.6%

<0.5
100%
100%
100%
100%
99.4%
100%
100%
100%
100%
100%
100%
100%
90.7%
100%
100%
47.6%
63.2%
100%

1,554,340

38.0%

99.2%

253,800

37.1%

37.1%

3,973,730

32.1

97.6%

2,776,890

28.4%

90%

524,310
3,797,540
4,473,640
3,723,230
383,890
4,076,790
1,451,710
2,909,230
5,336,050
3,680,630
344,220

25.2%
24.3%
20.9%
20.2%
17.9%
8.8%
7.5%
6.5%
6.4%
5.5%
5.1%

96.7%
100%
100%
100%
100%
35.3%
45.7%
56.3&
95%
100%
100%

585,280

3.1%

100%

4,424,800
1,524,350
959,960

2.9%
2.7%
2.5%

22.6%
89.8%
6.7%

680,790

2.2%

74.3%

359,460
305,320

1.8%
1%

82.7%
100%

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0— These results support the importance of future routine-biased technological change. Notably they anticipate the impactof automation encroaching on more cognitively advanced and complex occupations such as financial specialists.
— The predicted fall in retail sales workers and entertainment attendants, which between them account for a large
volume of employment, is consistent with an expansion in digitally provided goods and services.
— The transportation occupations represented may reflect a belief that driverless cars will disrupt the future workforce.
The rise of the sharing economy might reasonably be expected to lead to an increased demand for installation andreparation jobs, especially in areas such as transport, as cars and other assets are used more intensively, but thishypothesis is not supported here.

47

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

6.1.2. UK
Again, as per Section 4.2, our calculations are made at the finest level available for UK occupations; that is, the four-digit
UK SOC 2010. The percentage of the UK workforce as partitioned by the thresholds above is provided in Table 5.
Table 5: The fraction of the UK workforce above and below varying thresholds for the probability of increasing demand
Number of Occupations Employment
365
31,423,561

below 0.3
21.2%

above 0.5
51.8%

above 0.7
8.0%

Note: employment does not equal total UK employment as it excludes 4-digit SOC 2010 occupations corresponding to the 6-digit SOC occupations for which
O*NET data is missing. The latter consists of “All Other” titles in the BLS data, representing occupations with a wide range of characteristics which do not fit into
one of the detailed O*NET-SOC occupations. It also consists of occupations for which O*NET is either in the process collecting data e.g. underwriters or has
decided not to collect data e.g. legislators. Hence our analysis excludes occupations equivalent to roughly 1 percent of total UK employment.

In Figure 4, we describe (following Frey and Osborne, 2014) the distribution of current UK employment over its probabilityof future increased demand. We additionally disaggregate this employment by Major Group (that is, at one-digit level)in the UK SOC.
As with the US results, Figure 4 reveals that the model is notably more uncertain than in Frey and Osborne (2014). Notefrom Table 5 that, relative to the US, a larger fraction of the UK workforce is predicted to grow in share; however, a larger
fraction of the UK workforce also faces a high risk of declining in share.
Figure 4: The distribution of UK employment according to its probability of future increased demand.
Note that the total area under all curves is equal to total UK employment.

Management, Directors and Senior Officials
Professional Occupations
Associate Professional and Technical Occupations
Administrative and Secretarial Occupations
Skilled Trades Occupations
Caring, Leisure and Other Service Occupations
Sales and Customer Service Occupations
Process, Plant and Machine Operatives
Elementary Occupations48

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 6: For the UK, the minor occupation groups, or three-digit occupations, with the greatest
probabilities of future increased demand
For these occupations, we characterise the fraction of their current employment that has a probability ofincreased demand above two thresholds.
TITLE
Food Preparation and Hospitality Trades
Teaching and Educational Professionals
Sports and Fitness Occupations
Natural and Social Science Professionals
Managers and Proprietors in Hospitality and Leisure Services
Health and Social Services Managers and Directors
Artistic, Literary and Media Occupations
Public Services and Other Associate Professionals
Other Elementary Services Occupations
Therapy Professionals
Engineering Professionals
Media Professionals
Welfare Professionals
Electrical and Electronic Trades
Health Professionals

EMPLOYMENT
479,645
1,569,250
170,183
227,020
299,143
88,651
397,323
524,068
1,066,177
123,632
475,217
164,649
177,879
468,429545,874

>0.7
71.4%
57.3%
56.0%
55.0%
50.0%
44.1%
37.1%
31.9%
26.5%
22.6%
22.4%
19.3%
14.4%
12.4%
4.1%

>0.5
76.6%
94.5%
100.0%
100.0%
96.7%
100.0%
79.4%
100.0%
92.6%
100.0%
100.0%
100.0%
91.9%
100.0%
100.0%

We can glean several insights from Table 6.
—T he presence of health and education-related and other service occupations is consistent with Baumol’s cost disease
hypothesis – that is, lower productivity growth sectors should be expected to experience increasing workforce sharesfor a given increase in demand (Baumol et al., 2012).
—T he results also highlight the resilience of public sector occupations beyond health and education.21 It is unclear
whether this is because workshop participants believe these occupations will grow faster than others, or because they
believe that any change will be less volatile due to institutional factors, such as higher job security and unionisation,
enabling participants to have a higher degree of confidence in them. However understood, the finding is consistent withresearch by Acemoglu and Restrepo (2017a) showing that the public sector is among the few segments of the labour
market that holds up in areas affected by automation. Another possibility is that it reflects consumer concerns about
ethical, privacy and safety issues which, among other things, may affect the demand for regulation (World Economic
Forum (WEF), 2016; Jones, 2016).
—A ctivities that are not subject to international trade feature heavily in the list of occupation groups: food preparation,
elementary services, hospitality and leisure services and sports and fitness, and electrical and electronic trades.
This is also consistent with the high-tech multiplier effects identified by Moretti (2012) and Gregory et al. (2016).
—S ome of these occupations (food preparation and hospitality trades and other elementary service occupations) have
low skills requirements, but are associated with differentiated products which consumers value (Autor and Dorn, 2013).
They may therefore be ripe for job redesign to emphasise further product variety. Signs of this can be seen in the returnof artisanal employment: the remaking of goods and services such as barbering, coffee roasting, butchery, bartending,carpentry, textiles and ceramics, incorporating elements of craft-based technical skill which are higher-end – and more
expensive – than in the past. Workers also draw on deep cultural knowledge about what makes a good or service‘authentic’ and are able to communicate these values of ‘good’ taste to consumers. These markets benefit from theblurring of tastes between high- and low-brow culture and trends such as reshoring and the importance of localness
in production (Ocejo, 2017).
—T he craft phenomenon has been attributed, in part, to the consumption preferences of millennials. They may also
explain the support we find for occupations such as sports and fitness and therapy. Arguably, these activities represent
a broader definition of health – one which seeks to maintain people’s wellness through proper nutrition, exercise andmental health rather than than simply to respond to illness through the provision of acute, episodic care.
—C reative, digital, design and engineering occupations generally have a bright outlook. This can be seen from Table 6
but also Table B2 in the Appendix which shows the average probability of increased workforce share across minor
occupation groups. These activities are strongly complemented by digital technology but are also ones in which the
UK has a comparative advantage and benefits from trends such as urbanisation.
—A lthough ‘green’ occupations22 fall slightly below the 0.7 threshold, Table B2 shows that they are projected to growin share (with a mean probability of 0.62).
A number of other occupation groups have a high probability of experiencing a fall in workforce share, as detailedin Table 7.

49

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 7: For the UK, the minor occupation groups, or three-digit occupations, with the lowest
probabilities of future increased demand
We characterise for these occupations the fraction of their current employment that has a probability of increased
demand below two thresholds.
TITLE
Mobile Machine Drivers and Operatives
Elementary Administration Occupations
Elementary Sales Occupations
Elementary Storage Occupations
Customer Service Occupations
Customer Service Managers and Supervisors
Assemblers and Routine Operatives
Elementary Agricultural Occupations
Other Administrative Occupations
Printing Trades
Process Operatives
Metal Forming, Welding and Related Trades
Sales Assistants and Retail Cashiers
Animal Care and Control Services
Plant and Machine Operatives
Housekeeping and Related Services
Administrative Occupations: Finance
Other Skilled Trades
Administrative Occupations: Records
Secretarial and Related Occupations
Construction and Building Trades
Elementary Security Occupations
Elementary Process Plant Occupations
Managers and Proprietors in Other Services
Road Transport Drivers
Textiles and Garments Trades
Vehicle Trades
Elementary Cleaning Occupations
Other Drivers and Transport Operatives
Metal Machining, Fitting and Instrument-making Trades
Sales-related Occupations
Leisure and Travel Services
Building Finishing Trades
Agricultural and Related Trades
Business, Finance and Related Associate Professionals
Caring Personal Services

50

EMPLOYMENT
150,233
197,537
151,411
399,420
469,574
150,753
243,40992,209823,137
66,981
280,391
113,545
1,489,794
109,668
144,883
100,279
753,388111,153
396,852
673,395
837,300
280,115251,160
589,787
951,01155,975289,312
691,623
83,150299,920
166,780
193,102212,316
373,08688,927
1,327,903<0. 3
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
96.5%
94.7%
91.1%
90.5%
88.7%
84.3%
78.3%
77.1%
66.5%
59.2%
56.1%
44.6%
43.1%
39.9%
35.3%
34.7%
32.0%
31.2%
26.4%
22.0%
20.3%
20.4%
20.1%
18.5%
17.1%
14.6%
13.6%
7.3%
4.7%
1.8%

<0.5
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
77.1%
100.0%
100.0%
75.6%
79.2%
100.0%
100.0%
84.7%
100.0%
100.0%
58.0%
96.5%
100.0%
100.0%
100.0%
72.1%
100.0%
50.4%
27.0%
40.6%
7.3%
26.3%
89.6%

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 7 is suggestive of a number of interpretations.
— Technological advancements and globalisation may account for why many manufacturing production
occupations are predicted to see a fall in workforce share.
— The predicted decline in the workforce share of administrative, secretarial and, to some extent, sales
occupations is also consistent with routine-biased technological change. Customer service jobs which entailsocial interaction are perhaps harder to reconcile with this framework.
— Skilled trades occupations (SOC 2010 major group 5) exhibit a more heterogeneous pattern. Metal forming,
textiles, vehicle trades and, to an extent, construction and building trades are projected to fall in share; bycontrast, electrical and electronic trades, food preparation, building finishing and other skilled trades (forexample, glass and ceramics makers, decorators and finishers) fare much better (see Table B2). This suggeststhat there are likely to be pockets of opportunity for those lower down the skills ladder, depending on whichchoices are made.

51

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

6.2. SENSITIVITY ANALYSISFirstly, to test for the generalisability of our results, we perform a cross-validation exercise. In particular, we randomly
select a reduced training set of half the available data (corresponding to the labels for 15 occupations); the remaining
data form a test set. On the test set, we evaluate the receiver operating characteristic (ROC) curve (Murphy, 2012).
Given that we observe ternary, as opposed to binary, labels, the ROC curve is a surface (Waegeman et al., 2008).
An approximation is made to the volume under the surface (VUS), whereby the area under the curve (AUC) is calculated
separately for each ternary component. The volume under the ROC surface is taken to be the mean of the separate AUC
slices. As the workshop data or model prediction for an occupation is either an empirical or posterior distribution overternary labels respectively, samples are drawn and the mean VUS is calculated. Due to uncertainty in the distribution,a normalised VUS is calculated by dividing the test VUS by the ground truth VUS. We repeat this experiment for 50
random splits of the 30 occupations from the workshop set. Each experiment is composed of 10 training occupations
and 20 test occupations.
A high AUC/VUS (which ranges from 0.5 to 1) indicates that our model is able to reliably predict 15 occupations given a
distinct set of 15 occupations. This would suggest that the model is effective and that the training set is self-consistent.
It would also imply a degree of robustness of our results to the inclusion (or exclusion) of a small number of occupationsin that training set.
We also investigate how breaks in long-term trends as perceived by the workshop experts contribute to the findings above.
We do this by re-running the predictive model but using non-parametric extrapolation (as described in Section 5.6) of
occupational employment to label the occupations in place of the experts’ judgments.
As examples of extrapolated trends, Figure 5 plots the share of employment extrapolations for three UK occupations,
one each for higher, same and lower probability. Figure 6 shows the absolute employment extrapolations for three
US occupations, one each for higher, same and lower probability. The shaded areas give the 90% credibility intervalaround the mean.

52

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Figure 5: Examples of share employment number extrapolations for three UK occupations.
We show occupations with probability of higher, same and lower share in 2030.

Figure 6: Examples of absolute employment number extrapolations for three US occupations.
We show occupations with probability of higher, same and lower absolute employment numbers in 2030.

Note that, in what follows, all rankings of occupation groups from our model use the employment-weighted averageof probabilities of occupations within the group.
6.2.1. US
Firstly, Table 8 presents the results of our cross-validation exercise on US workshop data. The high VUS results
suggest that our model is robust, and that our results are not sensitive to small changes to the training set.

Table 8: The volume under the receiver operating surface from cross-validation of model on
US workshop data
Mean
0.949

Lower Quartile
0.943

Upper Quartile
0.956

Table 9 compares the probability of increased demand generated by trend extrapolation against that obtained from thejudgment of experts.

53THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 9: The percentage point (PP) differences (of trend extrapolation from our workshop-trained
predictions) in the probabilities of future demand at minor occupation group level
That is, each row is the probability produced by trend extrapolation minus the probability produced by theworkshop, multiplied by 100.

TITLE
Other Office and Administrative Support Workers
Legal Support Workers
Financial Clerks
Communications Equipment Operators
Financial Specialists
Information and Record Clerks
Secretaries and Administrative Assistants
Sales Representatives, Services
Entertainment Attendants and Related Workers
Retail Sales Workers
Rail Transportation Workers
Motor Vehicle Operators
Other Protective Service Workers
Material Recording, Scheduling, Dispatching and Distributing Worker
Sales Representatives, Wholesale and Manufacturing
Librarians, Curators and Archivists
Supervisors of Transportation and Material Moving Workers
Other Healthcare Support Occupations
Other Transportation Workers
Assemblers and Fabricators
Other Production Occupations
Lawyers, Judges and Related Workers
Baggage Porters, Bellhops and Concierges
Food and Beverage Serving Workers
Supervisors of Office and Administrative Support Workers
Printing Workers
Forest, Conservation and Logging Workers
Health Technologists and Technicians
Food Processing Workers
Supervisors of Sales Workers
Operations Specialties Managers
Supervisors of Personal Care and Service Workers
Cooks and Food Preparation Workers
Top Executives
Supervisors of Food Preparation and Serving Workers
Religious Workers
Other Sales and Related Workers
Other Construction and Related Workers
Media and Communication Workers
Plant and System Operators
Law Enforcement Workers
Business Operations Specialists
Textile, Apparel and Furnishings Workers
Counselors, Social Workers and Other Community and Social Service Specialists
Extraction Workers
Material Moving Workers
Other Management Occupations
Metal Workers and Plastic Worker
Supervisors of Production Workers
Water Transportation Workers
Fishing and Hunting Workers
Tour and Travel Guide
Nursing, Psychiatric and Home Health Aides
Other Teachers and Instructors
Health Diagnosing and Treating Practitioners

54

PP DIFFERENCE IN
PROBABILIT Y OF DEMAND
-52.2
-47.8
-47.0
-42.2
-39.6
-39.2
-38.7
-35.6
-34.4
-32.7
-27.7
-27.5
-27.4
-27.4
-26.3
-26.2
-26.1
-25.7
-25.4
-24.9
-23.6
-23.1
-22.8
-22.8
-22.4
-22.1
-21.6
-21.2
-21.1
-20.7
-20.4
-20.3
-20.2
-19.7
-19.6
-19.3
-19.2
-19.2
-18.9
-18.7
-18.5
-18.1
-18.1
-18.0
-17.9
-17.2
-17.1
-17.1
-16.8
-16.3
-15.9
-15.8
-15.6
-15.4
-15.4

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 9 (continued)

TITLE
Social Scientists and Related Workers
Supervisors of Protective Service Workers
Woodworkers
Vehicle and Mobile Equipment Mechanics, Installers and Repairers
Media and Communication Equipment Workers
Agricultural Workers
Other Food Preparation and Serving Related Workers
Supervisors of Construction and Extraction Workers
Occupational Therapy and Physical Therapist Assistants and Aides
Supervisors of Building and Grounds Cleaning and Maintenance Workers
Advertising, Marketing, Promotions, Public Relations and Sales Managers
Other Education, Training and Library Occupations
Life, Physical and Social Science TechniciansSupervisors of Farming, Fishing and Forestry Workers
Air Transportation Workers
Other Healthcare Practitioners and Technical Occupations
Preschool, Primary, Secondary and Special Education School Teachers
Other Personal Care and Service Workers
Fire Fighting and Prevention Workers
Entertainers and Performers, Sports and Related Workers
Funeral Service Workers
Other Installation, Maintenance and Repair Occupations
Construction Trades Workers
Drafters, Engineering Technicians and Mapping Technicians
Electrical and Electronic Equipment Mechanics, Installers and Repairers
Life Scientists
Art and Design Workers
Helpers, Construction Trades
Supervisors of Installation, Maintenance and Repair Workers
Architects, Surveyors and Cartographers
Mathematical Science Occupations
Computer Occupations
Grounds Maintenance Workers
Post-secondary Teachers
Physical Scientists
Personal Appearance Workers
Building Cleaning and Pest Control Workers
Animal Care and Service Workers
Engineers

55

PP DIFFERENCE IN
PROBABILIT Y OF DEMAND
-15.2
-15.2
-15.1
-14.8
-14.5
-14.4
-14.4
-14.2
-14.1
-13.9
-13.8
-13.7
-13.6
-13.5
-13.4
-13.3
-12.2
-11.9
-10.9
-10.8
-10.6
-10.4
-9.9
-8.4
-7.8
-7.3
-6.8
-6.5
-5.9
-5.8
-4.5
-4.2
-3.5
-3.3
-1.9
-1.0
4.2
4.4
8.8

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 10 further ranks occupation groups, using the same intermediate aggregation of major groups as described in
Section 6.1, by their probability of rising demand using our expert judgment training set. It compares these against rankings
based on independent quantitative forecasts for the year 2024 from the US BLS. (See Table A1 in the Appendix for majorgroup rankings).

Table 10: Relative rankings of intermediate aggregation of major occupation groups in the US by ourmodel, and by independent forecasts from the BLSR ANKING FROM EXPERT JUDGEMENTR ANKING FROM BLS PROJECTIONS 2014 –2024

Education, Legal, Community Service, Arts and Media Occupations

Healthcare Practitioners and Technical Occupations

Computer, Engineering and Science Occupations

Construction and Extraction Occupations

Healthcare Practitioners and Technical Occupations

Service Occupations

Management, Business and Financial Occupations

Computer, Engineering and Science Occupations

Construction and Extraction Occupations

Education, Legal, Community Service, Arts and Media Occupations

Service Occupations

Management, Business and Financial Occupations

Sales and Related Occupations

Installation, Maintenance and Repair Occupations

Farming, Fishing and Forestry Occupations

Sales and Related Occupations

Installation, Maintenance and Repair Occupations

Transportation and Material Moving Occupations

Office and Administrative Support Occupations

Office and Administrative Support Occupations

Transportation and Material Moving Occupations

Production Occupations

Production Occupations

Farming, Fishing and Forestry Occupations

Tables 9 and 10 reveal the following insights.
— Firstly, Table 9 makes it clear that expert judgment is considerably more pessimistic than trend extrapolation for
most minor occupation groups. (This can also be seen by inspecting the distribution of current employment over its
probability of future demand under trend extrapolation in Figure A1 in the Appendix and comparing it with Figure 3).
The divergence is largest for routine cognitive, as opposed to routine manual, occupations.
— We see considerable divergence across the three outlooks (our workshop-informed model, our trend extrapolation,and the BLS projection). This is arguably most marked for occupations in legal, architecture and engineering and arts,
design, entertainment, and sports and media. There is seemingly greater agreement, however, over occupations
projected to decline in workforce share.

56

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

6.2.2. UK
We begin by detailing in Table 11 the VUS scores from our cross-validation exercise on UK workshop data.
Again, the high results give some reassurance both that the model is robust and that the results are not sensitive
to minor alterations to the choice of occupations in the training set.

Table 11: The volume under the receiver operating surface from cross-validation of model on UK
workshop data

Mean
0.946

57

Lower Quartile
0.938

Upper Quartile
0.954

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 12 compares the probability of rising workforce share generated by trend extrapolation versus the judgmentof experts.
As with the US, although to a lesser degree, it shows that expert judgement is more pessimistic than trend extrapolation for
most minor occupation groups. (This can also be seen in the distribution of current employment over its probabilityof future demand under trend extrapolation in Figure A2 in the Appendix and comparing it with Figure 4). Expert judgement
is particularly more pessimistic about management and supervision roles. There are few cases where the experts are more
optimistic than trend extrapolation – construction and related occupations (not supervisors) are the main exception.
Table 12: The percentage point (PP) differences (of trend extrapolation from our workshop-trained
predictions) in the probabilities of future demand at minor occupation group level.
Each row is the probability produced by trend extrapolation minus the probability produced by the workshop,
multiplied by 100.

OCCUPATION TITLE
Construction and Building Trades Supervisors
Skilled Metal, Electrical and Electronic Trades Supervisors
Production Managers and Directors
Managers and Directors in Transport and Logistics
Mobile Machine Drivers and Operatives
Sales Supervisors
Managers and Directors in Retail and Wholesale
Senior Officers in Protective Services
Administrative Occupations: Office Managers and Supervisors
Elementary Storage Occupations
Conservation and Environmental Associate Professionals
Agricultural and Related Trades
Other Drivers and Transport Operatives
Chief Executives and Senior Officials
Managers and Proprietors in Agriculture-related Services
Elementary Sales Occupations
Financial institution Managers and Directors
Sales, Marketing and Related Associate Professionals
Public Services and Other Associate Professionals
Cleaning and Housekeeping Managers and Supervisors
Functional Managers and Directors
Administrative Occupations: FinanceManagers and Proprietors in Hospitality and Leisure Services
Elementary Administration Occupations
Conservation and Environment Professionals
Food Preparation and Hospitality Trades
Legal Professionals
Process Operatives
Managers and Proprietors in Health and Care Services
Managers and Proprietors in Other Services
Architects, Town Planners and Surveyors
Transport Associate Professionals
Protective Service Occupations
Elementary Cleaning Occupations
Sales Assistants and Retail Cashiers
Health and Social Services Managers and Directors
Business, Research and Administrative Professionals
Administrative Occupations: Records
Other Administrative Occupations
Elementary Agricultural Occupations
Elementary Process Plant Occupations
Hairdressers and Related Services
Sales-related Occupations
Business, Finance and Related Associate Professionals
Engineering Professionals
Assemblers and Routine Operatives
Animal Care and Control Services
Quality and Regulatory Professionals
Housekeeping and Related Services

58

PP DIFFERENCE IN
PROBABILIT Y OF DEMAND
-38.2
-31.9
-31.6
-27.8
25.1
-24.6
-24.6
-24.5
-24.0
-23.1
-21.4
-20.9
-20.9
-20.4
-20.4
-19.9
-19.1
-19.0
-19.0
-18.4
-17.9
-17.3
-17.3
-16.9
-16.6
-16.4
-16.2
-16.2
-15.6
-15.4
-15.0
-15.0
-15.0
-14.8
-14.7
-14.6
-14.2
-13.7
-13.4
-12.7
-12.6
-12.0
-11.7
-11.6
-10.4
-10.2
-9.9
-9.7
-9.6

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 12 (continued)

OCCUPATION TITLE
Road Transport Drivers
Other Elementary Services Occupations
Customer Service Occupations
Administrative Occupations: Government and Related Organisations
Elementary Security Occupations
Secretarial and Related Occupations
Research and Development Managers
Customer Service Managers and Supervisors
Leisure and Travel Services
Natural and Social Science Professionals
Legal Associate Professionals
Librarians and Related Professionals
Welfare and Housing Associate Professionals
Sports and Fitness Occupations
Information Technology and Telecommunications Professionals
Textiles and Garments Trades
Caring Personal Services
Printing Trades
Welfare Professionals
Artistic, Literary and Media Occupations
Media Professionals
Plant and Machine Operatives
Information Technology Technicians
Teaching and Educational Professionals
Health Associate Professionals
Nursing and Midwifery Professionals
Design Occupations
Science, Engineering and Production Technicians
Other Skilled Trades
Health Professionals
Construction Operatives
Vehicle Trades
Childcare and Related Personal Services
Metal Machining, Fitting and Instrument-making Trades
Metal Forming, Welding and Related Trades
Building Finishing Trades
Therapy Professionals
Elementary Construction Occupations
Construction and Building Trades
Draughtspersons and Related Architectural Technicians
Electrical and Electronic Trades

59

PP DIFFERENCE IN
PROBABILIT Y OF DEMAND
-9.6
-9.5
-9.5
-8.9
-8.1
-7.9
-7.5
-7.2
-5.9
-5.8
-5.6
-4.8
-4.6
-4.3
-3.8
-2.7
-2.2
-1.9
-1.8
-1.7
-1.5
-1.0
-1.0
-0.7
0.0
0.0
0.4
0.7
0.9
2.0
3.2
4.4
5.2
5.6
7.2
9.2
10.8
14.6
15.6
20.3
22.6

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 13 ranks major occupation groups by their probability of rising demand using our expert judgement training set,and compares against rankings based on independent quantitative forecasts for the year 2024 from the former UK
Commission for Employment and Skills (UKCES).
We comment on the rankings provided by our model and the UKCES, as described in Table 13.
— The rankings are broadly consistent across the two outlooks. However, there are important exceptions when
we drill down and examine minor group or three-digit occupations (see Table A2 in the Appendix for sub-majorgroup rankings).
— One difference is that the experts are a little less confident about rising demand for customer service occupations.
This places them closer to the results found in Frey and Osborne (2017).
— Our approach also reflects less optimism about occupations relating to personal care. While this may seem difficult toreconcile with an ageing population (care workers and home carers being the largest occupation within this group),
it may also reflect uncertainty over the fact that people are working longer and leading healthier, more independent
lives – a theme and tension which ran through the UK workshop discussions. Another difference is that our expertinformed model is more optimistic than UKCES on the prospects for certain medium-skilled jobs for example, skilled
metal, electrical and electronic trades and textiles, printing and other skilled trades – areas where apprenticeships
have traditionally been an important route to entry.
— Our model is also more optimistic about culture, media and sports, teaching and educational professionals andscience, research, engineering and technology occupations than UKCES.
Table 13: Relative rankings of major occupation groups in the UK by our model, trained on expert
judgement, and by independent forecasts from the UKCESR ANKING FROM EXPERT JUDGEMENTR ANKING FROM UKCESPROJECTIONS 2014 –2024

Professional Occupations

Managers, Directors and Senior Officials

Managers, Directors and Senior Officials

Professional Occupations

Associate Professional and Technical Occupations

Caring, Leisure and Other Service Occupations

Caring, Leisure and Other Service Occupations

Associate Professional and Technical Occupations

Skilled Trades Occupations

Elementary Occupations

Elementary Occupations

Sales and Customer Service Occupations

Administrative and Secretarial Occupations

Skilled Trades Occupations

Sales and Customer Service Occupations

Process, Plant and Machine Operatives

Process, Plant and Machine Operatives

Administrative and Secretarial Occupations

60

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 06.3. SKILLS
We now describe the findings of our study on the relationships between O*NET variables (which we referto as ‘features’ and occasionally informally refer to as ‘skills’) and future demand. Note that our methodology(see Section 5.4) provides employment-weighted results: as such, all the results that follow are robust to outlying
occupations with small employment.
We use two measures of the importance of features to future demand: the Pearson correlation coefficient(Section 5.4.1) and the average derivative (Section 5.4.2). In interpreting the values of these measures, note firstly
that the correlation coefficient lies between −1 and 1. The average derivative is calculated by considering thederivative of an unobservable real-valued function linked to demand. It is dimensionless; an average derivative’s
magnitude is significant only relative to that of another average derivative. For either measure, positive values areassociated with features whose increase is expected to increase demand, and negative values with features
whose increase is expected to decrease demand.
As described in Section 5.4.2, we exclude especially noisy features from consideration under the averagederivative measure.
For both the UK and US, these variables are:— Perceptual Speed (Abilities);— Building and Construction (Knowledge);— Food Production (Knowledge);— Production and Processing (Knowledge);— Control Precision (Abilities);— Biology (Knowledge); and— Fine Arts (Knowledge).
Note that the excluded variables are predominately knowledge features. The significance of knowledge,
perhaps more than other features differs considerably across occupations. It is hence not surprising thatthese features are more likely to be less reliable than others under the average derivative metric.

61

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

6.3.1. US
Table 14: A ranking, by Pearson correlation, of the importance of O*NET variables to future demandfor US occupations
R ANK
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

O*NET VARIABLE
Learning Strategies
Psychology
Instructing
Social PerceptivenessSociology and Anthropology
Education and Training
Coordination
Originality
Fluency of Ideas
Active Learning
Therapy and Counseling
Philosophy and Theology
Speaking
Service Orientation
Active Listening
Complex Problem Solving
Oral Expression
Communications and Media
Speech Clarity
Judgment and Decision-making
English Language Knowledge
Monitoring
Deductive Reasoning
Oral Comprehension
Critical Thinking
Systems Evaluation
History and Archeology
Inductive Reasoning
Persuasion Skills
Speech Recognition
Science
Negotiation
Management of Personnel Resources
Systems Analysis
Problem Sensitivity
Writing
Operations Analysis
Administration and Management
Biology
Fine Arts
Reading Comprehension
Memorization
Time Management
Foreign Language
Written Expression
Medicine and Dentistry
Technology Design
Personnel and Human Resources
Written Comprehension
Information Ordering
Time Sharing
Geography
Law and Government
Customer and Personal Service
Category Flexibility
Speed of ClosureManagement of Material Resources
Chemistry

62

CL ASS
SkillsKnowledge
SkillsSkillsKnowledge
Knowledge
SkillsAbilities
Abilities
SkillsKnowledge
Knowledge
SkillsSkillsSkillsSkillsAbilities
Knowledge
Abilities
SkillsKnowledge
SkillsAbilities
Abilities
SkillsSkillsKnowledge
Abilities
SkillsAbilities
SkillsSkillsSkillsSkillsAbilities
SkillsSkillsKnowledge
Knowledge
Knowledge
SkillsAbilities
SkillsKnowledge
Abilities
Knowledge
SkillsKnowledge
Abilities
Abilities
Abilities
Knowledge
Knowledge
Knowledge
Abilities
Abilities
SkillsKnowledge

PEARSON CORREL ATION
0.632
0.613
0.609
0.605
0.603
0.602
0.571
0.570
0.562
0.534
0.531
0.526
0.514
0.511
0.507
0.502
0.493
0.491
0.489
0.482
0.474
0.470
0.468
0.465
0.462
0.461
0.452
0.448
0.443
0.436
0.431
0.419
0.418
0.415
0.414
0.407
0.395
0.388
0.388
0.385
0.374
0.372
0.360
0.359
0.351
0.348
0.345
0.344
0.341
0.328
0.316
0.310
0.309
0.291
0.284
0.268
0.262
0.192

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 14 (continued)

R ANK
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120

O*NET VARIABLE
Public Safety and Security
Telecommunications
Computers and Electronics
Management of Financial Resources
DesignFlexibility of Closure
Physics
Programming
Engineering and Technology
Visualization
Sales and Marketing
Far Vision
Explosive Strength
Building and Construction
Selective Attention
Clerical Knowledge
Auditory Attention
Economics and Accounting
Mathematical Reasoning
Near Vision
Mathematics – Skills
Transportation
Mathematics – Knowledge
Number Facility
Dynamic Flexibility
Quality Control Analysis
Stamina
Food Production
Trunk Strength
Gross Body Coordination
Gross Body Equilibrium
Visual Color Discrimination
Installation
Dynamic Strength
Troubleshooting
Extent Flexibility
Equipment Selection
Static Strength
Hearing Sensitivity
Mechanical
Perceptual Speed
Depth Perception
Speed of Limb Movement
Spatial Orientation
Sound Localization
Multilimb Coordination
Production and Processing
Operation Monitoring
Night Vision
Peripheral Vision
Glare Sensitivity
RepairingResponse Orientation
Equipment Maintenance
Arm-Hand Steadiness
Reaction TimeOperation and Control
Finger Dexterity
Manual Dexterity
Rate Control
Wrist-Finger Speed
Control Precision
63

CL ASS
Knowledge
Knowledge
Knowledge
SkillsKnowledge
Abilities
Knowledge
SkillsKnowledge
Abilities
Knowledge
Abilities
Abilities
Knowledge
Abilities
Knowledge
Abilities
Knowledge
Abilities
Abilities
SkillsKnowledge
Knowledge
Abilities
Abilities
SkillsAbilities
Knowledge
Abilities
Abilities
Abilities
Abilities
SkillsAbilities
SkillsAbilities
SkillsAbilities
Abilities
Knowledge
Abilities
Abilities
Abilities
Abilities
Abilities
Abilities
Knowledge
SkillsAbilities
Abilities
Abilities
SkillsAbilities
SkillsAbilities
Abilities
SkillsAbilities
Abilities
Abilities
Abilities
Abilities

PEARSON CORREL ATION
0.189
0.189
0.186
0.160
0.146
0.133
0.126
0.122
0.121
0.120
0.118
0.105
0.099
0.078
0.069
0.047
0.036
0.036
0.035
0.016
0.008
0.004
-0.006-0.022-0.023-0.028-0.033-0.034-0.039-0.059-0.063-0.081-0.082-0.111-0.114-0.129-0.141-0.142-0.142-0.152-0.168-0.173-0.185-0.198-0.207-0.219-0.239-0.242-0.244-0.246-0.247-0.259-0.282-0.284-0.297-0.322-0.326-0.354-0.365-0.394-0.423-0.466

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Figure 7: The 10 most important O*NET features as ranked by Pearson correlation for the US

1. Learning Strategies

2. Psychology

3. Instructing

4. Social Perceptiveness5. Sociology andAnthropology

6. Education and Training

7. Coordination

8. Originality

9. Fluency of Ideas

10. Active Learning

64

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Figure 8: The 10 least important O*NET features as ranked by Pearson correlation for the US

1. Control Precision

2. Wrist Finger Speed

3. Rate Control

4. Manual Dexterity

5. Finger Dexterity

6. Operation and Control

7. Reaction Time

8. Arm Hand Steadiness

9. Equipment Maintenance

10. Response Orientation

65THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 14 ranks, by Pearson correlation coefficient, all 120 variables according to their association with a rising occupation
workforce share (in declining order of strength). Figures 7 and 8 plot, respectively, the top and bottom ten O*NETvariables as ranked by Pearson correlation. Table C1 in the Appendix also provides aggregate rankingsfor the average derivative.
— The results confirm the importance of 21st century skills in the US, with a particularly strong emphasis on
interpersonal competencies. This is underscored by the presence of skills such as Instructing, Social Perceptivenessand Coordination, and related knowledge domains such as Psychology and Sociology and Anthropology.
— This is consistent with the literature on the increasing importance of social skills – recall the fact that between 1980
and 2012, jobs with high social skills requirements grew by nearly 10 percentage points as a share of the US labour
force (Deming, 2015). There are good reasons to think that these trends will continue – not only as organisations
seek to reduce the costs of coordination but also as they negotiate the cultural context in which globalisation andthe spread of digital technology are taking place (Tett, 2017). Take over a variety of interventions targeted at different
stages of the life cycle have proven successful in fostering social skills. The evidence base is largest on the success ofearly programmes. Workplace-based internships and apprenticeships, however, also have a good track record due tothe need to learn informal or tacit knowledge and skills, and the bonds of attachment that can be formed between asupervisor and an apprentice (Kautz et al., 2014).
— The results also emphasise the importance of higher-order cognitive skills such as Originality and Fluency of Ideas.
Learning Strategies and Active Learning – the ability of students to set goals, ask relevant questions, get feedback
as they learn and apply that knowledge meaningfully in a variety of contexts – also feature prominently.
— Progress towards developing these skills as part of the formal education system has been slow due to difficulties
in understanding how they arise and develop over time and how they can be embedded in the curriculum and formal
assessments. Nonetheless, a number of initiatives have shown promise and are beginning to shape domestic andinternational policy dialogue (Schunk and Zimmerman, 2007; Lucas et al., 2013; OECD, 2016a). Strengthening theaffective aspects of education and a lifelong learning habit, especially among boys and students from disadvantaged
backgrounds who tend to have lower levels of motivation, is a further area of interest for policymakers. The research
literature shows that teachers can play an important role – both in raising student expectations and in rewarding theprocess of learning – for instance, in giving students opportunities to share the results of their work with others or
explain why what they learned was valuable to them, though they are unlikely to be sufficient in the absence of otherpolicies to promote educational excellence and equity (Covington and Müeller, 2001; Diamond et al., 2004; Weinstein,
2002; Hampden-Thompson and Bennett, 2013; OECD, 2017).
— In addition to knowledge fields related to social skills, English language, History and Archeology, Administrationand Management and Biology are all associated strongly with occupations predicted to see a rise in workforce
share, reminding us that the future workforce will have generic knowledge as well as skills requirements.
— Psychomotor and physical abilities are strongly associated with occupations with a falling workforce share.
Interestingly, this includes abilities such as Finger Dexterity and Manual Dexterity, which Frey and Osborne (2017)
identified as key bottlenecks to automation. Trade and offshoring offer a potential explanation for why these skills
might fall in demand – consistent with workshop participants having considered a broad range of trends. The mainfeature that makes a job potentially offshorable or vulnerable to import competition hinges less on a task’s routinenessor non-routineness than the cost advantages of producing overseas, and the marginal importance of face-to-faceinteractions in the production process.
— The correlations for variables associated with a rising occupation workforce share, are in general, stronger than those
associated with a falling occupation workforce share. This is perhaps not surprising: all things being equal, an increasein the value of any O*NET variable for an occupation makes it more skilled, and might broadly be expected to resultin greater demand (even if there are other reasons why the occupation will experience a fall in demand). It is also
fortunate: our core emphasis is on informing skills policy, which has a natural focus on those skills most strongly
linked to growing demand.

66

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

6.3.2. UK
Table 15: A ranking, by Pearson correlation, of the importance of O*NET variables to future
demand for UK occupations
R ANK
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

O*NET VARIABLE
Judgment and Decision-Making
Fluency of Ideas
Active Learning
Learning Strategies
Originality Abilities
Systems Evaluation
Deductive Reasoning
Complex Problem Solving
Systems Analysis
Monitoring
Critical Thinking
Instructing
Education and Training
Management of Personnel Resources
Coordination
Inductive Reasoning
Problem Sensitivity
Information Ordering
Active Listening
Administration and Management
Social PerceptivenessOperations Analysis
Psychology
Time Management
Oral Comprehension
Memorization
Speaking
Oral Expression
Category Flexibility
Sociology and Anthropology
Speed of Closure
Science
Writing
English Language
Written Comprehension
Personnel and Human Resources
Persuasion
Reading Comprehension
Communications and MediaManagement of Material Resources
Time Sharing
Speech Recognition
Negotiation
Speech Clarity
Written Expression
Technology Design
History and Archeology
Flexibility of Closure
BiologyManagement of Financial Resources
Fine Arts
Philosophy and Theology
Therapy and Counseling
Mathematics – Skills
Mathematical Reasoning
Service Orientation
Law and Government
Programming
Number Facility

67

CL ASS
SkillsAbilities
SkillsSkillsAbilities
SkillsAbilities
SkillsSkillsSkillsSkillsSkillsKnowledge
SkillsSkillsAbilities
Abilities
Abilities
SkillsKnowledge
SkillsSkillsSkillsSkillsAbilities
Abilities
SkillsAbilities
Abilities
Knowledge
Abilities
SkillsSkillsKnowledge
Abilities
Knowledge
SkillsSkillsKnowledge
SkillsAbilities
Abilities
SkillsAbilities
Abilities
SkillsKnowledge
Abilities
Knowledge
SkillsKnowledge
Knowledge
Knowledge
SkillsAbilities
SkillsKnowledge
SkillsAbilities

PEARSON CORREL ATION
0.752
0.732
0.721
0.715
0.710
0.703
0.672
0.671
0.670
0.663
0.658
0.657
0.636
0.635
0.620
0.611
0.601
0.575
0.571
0.559
0.556
0.555
0.551
0.550
0.545
0.530
0.528
0.526
0.520
0.516
0.504
0.502
0.492
0.491
0.481
0.476
0.467
0.465
0.463
0.462
0.452
0.446
0.443
0.440
0.439
0.420
0.415
0.412
0.408
0.402
0.394
0.393
0.383
0.382
0.380
0.379
0.357
0.337
0.335

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 15: Continued
R ANK
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120

O*NET VARIABLE
Computers and Electronics
Geography
Economics and Accounting
Mathematics – Knowledge
Visualization
Medicine and Dentistry
Near Vision
Chemistry
DesignSales and Marketing
Customer and Personal Service
Foreign Language
Physics
Selective Attention
Perceptual Speed
Engineering and Technology
Telecommunications
Food Production
Quality Control Analysis
Far Vision
Auditory Attention
Public Safety and Security
Building and Construction
Visual Color Discrimination
Clerical
Production and Processing
Hearing Sensitivity
Installation
Transportation
MechanicalTroubleshooting
Explosive Strength
Operation Monitoring
Equipment Selection
Gros Body Equilibrium
Depth Perception
Wrist-Finger Speed
Trunk Strength
Gross Body Coordination
StaminaFinger Dexterity
Repairing
Arm-Hand SteadinessSpatial Orientation
Extent Flexibility
Dynamic Strength
Equipment Maintenance
Dynamic Flexibility
Speed of Limb Movement
Response Orientation
Reaction Time
Glare Sensitivity
Sound Localization
Operation and Control
Night Vision
Multilimb Coordination
Peripheral Vision
Rate Control
Manual Dexterity
Static Strength
Control Precision

68

CL ASS
Knowledge
Knowledge
Knowledge
Knowledge
Abilities
Knowledge
Abilities
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Knowledge
Abilities
Abilities
Knowledge
Knowledge
Knowledge
SkillsAbilities
Abilities
Knowledge
Knowledge
Abilities
Knowledge
Knowledge
Abilities
SkillsKnowledge
Knowledge
SkillsAbilities
SkillsSkillsAbilities
Abilities
Abilities
Abilities
Abilities
Abilities
Abilities
SkillsAbilities
Abilities
Abilities
Abilities
SkillsAbilities
Abilities
Abilities
Abilities
Abilities
Abilities
SkillsAbilities
Abilities
Abilities
Abilities
Abilities
Abilities
Abilities

PEARSON CORREL ATION
0.334
0.319
0.306
0.304
0.300
0.289
0.258
0.248
0.246
0.242
0.236
0.235
0.227
0.224
0.216
0.212
0.207
0.175
0.143
0.141
0.099
0.083
0.069
0.058
0.057
0.039
-0.043-0.055-0.091-0.102-0.112-0.113-0.117-0.118-0.123-0.124-0.127-0.141-0.145-0.175-0.184-0.199-0.209-0.212-0.221-0.221-0.222-0.224-0.225-0.231-0.236-0.243-0.245-0.249-0.260-0.266-0.268-0.271-0.314-0.317-0.383

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Figure 9: The 10 most important O*NET features as ranked by Pearson correlation for the UK

1. Judgment and
Decision Making

2. Fluency of Ideas

3. Active Learning

4. Learning Strategies

5. Originality

6. Systems Evaluation

7. Deductive Reasoning

8. Complex Problem
Solving

9. Systems Analysis

10. Monitoring69

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Figure 10: The 10 least important O*NET features as ranked by Pearson correlation for the UK

1. Control Precision

2. Static Strength

3. Manual Dexterity

4. Rate Control

5. Peripheral Vision

6. Multilimb Coordination

7. Night Vision

8. Operation and Control

9. Sound Localisation

10. Glare Sensitivity

70

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 15 ranks, by Pearson correlation coefficient, all 120 variables according to their association with a rising
occupation workforce share (in declining order of strength). Figures 9 and 10 plot the top and bottom 10 O*NETvariables as ranked by Pearson correlation. Table C2 in the Appendix also provides aggregate rankings for the
average derivative.
— As in the US, the results confirm the importance of 21st century skills, though now with a particularly strongemphasis on cognitive competencies and learning strategies.
— Interestingly, systems skills, relatively underexplored in the literature, all feature in the top 10. Systems
thinking emphasises the ability to recognise and understand socio-technical systems – their interconnections
and feedback effects – and choose appropriate actions in light of them. It marks a shift from more reductionist
and mechanistic forms of analysis and lends itself to pedagogical approaches such as game design and casemethod with evidence that it can contribute to interdisciplinary learning (Tekinbas et al., 2014; Capra and Luisi,
2014; Arnold and Wade, 2015).
— The combined importance of these skills and interpersonal skills supports the view that the demandfor collaborative problem-solving skills may experience higher growth in the future (Nesta, 2017).
— Knowledge fields such as English language, Administration and Management, Sociology and Anthropology and
Education and Training are all associated strongly with occupations predicted to see a rise in workforce share,
again highlighting the importance of generic knowledge requirements.
— Like the US results, psychomotor and physical abilities, including Manual Dexterity and Finger Dexterity arestrongly associated with occupations with a falling workforce share.
— Also as in the US results, correlations for skills, knowledge areas and abilities associated with a rising occupation
workforce share are stronger than those associated with a falling occupation workforce share.

71

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

6.4. Relative importance of skills, abilities and knowledge
We now provide a comparison of the overall relative importance of skills, abilities and knowledge areas as captured in
O*NET. Figure 11 shows the results for a) the US and b) the UK. All figures feature on the horizontal axis the rank of all
O*NET features: the further to the right, the less important it is for demand. This importance is assessed using linear(Pearson correlation coefficient) and non-linear (average derivative) metrics.

Figure 11: The relative importance of skills, abilities and knowledge as assessed by Pearson correlation
coefficient(a) US Results

Abilities
Knowledge
SkillsAverage Correlation Coeﬃcient (weighted)

Proportion

0.8

0.4

0.6

0.2

0.4

0.0

0.2

-0.2

0.0

0

20

40

60

Ranking
O*NET
Ranking

80

100

Correlation Coeﬃcient (weighted)

0.6

1.0

-0.4

(b) UK Results
0.8
Abilities
Knowledge
SkillsAverage Correlation Coeﬃcient (weighted)

0.8

0.6

Proportion

0.4
0.6
0.2
0.4
0.0

0.2

0.0

72

-0.2

0

20

40

60

Ranking
O*NET
Ranking

80

100

-0.4

Correlation Coeﬃcient (weighted)

1.0

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Figure 12: The relative importance of skills, abilities and knowledge as assessed by average derivative.
(a) US Results
4
Abilities
Knowledge
SkillsAverage Gradient (weighted)

0.8

3

Proportion

2

0.6

1

0

0.4

Gradient (weighted)

1.0

-1
0.2
-2

0.0

0

20

40

60

Ranking
O*NET
Ranking

80

100

-3
120

(b) UK Results
2.0

1.0

Abilities
Knowledge
SkillsAverage Gradient (weighted)

1.0

0.5

0.6

0.0
0.4

-0.5

Gradient (weighted)

Proportion

0.8

1.5

-1.0
0.2
-1.5

0.0

0

20

40

60

Ranking
O*NET
Ranking

80

100

-0.2
120

In all the plots, it can be seen that abilities are broadly less important (weighted to the right). Perhaps the most interesting
insight, however, is that the non-linear metric (the average derivative) gives knowledge features more weight to the left.
That is, a non-linear measure ranks knowledge features more highly. Recall that the benefit of a non-linear metric is that itallows us to discover complementarities: skills that are only important if other skills take high values. As such, this result iscompatible with the intuition that knowledge features (such as Psychology and Foreign Language) are mostly valuable ascomplements. We find a similar pattern in a large number of Science, Technology, Engineering, and Mathematics (STEM) features (such as Science, Technology Design and Operations Analysis). They are not equally useful to all occupations (aswould be required to be assessed as important for our linear metric), but find use only for some specialised occupations
that have high values for other skills.

73

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 06.5. SKILL COMPLEMENTARITIESRecall from Section 5.4.2, that we say that an O*NET feature a is complementary to an O*NET feature b if increasinga increases demand for occupations with large values of b. Conversely, a is anti-complementary to an O*NETfeature b if increasing a decreases demand for occupations with large values of b. For each sub-major occupationgroup, for both the US and UK, we rank the three features that would most drive: (i) a rising workforce share fora unit increase in the feature (complementary features), and (ii) a falling workforce share for a unit increase in thefeature (anti-complementary features). As such, we are also able to establish which features are most importantfor different regions of the skills space. We represent the position of an occupation group in skills space by listing
its highest ranked features, which we term its current features.
6.5.1. US
We describe in Table 16 complementary and anti-complementary O*NET variables for US sub-major occupation groups.
Take Production Occupations, for example, which Figure 3 shows are predicted to see a fall in workforce share.
According to the O*NET data, Production and Processing, Near Vision and Problem Sensitivity are the threemost important or emblematic features for this occupation group. Our model predicts that increasing Customer
and Personal Service, Technology Design and Installation in the presence of these features will have the greatest
positive impact on future demand, while increasing Rate Control, Operation and Control and Quality Control
Analysis) will have the greatest negative impact. Looking across all occupation groups, Customer and Personal
Service and Technology Design (along with Science) appear to be the O*NET features most likely to appear
as positive complementary variables.
Of course, any reconfiguration of skills, abilities and knowledge requirements entails an evolution of the occupation.
Or, put differently, occupations may need to be redesigned in order to make effective use of skills andknowledge complements – and the results presented in Table 16 could be a useful guide in this exercise.

74

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 16: For US major occupation groups, ranked lists (the highest ranked, top, and lowest ranked,
bottom) of O*NET features that are: currently high-valued, complementary and anti-complementary

TITLE

CURRENTFEATURESCOMPLEMENTARYFEATURES

ANTI- COMPLEMENTARYFEATURES

11-0000

Management Occupations

Administration andManagement
Oral Expression
Oral Comprehension

Science
Philosophy and Theology
Sociology and Anthropology

Economics and Accounting
Medicine and Dentistry
Mathematics – Knowledge

13-0000

Business and Financial Operations
Occupations

Oral Comprehension
Written Comprehension
English Language

Science
Philosophy and Theology
Technology Design

Medicine and Dentistry
Economics and Accounting
Mathematics – Knowledge

15-0000

Computer andMathematical Occupations

Computers and Electronics
Critical Thinking
Problem Sensitivity

ScienceTechnology Design
DesignEconomics and Accounting
Design Rate Control
Medicine and Dentistry

17-0000

Architecture andEngineering Occupations

Engineering and Technology
Mathematics – Knowledge
DesignScienceTechnology Design
Operations Analysis

Operation and Control
Rate Control
Medicine and Dentistry

19-0000

Life, Physical and Social
Science Occupations

Written Comprehension
Oral Comprehension
Reading Comprehension

ScienceTechnology Design
Operations Analysis

Medicine and Dentistry
Rate Control
Operation and Control

21-0000

Community and Social
Service Occupations

Psychology
Therapy and Counseling
Active Listening

Operations Analysis
Science
Philosophy and Theology

Medicine and Dentistry
Reaction Time
Therapy and Counseling23-0000

Legal Occupations

Oral Expression
Law and Government
English Language

Science
Sociology and Anthropology
Philosophy and Theology

Economics and Accounting
Medicine and Dentistry
Mathematics – Knowledge

25-0000

Education, Training
and Library Occupations

Education and Training
Oral Expression Operations
English Language

Science
Analysis
Technology Design

Mathematics – Knowledge
Medicine and Dentistry
Economics and Accounting

27-0000

Arts, Design, Entertainment,
Sports, And Media Occupations

English Language
Oral Expression
Oral Comprehension

Science
Philosophy and Theology
Education and Training

Economics and Accounting
Rate Control
Mathematics – Knowledge

29-0000

Healthcare Practitioners
and Technical Occupations

Medicine and Dentistry
Technology Design
Customer and Personal Service Science
Oral Comprehension
Operations Analysis

31-0000

Healthcare Support Occupations

Customer and Personal Service Customer and Personal Service Rate Control
Oral Comprehension
Technology Design
Mathematics – Knowledge
English Language
Science
Computers and Electronics

33-0000

Protective Service Occupations

Public Safety and Security
Problem Sensitivity
English Language

SOC

75

Medicine and Dentistry
Rate Control
Operation and Control

Customer and Personal Service Rate Control
Technology Design
Operation and Control
Science Quality
Control Analysis

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 16: Continued

SOC

TITLE

CURRENTFEATURESCOMPLEMENTARYFEATURES

ANTI- COMPLEMENTARYFEATURES

35-0000

Food Preparation and Serving
Related Occupations

Customer and Personal
Service
Oral Comprehension
Oral Expression

Customer and Personal
Service
Static Strength
Service Orientation

Rate Control
Computers and Electronics
Operation and Control

37-0000

Building and Grounds

Customer and Personal
Service
Trunk Strength
English Language

Customer and Personal
Service
Static Strength
Service Orientation

Rate Control
Wrist-Finger Speed
Operation and Control

39-0000

Personal Care andService Occupations

Customer and Personal
Service
Oral Expression
Oral Comprehension

Customer and Personal
Service
Static Strength
Technology Design

Rate Control
Mathematics - Knowledge
Operation and Control

41-0000

Sales and Related Occupations

Customer and Personal
Service
Oral Expression
Oral Comprehension

Customer and Personal
Service
ScienceTechnology Design

Economics and Accounting
Mathematics – Knowledge
Rate Control

43-0000

Office and Administrative
Support Occupations

Customer and Personal
Service
Oral Comprehension
Oral Expression

Service Orientation
Customer and Personal
ServiceTechnology Design

Mathematics – Knowledge
Economics and Accounting
Rate Control

45-0000

Farming, Fishingand Forestry Occupations

Static Strength
Arm-Hand Steadiness
Multilimb Coordination

Customer and Personal
Service
Static Strength
Service Orientation

Rate Control
Wrist-Finger Speed
Operation and Control

49-0000

Installation, Maintenanceand Repair Occupations

Mechanical
Near Vision
Repairing

Installation
Customer and Personal
ServiceTechnology Design

Operation and Control
Rate Control
Quality Control Analysis

51-0000

Production Occupations

Production and Processing
Near Vision
Problem Sensitivity

Customer and Personal
ServiceTechnology Design
Installation

Rate Control
Operation and Control
Quality Control Analysis

53-0000

Transportation and Material
Moving Occupations

Multilimb Coordination
Near Vision
Control Precision

Customer and Personal
Service
Static Strength
Installation

Quality Control Analysis
Wrist-Finger Speed
Rate Control

76

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

6.5.2. UK
Table 17 provides the equivalent complementary and anti-complementary O*NET features for UK sub-major
occupation groups.
Here, take Customer Service and Sales Occupations, which according to the model are also likely to see a fall in future
demand. According to the O*NET data, Customer and Personal Service, Oral Comprehension and Oral Expression are
the three most important features for this group. Our model predicts that increasing Judgment and Decision-Making,Fluency of Ideas and Originality in the presence of these features will have the greatest positive impact on future
demand, while increasing Public Safety and Security, Law and Government, Operation and Control, Engineering andTechnology and Reading Comprehension will have the greatest negative impact.
Judgment and Decision-Making, Fluency of Ideas, Originality and Operations Analysis appear regularly across all
occupation groups and present an illustrative case where changes in organisational design may be required to takeadvantage of them. Without enhanced delegation of formal authority and employee involvement in decision-making
and the generation of ideas, the productivity gains from investing in these skills are likely to be modest. This is
supported by a large body of evidence on the role and complementarity of high-performance work practices (BenNer and Jones, 1995; Kruse et al., 2004; Lazear and Shaw, 2007). Decentralisation and the organisational structuresand skills which support them appear particularly important for firms closer to the technological frontier, firms in
more varied environments and younger firms (Acemoglu et al., 2007). The UK ranks around the OECD average interms of the share of jobs which employ these practices, though the level of task discretion, defined as employees’
immediate control over their work tasks has fallen sharply since the 1990s (Inanc et al., 2013; OECD, 2016d).
Science – defined here as the capacity to use scientific rules and methods to solve problems – is another crosscutting complement. We find it to be a complementary feature not only among prototypical high-skill occupations
but also Secretarial and Administrative occupations. Mastery of medium-skill science is already indispensable toa number of paraprofessional positions – from radiology technicians to electricians (Rothwell, 2013; Grinis, 2017).
Our results suggest that clerical occupations may be ripe for a similar transformation. One can envisage scenarios
where this is possible – for example, the credit controller occupation that increasingly applies aspects of data
science to help investigate the credit worthiness of borrowers and collect arrears of payment.
Table 17: For UK sub-major occupation groups, ranked lists (the highest ranked, top, andlowest ranked, bottom) of O*NET features that are: currently high-valued, complementary andanti-complementary

TITLE

CURRENTFEATURESCOMPLEMENTARYFEATURES

ANTI- COMPLEMENTARYFEATURES

1100

Corporate Managers
and Directors

Administration andManagement
Oral Expression
Oral Comprehension

Science
Operations Analysis
Originality

Public Safety and Security
Law and Government
Sound Localization

1200

Other Managers
and Proprietors

Customer and Personal Service Science
Oral Expression
Operations Analysis
Oral Comprehension
Originality

Public Safety and Security
Engineering and Technology
Law and Government

2100

Science, Research, Engineeringand Technology Professionals

Computers and Electronics
Written Comprehension
Oral Comprehension

Public Safety and Security
Sound Localization
Law and Government

2200

Health Professionals

Medicine and Dentistry
Operations Analysis
Customer and Personal Service Originality
Problem Sensitivity
Fluency of Ideas

SOC

77

Science
Operations Analysis
Fluency of Ideas

Public Safety and Security
Law and Government
Customer and Personal Service

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table 17: Continued

SOC

TITLE

CURRENT FEATURESCOMPLEMENTARYFEATURES

ANTI- COMPLEMENTARYFEATURES

2300

Teaching and Educational
Professionals

Education and Training
Oral Expression
English Language

Science
Operations Analysis
Originality

Public Safety and Security
Law and Government
Customer and Personal Service

2400

English Language
Business, Media and Public Service
Oral Expression
Professionals
Oral Comprehension

Science
Operations Analysis
Originality

Public Safety and Security
Law and Government
Customer and Personal Service

3100

Oral Comprehension
Science, Engineering and
Near Vision
Technology Associate Professionals
Computers and Electronics

Science
Operations Analysis
Fluency of Ideas

Public Safety and Security
Sound Localization
Law and Government

3200

Health and Social Care Associate
Professionals

Customer and Personal Service Science
Public Safety and Security
Oral Expression
Operations Analysis
Law and Government
Oral Comprehension
Judgment and Decision-Making Customer and Personal Service

3300

Protective Service Occupations

Public Safety and Security
Law and Government
English Language

Gross Body Equilibrium
Science
Gross Body Coordination

3400

Culture, Media and Sports
Occupations

English Language
Oral Expression
Oral Comprehension

Fluency of Ideas
Public Safety and Security
Originality
Law and Government
Judgment and Decision-Making Sound Localization

3500

Business and Public Service
Associate Professionals

English Language
Oral Comprehension
Oral Expression

Science
Operations Analysis
Fluency of Ideas

4100

Administrative Occupations

Customer and Personal Service Judgment and Decision-Making Public Safety and Security
Oral Expression
ScienceEngineering and Technology
ClericalFluency of Ideas
Mechanical

4200

Secretarial and
Related Occupations

ClericalEnglish Language
Oral Comprehension

Judgment and Decision-Making Public Safety and Security
ScienceEngineering and Technology
Fluency of Ideas
Operation and Control

5100

Skilled Agricultural
and Related Trades

Oral Comprehension
Oral Expression
Active Listening

Gross Body Equilibrium
Operations Analysis
Fluency of Ideas

78

Public Safety and Security
Law and Government
Engineering and Technology

Public Safety and Security
Law and Government
Customer and Personal Service

Sound Localization
Engineering and Technology
MechanicalT HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table 17: Continued

SOC

TITLE

CURRENT FEATURESCOMPLEMENTARYFEATURES

ANTI- COMPLEMENTARYFEATURES

5300

Skilled Construction
and Building Trades

Building and Construction
Manual Dexterity
Arm-Hand Steadiness

Gross Body Equilibrium
Sales and Marketing
Gross Body Coordination

Repairing
MechanicalComputers and Electronics

5400

Textiles, Printing
and Other Skilled Trades

Problem Sensitivity
Production and Processing
Oral Comprehension

Sales and Marketing
Gross Body Equilibrium
Operations Analysis

Engineering and Technology
Sound Localization
Public Safety and Security

6100

Caring Personal
Service Occupations

Oral Expression
Judgment and Decision-Making Public Safety and Security
Oral Comprehension
Fluency of Ideas
Engineering and Technology
Customer and Personal Service Originality
Operation and Control

6200

Leisure, Travel and Related
Personal Service Occupations

Customer and Personal Service Fluency of Ideas
Public Safety and Security
Oral Expression
Judgment and Decision-Making Engineering and Technology
Oral Comprehension
Originality
Reading Comprehension

7100

Sales Occupations

Customer and Personal Service Judgment and Decision-Making Public Safety and Security
Oral Comprehension
Fluency of Ideas
Engineering and Technology
Oral Expression
Originality
Reading Comprehension

7200

Customer Service Occupations

Customer and Personal Service Judgment and Decision-Making Public Safety and Security
Oral Expression
Fluency of Ideas
Law and Government
Oral Comprehension
Originality
Operation and Control

8100

Process, Plant
and Machine Operatives

Near Vision
Production and Processing
Problem Sensitivity

8200

Transport and Mobile Machine
Drivers And Operatives

Transportation
Gross Body Equilibrium
Far Vision
Sales and Marketing
Customer and Personal Service Gross Body Coordination

Engineering and Technology
MechanicalComputers and Electronics

9100

Elementary Trades andRelated Occupations

Manual Dexterity
Multilimb Coordination
Static Strength

RepairingComputers and Electronics
Reading Comprehension

9200

Elementary Administrationand Service Occupations

Customer and Personal Service Gross Body Equilibrium
Repairing
Oral Comprehension
Gross Body Coordination
Reading Comprehension
Oral Expression
Judgment and Decision-Making Computers and Electronics

79

Gross Body Equilibrium
Repairing
Sales and MarketingSound Localization
Judgment and Decision-Making Mechanical

Gross Body Equilibrium
Sales and Marketing
Economics and Accounting

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 06.6. New Occupations
It is also useful to think about the occupations which may emerge in the future in response to the drivers of labour
market change we consider in our study. These occupations correspond to high-demand locations in the feature
space and are not associated with existing occupations. The model allows us to identify a hypothetical occupation
which is ’almost certain’ (see Section 5.5 for a formal interpretation) to experience an increase in workforce shareand the combination of skills, abilities and knowledge features most associated with it.

6.6.1. US
For the US, the model identifies four hypothetical occupations which would almost certainly experience a rise in demand.
Table 18 ranks the top five O*NET features in declining order of feature value for each hypothetical occupation. (S) denotesthat the variable is an O*NET skills feature, (K) is an O*NET knowledge feature and (A) is an O*NET abilities feature.
We can understand something about these hypothetical occupations by looking at existing occupations that areclosest to them (in declining order of proximity), as described in Figure 12. Of the 20 occupations presented here, 11 aredefined by O*NET as enjoying a Bright Outlook and/or are expected to benefit from the growth of the green economy.23

Table 18: The four new occupations found by our model for the US, as described by their top
five O*NET features.

NEW
OCCUPATIONSFEATURE R ANK
1ST

2ND

3RD

4TH

5TH

1

Customer andPersonal Service (K)StaticStrength (A)Service Orientation(S)

Biology (K)Arm-Hand
Steadiness (A)2

Building andConstruction (K)Customer andPersonal Service (K)StaticStrength (A)Manual Dexterity (A)Arm-Hand
Steadiness (A)3Engineering andTechnology (K)Science (S)WrittenComprehension (S)CriticalThinking (S)Design (K)4

Education andTraining (K)Oral Comprehension(S)

SocialPerceptiveness (S)WrittenComprehension (S)Reading
Comprehension(S)

80

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0The employment time-series for four of these closest occupations is also plotted in Figure 13 for historical context.
Figure 12: ‘Closest’ occupations to hypothetical new high demand occupations for the US

Non-farm Animal Caretakers

Massage Therapists

Personal Care Aides

Maids and Housekeeping Cleaners

Home Health Aides

Construction Laborers

Painters, Construction

Helpers - Pipelayers, Plumbers,

Roofers

Drywall and Ceiling Tile Installers

and MaintenancePipefitters and Steamfitters

Electrical Engineers

Chemical Engineers

Nuclear Engineers

Engineering Teachers,

Aerospace Engineers

Post-secondary

Directors, Religious Activities
and Education

Training and Development
Specialists

81

Communications Teachers
Post-secondaryHome Economics Teachers
Post-secondary

Library Science Teachers
Post-secondaryTHE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Figure 13: Time-series of employment for four of the ‘closest’ occupations to new US occupations,as tabulated in figure 12(a) Non-farm Animal Caretakers(b) Construction Laborers(c) Aerospace Engineers(d) Directors Religious Activities and Education

These results provide another rejoinder to the view that jobs in the middle of the education and earnings distributions
will disappear in the future. Two of the four occupations can be plausibly viewed as middle-skill jobs. Our first hypotheticaloccupation – which has similarities to social care work – is particularly interesting. On the one hand, it is a textbook
example of a sector where the availability of low-skilled employees, the budgetary squeeze on government programmes– Medicare and Medicaid account for roughly 70% of all long-term care dollars – and the legacy of the politics of raceand gender have combined to create low-paid jobs with low status and precarious employment conditions (Institute ofMedicine, 2008; Duffy et al., 2015). However, the model points to bright demand prospects for care work which, requiresa mixture of tasks from across the skill spectrum, including formal knowledge and training which, in principle, would
support wage growth and job quality. Finally it is worth noting the extent to which interpersonal competencies featureacross these hypothetical occupations.

82

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

6.6.2. UK
For the UK, two new occupations are identified by the model. Table 19 shows the top five features of these occupations,in declining order of importance.

Table 19: The two new occupations found by our model for the UK, as described by their top five
O*NET features.

NEW

FEATURE R ANKOCCUPATIONS

1ST

2ND

3RD

4TH

5TH

1

Fine Arts (K)Originality (A)Design (K)Fluency of Ideas (A)Visualization (K)2Originality (A)Fluency of Ideas (A)Judgment and
Decision-Making (S)Active Learning (S)Oral Expression(A)

Again, we can learn something about these occupations by looking at existing occupations that are closest to them(in declining order of proximity). These closest occupations are described in Figure 14, and historical employment for twoof these are plotted in Figure 15. One of the occupations has high levels of creativity and combines traditional craft andtech-based skills; the other fits hospitality and sales occupations and requires originality, flexibility and management skills.

83THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Figure 14: The ‘closest’ occupations to hypothetical new high demand occupations for the UK.

Artists

Merchandisers andWindow Dressers

Florists

Tailors and Dressmakers

Photographers, audio-visual andbroadcasting equipment operators

Catering and bar managers

Publicans and managers
of licensed premises

Managers and directors in retail
and wholesale

Sales Supervisors

Chefs

Figure 15: Time-series of employment for two of the ‘closest’ occupations to new UK occupations,as tabulated in Figure 14(a) Artists

84(b) Catering and bar managers

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 085

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 086

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

7. LIMITATIONS OF THE ANALYSIS— While we believe that our research design has many
appealing features that increase the usefulness ofthe findings compared with previous studies, we
acknowledge that there are important limitations. First,
directional predictions may frustrate policymakers whoseek more detailed information on which to base theirdecisions. Experimenting with a larger number of labels
to achieve a finer distinction between different rates
of change might have value in this respect, though weneed to be mindful of the aforementioned cognitive
limits associated with prediction over a 15-year horizon.
— A second limitation is that we only assess theimplications for employment of structural shifts in
employer demand. In practice, however, employment
opportunities will arise when workers retire fromthe workforce (or leave for other reasons) and need
to be replaced. Indeed, replacement needs are
expected to provide significantly more job openings
than employment growth over the next decade(UK Commission for Employment and Skills, 2014;
US Bureau of Labor Statistics, 2016). Even those
occupations where employer demand is otherwise
expected to fall may still offer attractive careerprospects. As such, incorporating estimates of theage structure of the workforce to predict replacement
needs would complement our approach andassessment of future employment opportunities.
— Third, it would be useful to understand more about thecharacteristics of jobs that are anticipated to become
more important in coming years. Recent concerns thatfalling unemployment and the development of new
business models have not been accompanied by thecreation of ‘good’ jobs give this issue particular traction
and timeliness (Taylor, 2017). Earnings levels, career
progression, working environment, job security, voice inorganisational decisions, among other things, provide
objective and measurable benchmarks against whichto assess job quality (OECD, 2016c). And, in addition to
the value that jobs have for the people who hold them,
they also have potential side-effects, both positive andnegative, on the rest of society which a full assessment
would take into account.

87— Fourth, in future development of our analysis would
be to integrate trends more explicitly into the labelling
process – for instance, to choose occupations thatare most representative of the trends and likely to
encourage reflection about them (as opposed to,or possibly combined with, using the active learningalgorithm). Alternatively, workshop participants could
be asked to rank the trends by their importance or
relevance when labelling occupations as an input forour model, which would help sharpen interpretationof the results. Finally, it would be useful to explore
how estimates vary across countries (Hausmann etal., 2014; Beramendi et al., 2015). In the presenceof cross-country variations in resources, institutions
and technologies, even identical structural trends arelikely to be channelled in different ways, which in turngive rise to different labour market disruptions andopportunities.

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 088

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

8. CONCLUSIONSIn this report, we have presented a novel mixed-methods
approach for predicting the demand for skills, which we haveapplied to the US and UK economies in 2030. Specifically, we
have: generated directional predictions for occupation growthfor groups in the Standard Occupation Classification of the US
and UK; identified which skills, knowledge types and abilities
will, by association, most likely experience growth and decline;
and determined, at the occupational level, which humancapital investments will most likely boost future demand in
2030. We have grounded our analysis explicitly in the manysources of structural change likely to impact on US and UK
labour markets over this horizon.
Although there has been an explosion of reports looking
into the future of employment, we believe ours is themost comprehensive and methodologically ambitious andhas results that are actionable. It also contains the mostsophisticated treatment of uncertainty; this is important as
our finding that most jobs are associated with high levels of
uncertainty about future demand reminds us that the future
for most occupations is far from inevitable. Lastly, we makegreat efforts to benchmark our predictions – to tease out
our specific contributions to this important conversation– by comparing them with alternative forecasts. While thisnecessarily falls short of evaluation, we believe it further
separates our study from other recent exercises of this nature.
Our approach takes the labour market judgments, gleaned
at foresight workshops, of experts in a wide range of domain
areas where structural change is expected to impact on
employment, and combines it with a state of- the-art machinelearning algorithm. The model follows earlier studies in making
use of the US Department of Labor’s O*NET survey of morethan 1,000 occupations which asks detailed questions of every
occupation on skills, knowledge and abilities and the tasks andactivities which make up jobs. However, we depart from thesestudies in making use of all 120 skills, knowledge and abilities
features in the database.
We find that 9.6% (8.0%) of the current US [UK] workforce
is in an occupation that will very likely experience an increase
in workforce share and 18.7% (21.2%) in an occupation thatwill very likely experience a fall. These estimates imply thata large mass of the workforce in both the US and UK havehighly uncertain demand prospects (that is, a probability ofexperiencing a higher workforce share of close to 50:50).
This finding is significant. It contrasts sharply, for example, with
the U-shaped distribution in the studies of future automationof Frey and Osborne (2014 and 2017), with their implicationthat the overwhelming majority of US and UK workers are
employed in jobs with either very high or very low probabilityof automation. That our predictions are more uncertain is a
result of the distinctions of our methodology from previous
work: in particular, our foresight workshops force domain
experts to confront the uncertainties arising from structural
trends acting in complex and possibly offsetting ways.
The experts’ stated uncertainties reflecting this – andother sources – are explicitly factored into our machinelearning model.

89

Our skills results confirm the future importance of 21st
century skills – the combination of interpersonal andcognitive skills that has been an increasing preoccupation ofpolicymakers in recent years. In our US findings, there is aparticularly strong emphasis on interpersonal competencies,
consistent with the literature on the increasing importance ofsocial skills. In addition, a number of knowledge fields, suchas English Language, Administration and Management, andBiology are associated strongly with occupations predicted
to see rising demand – a reminder that the future workforce
will have generic knowledge as well as skills requirements.
In the UK, the findings support the importance of 21st
century skills too, though with an even stronger emphasis on
cognitive competencies and learning strategies. System skills –Judgment and Decision-making, Systems Analysis and Systems
Evaluation – feature prominently.
Our study makes contributions to all three literatures surveyed
in Section 2. First, the foresight workshops employ a novel
data collection methodology using active machine-learningalgorithms, which intelligently queries participants to maximise
the informativeness of the data collected. Second, the study
employs an innovative approach to generating predictions
about the future of skills, combining expert human judgmentwith machine-learning techniques that can flexibly respondto natural patterns in the data. Our approach thus permits
for richer, more complex, non-linear interactions between
variables – one that we exploit to assess complementarities
between skills and the implications for new occupations.
Third, the research is grounded in an explicit considerationof the diverse sources of structural change, any one of which
can be expected to have major impacts on future employer
skills needs. By making use of the detailed characterisation ofoccupations provided by the O*NET database, we are able
to provide a higher resolution treatment of skills, knowledge
types and abilities than is usually found in the skills literature.
Finally, our research serves as a potentially important
counterweight to the dominance of future automation inpolicy debates on employment.
This final point merits emphasis: it is tempting to focus on
the risks and dangers of the period ahead rather than theopportunities it offers. This is both dangerous and misleading.
It is dangerous because popular narratives matter for
economic outcomes and a storyline of relentless technologicaldisplacement of labour markets risks chilling growth andinnovation (Atkinson and Wu, 2017; Shiller, 2017). A backlash
against technology would be particularly dangerous at a time
when willingness to embrace risk is needed more than ever toimprove flagging productivity (Phelps, 2013; Erixon and Weigel,2016). It is misleading because our analysis points to theopportunities for boosting growth, though with one importantcaveat – that our education and training systems are agile
enough to respond appropriately. History is a reminder thatinvestments in skills must be at the centre of any long-term
strategy for adjusting to structural change. A precondition forthis is access to good information on skills needs – without
which policymakers risk flying blind. We hope this report is a
step towards improving understanding of this vital agenda.

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 090

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

APPENDIX A. SENSITIVITY ANALYSIS
US – Extrapolations
Figure A1: Using trend extrapolation, the distribution of US employment according to its probabilityof future increased demand
Note that the total area under all curves is equal to total US employment.

Management, Business and Financial Occupations
Computer, Engineering and Science Occupations
Education, Legal, Community Service, Arts and Media Occupations
Healthcare Practitioners and Technical Occupations
Service Occupations
Sales and Related Occupations
Office and Administrative Support Occupations
Farming, Fishing and Forestry Occupations
Construction and Extraction Occupations
Installation, Maintenance and Repair Occupations
Production Occupations
Transportation and Material Moving Occupations91THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

UK – Extrapolations
Figure A 2: Using trend extrapolation, the distribution of UK employment according to its probabilityof future increased demand
Note that the total area under all curves is equal to total UK employment.

Management, Directors and Senior Officials
Professional Occupations
Associate Professional and Technical Occupations
Administrative and Secretarial Occupations
Skilled Trades Occupations
Caring, Leisure and Other Service Occupations
Sales and Customer Service Occupations
Process, Plant and Machine Operatives
Elementary Occupations

92

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

US – Relative rankings
Table A1: Relative rankings of major occupation groups in the US by our model,
trained on expert judgment and by independent forecasts from the BLS.

R ANKING FROM EXPERT JUDGMENTR ANKING FROM BLS PROJECTIONS2014 -2024

Education, Training and Library Occupations

Healthcare Support Occupations

Community and Social Service Occupations

Healthcare Practitioners and Technical Occupations

Personal Care and Service Occupations

Personal Care and Service Occupations

Architecture and Engineering Occupations

Computer and Mathematical Occupations

Management Occupations

Community and Social Service Occupations

Arts, Design, Entertainment, Sports and Media Occupations

Construction and Extraction Occupations

Legal Occupations

Business and Financial Operations Occupations

Healthcare Support Occupations

Education, Training and Library Occupations

Healthcare Practitioners and Technical Occupations

Life, Physical and Social Science Occupations

Life, Physical and Social Science Occupations

Food Preparation and Serving Related Occupations

Computer and Mathematical Occupations

Installation, Maintenance and Repair Occupations

Building and Grounds Cleaning and Maintenance Occupations

Building and Grounds Cleaning and Maintenance Occupations

Construction and Extraction Occupations

Management Occupations

Protective Service Occupations

Legal Occupations

Business and Financial Operations Occupations

Sales and Related Occupations

Food Preparation and Serving Related Occupations

Transportation and Material Moving Occupations

Sales and Related Occupations

Protective Service Occupations

Farming, Fishing and Forestry Occupations

Arts, Design, Entertainment, Sports and Media Occupations

Installation, Maintenance and Repair Occupations

Architecture and Engineering Occupations

Office and Administrative Support Occupations

Office and Administrative Support Occupations

Transportation and Material Moving Occupations

Production Occupations

Production Occupations

Farming, Fishing and Forestry Occupations

93THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

UK – Relative rankings
Table A2: Relative rankings of sub-major occupation groups in the UK by our model,
trained on expert judgment and by independent forecasts from the KCES.

R ANKING FROM EXPERT JUDGMENTR ANKING FROM UKCES PROJECTIONS
TO 2024

Teaching and Educational Professionals

Customer Service Occupations

Culture, Media and Sports Occupations

Corporate Managers and Directors

Health Professionals

Caring Personal Service Occupations

Science, Research, Engineering and Technology Professionals

Business, Media and Public Service Professionals

Corporate Managers and Directors

Health and Social Care Associate Professionals

Business, Media and Public Service Professionals

Health Professionals

Textiles, Printing and Other Skilled Trades

Business and Public Service Associate Professionals

Skilled Agricultural and Related Trades

Culture, Media and Sports Occupations

Other Managers and Proprietors

Science, Research, Engineering and Technology Professionals

Business and Public Service Associate Professionals

Other Managers and Proprietors

Health and Social Care Associate Professionals

Teaching and Educational Professionals

Protective Service Occupations

Skilled Construction and Building Trades

Leisure, Travel and Related Personal Service Occupations

Science, Engineering and Technology Associate Professionals

Science, Engineering and Technology Associate Professionals

Elementary Administration and Service Occupations

Caring Personal Service Occupations

Skilled Agricultural and Related Trades

Skilled Metal, Electrical and Electronic Trades

Leisure, Travel and Related Personal Service Occupations

Skilled Construction and Building Trades

Transport and Mobile Machine Drivers and Operatives

Elementary Administration and Service Occupations

Elementary Trades and Related Occupations

Elementary Trades and Related Occupations

Protective Service Occupations

Administrative Occupations

Administrative Occupations

Sales Occupations

Sales Occupations

Transport and Mobile Machine Drivers and Operatives

Textiles, Printing and Other Skilled Trades

Secretarial and Related Occupations

Skilled Metal, Electrical and Electronic Trades

Customer Service Occupations

Process, Plant and Machine Operatives

Process, Plant and Machine Operatives

Secretarial and Related Occupations

94

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

APPENDIX B. MINOR OCCUPATIONGROUPS
US Minor Occupation Groups
Table B1: Probabilities of future increased demand for minor occupation groups
STANDARD
OCCUPATION
CL ASSIFICATION(SOC)

OCCUPATION TITLE

AVER AGE PROBABILIT Y(EMPLOYMENT-WEIGHTED)

39-2000
21-2000
25-2000
23-1000
25-1000
17-2000
21-1000
25-3000
39-9000
19-3000
11-2000
39-5000
25-9000
27-2000
11-9000
39-1000
31-2000
31-1000
43-1000
29-9000
29-1000
19-1000
19-2000
39-7000
17-1000
39-4000
27-3000
39-6000
11-3000
11-1000
13-1000
27-1000
33-1000
15-1000
37-1000
15-2000
47-2000
25-4000
37-2000
49-1000
47-3000
33-3000
37-3000
47-1000
53-2000
53-1000
41-1000

Animal Care and Service Workers
Religious Workers
Preschool, Primary, Secondary and Special Education School Teachers
Lawyers, Judges and Related Workers
Post-secondary Teachers
Engineers
Counselors, Social Workers and Other Community and Social Service Specialists
Other Teachers and Instructors
Other Personal Care and Service Workers
Social Scientists and Related Workers
Advertising, Marketing, Promotions, Public Relations and Sales Managers
Personal Appearance Workers
Other Education, Training and Library Occupations
Entertainers and Performers, Sports and Related Workers
Other Management Occupations
Supervisors Of Personal Care and Service Workers
Occupational Therapy and Physical Therapist Assistants and Aides
Nursing, Psychiatric and Home Health Aides
Supervisors of Office and Administrative Support Workers
Other Healthcare Practitioners and Technical Occupations
Health Diagnosing and Treating Practitioners
Life Scientists
Physical Scientists
Tour and Travel Guides
Architects, Surveyors and Cartographers
Funeral Service Workers
Media and Communication Workers
Baggage Porters, Bellhops and Concierges
Operations Specialties Managers
Top Executives
Business Operations Specialists
Art and Design Workers
Supervisors of Protective Service Workers
Computer Occupations
Supervisors Of Building and Grounds Cleaning and Maintenance Workers
Mathematical Science Occupations
Construction Trades Workers
Librarians, Curators and Archivists
Building Cleaning and Pest Control Workers
Supervisors Of Installation, Maintenance and Repair Workers
Helpers, Construction Trades
Law Enforcement Workers
Grounds Maintenance Workers
Supervisors Of Construction and Extraction Workers
Air Transportation Workers
Supervisors Of Transportation and Material Moving Workers
Supervisors Of Sales Workers

0.796
0.754
0.743
0.739
0.734
0.718
0.707
0.683
0.680
0.676
0.672
0.672
0.665
0.661
0.658
0.651
0.637
0.636
0.635
0.629
0.627
0.625
0.613
0.611
0.611
0.605
0.600
0.593
0.580
0.579
0.579
0.579
0.566
0.556
0.551
0.549
0.548
0.545
0.542
0.535
0.530
0.517
0.515
0.515
0.511
0.499
0.496

95

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table B1: Continued
SOC

OCCUPATION TITLE

33-2000
31-9000
41-9000
51-1000
35-3000
29-2000
35-1000
27-4000
41-4000
35-9000
17-3000
49-2000
33-9000
43-6000
19-4000
47-4000
45-2000
45-1000
41-2000
49-9000
43-4000
43-5000
53-5000
45-3000
39-3000
41-3000
35-2000
53-6000
53-3000
53-7000
23-2000
43-9000
13-2000
43-2000
53-4000
49-3000
47-5000
51-6000
51-3000
51-8000
45-4000
51-4000
51-7000
43-3000
51-2000
51-5000
51-9000

Fire Fighting and Prevention Workers
Other Healthcare Support Occupations
Other Sales and Related Workers
Supervisors of Production Workers
Food and Beverage Serving Workers
Health Technologists and TechniciansSupervisors of Food Preparation and Serving Workers
Media and Communication Equipment Workers
Sales Representatives, Wholesale and Manufacturing
Other Food Preparation and Serving Related Workers
Drafters, Engineering Technicians and Mapping Technicians
Electrical and Electronic Equipment Mechanics, Installers and Repairers
Other Protective Service Workers
Secretaries and Administrative AssistantsLife, Physical and Social Science Technicians
Other Construction and Related Workers
Agricultural Workers
Supervisors of Farming, Fishing and Forestry Workers
Retail Sales Workers
Other Installation, Maintenance and Repair Occupations
Information and Record Clerks
Material Recording, Scheduling, Dispatching and Distributing Workers
Water Transportation Workers
Fishing and Hunting Workers
Entertainment Attendants and Related Workers
Sales Representatives, Services
Cooks and Food Preparation Workers
Other Transportation Workers
Motor Vehicle Operators
Material Moving Workers
Legal Support Workers
Other Office and Administrative Support Workers
Financial Specialists
Communications Equipment Operators
Rail Transportation Workers
Vehicle and Mobile Equipment Mechanics, Installers and Repairers
Extraction Workers
Textile, Apparel and Furnishings Workers
Food Processing Workers
Plant and System Operators
Forest, Conservation and Logging Workers
Metal Workers and Plastic Workers
Woodworkers
Financial Clerks
Assemblers and Fabricators
Printing Workers
Other Production Occupations

96

AVER AGE PROBABILIT Y(EMPLOYMENT-WEIGHTED)
0.493
0.492
0.482
0.477
0.474
0.474
0.473
0.459
0.456
0.445
0.439
0.433
0.419
0.412
0.407
0.406
0.405
0.391
0.390
0.387
0.374
0.367
0.366
0.364
0.354
0.335
0.333
0.333
0.330
0.314
0.310
0.304
0.289
0.289
0.286
0.283
0.277
0.226
0.221
0.220
0.194
0.173
0.166
0.153
0.140
0.133
0.113

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

UK Minor Occupation Groups
Table B2: Probabilities of future increased demand for minor occupation groups
SOC

OCCUPATION TITLE

3440
1180
5430
2110
2220
1240
2310
1220
3420
2210
2140
2120
1110
3410
2230
1130
2150
1210
5240
2130
6240
3560
2440
2450
7130
1190
2470
9270
1150
2410
5110
2420
3550
1120
6210
3130
2430
1160
3230
2460
3540
1170
4160
6220
3530
3210
3310

Sports and Fitness Occupations
Health and Social Services Managers and Directors
Food Preparation and Hospitality Trades
Natural and Social Science Professionals
Therapy Professionals
Managers and Proprietors in Health and Care Services
Teaching and Educational Professionals
Managers and Proprietors in Hospitality and Leisure Services
Design Occupations
Health Professionals
Conservation and Environment Professionals
Engineering Professionals
Chief Executives and Senior Officials
Artistic, Literary and Media Occupations
Nursing and Midwifery Professionals
Functional Managers and Directors
Research and Development Managers
Managers and Proprietors in Agriculture-related Services
Electrical and Electronic Trades
Information Technology and Telecommunications Professionals
Cleaning and Housekeeping Managers and Supervisors
Public Services and Other Associate Professionals
Welfare Professionals
Librarians and Related Professionals
Sales Supervisors
Managers and Directors in Retail and Wholesale
Media Professionals
Other Elementary Services Occupations
Financial Institution Managers and Directors
Legal Professionals
Agricultural and Related Trades
Business, Research and Administrative Professionals
Conservation and Environmental Associate Professionals
Production Managers and Directors
Leisure and Travel Services
Information Technology Technicians
Architects, Town Planners and Surveyors
Managers and Directors in Transport and Logistics
Welfare and Housing Associate Professionals
Quality and Regulatory Professionals
Sales, Marketing and Related Associate Professionals
Senior Officers in Protective Services
Administrative Occupations: Office Managers and Supervisors
Hairdressers and Related Services
Business, Finance and Related Associate Professionals
Health Associate Professionals
Protective Service Occupations

97

AVER AGE PROBABILIT Y(EMPLOYMENT-WEIGHTED)
0.745
0.700
0.699
0.694
0.689
0.681
0.666
0.659
0.659
0.646
0.638
0.637
0.633
0.633
0.626
0.622
0.618
0.615
0.613
0.606
0.606
0.601
0.601
0.599
0.599
0.599
0.598
0.587
0.570
0.570
0.567
0.564
0.557
0.557
0.556
0.555
0.552
0.551
0.551
0.548
0.542
0.541
0.541
0.538
0.535
0.528
0.525

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table B2: Continued

SOC

OCCUPATION TITLE

3510
8140
9120
6120
3120
3110
5320
6140
1250
5250
7120
5330
8230
4110
5310
3520
5410
6230
4120
8210
5230
6130
5440
5220
4130
9240
4210
9230
7110
7220
7210
9210
9110
9130
4150
8120
8110
5420
5210
8220
8130
9250
9260

Transport Associate Professionals
Construction Operatives
Elementary Construction Occupations
Childcare and Related Personal Services
Draughtspersons and Related Architectural Technicians
Science, Engineering and Production Technicians
Building Finishing Trades
Caring Personal Services
Managers and Proprietors In Other Services
Skilled Metal, Electrical and Electronic Trades Supervisors
Sales-related Occupations
Construction and Building Trades Supervisors
Other Drivers and Transport Operatives
Administrative Occupations: Government and Related Organisations
Construction and Building Trades
Legal Associate Professionals
Textiles and Garments Trades
Housekeeping and Related Services
Administrative Occupations: FinanceRoad Transport Drivers
Vehicle Trades
Animal Care and Control Services
Other Skilled Trades
Metal Machining, Fitting and Instrument-making Trades
Administrative Occupations: Records
Elementary Security Occupations
Secretarial and Related Occupations
Elementary Cleaning Occupations
Sales Assistants and Retail Cashiers
Customer Service Managers and Supervisors
Customer Service Occupations
Elementary Administration Occupations
Elementary Agricultural Occupations
Elementary Process Plant Occupations
Other Administrative Occupations
Plant and Machine Operatives
Process Operatives
Printing Trades
Metal Forming, Welding and Related Trades
Mobile Machine Drivers and Operatives
Assemblers and Routine Operatives
Elementary Sales Occupations
Elementary Storage Occupations

98

AVER AGE PROBABILIT Y(EMPLOYMENT-WEIGHTED)
0.497
0.492
0.492
0.484
0.483
0.480
0.480
0.479
0.468
0.466
0.458
0.453
0.424
0.419
0.415
0.361
0.351
0.350
0.350
0.348
0.345
0.330
0.328
0.326
0.324
0.321
0.320
0.306
0.289
0.284
0.280
0.268
0.266
0.260
0.244
0.241
0.230
0.218
0.210
0.192
0.164
0.102
0.061

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

APPENDIX C. SKILLS RANKING BY AVERAGE DERIVATIVEUS Skills Ranking
Table C1: A ranking, by average derivative, of the importance of O*NET variables
to future demand for US occupations
R ANK
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

O*NET VARIABLE
Customer and Personal Service
Technology Design
Science
Service Orientation
Education and Training
Static Strength
Philosophy and Theology
Instructing
Installation
Sociology and Anthropology
Fluency of Ideas
Stamina
Personnel and Human Resources
Complex Problem Solving
Management of Material Resources
Extent Flexibility
Operations Analysis
DesignEquipment Selection
Psychology
Dynamic Strength
Originality
Management of Personnel Resources
Chemistry
Therapy and Counseling
Foreign Language
Arm-Hand Steadiness
Learning Strategies
Physics
Active Learning
Memorization
Administration and Management
Dynamic Flexibility
Time Sharing
Social PerceptivenessWriting
Manual Dexterity
Sound Localization
Multilimb Coordination
Gross Body Coordination
Engineering and Technology
Speaking
Reading Comprehension
Trunk Strength
Geography
Communications and Media
Telecommunications
Speech Recognition
Information Ordering
Inductive Reasoning
Active Listening
Coordination
Depth Perception
Far Vision
Mechanical
Written Comprehension
Problem Sensitivity

99

CL ASS
Knowledge
SkillsSkillsSkillsKnowledge
Abilities
Knowledge
SkillsSkillsKnowledge
Abilities
Abilities
Knowledge
SkillsSkillsAbilities
SkillsKnowledge
SkillsKnowledge
Abilities
Abilities
SkillsKnowledge
Knowledge
Knowledge
Abilities
SkillsKnowledge
SkillsAbilities
Knowledge
Abilities
Abilities
SkillsSkillsAbilities
Abilities
Abilities
Abilities
Knowledge
SkillsSkillsAbilities
Knowledge
Knowledge
Knowledge
Abilities
Abilities
Abilities
SkillsSkillsAbilities
Abilities
Knowledge
Abilities
Abilities

AVER AGE DERIVATIVE
2.578
2.565
2.557
2.229
2.087
1.965
1.953
1.847
1.843
1.655
1.602
1.570
1.544
1.377
1.227
1.226
1.189
1.170
1.162
1.071
1.067
1.048
1.041
1.040
1.016
1.012
1.008
0.985
0.971
0.940
0.914
0.902
0.844
0.841
0.745
0.737
0.721
0.659
0.652
0.634
0.631
0.622
0.580
0.552
0.533
0.527
0.514
0.510
0.454
0.441
0.391
0.379
0.351
0.348
0.341
0.332
0.330

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table C1: Continued

R ANK
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

O*NET VARIABLE
Monitoring
Time Management
Deductive Reasoning
Written Expression
History and Archeology
Visual Color Discrimination
Finger Dexterity
Glare Sensitivity
Judgment and Decision-making
Oral Expression
Peripheral Vision
Visualization
Persuasion
Gross Body Equilibrium
Oral Comprehension
Spatial Orientation
Public Safety and Security
Explosive Strength
Management of Financial Resources
Critical Thinking
Programming
Speech Clarity
Speed of Limb Movement
Speed of Closure
Transportation
Troubleshooting
Systems Analysis
Selective Attention
Sales and Marketing
Near Vision
Category Flexibility
Negotiation
Equipment Maintenance
Systems Evaluation
ClericalNight Vision
RepairingResponse Orientation
Response Orientation
Auditory Attention
Operation Monitoring
Flexibility of Closure
Hearing Sensitivity
Mathematics – Skills
Law and Government
Mathematical Reasoning
English Language
Medicine and Dentistry
Number Facility
Reaction Time
Quality Control Analysis
Economics and Accounting
Computers and Electronics
Wrist-Finger Speed
Operation and Control
Mathematics – Knowledge

100

CL ASS
SkillsSkillsAbilities
Abilities
Knowledge
Abilities
Abilities
Abilities
SkillsAbilities
Abilities
Abilities
SkillsAbilities
Abilities
Abilities
Knowledge
Abilities
SkillsSkillsSkillsAbilities
Abilities
Abilities
Knowledge
SkillsSkillsAbilities
Knowledge
Abilities
Abilities
SkillsSkillsSkillsKnowledge
Abilities
SkillsAbilities
Abilities
Abilities
SkillsAbilities
Abilities
SkillsKnowledge
Abilities
Knowledge
Knowledge
Abilities
Abilities
SkillsKnowledge
Knowledge
Abilities
SkillsKnowledge

AVER AGE DERIVATIVE
0.267
0.210
0.171
0.162
0.160
0.155
0.142
0.091
0.069
0.050
0.046
0.043
0.034
0.012
0.012
-0.053-0.081-0.103-0.138-0.176-0.182-0.299-0.326-0.328-0.365-0.367-0.391-0.424-0.434-0.440-0.517-0.559-0.561-0.572-0.601-0.701-0.715-0.737-0.737-0.822-0.910-0.924-0.944-0.944-0.949-1.024-1.079-1.233-1.399-2.014-2.027-2.043-2.052-2.053-2.334-2.365

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

UK Skills Ranking
Table C2: A ranking, by average derivative, of the importance of O*NET variables to future
demand for UK occupations
R ANK
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

O*NET VARIABLE
Judgment and Decision-making
Fluency of Ideas
Originality
Science
Operations Analysis
Gross Body Equilibrium
Gross Body Coordination
Medicine and Dentistry
Economics and Accounting
Sales and Marketing
Psychology
Complex Problem Solving
Sociology and Anthropology
Active Learning
Foreign Language
Systems Evaluation
Education and Training
Service Orientation
Management of Personnel Resources
Learning Strategies
Stamina
Programming
Manual Dexterity
Information Ordering
Time Management
Information Ordering
Time Management
Trunk Strength
Dynamic Strength
Finger Dexterity
Quality Control Analysis
Visual Colour Discrimination
Physics
Far Vision
Visualisation
Extent Flexibility
Arm-Hand Steadiness
Deductive Reasoning
History and Archeology
Coordination
Geography
Therapy and Counselling
Systems Analysis
Explosive Strength
Chemistry
Administration and Management
Management of Material Resources
Dynamic Flexibility
Oral Expression
Spatial Orientation
Communications and Media
Near Vision
Mathematics – Knowledge
Social Perceptiveness
Active Listening
Category Flexibility
Critical Thinking

101

CL ASS
SkillsAbilities
Abilities
SkillsSkillsAbilities
Abilities
Knowledge
Knowledge
Knowledge
Knowledge
SkillsKnowledge
SkillsKnowledge
SkillsKnowledge
SkillsSkillsSkillsAbilities
SkillsAbilities
Abilities
SkillsAbilities
SkillsAbilities
Abilities
Abilities
SkillsAbilities
Knowledge
Abilities
Abilities
Abilities
Abilities
Abilities
Knowledge
SkillsKnowledge
Knowledge
SkillsAbilities
Knowledge
Knowledge
SkillsAbilities
Abilities
Abilities
Knowledge
Abilities
Knowledge
SkillsSkillsAbilities
SkillsAVER AGE DERIVATIVE
4.528
4.366
4.229
4.228
3.976
3.708
3.225
2.896
2.851
2.834
2.828
2.725
2.616
2.493
2.463
2.351
2.283
2.116
2.107
2.077
2.047
1.873
1.843
1.831
1.802
1.831
1.802
1.710
1.645
1.612
1.578
1.453
1.439
1.347
1.235
1.196
1.190
1.136
1.132
1.095
0.940
0.938
0.935
0.916
0.887
0.826
0.814
0.796
0.796
0.779
0.772
0.710
0.697
0.657
0.615
0.462
0.439

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table C2: Continued

R ANK
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

O*NET VARIABLE
Equipment Selection
Problem Sensitivity
Management of Financial Resources
WritingInductive Reasoning
Telecommunications
Oral Comprehension
Technology Design
Philosophy and Theology
Installation
Personnel and Human Resources
Monitoring
Memorisation
Rate Control
Time Sharing
Speed of Limb Movement
Speed of Closure
Auditory Attention
Peripheral Vision
Selective Attention
Reaction Time
Wrist-Finger Speed
Written Expression
Clerical
Depth Perception
Night Vision
Speaking
Speech Recognition
Persuasion
Multilimb Coordination
Customer and Personal Service
English Language
Glare Sensitivity
Instructing
Flexibility of Closure
Transportation
Operation Monitoring
Number Facility
Hearing Sensitivity
Mathematical Reasoning
Negotiation
Response Orientation
DesignTroubleshooting
Mathematics – Skills
Mathematical Reasoning
English Language
Medicine and Dentistry
Number Facility
Reaction Time
Quality Control Analysis
Economics and Accounting
Computers and Electronics
Wrist-Finger Speed
Operation and Control
Mathematics – Knowledge
Rate Control
Mathematics – Knowledge

102

CL ASS
SkillsAbilities
SkillsSkillsAbilities
Knowledge
Abilities
SkillsKnowledge
SkillsKnowledge
SkillsAbilities
Abilities
Abilities
Abilities
Abilities
Abilities
Abilities
Abilities
Abilities
Abilities
Abilities
Knowledge
Abilities
Abilities
SkillsAbilities
SkillsAbilities
Knowledge
Knowledge
Abilities
SkillsAbilities
Knowledge
SkillsAbilities
Abilities
Abilities
SkillsAbilities
Knowledge
SkillsSkillsAbilities
Knowledge
Knowledge
Abilities
Abilities
SkillsKnowledge
Knowledge
Abilities
SkillsKnowledge
Abilities
Knowledge

AVER AGE DERIVATIVE
0.434
0.404
0.353
0.330
0.265
0.205
0.201
0.194
0.181
0.159
0.150
0.108
0.087
0.070
0.037
0.035
-0.030-0.070-0.114-0.148-0.151-0.165-0.200-0.203-0.219-0.225-0.267-0.326-0.431-0.432-0.522-0.571-0.665-0.678-0.696-0.698-0.780-0.808-0.848-0.868-1.058-1.174-1.230-1.277-1.320-1.024-1.079-1.233-1.399-2.014-2.027-2.043-2.052-2.053-2.334-2.365-2.684-2.365

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

APPENDIX D. UK–US OCCUPATION CROSSWALKMETHODOLOGY
Given that the UK lacks a comprehensive system forcollecting and disseminating information on occupationaland skills requirements, it is necessary to exploit the US
information that is already collected for O*NET. This isdone by mapping, or ‘crosswalking’, the US and UK SOC
taxonomies. Specifically, our crosswalk is based on theapplication programming interface (API) for the LMI for
All/O*NET SOC to UK SOC crosswalk (UK Commission for
Employment and Skills, 2017b) (under o-net/soc2onet/ ).
A feature of O*NET is that there are significantly more
occupations in the database than in the UK SOC (atfour-digit level). As a result, some UK occupations arecrosswalked to more than one equivalent US occupationin LMI for All. Where a direct single match is not possible,
we match the UK SOC to the occupation with the highest
employment. This simple rule is justified on the grounds
that it is more likely to be representative of the otheroccupations in the group.
To generate employment estimates, we use the BLS
Occupational Employment Statistics (OES), a semi-annual
survey of approximately 200,000 non-farm businessestablishments. As the O*NET occupational classification(eight-digit) is slightly more detailed than the six-digit 2010
SOC system, for which employment is reported, we are
required to ignore the last two digits. After investigation,
we believe that the information loss associated withthis approach is minimal and superior to more complex
procedures. We use May 2015 employment data – as in
the rest of the analysis – and compare it with estimatesfrom more recently available data (May 2016) and from2006 (May 2006) to ensure that the results are robust
over time.

103

In a small number of cases, two or more occupations
account for the highest employment in a group. This
arises from the fact while some occupation codes differ
at the eight-digit level, they are identical at the six-digit
level. As a general rule in such cases we select the moregeneric occupation since it better approximates the levelof detail found in the UK SOC. In many cases this is easy toestablish from the respective position of the occupationsin the SOC hierarchy. For example, Statisticians (152041.00) is chosen over Biostatisticians (15-2041.01).
In other cases, we apply our judgment to determine
which code is most generic.
We assess the degree of error introduced by using
only one O*NET code as a result of our ‘highest
employment’ rule. Ideally, the code with the highest
employment should account for all the employment
in the group. We find evidence that, in many cases, thesecodes do account for the lion’s share of employment.
In sum, our one-to-one crosswalk generated using thisrule contains 283 unique US SOC codes, which accountfor 70% of total US employment.
The LMI for All API does not provide a crosswalk for
23 UK occupations. As a result, we manually choose
23 satisfactory US matches (designated by * below).
Four UK occupations have no US counterpart with
relevant job tasks and skills data in O*NET and areconsequently excluded.

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table D1: The UK-to-US Occupation Crosswalk basedon LMI for All data

UK SOC
1115
1121
1122
1123*
1131
1132
1133
1134
1135*
1136

UK TITLE
Chief executives and senior officials
Production managers and directors
in manufacturing

US O*NET SOC
11-1011.00

US TITLE
Chief Executives

11-1021.00

General and Operations Managers

Production managers and directors in construction11-9021.00
Production managers and directors in mining
and energy
Financial managers and directors
Marketing and sales directors
Purchasing managers and directors
Advertising and public relations directors
Human resource managers and directors
Information technology and telecommunications
directors

Construction Managers

11-3051.00

Industrial Production Managers

11-3031.02
11-2021.00
11-9199.04
11-2031.00
11-3121.00

Financial Managers, Branch or Department
Marketing Managers
Supply Chain Managers
Public Relations and Fundraising Managers
Human Resources Managers

11-3021.00

Computer and Information Systems Managers

1139

Functional managers and directors n.e.c.

43-1011.00

1150

Financial institution managers and directors
Managers and directors in transportand distribution
Managers and directors in storage
and warehousing
Senior police officers
Senior officers in fire, ambulance,
prison and related services

11-3031.02

First-line Supervisors of Office and Administrative
Support Workers
Financial Managers, Branch or Department

11-3071.01

Transportation, Storage and Distribution Managers

1161
1162
1172
1173
1181
1184
1190
1211

Health services and public healthmanagers and directors
Social services managers and directors
Managers and directors in retail and wholesale
Managers and proprietors in agriculture andhorticulture

33-1012.00

First-line Supervisors of Helpers, Laborers andMaterial Movers, Hand
First-line Supervisors of Police and Detectives

33-2022.00

Forest Fire Inspectors and Prevention Specialists

11-9111.00

Medical and Health Services Managers

11-9151.00
41-1011.00

Social and Community Service Managers
First-line Supervisors of Retail Sales Workers

19-1031.02

Range Managers

53-1021.00

1213*

Managers and proprietors in forestry, fishing and
19-1032.00
related services

Foresters

1221

Hotel and accommodation managers andproprietors

Residential Advisors

1223
1224
1225
1226*
1241
1242

Restaurant and catering establishment managers
and proprietors
Publicans and managers of licensed premises
Leisure and sports managers
Travel agency managers and proprietors
Health care practice managers
Residential, day and domiciliary caremanagers and proprietors

39-9041.00

11-9051.00
39-1021.01
41-3041.00
11-9111.00

First-line Supervisors of Food Preparation andServing Workers
Food Service Managers
Spa Managers
Travel Agents
Medical and Health Services Managers

11-9111.00

Medical and Health Services Managers

35-1012.00

1251

Property, housing and estate managers

11-9141.00

1252

Garage managers and proprietors

49-1011.00

1259
2111
2112

Hairdressing and beauty salon managers
and proprietors
Shopkeepers and proprietors – wholesale andretailWaste disposal and environmental services
managers
Managers and proprietors in other services n.e.c.
Chemical scientists
Biological scientists and biochemists

2113

Physical scientists

19-2042.00

2114

Social and humanities scientists

19-4061.00

1253
1254
1255

104

Property, Real Estate and Community Association
Managers
First-line Supervisors of Mechanics, Installers andRepairers

39-1021.00

First-line Supervisors of Personal Service Workers

41-2031.00

Retail Salespersons

17-2081.00

Environmental Engineers

21-2021.00
19-2031.00
17-2199.01

Directors, Religious Activities and Education
Chemists
Biochemical Engineers
Geoscientists, except Hydrologists andGeographers
Social Science Research Assistants

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table D1: Continued

UK SOC
2119
2121
2122
2123
2124
2126*
2127
2129
2133
2134*
2135*

UK TITLE
Natural and social science professionals n.e.c.
Civil engineers
Mechanical engineers
Electrical engineers
Electronics engineers
Design and development engineers
Production and process engineers
Engineering professionals n.e.c.
IT specialist managers
IT project and programme managers
IT business analysts, architects and systems
designers

US O*NET SOC
11-9121.00
17-2051.00
17-2141.00
17-2071.00
17-2072.00
17-2112.00
17-2112.00
13-1199.01
11-3021.00
15-1199.09

US TITLE
Natural Sciences Managers
Civil Engineers
Mechanical Engineers
Electrical Engineers
Electronics Engineers, except Computer
Industrial Engineers
Industrial Engineers
Energy Auditors
Computer and Information Systems Managers
Information Technology Project Managers

15-1199.09

Information Technology Project Managers

15-1131.00

Computer Programmers

43-9031.00

Desktop Publishers

15-1143.01

Telecommunications Engineering Specialists

2141

Programmers and software development
professionals
Web design and development professionals
Information technology and telecommunications
professionals n.e.c.
Conservation professionals

19-1031.01

2142

Environment professionals

19-2041.00

2150*
2211
2212
2213
2214
2215
2216
2217
2218
2219
2221
2222
2223
2229
2231
2232
2311
2312

Research and development managers
Medical practitioners
Psychologists
Pharmacists
Ophthalmic opticians
Dental practitioners
Veterinarians
Medical radiographers
Podiatrists
Health professionals n.e.c.
Physiotherapists
Occupational therapists
Speech and language therapists
Therapy professionals n.e.c.
NursesMidwives
Higher education teaching professionals
Further education teaching professional

11-9121.01
29-1069.02
19-3031.02
29-1051.00
29-1041.00
29-1021.00
29-1131.00
29-2032.00
29-1081.00
31-9092.00
29-1123.00
29-1122.00
29-1127.00
29-1126.00
29-2061.00
29-9099.01
25-1011.00
25-1194.00

2314

Secondary education teaching professionals

25-2031.00

Soil and Water Conservationists
Environmental Scientists and Specialists, Including
HealthClinical Research Coordinators
Dermatologists
Clinical Psychologists
Pharmacists
Optometrists
Dentists, General
Veterinarians
Diagnostic Medical Sonographers
Podiatrists
Medical Assistants
Physical Therapists
Occupational Therapists
Speech-Language Pathologists
Respiratory Therapists
Licensed Practical and Licensed Vocational Nurses
Midwives
Business Teachers, Postsecondary
Vocational Education Teachers, Post-secondary
Secondary School Teachers, except Special andCareer/Technical Education
Elementary School Teachers, except Special
Education
Special Education Teachers, Middle School
Education Administrators, Post-secondary
Education Administrators,
Elementary and Secondary School

2136*
2137
2139*

2316*
2317

Primary and nursery education teachingprofessionals
Special needs education teaching professionals
Senior professionals of educational establishments

25-2053.00
11-9033.00

2318

Education advisers and school inspectors

11-9032.00

2315

2319
2412
2413
2419
2421
2423
2424
2425
2426

Teaching and other educational professionals
n.e.c.
Barristers and judges
Solicitor
Legal professionals n.e.c.
Chartered and certified accountants
Management consultants and business analysts
Business and financial project management
professionals
Actuaries, economists and statisticians
Business and related research professionals

105

25-2021.00

25-3021.00

Self-enrichment Education Teachers

13-1041.06
23-1011.00
23-1011.00
13-2011.01
13-2099.02

Coroners
Lawyers
Lawyers
Accountants
Risk Management Specialists

13-1111.00

Management Analysts

15-2041.00
33-3021.03

Statisticians
Criminal Investigators and Special Agents

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table D1: Continued

UK SOC

2442

UK TITLE
Business, research and administrative
professionals n.e.c.
Architects
Town planning officers
Quantity surveyors
Chartered surveyors
Chartered architectural technologists
Construction project managers
and related professionals
Social workers

2443

Probation officers

21-1092.00

2444
2449*
2451
2452
2461
2462

ClergyWelfare professionals n.e.c.
Librarians
Archivists and curators
Quality control and planning engineers
Quality assurance and regulatory professionals

21-2011.00
11-9151.00
25-4021.00
25-4012.00
17-2199.02
11-9199.02

2463

Environmental health professionals

19-2041.00

2471
2472

3121
3122
3131
3132*
3213
3216
3217
3218

Journalists, newspaper and periodical editors
Public relations professionals
Advertising accounts managers
and creative directors
Laboratory technicians
Electrical and electronics techniciansEngineering technicians
Building and civil engineering technicians
Quality assurance technicians
Planning, process and production technicians
Science, engineering and production
technicians n.e.c.
Architectural and town planning techniciansDraughtspersons
IT operations technicians
IT user support technicians
Paramedics
Dispensing opticians
Pharmaceutical technicians
Medical and dental technicians

3219

Health associate professionals n.e.c.

29-1122.01

3231
3233

Youth and community workers
Child and early years officers

21-1093.00
21-1021.00

3234

Housing officers

11-9141.00

3235

Counsellors

21-1012.00

3239
3312
3313
3314
3315
3319

Welfare and housing associate professionals n.e.c.
Police officers (sergeant and below)
Fire service officers (watch manager and below)
Prison service officers (below principal officer)
Police community support officers
Protective service associate professionals n.e.c.

43-4051.03
33-3051.01
33-2011.01
33-3012.00
33-3051.01
11-9199.08

3411

Artists

27-1013.00

3412
3413

Authors, writers and translators
Actors, entertainers and presenters27-3042.00
27-2011.00

2429
2431
2432
2433
2434
2435
2436

2473
3111
3112
3113
3114
3115
3116
3119

106

US O*NET SOC

17-1011.00
19-3051.00
13-1051.00
17-1022.00
17-3011.01

US TITLE
First-line Supervisors of Office andAdministrative Support Workers
Architects, except Landscape and Naval
Urban and Regional Planners
Cost Estimators
Surveyors
Architectural Drafters

19-3099.01

Transportation Planners

21-1021.00

27-3041.00
27-3031.00

Child, Family and School Social Workers
Probation Officers andCorrectional Treatment Specialists
ClergySocial and Community Service Managers
Librarians
Curators
Validation Engineers
Compliance Managers
Environmental Scientists and Specialists,
Including Health
Editors
Public Relations Specialists

27-1011.00

Art Directors

29-2011.00
17-3023.01
17-3023.03
17-3022.00
19-4099.01
17-3029.09

Medical and Clinical Laboratory Technologists
Electronics Engineering Technicians
Electrical Engineering Technicians
Civil Engineering Technicians
Quality Control Analysts
Manufacturing Production Technicians

19-4099.03

Remote Sensing Technicians

19-4061.01
17-3011.01
15-2041.02
15-1151.00
29-2041.00
29-2081.00
29-2052.00
29-2021.00

City and Regional Planning Aides
Architectural Drafters
Clinical Data Managers
Computer User Support Specialists
Emergency Medical Technicians and Paramedics
Opticians, Dispensing
Pharmacy Technicians
Dental Hygienists
Low Vision Therapists, Orientation and Mobility
Specialists and Vision Rehabilitation Specialists
Social and Human Service AssistantsChild, Family and School Social Workers
Property, Real Estate and Community
Association Managers

43-1011.00

Educational, Guidance, School,
and Vocational Counselors
Patient Representatives
Police Patrol Officers
Municipal Firefighters
Correctional Officers and Jailers
Police Patrol Officers
Lost Prevention Managers
Fine Artists, Including Painters, Sculptors,
and Illustrators
Technical Writers
ActorsT HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table D1: Continued

UK SOC
3414
3415
3416

US O*NET SOC
27-2031.00
27-2042.02
27-2012.01

US TITLE
Dancers
Musicians, Instrumental
Producers

27-4021.00

Photographers

3421
3422
3441
3442
3443
3538
3511
3512
3513
3520

UK TITLE
Dancers and choreographersMusicians
Arts officers, producers and directors
Photographers, audiovisual and broadcasting
equipment operators
Graphic designers
Product, clothing and related designers
Sports players
Sports coaches, instructors and officials
Fitness instructors
Financial accounts managers
Air traffic controllers
Aircraft pilots and flight engineers
Ship and hovercraft officers
Legal associate professionals

27-1024.00
27-1025.00
27-2021.00
27-2022.00
39-9031.00
11-9199.03
53-2021.00
53-2011.00
53-5021.01
23-2011.00

Graphic Designers
Interior Designers
Athletes and Sports Competitors
Coaches and Scouts
Fitness Trainers and Aerobics Instructors
Investment Fund Managers
Air Traffic Controllers
Airline Pilots, Copilots and Flight Engineers
Ship and Boat Captains
Paralegals and Legal Assistants

3531

Estimators, valuers and assessors

13-1031.02

Insurance Adjustors, Examiners and Investigators

3532
3534
3535
3536
3537
3539

Brokers
Finance and investment analysts and advisers
Taxation experts
Importers and exporters
Financial and accounting technicians
Business and related associate professionals n.e.c.

13-1199.03
13-2051.00
13-2082.00
13-1199.03
43-3031.00
13-1111.00

3541

Buyers and procurement officers

13-1023.00

Customs Brokers
Financial Analysts
Tax Preparers
Customs Brokers
Bookkeeping, Accounting and Auditing Clerks
Management Analysts
Purchasing Agents, except Wholesale, Retail andFarm Products

3542

Business sales executives

41-4012.00

3543
3544

Marketing associate professionals
Estate agents and auctioneers
Sales accounts and business development
managers

41-3011.00
41-9022.00

Sales Representatives, Wholesale and Manufacturing,
except Technical and Scientific Products
Advertising Sales Agents
Real Estate Sales Agents

11-2022.00

Sales Managers

13-1121.00

Meeting, Convention and Event Planners

3417

3545
3546
3550
3561*
3562
3563
3564
3565
3567
4112
4113
4114*
4121
4122
4123
4124
4129
4131
4132
4133
4134
4135
4138
4151
4159

Conference and exhibition managers andorganisers
Conservation and environmental associate
professionals
Public services associate professionals
Human resources and industrial relations officers
Vocational and industrial trainers and instructors
Careers advisers and vocational guidance
specialists
Inspectors of standards and regulations
Health and safety officers
National government administrative occupations
Local government administrative occupations
Officers of non-governmental organisations
Credit controllers
Book-keepers, payroll managers and wages clerks
Bank and post office clerks
Finance officers
Financial administrative occupations n.e.c.
Records clerks and assistants
Pensions and insurance clerks and assistants
Stock control clerks and assistants
Transport and distribution clerks and assistants
Library clerks and assistants
Human resources administrative occupations
Sales administrators
Other administrative occupations n.e.c.

107

19-4091.00
11-3011.00
13-1041.03
25-9031.00
21-1012.00
13-1041.01
29-9011.00
43-4061.00
43-4031.02
11-1011.00
43-4041.01
43-3031.00
43-3071.00
43-3031.0
11-3031.01
43-5061.00
43-9041.01
43-5081.01
43-5071.00
43-4121.00
43-3051.00
43-4151.00
43-9061.00

Environmental Science and Protection Technicians,
Including HealthAdministrative Services Managers
Equal Opportunity Representatives and Officers
Instructional Coordinators
Educational, Guidance, School and VocationalCounsellors
Environmental Compliance Inspectors
Occupational Health and Safety Specialists
Eligibility Interviewers, Government Programmes
Municipal Clerks
Chief Executives
Credit Authorizers
Bookkeeping, Accounting and Auditing Clerks
Tellers
Bookkeeping, Accounting and Auditing Clerks
Treasurers and Controllers
Production, Planning and Expediting Clerks
Insurance Claims Clerks
Stock Clerks, Sales Floor
Shipping, Receiving and Traffic Clerks
Library Assistants, Clerical
Payroll and Timekeeping Clerks
Order Clerks
Office Clerks, General

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table D1: Continued

UK SOC
4161

UK TITLE
Office managers

US O*NET SOC
11-3011.00

4162

Office supervisors43-1011.00

4211
4212

Medical secretaries
Legal secretaries

43-6013.00
43-6012.00

4213

School secretaries

43-6014.00

4214

Company secretaries

43-6014.00

4215

Personal assistants and other secretaries

43-6014.00

4216
4217
5111

Receptionists
Typists and related keyboard occupations
Farmers

43-4171.00
43-3021.01
19-4099.02

5112

Horticultural trades

45-1011.07

5113

Gardeners and landscape gardeners

37-1012.00

5114
5119

Groundsmen and greenkeepers
Agricultural and fishing trades n.e.c.

37-3011.00
45-1011.06

5211

Smiths and forge workers

51-4022.00

5212

Moulders, core makers and die casters

51-4072.00

5213
5214
5215
5216

Sheet metal workers
Metal plate workers and riveters
Welding trades
Pipe fitters47-2211.00
47-2011.00
51-4121.06
47-2152.01

5221

Metal machining setters and setter-operators

51-4031.00

5222
5223
5224
5225
5231
5232
5234
5235
5236
5237
5241

Tool makers, tool fitters and markers-out
Metal working production and maintenance fitters
Precision instrument makers and repairers
Air-conditioning and refrigeration engineers
Vehicle technicians, mechanics and electricians
Vehicle body builders and repairers
Vehicle paint technicians
Aircraft maintenance and related trades
Boat and ship builders and repairers
Rail and rolling stock builders and repairers
Electricians and electrical fitters

51-4111.00
49-9041.00
49-9062.00
49-9021.02
49-3023.01
49-3021.00
51-9122.00
49-3011.00
49-3051.00
53-4011.00
47-2111.00

5242

Telecommunications engineers

49-2022.00

5244
5245
5249

TV, video and audio engineers
IT engineers
Electrical and electronic trades n.e.c.
Skilled metal, electrical and electronic
trades supervisors
Steel erectors
Bricklayers and masonsRoofers, roof tilers and slaters
Plumbers and heating and ventilating engineers
Carpenters and joiners
Glaziers, window fabricators and fitters
Construction and building trades n.e.c.
Plasterers
Floorers and wall tilers
Painters and decorators

27-4011.00
17-2061.00
49-9051.00

5250
5311
5312
5313
5314
5315
5316
5319
5321
5322
5323

108

51-1011.00
47-2221.00
47-2081.00
47-2181.00
47-2152.02
47-2031.01
47-2121.00
47-4031.00
47-2161.00
47-2044.00
47-2141.00

US TITLE
Administrative Services Managers
First-line Supervisors of Office and Administrative
Support Workers
Medical Secretaries
Legal Secretaries
Secretaries and Administrative Assistants, except
Legal, Medical and Executive
Secretaries and Administrative Assistants, except
Legal, Medical and Executive
Secretaries and Administrative Assistants, except
Legal, Medical and Executive
Receptionists and Information Clerks
Statement Clerks
Precision Agriculture TechniciansFirst-line Supervisors of Agricultural Crop andHorticultural Workers
First-line Supervisors of Landscaping, Lawn Service
and Groundskeeping Workers
Landscaping and Groundskeeping Workers
First-line Supervisors of Aquacultural Workers
Forging Machine Setters, Operators and Tenders,
Metal and Plastic
Molding, Coremaking and Casting Machine Setters,
Operators and Tenders, Metal and Plastic
Sheet Metal Workers
Boilermakers
Welders, Cutters and Welder Fitters
Pipefitters and Steamfitters
Cutting, Punching and Press Machine Setters,
Operators and Tenders, Metal and Plastic
Tool and Die Makers
Industrial Machinery Mechanics
Medical Equipment Repairers
Refrigeration Mechanics and Installers
Automotive Master Mechanics
Automotive Body and Related Repairers
Painters, Transportation Equipment
Aircraft Mechanics and Service Technicians
Motorboat Mechanics and Service Technicians
Locomotive Engineers
Electricians
Telecommunications Equipment Installers andRepairers, except Line Installers
Audio and Video Equipment Technicians
Computer Hardware Engineers
Electrical Power-line Installers and Repairers
First-line Supervisors of Production and Operating
Workers
Structural Iron and Steel Workers
Drywall and Ceiling Tile Installers
Roofers
Plumbers
Construction Carpenters
Glaziers
Fence Erectors
Plasterers and Stucco Masons
Tile and Marble Setters
Painters, Construction and MaintenanceT HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table D1: Continued

UK SOCUK TITLE

US O*NET SOC

5330

Construction and building trades supervisors47-1011.00

5411

Weavers and knitters

51-6063.00

5412
5413
5414
5419
5421*
5422*
5423*
5431
5432
5433
5434
5435
5436

51-6093.00
51-6041.00
51-6052.00
51-9031.00
51-5111.00
51-5112.00
51-5113.00
51-3021.00
51-3011.00
51-3022.00
35-1011.00
35-2014.00
11-9051.00
51-9195.05

Potters, Manufacturing

5442
5443

Upholsterers
Footwear and leather working trades
Tailors and dressmakers
Textiles, garments and related trades n.e.c.
Pre-press technicians
Printers
Print finishing and binding workers
Butchers
Bakers and flour confectioners
Fishmongers and poultry dressers
Chefs
Cooks
Catering and bar managers
Glass and ceramics makers, decorators andfinishers
Furniture makers and other craft woodworkers
Florists

Textile Knitting and Weaving Machine Setters,
Operators and Tenders
Upholsterers
Shoe and Leather Workers and Repairers
Tailors, Dressmakers and Custom Sewers
Cutters and Trimmers,Hand
Prepress Technicians and Workers
Printing Press Operators
Print Binding and Finishing Workers
Butchers and Meat Cutters
BakersMeat, Poultry and Fish Cutters and Trimmers
Chefs and Head Cooks
Cooks, Restaurant
Food Service Managers

51-7011.00
27-1023.00

5449

Other skilled trades n.e.c.

51-9121.00

6121
6122
6123
6125
6126
6131
6132
6139
6141

Nursery nurses and assistantsChildminders and related occupations
Playworkers
Teaching assistants
Educational support assistants
Veterinary nurses
Pest control officers
Animal care services occupations n.e.c.
Nursing auxiliaries and assistants

39-9011.00
39-9011.00
39-9011.00
25-9041.00
25-9041.00
29-2056.00
37-2021.00
39-2021.00
31-9099.01

6142

Ambulance staff (excluding paramedics)

53-3011.00

6143
6144
6145
6146
6147*

31-9091.00
39-9041.00
39-9021.00
39-1021.00
39-9021.00
39-4021.00

Funeral Attendants

6211

Dental nurses
Houseparents and residential wardens
Care workers and home carers
Senior care workers
Care escorts
Undertakers, mortuary and crematorium
assistants
Sports and leisure assistants

Cabinetmakers and Bench Carpenters
Floral Designers
Coating, Painting and Spraying Machine Setters,
Operators and Tenders
Childcare Workers
Childcare Workers
Childcare Workers
Teacher Assistants
Teacher Assistants
Veterinary Technologists and Technicians
Pest Control Workers
Nonfarm Animal Caretakers
Speech-language Pathology Assistants
Ambulance Drivers and Attendants,
except Emergency Medical Technicians
Dental Assistants
Residential Advisors
Personal Care Aides
First-line Supervisors of Personal Service Workers
Personal Care Aides

39-9032.00

6212

Travel agents

43-4181.00

6214*
6215
6219*
6221
6222
6231

Air travel assistants
Rail travel assistants
Leisure and travel service occupations n.e.c.
Hairdressers and barbers
Beauticians and related occupations
Housekeepers and related occupations

53-2031.00
53-4031.00
43-4051.00
39-5012.00
39-5092.00
39-9021.00

6232

Caretakers

37-2011.00

Recreation Workers
Reservation and Transportation
Ticket Agents and Travel Clerks
Flight Attendants
Railroad Conductors and Yardmasters
Customer Service Representatives
Hairdressers, Hairstylists and Cosmetologists
Manicurists and Pedicurists
Personal Care Aides
Janitors and Cleaners, except Maids andHousekeeping Cleaners

5441

6148

6240
7111
7112
7113

Cleaning and housekeeping managers andsupervisors
Sales and retail assistants
Retail cashiers and checkout operators
Telephone salespersons

109

US TITLE
First-line Supervisors of Construction Trades andExtraction Workers

39-1021.00

First-line Supervisors of Personal Service Workers

41-2031.00
41-2011.00
41-9041.00

Retail Salespersons
Cashiers
Telemarketers

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Table D1: Continued

UK SOC
7114
7115
7121
7122
7123

UK TITLE
Pharmacy and other dispensing assistants
Vehicle and parts salespersons and advisers
Collector salespersons and credit agents
Debt, rent and other cash collectors
Roundspersons and van salespersons

US O*NET SOC
31-9095.00
41-2022.00
41-3021.0
43-3011.00
53-3031.00

7124

Market and street traders and assistants

41-9091.00

7125
7129
7130
7211

Merchandisers and window dressers
Sales-related occupations n.e.c.
Sales supervisors
Call and contact centre occupations

27-1026.00
41-9011.00
41-1011.00
43-4051.00

7213

Telephonists

43-2011.00

7214
7215
7219
7220
8111
8112

Communication operators
Market research interviewers
Customer service occupations n.e.c.
Customer service managers and supervisors
Food, drink and tobacco process operatives
Glass and ceramics process operatives

43-5031.00
43-4111.00
43-4051.00
43-4051.00
51-3092.00
51-9195.04

8113

Textile process operatives

51-6064.00

8114

Chemical and related process operatives

51-9023.00

8115
8116

Rubber process operatives
Plastics process operatives

51-9197.00
51-2091.00

8117

Metal making and treating process operatives

51-4021.00

8118

Electroplaters

51-4193.00

8119

Process operatives n.e.c.

51-9195.07

8121

Paper and wood machine operatives

51-9196.00

8122
8123
8124
8125

Coal mine operatives
Quarry workers and related operatives
Energy plant operatives
Metal working machine operatives

47-5061.00
47-5013.00
51-8013.00
51-4041.00

8126

Water and sewerage plant operatives

51-8031.00

8127*

Printing machine assistants

51-5112.00

8129

Plant and machine operatives n.e.c.

51-9041.00

8131
8132

Assemblers (electrical and electronic products)
Assemblers (vehicles and metal goods)

51-4121.07
51-2031.00

8133

Routine inspectors and testers

51-9061.00

8134
8135
8137
8139
8141
8142

Weighers, graders and sorters
Tyre, exhaust and windscreen fitters
Sewing machinists
Assemblers and routine operatives n.e.c.
Scaffolders, stagers and riggers
Road construction operatives

45-2041.00
49-3093.00
51-6031.00
51-2092.00
49-9096.00
47-2051.00

8143

Rail construction and maintenance operatives

47-4061.00

8149
8211
8212
8213
8214
8215

Construction operatives n.e.c.
Large goods vehicle drivers
Van drivers
Bus and coach drivers
Taxi and cab drivers and chauffeurs
Driving instructors

47-4041.00
53-3032.00
53-3033.00
53-3022.00
53-3041.00
25-3021.00

110

US TITLE
Pharmacy Aides
Parts Salespersons
Insurance Sales AgentsBill and Account Collectors
Driver/Sales Workers
Door-to-Door Sales Workers, News and Street
Vendors and Related Workers
Merchandise Displayers and Window Trimmers
Demonstrators and Product PromotersFirst-line Supervisors of Retail Sales Workers
Customer Service Representatives
Switchboard Operators, Including Answering
Service
Police, Fire and Ambulance Dispatchers
Interviewers, except Eligibility and Loan
Customer Service Representatives
Customer Service Representatives
Food Batchmakers
Glass Blowers, Molders, Benders and Finishers
Textile Winding, Twisting and Drawing Out Machine
Setters, Operators and Tenders
Mixing and Blending Machine Setters,
Operators and Tenders
Tire Builders
Fiberglass Laminators and Fabricators
Extruding and Drawing Machine Setters, Operators
and Tenders, Metal and Plastic
Plating and Coating Machine Setters, Operators
and Tenders, Metal and Plastic
Molding and Casting Workers
Paper Goods Machine Setters, Operators andTenders
Roof Bolters, Mining
Service Unit Operators, Oil, Gas and Mining
Power Plant Operators
Machinists
Water and Wastewater Treatment Plant andSystem Operators
Printing Press Operators
Extruding, Forming, Pressing and Compacting
Machine Setters, Operators and Tenders
Solderers and Brazers
Engine and Other Machine Assemblers
Inspectors, Testers, Sorters, Samplers andWeighers
Graders and Sorters, Agricultural Products
Tire Repairers and Changers
Sewing Machine Operators
Team Assemblers
Riggers
Cement Masons and Concrete Finishers
Rail-track Laying and Maintenance Equipment
Operators
Hazardous Materials Removal Workers
Heavy and Tractor–Trailer Truck Drivers
Light Truck or Delivery Services Drivers
Bus Drivers, School or Special Client
Taxi Drivers and Chauffeurs
Self-enrichment Education Teachers

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Table D1: Continued

UK SOC
8221
8222
8223

UK TITLE
Crane drivers
Fork-lift truck drivers
Agricultural machinery drivers

US O*NET SOC
53-7021.00
53-7051.00
45-2091.00

8229

Mobile machine drivers and operatives n.e.c.

47-2073.00

8231
8232
8233
8234

Train and tram drivers
Marine and waterways transport operatives
Air transport operatives
Rail transport operatives

53-4041.00
53-5021.02
53-2022.00
53-4021.00

8239

Other drivers and transport operatives n.e.c.

53-1031.00

9111
9112

Farm workers
Forestry workers
Fishing and other elementary agriculture
occupations n.e.c.
Elementary construction occupations
Industrial cleaning process occupations
Packers, bottlers, canners and fillers
Elementary process plant occupations n.e.c.
Postal workers, mail sorters, messengers
and couriers

45-2092.02
37-3013.00

US TITLE
Crane and Tower Operators
Industrial Truck and Tractor Operators
Agricultural Equipment Operators
Operating Engineers and Other Construction
Equipment Operators
Subway and Streetcar Operators
Mates – Ship, Boat and Barge
Airfield Operations Specialists
Railroad Brake, Signal and Switch Operators
First-line Supervisors of Transportation and Materialmoving Machine and Vehicle Operators
Farmworkers and Laborers, Crop
Tree Trimmers and Pruners

37-3011.00

Landscaping and Groundskeeping Workers

47-2061.00
47-4071.00
53-7064.00
51-9198.00

Construction Laborers
Septic Tank Servicers and Sewer Pipe Cleaners
Packers and Packagers, Hand
Helpers–Production Workers

43-5052.00

Postal Service Mail Carriers

9219

Elementary administration occupations n.e.c.

43-9051.00

Mail Clerks and Mail Machine Operators, except
Postal Service

9231

Window cleaners

37-2011.00

9232
9233
9234
9235
9236

Street cleaners
Cleaners and domestics
Launderers, dry cleaners and pressers
Refuse and salvage occupations
Vehicle valeters and cleaners

47-4051.00
37-2012.00
51-6011.00
53-7081.00
53-7061.00

9239

Elementary cleaning occupations n.e.c.

37-2011.00

9241
9242
9244
9249

Security guards and related occupations
Parking and civil enforcement occupations
School midday and crossing patrol occupations
Elementary security occupations n.e.c.

33-9032.00
53-6021.00
33-9091.00
33-3011.00

9251

Shelf fillers

53-7062.00

9259

Elementary sales occupations n.e.c.

43-5081.04

9260

Elementary storage occupations

53-7062.00

9271*

Hospital porters

31-9092.00

9272

Kitchen and catering assistants

35-3021.00

9273
9274
9275
9279

Waiters and waitresses
Bar staff
Leisure and theme park attendants
Other elementary services occupations n.e.c.

35-3031.00
35-3011.00
39-3091.00
39-3031.00

9119
9120
9132
9134
9139
9211

111

Janitors and Cleaners, except Maids andHousekeeping Cleaners
Highway Maintenance Workers
Maids and Housekeeping Cleaners
Laundry and Dry-Cleaning Workers
Refuse and Recyclable Material Collectors
Cleaners of Vehicles and Equipment
Janitors and Cleaners, except Maids andHousekeeping Cleaners
Security Guards
Parking Lot Attendants
Crossing Guards
Bailiffs
Laborers and Freight, Stock and Material Movers,
Hand
Order Fillers, Wholesale and Retail Sales
Laborers and Freight, Stock and Material Movers,
Hand
Medical Assistants
Combined Food Preparation and Serving Workers,
Including Fast Food
Waiters and Waitresses
Bartenders
Amusement and Recreation Attendants
Ushers, Lobby Attendants and Ticket Takers

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0112

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

REFERENCES
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro,
C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S.,
Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R.,
Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore,
S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B.,
Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan,
V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke,

Autor, D. H., 2013. The “task approach” to labor markets: an
overview. Tech. Rep. 7178, IZA discussion paper.
Autor, D. H., Dorn, D., 2013. The growth of low-skill service
jobs and the polarization of the US labor market. American
Economic Review 103 (5), 1553–97. URL http://www.aeaweb.
org/articles?id=10.1257/aer.103.5.1553

M., Yu, Y., Zheng, X., 2015. TensorFlow: Largescale machine

Autor, D.., L. F., Murnane, R., 2003. The skill content of recent

learning on heterogeneous systems. Software available fromtechnological change: an empirical exploration. Quarterly

tensorflow.org

Journal of Economics 116 (4), 1279–1333.

Abeliansky, A., Prettner, K., 2017. Automation andBaehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M.,demographic change. Tech. Rep. 05-2017, Hohenheim

Hansen, K., Muller, K.-R., 2010. How to Explain Individual

Discussion Papers in Business, Economics and Social

Classification Decisions. Journal of Machine Learning

Sciences Discussion Paper.

Research 11, 1803–1831. URL https://is.tuebingen.mpg.de/

Acemoglu, D., Aghion, P., Lelarge, C., Van Reenen, J., Zilibotti, F.,fileadmin/user{_}upload/ files/publications/baehrens10a{_}[0].pdf

2007. Technology, information and the decentralization of

Banerjee, A., Duflo, E., 2008. What is middle class about thethe firm*. The Quarterly Journal of Economics 122 (4), 1759.

middle classes around the world? Journal of Economic

URL +http://dx.doi.org/10.1162/qjec.2007.122.4.1759

Perspectives 22 (2), 3–28.

Acemoglu, D., Autor, D., 2011. Skills, tasks and technologies:
Implications for employment and earnings. In: Handbook ofLabor Economics. Vol. 4b.
Acemoglu, D., Restrepo, P., 2017a. Robots and jobs: Evidence
from US labor markets.
Acemoglu, D., Restrepo, P., 2017b. Secular stagnation?
the effect of aging on economic growth in the age of
automation.
Adalet McGowan, M. andrews, D., 2015. Labour market

Bank of America Merrill Lynch, 2017. Overdrive – global future
mobility primer.
Barany, Z. L., Siegel, C., Mar. 2017. Job polarizationand structural change. American Economic Journal:
Macroeconomics. URL https://kar.kent.ac.uk/61271/
Baumol, W. J. and Bowen, W. G. 1966. Performing Arts: The
Economic Dilemma. New York: The Twentieth Century Fund.
Baumol, W. J., de Ferranti, D., Malach, M., Pablos-Méndez, A.,
Tabish, H., Wu, L. G., 2012. The cost disease: why computers

mismatch and labour productivity: Evidence from PIAAC

get cheaper and health care doesn’t. Yale university press.

data. Tech. Rep. 1209, OECD Economics Department

URL http://www.jstor.org/stable/j.ctt32bhj9

Working Papers.
Alpert, A., Auyer, J., October 2003. Evaluating the BLS 1988–
2000 employment projections. BLS Monthly Labor Review.
Armstrong, S., Sotala, K., Ó hÉigeartaigh, S. S., Jul. 2014. The

Beblavý, M., Maselli, I., Veselkova, M., 2015. Green, Pink &Silver? The Future of Labour in Europe. SSRN Scholarly
Paper ID 2577743, Social Science Research Network,
Rochester, NY.

errors, insights and lessons of famous AI predictions – andBecker, S., Hornung, E., Woessmann, L., 2009. Catch me if you

what they mean for the future. Journal of Experimental &can – education and catch-up in the industrial revolution.

Theoretical Artificial Intelligence 26 (3), 317–342.

Tech. Rep. 2816, CESifo Working Paper Series.

Arnold, R. D., Wade, J. P., 2015. A definition of systems thinking:Becker, S. O. and Muendler, M. 2015. Trade and tasks:a systems approach. Procedia Computer Science 44,

an exploration over three decades in Germany, Economic

669–678.

Policy, 30(84), 589-641.

Arntz, M., Gregory, T., Zierahn, U., 2016. The risk of automation

Ben-Ner, A., Jones, D. C., 1995. Employee participation, ownershipfor jobs in OECD countries: A comparative analysis. Tech.

and productivity: A theoretical framework. Industrial Relations:

Rep. 189, OECD Social, Employment and Migration Working

A Journal of Economy and Society 34 (4), 532–554.

Papers.
Atkinson, R. D., Wu, J., 2017. False alarmism: technologicaldisruption and the U.S. Labor market, 1850–2015. Tech.
rep., Information Technology and Innovation Foundation.
Autor, D., Katz, L., Kearney, M., 2006. The polarization of the US
labor market.

Beramendi, P., Häusermann, S., Kitschelt, H., Kriesi, H., Apr.
2015. The Politics of Advanced Capitalism. Cambridge
University Press.
Bix, A., 2000. Inventing ourselves out of jobs?: America’s
debate over technological unemployment. Johns Hopkins
University Press.

Autor, D., Katz, L., Kearney, M., 2008. Trends in US wage

Black, S., Spitz-Oener, A., 2010. Explaining women’s success:inequality: Reassessing the revisionists. Review ofTechnological change and the skill content of women.

Economics and Statistics 90 (2), 300– 323.

Review of Economics and Statistics 92, 187–194.

113

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Brandes, P., Wattenhofer, R., 2016. Opening the Frey/David, P., Wright, G., 1999. General purpose technologies andOsborne black box: Which tasks of a job are susceptible tosurges in productivity: Historical reflections on the futurecomputerization? arXiv:1604.08823 [cs.CY].

of the ICT revolution. Tech. Rep. 31, Oxford University

Broadbent, B., 2013. Forecast errors. Speech given at the MileEnd Group of Queen Mary, University of London, Wednesday
1 May 2013.
Byrd, R. H., Lu, P., Nocedal, J., Zhu, C., 1995. A limited memoryalgorithm for bound constrained optimization. SIAM Journal
on Scientific Computing 16 (5), 1190–1208.
Cappelli, P., 2015. Skill gaps, skill shortages and skill
mismatches: Evidence and arguments for the United States.
ILR Review 68 (2), 251–290.
Capra, F., Luisi, P. L., 2014. The systems view of life: a unifying
vision. Cambridge University Press.
Caprettini, B., Voth, H.-J., 2017. Rage against the machines:

Discussion Papers in Economic and Social History.
Davis, S., Haltiwanger, J., 2014. Labor market fluidity andeconomic performance. Tech. Rep. 20479, NBER Working
Paper.
Deming, D., 2015. The growing importance of social skills in
the labor market. Tech. Rep. 21473, NBER Working Paper.
Dessein, W., Santos, T., 2006. Adaptive organizations. Journalof Political Economy 114 (5), 956–995.
Diamond, J. B., Randolph, A., Spillane, J. P., 2004. Teachers’
expectations and sense of responsibility for student
learning: the importance of race, class and organizational
habitus. Anthropology & Education Quarterly 35 (1), 75–98.

labour- saving technology and unrest in england, 1830-32.

Duffy, M., Armenia, A., Stacey, C. L., 2015. Caring on the clock:

SSRN Scholarly Paper ID 2904322, Social Science Research

the complexities and contradictions of paid care work.

Network, Rochester, NY.

Rutgers University Press.

Carnevale, A., Smith, N., Strohl, J., 2010. Help wanted:

Dustmann, C., Ludsteck, J., Schonberg, U., 2009. Revisiting the

Projections of jobs and education requirements throughGerman wage structure. Quarterly Journal of Economics

2018. Tech. rep., Georgetown Center on Education and the

124 (2), 809–842.

Workforce.

Ecken, P., Von Der Gracht, H., Gnatzy, T., 2011. Desirability bias

CEDEFOP (European Centre for the Development of Vocationalin foresight: Consequences for decision quality based on

Training), 2008. Systems for anticipation of skills needs in

Delphi results. Technological Forecasting and Social Change

the EU member states. Tech. Rep. 1, CEDEFOP Working

78 (9), 1654–1670.

Paper.
Chadha, J., 2017. Why forecast. Tech. Rep. 239, National
Institute Economic Review.
Christodoulou, D., 2014. Seven myths about education.
Routledge Press.

Erixon, F., Weigel, B., 2016. The innovation illusion: how solittle is created by so many working so hard. Yale University
Press.
Freeman, R. B., 2015. Who owns the robots rules the world.
IZA World of Labor 5.

Chu, W., Ghahramani, Z., 2005. Gaussian processes for ordinalFrey, C., Berger, T., 2016. Industrial renewal in the 21st century:

regression. Journal of Machine Learning Research 6 (Jul),

Evidence from US cities. Regional Studies, forthcoming.

1019–1041.

Frey, C. B., Osborne, M. A., Oct. 2014. Agiletown: the relentless

Clark, B., Joubert, C., Maurel, A., 2014. The career prospects

march of technology and London’s response. Tech. rep.,of overeducated Americans. Tech. Rep. 20167, National

Deloitte. URL http://www.deloitte.com/view/en_GB/uk/market-

Bureau of Economic Research Working Paper.

insights/ uk-futures/london-futures/index.htm

Consoli, D., Marin, G., Marzucchi, A., Vona, F., 2016. Do green

Frey, C. B., Osborne, M. A., 2017. The future of employment:

jobs differ from non-green jobs in terms of skills and human

how susceptible are jobs to computerisation? Technologicalcapital? Research Policy, 45 (5), 1046–60.

Forecasting and Social Change 114, 254–280.

Covington, M. V., Müeller, K. J., 2001. Intrinsic Versus Extrinsic

Galor, O., Moav, O., 2006. Das human-kapital: A theory of the

Motivation: An Approach/Avoidance Reformulation.

demise of the class structure. Review of Economic Studies

Educational Psychology Review 13 (2), 157–176.

73 (1), 85–117.

Canadian Scholarship Trust (CST), 2017. CST Careers 2030.
URL http://careers2030.cst.org.
Davenport, T. H., Kirby, J., 2016. Only humans need apply:
winners and losers in the age of smart machines. Harper
Business.
David, P., 1990. The dynamo and the computer: An historicalperspective on the modern productivity paradox. American
Economic Review, 80 (2), 355–361.

Gambin, L., Hogarth, T., Murphy, L., Spreadbury, K., Warhurst,
C., Winterbotham, M., 2016. Research to understand theextent, nature and impact of skills mismatches in theeconomy. Tech. Rep. 265, BIS Research Paper.
Gathmann, C., Schonberg, U., 2010. How general is humancapital? a taskbased approach. Journal of Labor Economics
28 (1), 1–49.
Ghahramani, Z., 2013. Bayesian non-parametrics and theprobabilistic approach to modelling. Phil. Trans. R. Soc. A
371 (1984), 20110553.

114

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Gigerenzer, G., 2010. Moral satisficing: Rethinking moral

Hirsch, E. D., 2016. Why knowledge matters: rescuing

behavior as bounded rationality. Topics in Cognitive Science

our children from failed educational theories. Harvard

2 (3), 528–554.

Education Press.

Goldin, C., Katz, L., 2009. The race between education andtechnology. Harvard University Press.

Hyatt, H., Spletzer, J., 2013. The recent decline in employment
dynamics. IZA Journal of Labor Economics 2 (5).

Goodwin, P., Wright, G., 2010. The limits of forecasting

Inanc, H., Felstead, A., Gallie, D., Green, F., 2013. Job control inmethods in anticipating rare events. Technological

Britain: First findings from the Skills and Employment Survey

Forecasting and Social Change 77 (3), 355–368.

2012.

Goos, M., J., K., Vandeweyer, M., 2015. Employment growth in

Institute of Medicine, 2008. Retooling for an aging America:

europe: The roles of innovation, local job multipliers andbuilding the health care workforce. The National Academies

institutions. Tech. Rep. 10, Utrecht School of Economics

Press, Washington, DC.

Discussion Paper Series.
Goos, M., Manning, A., 2007. Lousy and lovely jobs: the risingpolarization of work in Britain. Review of Economics andStatistics, 89 (1).
Graetz, G., Michaels, G., 2015. Robots at work. Tech. Rep.
10477, CEPR Discussion Paper.
Granovetter, M., 2017. Society and Economy: Framework andPrinciples. Harvard University Press.
Gregory, T., Salomons, A., Zierahn, U., 2016. Racing with oragainst the machine? Evidence from europe. Tech. Rep.
16-053.
Grinis, I., 2017. The STEM requirements of ‘Non-STEM’ jobs:
evidence from UK online vacancy postings and implications
for skills & knowledge shortages. Mimeo. URL https://papers.
ssrn.com/sol3/papers.cfm?abstract_id= 2864225
Guyon, I., Elisseeff, A., 2003. An introduction to variable andfeature selection. Journal of Machine Learning Research 3(Mar), 1157–1182.
Hajkowicz, S. A., Reeson, A., Rudd, L., Bratanova, A., Hodgers,
L., Mason, C., Boughen, N., 2016. Tomorrow’s digitallyenabled workforce: Megatrends and scenarios for jobs andemployment in Australia over the coming twenty years.
Australian Policy Online.
Hampden-Thompson, G., Bennett, J., 2013. Science teaching
and learning activities and students’ engagement inscience. International Journal of Science Education 35 (8),1325–1343.
Handel, M., 2012. Trends in job skill demands in OECD
countries. Tech. Rep. 143, OECD Social, Employment andMigration Working Papers.
Handel, M., 2016. Dynamics of occupational change:
Implications for the occupational requirements survey.
Mimeo.
Hausmann, R., Hidalgo, C. A., Bustos, S., Coscia, M., Simoes,A., Yildirim, M. A., 2014. The atlas of economic complexity:
mapping paths to prosperity. MIT Press.
Heckman, J., 1995. Lessons from the bell curve. Journal of
Political Economy 103 (5), 1091–1120.
Heckman, J., Kautz, T., 2012. Hard evidence on soft skills.
Labour Economics 19 (4), 451–464.

115

International Monetary Fund (IMF), 2017. World economic
outlook, April 2017: Gaining momentum?
Jones, C. I., 2016. Life and growth. Journal of political Economy
124 (2), 539–578.
Kahneman, D., 2011. Thinking, fast and slow. Penguin Press.
Kambourov, G., Manovskii, I., 2009. Occupational specificity
of human capital. International Economic Review 50 (1),
63–115.
Kasparov, G., 2017. Deep thinking: where machine intelligenceends and human creativity begins. John Murray.
Kautz, T., Heckman, J. J., Diris, R., ter Weel, B., Borghans, L., Dec.
2014. Fostering and measuring skills: improving cognitive
and non-cognitive skills to promote lifetime success.
Working Paper 20749, National Bureau of Economic
Research.
Keynes, J. M., 1930. Economic possibilities for our
grandchildren. Essays in Persuasion (1963), pp. 358-73.
W.W. Norton.
King, M., Ruggles, S., Alexander, J. T., Flood, S., Genadek, K.,
Schroeder, M. B., Trampe, B., Vick, R., 2010. Integrated
public use microdata series, current population survey:
Version 3.0.[machine-readable database]. Minneapolis:
University of Minnesota 20. URL http://doi.org/10.18128/
D030.V4.0
Kruse, D., Freeman, R., Blasi, J., Buchele, R., Scharf, A., Rodgers,
L., Mackin, C., 2004. Motivating employee-owners in ESOP
firms: Human resource policies and company performance.
In: Employee Participation, Firm Performance and Survival.
Emerald Group Publishing Limited, pp. 101–127.
Lazear, E. P., Shaw, K. L., 2007. Personnel economics: The
economist’s view of human resources. The Journal of
Economic Perspectives 21 (4), 91–114.
Levy, F., Murnane, R., 2004. The new division of labor: how
computers are creating the next job market. Princeton
University Press.
Lin, J., 2011. Technological adaptation, cities and new work.
Review of Economics and Statistics 93 (2), 554–574.
Liu, Y., Grusky, D., 2013. The payoff to skill in the third
industrial revolution. American Journal of Sociology 118 (5),1330–74.

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

Lucas, B., Claxton, G., Spencer, E., 2013. Progression in student
creativity in school. OECD education working papers, Organisation
for Economic Co-operation and Development, Paris.
MacCrory, F., Westerman, G., Alhammadi, Y., Brynjolfsson,E., 2014. Racing with and against the machine: Changes
in occupational skill composition in an era of rapid
technological advance. Thirty-fifth International Conferenceon Information Systems, Auckland.
MacKay, D. J., 2003. Information theory, inference and learningalgorithms. Cambridge University Press.
ManpowerGroup, 2016. The talent shortage survey. URL http://
manpowergroup.com/talent-shortage-2016
Matthews, A. G. d. G., van der Wilk, M., Nickson, T., Fujii,K., Boukouvalas, A., León Villagrá, P., Ghahramani, Z.,
Hensman, J., 2016. GPflow: A Gaussian process library using
TensorFlow. arXiv preprint 1610.08733.
McKinsey Global Institute, 2017. A future that works:
Automation, employment and productivity.
Mercier, H., Sperber, D., 2017. The enigma of reason. Harvard
University Press.
Michaels, G., Natraj, A., Reenen, J. V., 2009. Has ICT polarized

Nesta, 2017. Solved! Making the case for collaborative
problem-solving.
Occupational Information Network (O*NET), 2017. O*NET
online. URL https://www.onetonline.org
Ocejo, R. E., 2017. Masters of craft: old jobs in the new urban
economy. Princeton University Press.
Organisation for Economic Co-operation and Development(OECD), 2012. Better Skills, Better Jobs, Better Lives: A
Strategic Approach to Skills Policies.
OECD, 2016a. Fostering and assessing students’ creativity andcritical thinking in higher education. In: Workshop Summary
Report.
OECD, 2016b. Getting skills right: assessing and anticipatingchanging skill needs.
OECD, 2016c. How good is your job? Measuring and assessing
job quality. In: OECD Employment Outlook. pp. 79–139.
OECD, 2016d. OECD Employment Outlook 2016.
OECD, 2017. PISA 2015 Results (Volume 1): Excellence and Equity.
Office for National Statistics (ONS), 2017a. emp04:
employment by occupation. URL https://www.ons.

skill demand? Evidence from eleven countries over 25 years.

gov.uk/employmentandlabourmarket/ peopleinwork/

Review of Economics and Statistics 96 (1), 60–77.

employmentandemployeetypes/datasets/

Miller, R., 2006. Futures studies, scenarios and the “possibilityspace” approach. Schooling for Tomorrow, 93–105.
Mims, C., 2016 September 25. Self-driving hype doesn’t reflectreality. The Wall Street Journal.
Mitchell, T., Brynjolfsson, E., 2017. Track how technology istransforming work. Nature 544, 290–292.
Mokyr, J., Vickers, C., Ziebarth, N., 2015. The history oftechnological anxiety and the future of economic growth:
Is this time different? Journal of Economic Perspectives 29(3), 31–50.
Montt, G., 2015. The causes and consequences of field-of-

employmentbyoccupationemp04
ONS, 2017b. Labour force survey 2017. URL https://discover.
ukdataservice.ac.uk/series/?sn=2000026
Pearson, K., 1901. On lines and planes of closest fit to systemsof points in space. Philosophical Magazine Series 6 2 (11),559–572. URL http://dx.doi.org/10.1080/14786440109462720
Phelps, E. S., 2013. Mass Flourishing: How Grassroots
Innovation Created Jobs, Challenge and Change. Princeton
University Press.
Pierson, P., 2004. Politics in time: history, institutions andsocial analysis. Princeton University Press.

study mismatch: An analysis using PIAAC. Tech. Rep. 167,

Poletaev, M., Robinson, C., 2008. Human capital specificity:

oECD Social, Employment and Migration Working Papers.

Evidence from the dictionary of occupational titles andMoretti, E., 2012. The new geography of jobs.
Houghton Mifflin Harcourt.
Mosca, I., Wright, R., 2013. Is graduate under-employment
persistent? evidence from the United Kingdom. Tech. Rep.
6177, IZA Discussion Paper.
Murphy, K. P., 2012. Machine learning: a probabilistic
perspective. MIT press.
National Research Council. 2012. Education for life andwork: Developing transferable knowledge and skills in the
21st century. The National Academies Press. https://doi.
org/10.17226/13398.
Nemet, G., Anadon, L., Verdolini, E., 2016. Quantifying theeffects of expert selection and elicitation design on
experts’ confidence in their judgments about future energy
technologies. Risk Analysis 7 (2), 315–330.

116

displaced worker surveys, 1984- 2000. Journal of Labor
Economics 26 (2), 387–420.
Poropat, A., 2009. Meta-analysis of the five-factor model of
personality and academic performance. Psychological
Bulletin 135 (3), 322–338.
PwC, 2016. The future of work – A journey to 2022.
PwC, 2017. Consumer spending prospects and the impact ofautomation on jobs. UK Economic Outlook, March.
Rasmussen, C., Williams, C., 2006. Gaussian processes formachine learning. MIT Press.
Reimers, F., Chung, C. (Eds.), 2016. Teaching and learning forthe twenty- first century: educational goals, policies andcurricula from six nations. Harvard Education Press.

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Robinson, C., 2011. Occupational mobility, occupation distance

UK Commission for Employment and Skills, 2014. The future ofand basic skills: Evidence from job-based skill measures.

work: Jobs and skills in 2030. Tech. Rep. Evidence Report 84.

Tech. Rep. 2011–5, cIBC Working Paper Series.
Rosen, S., 1983. Specialization and human capital. Journal ofLabor Economics 1 (1), 43–49.
Rothwell, J., 2013. The hidden STEM economy. Metropolitan
Policy Program at Brookings. URL http://kstp.com/kstpImages/
repository/cs/files/ TheHiddenSTEMEconomy610.pdf

Şahin, A., Song, J., Topa, G., Violante, G., 2014. Mismatch
unemployment. American Economic Review, 104 (11),3529–64.
Schleicher, A., 2015 December 17. How can we equip thefuture workforce for technological change?. World Economic
Forum Agenda (blog).
Schneider, P., Armstrong, H., Bakhshi, H., 2017. The Futureof US Work and Skills: trends impacting on employment in
2030. Tech. rep., Nesta/- Pearson.
Schunk, D. H., Zimmerman, B. J., 2007. Influencing children’s
self- efficacy and self-regulation of reading and writing
through modeling. Reading & Writing Quarterly 23 (1), 7–25.
Shackle, G., 1972. Epistemics and economics: a critique of
economic doctrines. Transaction Publishers.
Shah, J., Wiken, J., Williams, B., Breazeal, C., 2011. Improved

UK Commission for Employment and Skills, 2017a. LMI for all.
URL http://www.lmiforall.org.uk/about-lmi-for-all/
UK Commission for Employment and Skills, 2017b. LMI for all.
URL http://api.lmiforall.org.uk
US Bureau of Labor Statistics, 2010. 2010 SOC user guide. URL
https://www.bls.gov/soc/soc_2010_user_guide.pdf
US Bureau of Labor Statistics, 2015. May 2015 occupational
employment statistics. URL https://www.bls.gov/oes/2015/
may/oes_nat.htm
US Bureau of Labor Statistics, 2016. Replacement Needs. URL
https://www. bls.gov/emp/ep_table_110.htm.
Van Rens, T., 2015. The skills gap: Is it a myth?. Global
Perspectives Series: Paper 5, Social Market Foundation.
Waegeman, W., De Baets, B., Boullart, L., 2008. ROC analysis in
ordinal regression learning. Pattern Recognition Letters 29(1), 1–9.
Weaver, A., Osterman, P., 2017. Skill demands and mismatch in
US manufacturing. ILR Review 70 (2), 275–307.
Weinstein, R. S., 2002. Reaching Higher. Harvard University
Press.

humanrobot team performance using chaski, a human-

Woolley, A., Chabris, C., Pentland, A., Hashmi, N., Malone,

inspired plan execution system. In: Proceedings of the 6thT., 2010. Evidence for a collective intelligence factor inInternational Conference on Human–Robot Interaction. HRI

the performance of human groups. Science 330 (6004),’11. ACM, New York, NY, USA, pp. 29–36.

686–688.

Shiller, R. J., 2017. Understanding Today’s Stagnation. URL
https://www.project-syndicate.org/commentary/ secularstagnation-future-of-work-fears-by-robert-j--shiller-2017-05.
Simonite, T., 2016. Prepare to be underwhelmed by 2021’s
autonomous cars. MIT Technology Review.
Spitz-Oener, A., 2006. Technical change, job tasks and rising
educational demands: Looking outside the wage structure.
Journal of Labor Economics 24 (2), 235–270.
Sutherland, J., 2012. Qualifications mismatch and skills
mismatch. Education + Training 54 (7), 619–632.
Syverson, C., 2013. Will history repeat itself? comments on "Is
the information technology revolution over?" International
Productivity Monitor 25, 37–40.
Taylor, M., 2017. Good work: The Taylor review of modern
working practice. Tech. rep., Department for Business,
Energy and Industrial Strategy.
Tekinbas, K. S., Gresalfi, M., Peppler, K., Santo, R., Gee, J.
P., 2014. Gaming the system: designing with gamestar
mechanic. MIT Press.
Tett, G., 2017. An anthropologist in the boardroom.
Financial Times
Tichy, G., 2004. Over-optimism among experts in assessmentand foresight. Technological Forecasting and Social Change
71 (4), 341–363.

117

World Bank, 2013. World Development Report 2013: Jobs.
World Bank Publishing.
World Economic Forum (WEF), 2016. The future of jobs:
Employment, skills and workforce strategy for the fourthindustrial revolution. World Economic Forum.
Wyatt, I., September 2010. Evaluating the 1996-2006
employment projections. BLS Monthly Labor Review.

THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

TECHNICAL TERMS GLOSSARY
Application Programming Interface (API)A system of tools and resources in an operating system,enabling developers to create software applications
Area Under The Curve (Auc)The area between a graph curve and the ‘x’ axis, between
two given ‘x’ values, regardless of whether the area isabove or below the ‘x’ axis.
Baumol’s cost disease hypothesis
Is the rise of salaries in jobs that have experienced noincrease of labor productivity, in response to rising salaries
in other jobs that have experienced the labor productivity
growth [also known as the baumol effect]
Bayesian non-parametric model
A term used in statistics and the creation of machinelearning algorithms. A key problem in statistical modelingis how to choose a model at an appropriate level ofcomplexity. Bayesian nonparametric methods, are a classof statistical methods that enables the data to inform thecomplexity of the model.
Limited memory Broyden–Fletcher–Goldfarb–
Shanno (BFGS)An optimisation algorithm used in machine learning tosolve mathematical challenges that are non linear). It isparticularly well suited for optimisation problems with alarge number of variables.
Covariance Matrix
In statistics, a covariance matrix is generated to investigate
the similarities or differences of two variables acrossmultiple dimensions.
CrosswalkA term deployed to describe a mechanism or approach
to translating, comparing or moving between metadata standards (http://marinemetadata.org/guides/
mdatastandards/crosswalks) or converting skills or content
from one discipline to another.
Crosswalked
A process for matching up the elements or variables of
one list with the closest equivalent on another. In the caseof this study, UK occupation categories were “crosswalked”
to US occupation categories so that the US-based O*NETdata set (containing occupation skills, knowledge areas
and abilities), could be applied across both countries.
Delphi Method
A forecasting method based on the results ofquestionnaires sent to a panel of experts. Several roundsof questionnaires are sent out and the anonymous
responses are aggregated and shared with the group after
each round.

118

Dimensionality Reduction
In machine learning and statistics, dimensionality
reduction or dimension reduction is the process of
reducing the number of random variables underconsideration, via obtaining a set of principal variables.
It can be divided into feature selection and feature
extraction.
Feature Selection
In machine learning and statistics, feature selection is
the process of selecting a subset of relevant features(variables, predictors) for use in model construction
Gaussian Process
In probability theory and statistics, a Gaussian process isa particular kind of statistical model where observations
occur in a continuous domain, e.g. time or space. In a
Gaussian process, every point in some continuous input
space is associated with a normally distributed randomvariable.
Green Economy
Defined as an economy that aims at reducing
environmental risks and ecological scarcities and thataims for sustainable development without degrading
the environment. It is closely related with ecologicaleconomics, but has a more politically applied focus.
Heteroskedastic
in statistics, heteroskedasticity is when the standard
deviations of a variable, monitored over a specific amountof time, are nonconstant.
Information-Theoretic Approach
information-theoretic approach: a model for testing
the data which simultaneously evaluates hypotheses bybalancing between model complexity and goodness of fit.
Labour Market Information (LMI)
Describes all kinds of information used to make labour
market decisions. LMI can be a compilation of detailed
statistical data on jobs and salaries, employers andemployees, sectors, current employment conditions andfuture trends.
Machine-Learning
Is an application of artificial intelligence (AI) that provides
systems the ability to automatically learn and improvefrom experience without being explicitly programmed.
Machine learning focuses on the development ofcomputer programs that can access data and use it learn
for themselves.
Matérn CovarianceIn statistics, the Matérn covariance (named after the
Swedish forestry statistician Bertil Matérn) is a covariancefunction used in spatial statistics, geostatistics, machinelearning, image analysis and other applications ofmultivariate statistical analysis on metric spaces.

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0Multinomial Distribution
A distribution that shows the likelihood of the possible
results of an experiment with repeated trials in which eachtrial can result in a specified number of outcomes that isgreater than two.
Noisy
statistical noise is a term that refers to the unexplained
variation or randomness that is found within a given data
sample.
Non-Parametric Extrapolation
extrapolation is the action of estimating or concluding
something by assuming that existing trends will continue.
When using a non-parametric method to do this, theboundaries of what is possible come from the data (or the
training set), rather than the statistical model.
Occam’s Razor
A scientific and philosophical rule that entities should
not be multiplied unnecessarily which is interpreted as
requiring that the simplest of competing theories be
preferred to the more complex or that explanations of
unknown phenomena be sought first in terms of known
quantities.
Ordinal Regression Modela type of regression analysis used for predicting an ordinalvariable, i.e. a variable whose value exists on an arbitraryscale where only the relative ordering between different
values is significant.
Over-Fitting
A modeling error which occurs when a function is too
closely “fit” to a specific set of data points, limiting its
generalisability.
Python Library
A Python library is a collection of functions and methods
that allows you to perform lots of actions without writing
your own code.
Principal Component Analysis (Pca)
Principal component analysis is an approach to factor
analysis that considers the total variance in the data,
which is unlike common factor analysis and transforms theoriginal variables into a smaller set of linear combinations.
Time-Series
A time series is a sequence of numerical data points insuccessive order.
Uncertainty Sampling
An Active Learning approach in which the Machine
Learning Algorithm selects the Documents as to which itis least certain about Relevance, for Coding by the Subject
Matter Expert(s) and addition to the Training Set.

119THE F UT URE OF SKI L L S: EMPL OYMEN T I N 2 0 3 0

ENDNOTES
automatability at a more granular level. Second, differences
may arise from the classification method used by Arntz
et al (2016). Their modified logistic regression implies alinear relationship between features and whether a job isautomatable. This is a simpler and less flexible model thanFrey and Osborne’s and again is likely to default towards
predictive probabilities in the middle. Third, the authors
include a number of variables such as gender, education,
income, sector and firm size as predictors of automatability,
even though these are not obviously supported orinterpreted in terms of economic theory. In addition PwC(2017) finds that some of these results are an artefact ofwhich variables from the PIAAC database are used. Its own
estimates, using a different set of occupational features, arestill lower than those of Frey and Osborne, but still muchcloser to them than to the estimates of Arntz et al.

Emblematic of this interest was the publication of the
BLS Occupational Outlook Handbook and the OECD’s
Mediterranean Regional Project which popularised
the use of manpower planning in developed anddeveloping countries.

1

2

Ireland, for example, uses foresight in its various sector
studies, while Germany’s BIB-BIAB Qualification andOccupational Fields programme produces qualitative
scenarios to contrast their baseline quantitative
projections.
Making predictions about technological progress is also
notoriously difficult (Armstrong et al., 2014).

3

The direction of employment change – in both absoluteand relative terms – was projected accurately for 70% of
detailed occupations included in Handel’s evaluation.

4

An interesting study on the short-run dislocations oflabour-saving technology is Caprettini and Voth (2017).
It examines the diffusion of threshing machines throughthe English countryside during the 1830s and its impact
on social unrest, the so-called ‘Captain Swing’ riots. To
measure diffusion, the study uses farm advertisements
in local newspapers, which provide detail on the location
and use intensity of new threshing machines. The take-up
of new technology was not exogenous, making causal
assignment difficult. For example, landlords, alarmed by
the outbreak of violence, might have introduced fewer
machines, which would bias estimates downwards. To
identify causality, the authors use soil suitability for wheat
as an instrument for the adoption of threshing technology.
The reason for this is that wheat was the only grain that
could be threshed profitably by early threshing technology.
This instrument is valid insofar as it does not affectrural workers’ propensity to riot other than through its
effect on the adoption of technology. Thus, among otherthings, wheat-growing areas were not poorer than otherareas. The authors find that areas more suited to wheat
cultivation exhibit both greater adoption of threshingmachines and significantly higher incidence of riots.

5

By mimic, we refer to the ability of a machine to replicate
or surpass the results of a human rather than achieve
those results in the same way.

6This poses a puzzle – if Arntz et al. (2016) proceed froma similar assessment of occupation level automatability
as Frey and Osborne (2017), why do they arrive at such
sharply different results? While the task-based approach
may explain some of the difference, it is arguably
overstated by aspects of their research design. This canbe understood on three levels. First, the PIAAC data
is available only at a two-digit International Standard
Classification of Occupations (ISCO) level, in contrast
to data on detailed occupations used by Frey andOsborne. Studying occupations in aggregate is likely topush employment towards the medium risk category
insofar as it washes out variation between occupations’

7

120

8

See also Rosen (1983) on the indivisibility of occupations.

9In the 19th century, 98% of the labour required to weave
cloth was automated but employment in the weaving
industry still increased due to increased demand forcheaper clothes.

10

Exceptions include the public sector and non-automatedmanufacturing industries such as recycling, basic metals,

The occupations labelled come from the US Bureau ofLabor Statistics (BLS) 2010 SOC and the UK Office for
National Statistics (ONS) SOC 2010.

11

The selection of the first 10 occupations presented tothe experts was random: specifically, 10 occupations
were randomly selected, but individual occupations were
replaced with another randomly selected occupation
when historical time-series were not available at least backto 1983 for the US and 2001 for the UK. This constraint
meant that, in the US, we drew the occupations from 125of the total 840 six-digit SOC codes and from 163 of thetotal 369 four-digit SOC codes in the UK. textiles, paper,
furniture and transportation equipment.

12

As explained in 4.2, we adopt six-digit US SOC codes for
US data. When we use US O*NET data for similar UK
occupations, we present UK occupations at the four-digit
UK SOC code level. The US and UK systems are the samelevel of detail, in that they refer to detailed occupations.

13 

According to O*NET, skills represent developed
capacities which facilitate learning or the more rapid
acquisition of knowledge; abilities are enduring
attributes of the individual which influence performanceand knowledge refers to organised sets of principles andfacts applying in general domains.

14 

No dataset equivalent to O*NET exists for UK
occupations. Instead, we perform a bespoke crosswalk
from UK occupations to the closest match US
occupation, as determined by using the 'LMI for All'
database (UK Commission for Employment and Skills,
2017a) with some custom changes that we explain.

15 

T HE FUT URE OF SKI L L S: EM PL OYM ENT I N 2 0 3 0

Consider engineers and metal workers and plastic
workers – two occupation groups which the BLS predicts
will decline in share between 2014 and 2024. Over
this period, metal workers and plastic workers are
anticipated to lose 99,000 jobs, whereas engineers are
expected to add 65,000 jobs.

16 

17

 bservations consisted only of the ternary-valued
Olabels and participant uncertainty was not incorporated.
Reflecting the dataset used in Frey and Osborne (2017)
only nine features were used, namely: Originality Systems
Evaluation, Hearing Sensitivity, Arm-Hand Steadiness,
Learning Strategies, Oral Comprehension, Social
Perceptiveness, Manual Dexterity, Problem Sensitivity.

18

 ote that the Pearson correlation coefficient is oftenNused for feature selection (as a filter), an exception to
the inappropriate information-theoretic methods offeature selection we generically describe above.

19

 o be precise, this entails only redefining theT
expectations above as employment-weighted sums
over that subset of occupations, rather than over
all occupations.

20

 ur definition of complementarity is loosely relatedOto that used by economists: two features arecomplementary if the marginal value product of one
is increasing in the level of the second. This definition
fails to meet our needs. The first problem is that thedefinition makes no accommodation for location infeature space. The economics definition is a statement
about the second-order derivative of the function
with respect to the two features being positive; for an
arbitrary function, as may be learned by our flexible
non-parametric model, the second-order derivatives
may be in very different regions of space. The second
related problem with the definition is that it may leadto highlighting feature combinations which, even if thesecond-order derivatives are positive and constantacross space, are actively harmful. As a simple example,
for a bivariate quadratic function with positive-definite
Hessian (a convex bowl), an occupation on the wrong
side of the critical point (the minimiser, or the locationof the bottom of the bowl) would see its demanddecreased with increases in either or both of thefeatures. Our means of assessing complementarity,
however, will more correctly identify the differing
importances of features combinations at any point infeature space.

21

 his is clearest in the case of ‘Public services and otherT
associate professionals’ and ’Welfare professionals’.
However all other three-digit public sector occupations
i.e. ‘Senior officers in protective services’, ’Protective
services’, ‘Quality and regulatory professionals’, ’Healthassociate professionals’ and ’Welfare and housing
associate professionals’ have an average probabilityof increased workforce share > 0.5.

121

22

 reen occupations are defined here as ‘Waste disposalGand environmental services’, ‘Conservation professionals’,‘Environment professionals’, ‘Environmental healthprofessionals’ and ‘Conservation and environmental
associate professionals’ (four-digit level).
Specifically, Bright Outlook occupations are ones
that: are projected to grow much faster than average(employment increase of 14% or more) over the period
2014–2024; have 100,000 or more job openings overthe period 2014–2024; or are new and emerging
occupations in a high-growth industry.

23 

124