NBER WORKING PAPER SERIESREMOTE WORK ACROSS JOBS, COMPANIES, AND SPACE
Stephen HansenPeter John Lambert
Nicholas Bloom
Steven J. Davis
Raffaella Sadun
Bledi Taska
Working Paper 31007
http://www.nber.org/papers/w31007

NATIONAL BUREAU OF ECONOMIC RESEARCH
1050 Massachusetts Avenue
Cambridge, MA 02138
March 2023
Thanks for outstanding research assistance to Yabra Muvdi, who built and estimated theclassification algorithm, and to Miaomiao Zhang and Kelsey Shipman, who supported the data
analysis. Hansen gratefully acknowledges financial support from ERC Consolidator Grant
864863, Lambert from the London School of Economics STICERD PhD research grant and theCommonwealth Scholarship Commission, Bloom from the Smith Richardson and John
Templeton Foundations, Davis from the Templeton Foundation and the Booth School of Businessat the University of Chicago, and Sadun from Harvard Business School. Selected visualisations
and data accompanying this paper can be found at www.WFHmap.com. The views expressedherein are those of the authors and do not necessarily reflect the views of the National Bureau of
Economic Research.
At least one co-author has disclosed additional relationships of potential relevance for thisresearch. Further information is available online at http://www.nber.org/papers/w31007
NBER working papers are circulated for discussion and comment purposes. They have not been
peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies
official NBER publications.
© 2023 by Stephen Hansen, Peter John Lambert, Nicholas Bloom, Steven J. Davis, Raffaella
Sadun, and Bledi Taska. All rights reserved. Short sections of text, not to exceed two paragraphs,
may be quoted without explicit permission provided that full credit, including © notice, is given
to the source.

Remote Work across Jobs, Companies, and Space
Stephen Hansen, Peter John Lambert, Nicholas Bloom, Steven J. Davis, Raffaella Sadun,
and Bledi Taska
NBER Working Paper No. 31007
March 2023
JEL No. C55,E24,M54,O33,R3
ABSTRACTThe pandemic catalyzed an enduring shift to remote work. To measure and characterize this shift,
we examine more than 250 million job vacancy postings across five English-speaking countries.
Our measurements rely on a state-of-the-art language-processing framework that we fit, test, andrefine using 30,000 human classifications. We achieve 99% accuracy in flagging job postings thatadvertise hybrid or fully remote work, greatly outperforming dictionary methods and also
outperforming other machine-learning methods. From 2019 to early 2023, the share of postings
that say new employees can work remotely one or more days per week rose more than three-foldin the U.S and by a factor of five or more in Australia, Canada, New Zealand and the U.K. Thesedevelopments are highly non-uniform across and within cities, industries, occupations, andcompanies. Even when zooming in on employers in the same industry competing for talent in thesame occupations, we find large differences in the share of job postings that explicitly offer
remote work.
Stephen Hansen
University College London
30 Gordon Street
Drayton House
London WC1H 0AX
United Kingdom
stephen.hansen@ucl.ac.uk

Steven J. Davis
Booth School of Business
The University of Chicago
5807 South Woodlawn Avenue
Chicago, IL 60637
and NBER
Steven.Davis@ChicagoBooth.edu

Peter John Lambert
London School of Economics
and Political Science (LSE)
p.j.lambert@lse.ac.uk

Raffaella Sadun
Harvard Business School
Morgan Hall 233
Soldiers Field
Boston, MA 02163
and NBER
rsadun@hbs.edu

Nicholas Bloom
Stanford University
Department of Economics
579 Jane Stanford Way
Stanford, CA 94305-6072
and NBER
nbloom@stanford.edu

Bledi Taska
Lightcast
66 Long Wharf, Floor 2
Boston, MA 02110
bledi.taska@emsibg.com

A data repository is available at https://wfhmap.com

1

Introduction

The COVID-19 pandemic propelled an enormous uptake in hybrid and fully remote work.
Over time, it has become clear that this shift will endure long after the initial forcing event.
Looking forward, U.S. survey data say that one-quarter of full workdays will happen at homeor other remote location after the pandemic ends, five times the pre-pandemic rate (Barrero
et al., 2021). The pandemic also drove large, enduring increases in remote work in dozens
of other countries (Criscuolo et al. 2021, Aksoy et al. 2022). There are few, if any, modern
precedents for such an abrupt, large-scale shift in working arrangements.
Most previous efforts to quantify and characterize this shift rely on surveys of workers
and employers and assessments of remote-work feasibility by occupation. We rely insteadon the information contained in job vacancy postings. Specifically, we consider the full textof over 250 million postings in five English-speaking countries. In doing so, we apply stateof-the-art language-processing methods to analyze the text and determine whether the job
allows for remote work. We fit, test, and refine our language-processing model using 30,000
classifications generated by human readings. We also identify the city, employer, industry,
occupation, and other attributes associated with each job vacancy.
Vacancy postings pertain to the flow of new jobs rather than the stock.1 In addition,
postings that promise remote work two days a week, for example, entail a commitment–or atleast a statement of intent–that extends into the future. For both reasons, postings need notshow the same pattern of remote work as the currently employed. Indeed, the remote-workshare of postings lags far behind the remote-work share of employment in the pandemic’s
early stages. And while the incidence of remote work among the employed fell markedly inthe two years after spring 2020, we show that the remote-work share of postings rose sharply
over the same period.
Our approach to studying the remote-work phenomenon has several noteworthy strengths.
First, our data cover almost all vacancies posted online by job boards, employer websites,
and vacancy aggregators from 2014 to 2022 in our five countries. Coverage on this scale isinfeasible with survey methods. Second, postings typically describe the job and its attributes
in considerable detail, as suggested by a median posting length of 347 words. Comparable
1

Vacancy distributions by industry, employer size, and worker turnover rates differ greatly from the corresponding employment distributions, and the differences are highly sensitive to labor market conditions. See
Davis et al. (2012).

1

detail is hard to obtain from other sources, especially at scale.2 Third, we apply frontier
methods to develop a language-processing model that reads and classifies postings in an
automated manner. The model achieves a 99% accuracy rate in flagging jobs that allow for
remote work, greatly outperforming dictionary methods. Our model also outperforms a variety of other methods. Fourth, the combination of scale, rich text data, and automation lets
us characterize the shift to remote work in a highly granular manner. We track the evolution of remote work at a monthly frequency in hundreds of occupations, thousands of cities,
tens-of-thousands of employers, and in city-by-occupation and employer-by-occupation cells.
We post many of these statistics at www.WFHmap.com, with frequent updates.
The share of postings that say new employees can work remotely one or more days per
week was tiny before the pandemic: 1% or less in Australia, Canada and New Zealand as of2019, about 3% in the U.K., and about 4% in the U.S. From 2019 to 2022, this remote-workshare rose more than three-fold in the U.S and five-fold or more more in the other countries.
As of January 2023, the remote-work share exceeds 10% of postings in Australia, Canada,
the U.K, and the U.S., and it appears to be on an upward trajectory in all five countries.
Remote-work posting shares vary greatly across occupations, industries, and cities. Looking across occupations, the remote-work share correlates positively with computer use, education, and earnings. Finance, Insurance, Information and Communications have especiallyhigh remote-work shares. Chicago, London, New York, San Francisco, Toronto, and other
cities that function as business service hubs have high remote-work shares. These differences
have widened since the pandemic struck. According to a linear least-squares regression, 63%of the variation across occupations in 2022 remote-work shares is accounted for by their 2019
shares. In contrast, just 28% of city-level variation in 2022 remote-work shares is accounted
for by 2019 shares.
We also find that the shift to remote work is highly non-uniform across same-industry
employers, even when they are recruiting in the same occupational category. This emergent
heterogeneity on the demand side expands opportunities to satisfy preferences over remote
work on the supply side.3 Our non-uniformity result also carries another important message:
in many occupations, it is misleading to think of remote-work suitability as a purely technological constraint. Remote-work intensity is, instead, an outcome of choices about job design
2Previous research exploits the detail in vacancy postings to study technical change, the cyclicality of skill
requirements, their relationship to wages, how compensation and other job attributes affect applicant flows
and, of course, to classify jobs in a fine-grained manner. Examples include Modestino et al. (2016), Deming
and Kahn (2018), Hershbein and Kahn (2018), Davis and Samaniego de la Parra (2020), Forsythe et al.
(2020), Marinescu and Wolthoff (2020), and Acemoglu et al. (2022).
3
On preference heterogeneity in regards to remote work, see Bloom et al. (2015), Mas and Pallais (2017),
Wiswall and Zafar (2018), Barrero et al. (2021), and Aksoy et al. (2022).

2and how to operate an organization. These choices are influenced by the external environment and subject to shock-induced shifts. In line with this view, Aksoy et al. (2022) findthat employers plan higher levels of work from home after the pandemic ends in countries
that experienced longer and stricter government-mandated lockdowns during the pandemic.
Several recent papers use job postings to study the remote-work phenomenon. See Draca
et al. (2022) for the U.K., Alipour et al. (2020) for Germany, and Bamieh and Ziegler (2022)
for Austria. Bai et al. (2021) use pre-pandemic postings in the U.S. to construct firmlevel indexes of remote-work feasibility, which they relate to post-pandemic performance asmeasured by sales, net income, and equity returns. Perhaps the closest forerunner to ourpaper is Adrjan et al. (2021), who use postings data at the country-sector-month level tostudy remote work from January 2019 to September 2021. Previous studies use dictionary
methods (keyword search criteria) to identify postings that allow for remote work.
In contrast, we apply a large-scale language-processing framework to the task, an approach that is rapidly diffusing in data-science applications but, thus far, is little used ineconomics. First, we pre-train the DistilBERT framework (Sanh et al., 2020) on one million text chunks drawn from vacancy postings.4 This step familiarizes DistilBERT with the(heterogeneous) structure of job ads. Second, we consider 10,000 text chunks drawn fromvacancy postings and assign three human readers to label each one. The reader assigns apositive label if the text says the job allows work from home (or other remote location) one
or more days per week. Third, we fit the pre-trained DistilBERT framework to the humanlabels to obtain a model that classifies each posting as follows: the job allows hybrid or
fully-remote work arrangements, or it does not. Finally, we apply the resulting “Work from
Home Algorithmic Measure” (WHAM) to classify all 250 million job postings.
WHAM greatly outperforms a previously used dictionary in classifying the remote-workstatus of vacancy postings. The dictionary method yields high classification error rates thatvary greatly over time and across occupations. Expressions like “home or office working
possible” and “work from home care facilities” and “requires a Home Office work permit”
suggest some of the difficulties that arise when applying dictionary methods to job ads.
Logistic regressions and GPT-3 offer large improvements over dictionary methods. WHAM
offers even larger improvements, substantially outperforming all other methods when applied
to vacancy postings, as measured by accuracy rates, precision, and F1 scores.
Few previous works in economics combine a frontier language-processing framework with
4

DistilBERT is a smaller, faster version of the BERT framework introduced by Devlin et al. (2019), whichhas 60,000 Google Scholar citations as of February 2023. BERT and DistilBERT exploit machine-learning
tools and are pre-trained on the full English-language Wikipedia corpus and the Toronto Book Corpus. Fora helpful non-technical overview of BERT, see Luktevich (2022).

3

human-generated labels to develop an automated classification model and to quantify its
performance. In the only example we know, Shapiro et al. (2022) develops a BERT-basedmodel and finds little gain relative to dictionaries in detecting news sentiment.5 However,they use fewer than 1,000 human-labeled text examples in fitting their BERT-based model,
which may explain why it yields small performance gains.
A prominent line of research classifies occupations as suitable or unsuitable for remote
work based on descriptions of work activities and experiences.6 Our analysis highlights
some limitations of this approach. First, remote-work intensity is a malleable feature ofjobs, occupations, and organizations. Second, classifications based on suitability assessments
explain little of the variation in remote-work posting shares. For occupations that Dingel andNeiman (2020) classify as unsuitable to be done entirely from home, the remote-work shareof U.S. postings in 2022 ranges from 0 to 50% with a mean of 5% and standard deviation of
7%. For occupations they classify as suitable for work from home, the share ranges from 0.3
to 74% with a mean of 18% and standard deviation of 12%.
Another prominent line of research surveys workers and employers to study working
arrangements. Barrero et al. (2020), Bartik et al. (2020), Bick et al. (2022) and Brynjolfsson
et al. (2020) document and characterize the enormous uptake in work from home in spring2020. Bartik et al. (2020), Barrero et al. (2021) and Ozimek (2020) use employer plansand other forward-looking survey data to forecast that the big shift to remote work will
endure. Relative to our approach, the survey-based approach is more useful for eliciting
the perceptions, attitudes, and expectations of workers and employers. Our approach offers
several other distinct advantages, as discussed above.
The next section describes our vacancy posting data and develops our classification model.
Section 3 assesses the model’s performance in absolute terms, and relative to other approaches. Section 4 sets forth our main findings related to remote-work intensity over timeand across countries, cities, occupations, and more. We also compare our remote-working
posting shares to survey-based measures of remote work. Section 5 concludes.
5

Bajari et al. (2021) and Bana (2022) use BERT to predict prices from Amazon product reviews and wages
from job posting text, respectively. Each paper achieves high predictive performance. Their applications
don’t involve the use of human-generated labels.
6
Dingel and Neiman (2020) is the most influential example. Other examples include del Rio-Chanona et al.
(2020), Mongey et al. (2021), and Adams-Prassl et al. (2022). Like us, Adams-Prassl et al. (2022) concludes
that remote-work intensity varies greatly across jobs within occupations.

42

Data and Measurement

To measure remote-work posting shares, we exploit a near-universe of online job postingsfrom January 2014 through January 2023 for our five countries.
We extract 10,000 text sequences from selected postings and ask humans to read them.
Each sequence is about 45 words long, and the average posting has about six sequences.
Breaking postings into sequences facilitates human and algorithmic classification at scale,as we discuss below. Our human readers answer this question: ‘Does this text explicitlyoffer an employee the right to remote-work one or more days a week?’, yielding a binaryclassification. The pairwise agreement rate between readers exceeds 90 percent.
We turn to the ‘DistilBERT’ language-processing framework to build a text-classification
model for our purposes.7 DistilBERT is pre-trained on thousands of books and the Englishlanguage Wikipedia corpus, which helps it interpret the intended meaning of a given document or passage. We further pre-train on roughly one million text sequences drawn from our
corpus of vacancy postings. This further pre-training step familiarizes the framework with
the nature of the text in vacancy postings.
After pre-training, we use the human labels to train, or fit, a bespoke classification
model for predicting these labels. We call this model the ‘Working-(from)-Home Algorithmic
Measure’ (WHAM ). We will show that WHAM achieves near-human performance in its
classification task, and that it outperforms a variety of other approaches. We describe ourapproach in some detail, because we think it has useful applications to many other textanalysis tasks in economics and other fields.

2.1

Job Vacancy Data

We examine online vacancy postings collected by Lightcast (formerly Emsi Burning Glass),
an employment analytics and labor market information firm. Lightcast scrapes postingsfrom more than fifty thousand online sources that incluide vacancy aggregators, government
job boards, and employer websites. Lightcast claims to cover a “near-universe” of online
postings in our five countries during the period covered by our analysis. See Appendix A fora detailed description of our data and pre-processing steps.
7

DistilBERT is a direct descendant of BERT, which is widely used in industry. BERT stands for Bidirectional
Encoder Representations from Transformers. Transformers are a deep-learning method in which every
output element is connected to every input element of a text sequence. This allows the meaning of aparticular word to depend on the context of surrounding words, which as we show below is crucial in our
setting. See Phuong and Hutter (2022) for a formal overview of how Transformers work. Vaswani et al.
(2017) is the seminal contribution.

5

Burke et al. (2020) compare vacancy postings in Lightcast data for the United States tojob vacancy data from the U.S. Job Openings and Labor Turnover Survey (JOLTS). The twosources are reasonably well aligned, but the JOLTS data show larger vacancy shares in foodservices, public administration and construction and smaller shares in finance, insurance,
healthcare, social assistance and educational services.
For each online vacancy posting in our dataset, we have access to a plain text document
scraped from the job listing. We also observe the posting date, employer name, occupation,
location of the employer, industry, and more. We consider postings listed from January
2014 and January 2023, dropping those with an unknown occupation (less than 1%. We
use a 5% random sample of postings before January 2019, and the universe of postings
thereafter. The resulting dataset covers more than 250 million online vacancy postings infive countries, spanning 5.2 million employers and nearly 40 thousand cities. Table 1 provides
more information.
For our baseline results, we re-weight the postings in each country-month cell to match
the U.S. occupational distribution of new online vacancy postings in 2019. Appendix B
reports selected results for alternative weighting schemes.

2.2

The Measurement Problem

The measurement problem we face is to determine whether each job posting allows a new hire
to work remotely, understood here to encompass both fully remote and hybrid positions. We
adopt a binary classification approach, and refer to a ‘positive’ posting as one that mentions
the ability to work remotely, and a ‘negative’ posting as one that does not. For positions
that offer hybrid working arrangements, we use a threshold of at least one day per week
for our positive classification8 This approach effectively measures an employer’s willingness
to commit ex ante to offering flexibility in work location. Negative postings may in fact
be associated with work-from-home positions, for example because the ability to work fromhome is assumed by market participants to be feasible in particular jobs, or because theemployer prefers to bargain over work arrangements during the hiring process rather thanmake a prior binding commitment. We return below to discuss the interpretation of ourmeasure, and first focus on developing an accurate and robust classification.
The most precise way of classifying postings is arguably via direct human reading. Given
the size of our data, however, this approach is not feasible to scale and some means ofautomated classification is required. The most standard approach adopted in the text-as8

In principle our measurement approach could be extended to the intensive margin (days per week), but for
simplicity we begin with the this binary classification.

6

data literature in economics is to use a dictionary of keywords whose presence is assumed
to indicate a positive classification. As an initial step, we use the keywords in Table C.1
to classify job postings as positive or negative. While we do not claim the dictionary ofterms is fully optimized, it is in line with others in the literature for classifying postings aswork-from-home (Adrjan et al., 2021).
An issue that becomes immediately apparent upon inspecting job postings that are classified by keywords is the presence of notable errors, which Table 2 illustrates. False positives
include references to companies’ home offices and working in homes dedicated to healthcare provision. A second, and perhaps most worrying, source of false positives is that thestructure of job ads shifts during COVID-19 in a way correlated with the presence of false
positives. This is due to the fact that, after early 2020, many postings feature a new text
field indicating whether home work is allowed, and then explicitly state it is not—a naive
application of the dictionary method would infer from this text field that the job postingallows working from home.9 Table 2 also lists examples of false negatives, which illustrates
the many and complex ways that companies can use to describe remote work. Accounting
for this linguistic variety with a fixed set of keywords is a major challenge.

2.3

Our Approach to Classification

Our approach to address the classification errors in the dictionary approach has three steps.
First, we use at least three human auditors to read and classify 10,000 pieces of text extracted
from job ads which produces 30,000 labels. Second, we train a modern machine learningalgorithm using these human classifications. Third, we take this predictive model out-ofsample to classify each job ad as either positive or negative. The hope is to scale theaccuracy of human reading—which can only be deployed on a small fraction of data—to theentire dataset. While this approach is common in the machine learning literature, it is notoften used in economics, even though it appears to hold great promise. We call the final
model used in this paper the Working-from-Home Algorithmic Measure (or WHAM) model.
The main text contains an overview of our methodology, with further details in Appendix
C.
9

One approach to correcting this problem is to extend the dictionary to incorporate negation (e.g. to treat
as a negative classification the phrase ‘this is not a remote work position’). In section 3 we show that thisindeed improves measurement accuracy but not by as much as our proposed solution below.

72.3.1

Breaking Up Job-Ad Text into Sequences

While we ultimately wish to classify job postings, we initially label and classify smaller unitsof text we refer to as sequences. The first reason for doing so is that human labeling of entire
job postings is prone to a high error rate because of their length and complexity. The second
reason is that the typical posting has a great deal of information unrelated to remote work,for example descriptions of the skills required for the job, the tasks involved, etc. Mixing
text relevant for work-from-home with a great deal of irrelevant text introduces noise into
the classification algorithm.
The procedure for generating sequences has three salient features. First, postings always
begin with a job title, e.g. “Software Programmer familiar with R and Python.” We extract
these as a single sequence. Second, the beginning of each posting typically has a number ofbullet points or other structured fields. In most cases, these also form a single sequence.10
Finally, the remainder of a posting is typically structured like standard prose with a succession of paragraphs. Each paragraph is taken as a single sequence, unless it passes a length
threshold. In this case, we break it into multiple sequences of consecutive sentences.
This procedure produces approximately 1.6 billion sequences out of the over 250 million
job postings.
2.3.2

Human Labels for Training and Evaluation

From the sample of sequences, we first chose 10,000 to label manually. One quarter of thesesequences was chosen at random from the set of sequences that contained a set of dictionary
terms listed in Table C.1. Another quarter was chosen to contain a broad set of terms
that might reflect work-from-home language, including the generic terms ‘remote’, ‘home’,‘work’, ‘location’; any word that begins with ‘tele’; and any two-word sequence that begins
with ‘remote’. Another quarter consisted of sequences that might confound a classifier,including ‘home repairs’, ‘nursing home’, ‘remote construction’, etc. The final quarter was arandom sample of sequences not satisfying the three aforementioned criteria. Each portionof the label sample is balanced across year-quarter from 2014Q1 through 2021Q3. We alsobalance the sample evenly across countries11 to account for varying English idioms in different
geographic locations.
We used Amazon Mechanical Turk to generate labels. To ensure high-quality workers,
10

The exception is if the number of distinct structured fields is too large, in which case we split them into
multiple sequences.
11
We draw one quarter of this dataset from each of USA, UK, Canada, and a further one quarter from thepooled Australia and New Zealand data.

8

we set up an initial screening test that required prospective workers to label 20 sequencesthat we had previously manually classified. Only workers that made at most one error wereallowed to proceed to label the full set. Another quality control strategy was to pay around
25% above typical market rates for labeling tasks. This motivated workers who passed initial
screening to continue on the project.12
Each of the 10,000 sequences were labeled by three distinct workers. There is a highagreement rate among workers: 66.9% of examples are unanimous negative examples and25.5% are unanimous positive examples. The remaining 7.6% examples are evenly balanced
between one dissenting vote for either positive or negative. Note that, while half of thesample was chosen to contain a word with the potential to denote work-from-home, only
29.2% of the sequences receive a majority of positive votes.
2.3.3

Developing the WHAM model

In the last five years, the field of natural language processing has been revolutionized bymodels that allow the meaning of word sequences to arise by how they interact. Consider thesentences ‘Some of the deep-sea wells we operate are in remote locations’ and ‘We are pleased
to offer opportunities for remote work’. Each includes the word ‘remote’ but only the latter
is a positive example of remote work. The important point is that the interaction of ‘remote’with surrounding context words determines the overall meaning of these sentences. Moreover,
not all context words are equally informative: for example, in the first sentence ‘deep-sea’,‘wells’, and ‘locations’ are more important than ‘some’ and ‘we’ in understanding the meaningof ‘remote’. Self-attention (Vaswani et al., 2017) is a mathematical construct that allows
vector representations of individual words to interact with each other to form new vectorsthat encode the meaning of sequences. These interaction weights effectively determine which
words should be “paid attention to” in resolving these meanings. Self-attention is the key ideathat powers models such as BERT, RoBERTa, GPT, GPT-3, PALM, and, most famously,
ChatGPT. For more details on these models, collectively called Transformers, see Ash and
Hansen (2023).
The particular Transformer model we adopt in the development of WHAM is DistilBERT(Sanh et al., 2020). DistilBERT is based on Google’s BERT model (Bidirectional Encoder
Representations from Transformers, Devlin et al., 2019), which when originally released set
important new performance benchmarks for common NLP tasks (since eclipsed by larger12

In general workers appeared engaged and focused on the labeling task. We received communication frommultiple workers seeking to clarify ambiguous cases, which went above and beyond what AMT requiredfor payment.

9scale models). Since 2020, Google has used BERT to process its online search queries.
DistilBERT ‘distills’ the information in BERT by training a simplified model to reproduce
the same output as BERT. The main advantage is that DistilBERT obtains an expressive
language model with far fewer parameters which reduces processing time and estimation
costs.
We make two main modifications to the off-the-shelf DistilBERT model to build WHAM.
First, the initial set of parameters of off-the-shelf DistilBERT is obtained by predicting
randomly deleted words in generic English from surrounding context words. We instead
update these parameters to predict randomly deleted words in a sample of 900,000 job
posting sequences which is balanced across all years and countries. This step creates word
representations that are specific to the language of job postings.
Second, we further modify off-the-shelf DistilBERT to predict human labels from vector
representations of job posting sequences. We split our labeled sequences into training andtest sets of 5,950 and 4,050 sequences, respectively. The prediction problem is conducted at
the label rather than sequence level, so there are 3 * 5,950 = 17,850 total observations in thecore training sample of labeled data.13 Appendix C details how we specify the prediction
model’s hyper-parameters. Table 3 provides an illustration of which words are influential
in the classification problem, and compares the WHAM approach to a dictionary approach.
Crucially, the weights attached to words are learned by the algorithm rather than imposed
ex ante by researchers, and the weight on a particular word depends on the surroundingwords. In the following section, we compare the test-set accuracy of the estimated modelwith that of other algorithms in the literature and show its performance is outstanding.
2.3.4

Predicting Remote Work Language at Scale

Finally, we use the estimated prediction model to assign a continuous probability to all
sequences in our corpus. The higher the probability, the more confidence the model has that
this sequence denotes an offer of remote work arrangements. Figure B1 plots a histogram ofthe share of sequences that fall in different probability intervals. The distribution is bimodal
at the lowest and highest probability bins, with the former dominating the distribution. As
expected, most sequences do not contain work-from-home language because, as we show
below, most job postings do not explicitly mention the possibility to work from home and,among those that do, the majority of sequences discuss other features of the position. The
13

During an initial exploratory phase, we labeled a sample of around 10,000 additional sentences (rather
than sequences) using a combination of Mechanical Turk, hired research assistants, and ourselves. Since
these are also potentially informative, we include them in the training set. In most cases, these sentences
only received a single label and so in total generate 11,574 additional labels in the training set.

10

bimodality of the distribution shows that the classification algorithm typically produces aclear prediction, in line with human labelers’ high agreement rates. We use an 0.5 thresholdfor assigning a sequence a positive classification according to WHAM’s predicted probability,
but the properties of the predicted probability distribution imply that our results are notsensitive to this particular cutoff.
2.3.5

Aggregating Measurement back to Job Postings

We have conducted all the analysis so far at the sequence level, but are ultimately interested
in a job-posting-level classification. For this, we use a simple ‘max’ rule and positively
classify a job posting if it contains one or more positive sequences. Table B.1 shows thenumber of positively classified sequences in each job ad. We can see that among the positive
job ads (those with one or more positive sequence), the majority have just a single positive
sequence. This reduces concern that the algorithm produces correlated false positive hits
at the posting level.14 This posting-level classification constitutes the final output from our
WHAM model, which we use to study the adoption of remote work.
2.3.6

Public Access to WHAM

To allow researchers to interact with and study the properties of our model, we make available
a simple online tool that allows one to input arbitrary text and receive a predicted probability
as output. The URL is https://huggingface.co/spaces/yabramuvdi/wfh-app-v2, which
will reproduce the same probabilities as in the paper.15 One can verify that the examples inTable Table 2 that confound dictionary approaches are correctly classified by WHAM.
2.3.7

Computational Performance of WHAM

One constraint on implementing large-scale NLP models is computational. To provide some
performance guidelines, Table Table B.2 tabulates the hardware we use for each step of development, the time taken, and the cost involved. All estimation is done on the Google Cloud
Platform. In neither time nor money terms is the implementation of WHAM particularly
14

We have manually read a number of randomly drawn postings with more than five positive sequences, andfound no instance of the algorithm failing. In some cases, the scraping procedure that gathers data fromonline job portals appears to have identified as a single job ad a succession of postings by recruitment
agencies. In other words, the measurement error arises from the data itself rather than the classification
approach.
15
Many thanks to Yabra Muvdi for estimating the model and making it accessible. The model is subject torevision, at which point the predicted probabilities for a given text may change. Users who find systematic
biases in the predictions are welcome to contact the authors with their findings, which can be incorporated
into future work.

11costly from a computational perspective: the total run time for all steps is 51 hours, and thetotal cost is approximately $1,500. Our view is that researchers should therefore not view
computational costs as a major impediment to adopting large language models.
On request, we make available all code to efficiently train WHAM and apply it out-ofsample. Interested researchers should register interest at WFHmap.com.

3

Assessing the Performance of WHAM

Above we highlight instances in which the presence or absence of keywords is insufficient
to correctly classify a selection of job posting texts due to the complexity of surrounding
context. In order to quantify the gains from adopting our approach, we now undertake asystematic comparison of the ability of different algorithms to correctly classify unseen texts.
To do so, we adopt a standard approach in the machine learning literature and randomly
split the 10,000 human-labeled sequences into training and test sets (of sizes 5,950 and 4,050,
respectively). We then train WHAM just on the training data and use the fitted model toassign a predicted value to each test-set observation. By way of comparison, we also use
the following alternative methods for classifying test-set observations (full details of eachapproach are in Appendix C):
1. All Zero. Each test-set observation is assigned a 0 to match the modal outcome.
2. Dictionary. We use a term set similar to that from Adrjan et al. (2021),16 and countan observation as positive if it contains a term from this set.
3. Dictionary with Negation. Shapiro et al. (2022) shows that accounting for negation can
improve the performance of dictionaries. We adopt a similar method and only countthe presence of a dictionary term as indicating remote work when a negation term does
not appear in the surrounding context.
4. Logistic Regression. Adams-Prassl et al. (2020) uses Lightcast data from the UK tomeasure the prevalence of flexible work schedules, i.e. the times at which work must becompleted, from job posting text. The paper uses humans to manually annotate 7,000
texts, and fits a (penalized) logistic regression model for classification. The features ofthe logistic regression are the word frequencies in a given document. We implement asimilar logistic model on our training data and use it to classify test data.
16

The terms are reported in Table Table C.1. The remote work measures in Adrjan et al. (2021) are basedon data from Indeed which potentially has a different structure from the Lightcast data.

125. Logistic Regression with Negation. We expand the feature set of the logistic regression
to incorporate negation and re-estimate it on the training data.
6. GPT-3. Brown et al. (2020) introduced GPT-3, a large language model capable ofperforming a variety of natural language tasks with limited or no training examples
to learn from. We query the model with the prompt “Identify if the text offers thepossibility of remote work at least one day per week” and convert the answer into a
0/1 classification.17
7. WHAM with Generic English. Rather than pre-train DistilBERT using job posting
text, we use its off-the-shelf word embeddings trained on general English.
Table 4 reports the test-set performance for all methods. A straightforward metric is theerror rate, i.e. the fraction of mis-classified texts. On this measure, WHAM outperforms
all other methods with an error rate of 0.02.18 GPT-3 has an error rate three times that of
the baseline model, while the dictionary method’s error rate is eight times higher. On theother hand, the pre-training of the model with job posting text generates only a modest an
improvement over generic English.
A more standard performance metric in the machine learning literature is the F1 score
which accounts for both a classifier’s ability to recover the true positives (recall ) as well as
the share of predicted positives that are true positives (precision).19 The F1 score varies
between 0 and 1, where higher values indicate better performance. Again, we observe that
WHAM substantially outperforms all other measures.20
One concern is that the distribution of positive and negative postings in the test data does
not correspond to that of the full population of job postings: the data extracted for labelingis specifically designed to over-represent positive cases. To obtain a sense of classification
17

Deploying GPT-3 on the full Lightcast dataset would be prohibitively expensive at current costs, but westill report its test-set performance for benchmarking purposes. More recently, ChatGPT, a successor
model to GPT-3, has generated a great deal of public interest. ChatGPT is largely built on an underlying
model that OpenAI calls text-davinci-03 whereas GPT-3 is built on text-davinci-02. In our experiments,
the former outperforms the latter, so we only report results for GPT-3.
18
This error rate is consistent across countries and years. When broken down by country, the test set error
rate is 0.02 in each case. When broken down by year, the set error rate is 0.02 in each year except for 2015
(error rate 0.03) and 2014 (error rate 0.01).
19
Among a set of classified observations, let TP, FP, TN, and FN be the number of true positives, false
positives, true negatives, and false negatives, respectively. Precision is TP / (TP + FP), and recall is TP/ (TP + FN).
20
An alternative dictionary for measuring remote work adoption is proposed in Draca et al. (2022) whichuses our same UK Lightcast sample. The overall error rate of this dictionary in the full test data set
is 0.19 and for the test data set arising from the UK is 0.17. Interestingly, the F1 score we obtain forlogistic regression (0.81) is similar to that reported by Adams-Prassl et al. (2020) for classifying flexible
work scheduling (0.83, see Table 3 of that paper).

13

accuracy on the full population, we create a simulated dataset of 1000 * 4,050 = 4,050,000
observations, 3% (97%) of which are sampled with replacement from the set of positive(negative) test set examples. Table 4 reports the same metrics as Table 4 but computed on
this more unbalanced dataset. Again, we find that WHAM outperforms all other methods,
but in this case the difference in F1 scores is even starker. Our baseline WHAM achieves a0.85 F1 score, while the F1 score of GPT-3 falls to 0.52 and other methods have even worse
performance. Moreover, pre-training becomes more important as the F1 score for WHAM
with generic embeddings drops to 0.78. These results arise because, as Table 4 shows, WHAM
has a particularly low false positive (FP) rate compared to other methods. When negative
examples dominate the evaluation sample, correctly classifying them becomes importantfor overall performance and WHAM is strong in this dimension. Since this sample’s label
composition is more in line with the expected composition of the universe of job postings,
our findings highlight the potential gains in accuracy of using our approach.
We view these results as methodologically important because they are among the first,
to our knowledge, to quantify the gains of adopting modern NLP methods for text classification in economic environments. There are very few papers in the economics literaturethat systematically compare different classification approaches, and those that do have notfound that large language models outperform simpler approaches. For example, Shapiro
et al. (2022) does not report large gains from using BERT over simpler models for classifying news sentiment. One reason that we, in contrast, do find large gains is the size of ourtraining data. Shapiro et al. (2022) trains BERT on 800 labeled articles whereas we havean order of magnitude more training data, which provides more information for estimating
the complex ways in which word sequences map into outcomes. We conjecture that otherprediction problems using text in economics might similarly benefit from a large training
sample combined with sequence embedding models.
A separate question is how WHAM compares to alternative methods on the full data
sample. Rather than consider all alternatives, we focus on how WHAM compares to the
Dictionary method, which is most common in the literature measuring remote work adoption
from job posting text. Figure 1 plots monthly time series of the share of remote work postings
in the US sample from 2019 through early 2023.21 The patterns present in both series differ
markedly. According to the Dictionary method, the remote work share surged at the onsetof the COVID-19 pandemic, peaked in early 2021, and fell markedly throughout 2021 before
stabilizing in 2022. In contrast, the WHAM method suggests a more modest immediate
21

These time series are computed using the approach we adopt for the baseline results discussed in the next
section, and are not the simple raw positive share.

14

reaction to the pandemic followed by a steady growth rate thereafter. Two features of theDictionary series are of note: First, the initial COVID-19 shock drove a large number ofboth real and negated mentions of remote work arrangements, so this series increases much
more dramatically than the WHAM series. Second, towards the end of 2022 a handfulof very large job boards altered their structure to partially address this issue of negation.
Importantly, this second event appears not to have induced a discontinuity in our WHAM
measure, likely because it is robust to changes in structure so long as the intended meaning
remains consistent. Clearly, then, the choice of measurement approach can have important
quantitative implications even in aggregate.22
Of course, aggregate comparisons between methods can mask underlying differences atmore granular levels. To illustrate this, we compute the growth rate in remote work adoption
according to the Dictionary method and WHAM from 2019 to 2022 for individual SOC2
occupations, pooling all 2019 postings and 2022 postings together. In these two years, theDictionary method appears similar to WHAM but with an upward shift of around five
percentage points. However, as Figure 2 reveals, there are large differences in the specificoccupations that each method associated with growth in remote work adoption. According
to the Dictionary method, the ‘Food Preparation and Serving’ occupation has experienced
highest growth in adoption, while for WHAM the highest-growth occupation is ‘Computerand Mathematical’. Moreover, according to WHAM all occupations experienced positive
growth in adoption, whereas adoption rates fall for the ‘Farming, Fishing, and Forestry’occupation according to the Dictionary method. The higher accuracy of WHAM in thesample of human labels suggests its ranking of occupations is more reliable. In the next
section we provide a more in-depth analysis of occupation-level heterogeneity according to
WHAM.
In sum, the WHAM model displays a very high classification accuracy—relative to humanlabels—and differs markedly from the most popular alternative approach in the literature
based on keyword search. This difference is especially pronounced since 2020, even at theaggregate level. We believe our approach to measurement provides a highly accurate classification of remote work offers in the text of job postings, and base the remainder of thepaper on analyzing its output.
22

The patterns in the Dictionary series need not match those from Adrjan et al. (2021) even though we use
a similar set of keywords, as the structure of the Lightcast data could differ in important ways from thatof the Indeed data that Adrjan et al. (2021) use.

15

4

Results

In this section we document how the percent of new remote work vacancies–the fraction of
all new vacancies which explicitly offer the right to work remotely one or more days per
week–has changed over time. We document this across countries, occupations, cities, andemployers. This covers both hybrid and fully remote work.
This section is organised as follows: First, we look at the percent of remote work vacancies
across each of our five countries. We plot this as a monthly time series, spanning January
2014 to January 2023. Second, we compare the percent of new remote work vacancies acrossbroad and narrowly defined occupations, contrasting our measurements in 2019 2022. Weshow that the substantial rise since the onset of COVID is highly uneven across occupations,and find that occupations with the highest 2019 percentage of remote work were the most
likely to top the list in 2022. We also compare occupation-level classifications used in theliterature to our measurement. Third, we compare the percentage of new vacancies offering
remote work arrangements across cities. We show that cities with higher remote work percentages in 2019 do not strongly predict higher percentages by 2022 (unlike occupations).
This suggests that additional confounding city-level characteristics have played an importantrole in the adoption of remote work. We also compare a monthly time series across a selection of US cities. Fifth, we compare our measures to survey information from the American
Communities Survey (ACS). We show that MSA’s which have a high remote work share ofvacancies in our data also have high fractions of the population who selected “Worked fromhome” when asked about their commuting methods. Fifth, we show that the percentage of
remote work vacancies posted by employers who operate in the same industry, and search
for the same talent, can vary widely.

4.1

Remote Work across Countries

How did the share of advertised hybrid and fully remote work differ across countries priorto, during and after the pandemic? In Figure 3 we plot the monthly time series of the shareof advertised remote work for the US, UK, Canada, Australia and New Zealand. For eachcountry and in each month, this figure reports the weighted-mean of the percent of remote
work vacancies across nearly 800 narrow occupation groups. We weight each group basedon the share of vacancies in this group in the USA during 2019. Our baseline results utilise
this method to reduce the impact of compositional differences, both across time and acrosscountries. Three high-level facts emerge:
1. Unprecedented and sharp increase of advertised remote work at the onset
16

of COVID-19
In March-April 2020, the share of new job vacancies which advertised remote work
saw a sharp rise across all countries. On average, the increase from February 2020 toApril 2020 was 200%. While this immediate increase occurred across all our countries,
the level-change was most pronounced in countries with a more severe initial COVID
outbreak (USA, UK and Canada).
2. Sustained growth thereafter
Since the large spike in March-April 2020, there has been sustained growth in the percent of advertised remote work. In level-terms, this growth has been most pronounced
in the UK (where COVID lockdowns most lingered and were most severe relative tothe other countries in the sample). We also see evidence of higher growth rates in Australia and NZ, as their pandemic experience worsened during 2021. In all countries,
the growth in advertised remote work has continued long after the forcing event of thepandemic subsided. An additional reason for this high and persistent growth is that
our measure of new job vacancies lags the stock of employees actually working from
home, possibly because employers were slow to accept this as a permanent practice.
3. Substantial heterogeneity across countries, even before the pandemic
The USA had nearly 4% advertised remote work share in 2019, the highest of any
country. The UK was only marginally lower, where as Australia, Canada and New
Zealand had respectively half, a third, and a tenth the share of the US in 2019. By
mid 2022 the spread in levels is much greater, but proportional differences have reduced.
In our robustness exercises, we also look at the raw shares of remote work, i.e. without
the re-weighting applied to our baseline Figure 3. Comparing the unweighted Figure B.2
to Figure 3 tells us the direction and magnitude of the impact that occupation composition
plays in our results. For example, in mid 2022 the difference between the UK and USA is
8 percentage points using the raw data and 4 percentage points after re-weighting. Thissuggests that roughly half of the difference in advertised remote work shares between the
US and UK is accounted for by differences in the types of jobs being advertised, which isunsurprising as the UK is on the whole more skewed towards white-collar jobs with a higher
propensity to be worked from home.

17

4.2

Remote Work across Jobs

We first show the share of advertised remote work by grouping job ads into broad occupationgroups (based on two-digit SOC 2010 classifications), which Figure 4 reports. For this, welook only at data from the United States. The differences across broad occupation groups
varies greatly. In 2019, we see that just one-in-twenty positions of all job ads in ‘Computerand Mathematical’ occupations explicitly offered remote work arrangements in their postings,
whereas in 2022 this share raises to a more one-third of new ads offering remote work. As
one might expect, the share of advertised remote work correlates positively with computeruse, education, and earnings and is lower in occupation groups which require specialised
equipment or customer interactions. Lastly, Figure 4 provides some evidence that the 2019
shares of remote work correlates with post-pandemic shares.
To investigate the relationship between 2019 and 2022 shares further we next turn toan analysis at the detailed ONET occupation-level. We group our US job vacancies into
granular occupations (using O*NET definitions), and plot both the 2019 and 2022 percent
of advertised remote work (on a log-scale), presented in Figure 5. After dropping a handfulof data points with fewer than 250 postings in 2019, we retain 875 O*NET occupations.23
Figure 5 also shows the feasibility classification according to Dingel and Neiman (2020). A
black circle represents jobs which these authors classify as ‘not suitable for full-time telework’,and an orange triangle denotes the opposite 24 . An unweighted ordinary-least-squares trendline is also depicted in blue. Our main takeaways from Figure 5 are:
• The bivariate unweighted-OLS fit using a log-log specification yields an R2 of 0.63,
which shows that–for a given ONET occupation–the share of vacancies which advertised remote work pre-pandemic was strongly predictive of the share post-pandemic.
• The slope coefficient from the bivariate unweighted-OLS model shows that the elasticityof 2019 percent to 2022 percent was 0.76%.25• Across all ONET occupations depicted, the mean share of new postings which advertised remote work was 4% in 2019 and 10% in 2022.
• There is substantial variation in the share of advertised remote work across occupations,which grows over time. Across all ONET occupations, the standard deviation in the
23

In total, there are 1,016 O*NET occupations. Our sample of O*NET codes which have greater than 250
vacancy postings in 2019 is 875. This attrition is expected, for example a number of military occupations
are not present in our data.
24
These are taken from the authors replication data, accessed April 2022, which can be found here.
25
Our ordinary least-squares estimates impose a power-law coefficient, given the log-log specification.

18

shares of advertised remote work was 5% in 2019 and 11% in 2022.
• Dingel and Neiman (2020)’s classification can account for a small part of the variationin the 2022 levels of advertised remote work. For occupations that they classify as
unsuitable to be done entirely remotely, the share of advertised remote work in 2022
ranges from 0 to 51% with a mean of 5% and standard deviation of 7%. For occupations
they classify as suitable for telework, the share ranges from 0.3 to 74% with a mean of
18% and standard deviation of 12%.26
We view three key points of difference between our measurement approach and thosemeasures which assess telework feasibility for each occupation. First, since our measurement
works at the job vacancy level and not the occupation level, our measure offers more variation
and a signal heterogeneity in remote work feasibility within occupations and across firms.
Second, whereas the feasibility measures treat each job as a collection of tasks, our measure
combines both task-feasibility as well as employer and employee preferences, labour market
forces, past experience with remote arrangements, and so on.27 The third reason for the discrepancy is that our measurement exercise will likely have some amount of under-reporting,as employers may not explicitly advertise remote work in their vacancies but none-the-lessallow such arrangements.

4.3

Remote Work across Cities

Next we compare the percent of new vacancy postings which advertised remote work acrosscities. Job posting are matched to a city based on specific locations listed on the website
from which it was scraped, or else mentioned in-text.28
26

In a few cases, the D&N machine classification appears very inaccurate. For example, travel agents
have been classified as ‘not teleworkable’, although both before and after the pandemic roughly 1-in-3
jobs advertised remote work. This is likewise the case for ‘Advertising Sales Agents’ and ‘Interpreters &Translators’. Some of these outliers appear to be resolved by the hand coded measure, but these data are
only available at a higher level of occupational-aggregation.
27
A clear example of the differences between our measurement approach and Dingle and Neiman (2020) is
for teaching jobs. For example, while D&N correctly classify jobs for “physical education teaching” asbeing feasible for full time home working (i.e. via a virtual class room), we know anecdotally that thisarrangement was very taxing on staff and avoided as soon as normal schooling resumed. We find thatteaching jobs in general (and “physical education teachers” in particular) have some of the lowest sharesof advertised remote work of any job, highlighting that feasibility and actual behaviour can vary markedly.
28
Since the predominant remote work arrangements are hybrid, the location of the work site remains a key
feature of most jobs. However, in the case of a ‘fully remote’ position this analysis becomes less precise. We
plan to refine our measurement approach in future work to distinctly classify ‘hybrid’ vs ‘fully remote’ work
arrangements, but have thus far concluded that the majority of remote work jobs offer hybrid arrangements

19

Figure 6 shows the percent of advertised remote work across a selection of large international cities, both for 2019 and 2022. We see that the percentages vary widely. For example,
in 2022, 1-in-4 new job postings in Washington (DC) advertised remote work arrangements,
compared to 1-in-14 in Perth, Australia. The substantial increases as well as the large
heterogeneity in these shifts can be seen both across- and within-countries.
Further evidence of the large shift in both levels and spread of remote work ads is shownin Figure 7, which plots all cities in our data with more than 250 new vacancies in 2019.
The mean (standard deviation) increased from 4% (2%) in 2019 to 10% (5%) in 2022. An
unweighted OLS regression line fitted to these city-level data show a much lower coefficientof determination (R2 ) of 0.28, compared to the value of 0.63 when running the same exercise across occupation-groups. This highlights that the 2019 shares are far less informative
predictor of post-pandemic shares at the city level.
This sizable increase in the levels and spread of remote work across cities, as well as
the weak relationship between 2019 and 2022 shares, poses an interesting question: What
are the city-level determinants of remote work adoption? We hypothesize that a mix of
institutional features, infrastructure quality, pandemic severity (both in disease and policy)
and the composition of jobs and firms in each city are all important factors. We leave a moreformal tests of these predictions to future work.
We next turn to more granular monthly time series for selected US cities, shown in Figure
8. As well as illustrating the granularity of our data, a number of interesting features emerge
from these time series:• Cities from the North-East and West regions (e.g. San Francisco, Boston, New York)
all experience similar increases at the outset of the pandemic, but have very different
growth levels subsequently. By 2023, these differential growth rates result in very
dispersed levels.
• We also see substantial fluctuations over time in these North-East and Western cities.
These fluctuations appear to be correlated across series, for example the July 2021 dip
occurs in SF, Boston, Colorado, and to a lesser extent NYC.
• By contrast, cities from the South show far less growth since COVID and also lessvolatility. Savannah and Miami Beach appear to have partially reverted back to prepandemic shares of advertised remote work.
• Note that in this exercise, we do not re-weight the data, such that much of the variation
across cities is likely to be driven by differences in occupation and industry composition.
20

We leave as future work a mapping from our time-series measures and forcing events,
such as shelter-in-place orders.

4.4

Comparing Job Advert Measurement to Survey Responses
across MSAs

Our measurement of remote working utilises new job postings, which is conceptually a very
different empirical object to measures of the share of employees / work days conducted inpeoples homes. To understand how these different measurement approaches might relate (ifat all) to one another, we utilise recent survey evidence form the American Communities
Survey (ACS).29 Specifically, we use the (survey weighted) share of 2021 employees acrossMetropolitan Statistical Area’s (MSAs) who, when asked about their commuting method,
respond that they ‘Worked from home’30 . We compare this to the fraction of new job ads
from each MSA which advertised remote work in 2022.
Figure 9 compares the our measure of remote work from job ads to the ACS’s survey
measure. A least-squares regression line (shown in blue) has a slope coefficient of 0.36.
Strictly interpreted, this suggests that ceteris paribus an MSA with 10% more employees
who respond ‘Working at home’ to the ACS commute question would accompany a 3.5%
increase in the percent of new job ads offering hybrid or fully remote arrangements. The
overall fit of this least squares regression line is rather high, with a coefficient of determination(R2 ) of 0.55. Taken in tandem, this evidence suggests our measurement approach relates
to the stock of remote workers. This, along with the many advantages of working with job
postings (large scale, very high granularity, long historical time series) support the use-casefor our data.

4.5

Remote Work across Companies

Ultimately, the decision to advertise remote work arrangements is made by each employer
who is searching for talent. By-and-large, workers value the flexibility to work some days
remotely, with survey evidence estimating that a typical worker would sacrifice 6% of their29

The American Community Survey (ACS) is a demographics survey program conducted by the U.S. Census
Bureau. The ACS regularly gathers information previously contained only in the long form of the decennial
census, such as ancestry, citizenship, educational attainment, income, language proficiency, migration,
disability, employment, and housing characteristics.
30
ACS respondents are instructed to “Mark (X) ONE box for the method of transportation used for most of
the distance” which suggests that only those who work in a fully remote capacity should select this box,
since persons with 1+ days of commute per week have more mileage from that commute mode.

21

salary to receive this amenity (Barrero et al., 2021). Thus, one important reason why employers have increasingly chosen to offer remote work arrangements even after the pandemic
is to attract workers. Similarly, remote work arrangements can also lessen the burden of
distance and allow firms to recruit for talent in wider geographic areas. Again, this deepens
the labour market and may facilitate matching with better candidates. Another reason why
we see that employers are offering remote work arrangements in their vacancy listings might
be due to learning. Most CEO’s comment that mass remote-work of staff would have beenunthinkable prior to 2020, yet the forced experimentation during COVID-19 has left manywith at least an indifference to such practices and at most tangible evidence of the productivity benefits these bring. Finally, firms–especially those who are expanding quickly–may
see remote work arrangements as a way to reducing office space and energy consumption. On
the other hand, the need to adjust internal processes to a fully or partially remote workforce
may also inhibit firms from explicitly committing to this work arrangement.
Our analysis of employers is by no means exhaustive, and we leave for future work amore in-depth match to firm-level covariates. The first piece of analysis illustrates thatthe prevalence of employers who explicitly offer remote work arrangements in their vacancy
postings varies greatly, even among same-industry firms recruiting in the same occupational
category. Figure 10 takes selected employers, and finds:• Figure 10: Panel A shows the share of remote work vacancies posted by four large
aerospace manufacturing firms (NAICS code 3364). We consider only management
occupations in this panel, and find that both Boeing and Lockheed Martin explicitlyoffer remote arrangements in half of their postings in 2022.31 We further see that
Northrop Grumman makes explicit offers of remote work in less than one-in-four management job vacancy postings. In contrast, SpaceX made no explicit offers for sucharrangements in any of its new job listings in 2022. All of these firms explicitly offered
minimal amounts of remote work in 2019.
• Figure 10: Panel B shows selected insurance firms who advertise vacancies for workers
in the mathematical science occupations. We chose this occupation because we know
it has a high national share of remote work vacancy postings. We see that United
Health had a sizable fraction (52%) of vacancies which explicitly offered remote work,
31

Without further analysis, we cannot say if a 50% share of remote work vacancies in 2022 results fromsome switch in behaviour (e.g. if firms post uniformly in time, and switch to fully remote halfway throughthe year, we would calculate a 50% share) or else if this is driven by some more granular cross-sectional
difference between jobs with and without remote arrangements on offer. This will be addressed in futurerevisions.

22

even pre-pandemic in 2019, which grew to 80% by 2022. Mutual of Omaha had amore modest pre-pandemic remote work share, but by 2022 mentions such practices innearly all vacancies targeting mathematicians. Humana saw more than a doubling in
its remote work share, but remains substantially lower than its peers.
• Figure 10: Panel C conducts the same exercise for selected auto manufacturing firms
who hire engineers. Almost no explicit offers of remote work were made in 2019. During2022, Honda explicitly offers 1-in-2 new engineering hires the right to work remotely
at least one day per week. GM offers less than half this number, and Ford less than a6th. Tesla job postings make almost no offers of remote work in either 2019 or 2022.

5

Conclusion

This paper’s first contribution is to develop a methodology for classifying job postings asoffering fully remote or hybrid work arrangements. We take an off-the-shelf, large-scale
Transformer model and adjust it to both account for the specific language structure of
postings and, more importantly, to predict tens of thousands of human-labeled sequences.
The resulting WHAM algorithm substantially outperforms existing methods in terms of outof-sample classification accuracy, including the language models that underlie GPT-3 andChatGPT. To the best of our knowledge, this is the first attempt in the literature to assess
the gains from adopting large language models for economic measurement, and our results
suggest the promise of the method more broadly.
We then apply WHAM to the full universe of Lightcast data across five English-speaking
countries (USA, UK, Canada, Australia, and New Zealand). This generates a dataset of
remote and hybrid work adoption whose scale, granularity, and high frequency extend wellbeyond what is possible to achieve with surveys. We use this to zoom in on countries, occupations, industries, cities, and firms and, in each case, document a high degree of heterogeneity
in remote work adoption since the pandemic. Moreover, this heterogeneity is not simply afunction of pre-pandemic conditions. For example, the incidence of remote and hybrid work
across cities in 2019 explains relatively little of the cross-city increase in adoption since.
We conjecture that the heterogeneity we document has its roots in myriad forces, including
worker and firm preferences, competitive pressures in the labour market, and local norms.
An important topic for future research, which our measures can help advance, will be toquantify the relative importance of these factors.
Many of the data series in this paper are available through a companion website WFHmap.com
which we will continue to update regularly going forward.
23

References
Acemoglu, D., Autor, D., Hazell, J., and Restrepo, P. (2022). Artificial Intelligence andJobs: Evidence from Online Vacancies. Journal of Labor Economics, 40(S1):S293–S340.
Adams-Prassl, A., Balgova, M., and Qian, M. (2020). Flexible Work Arrangements in Low
Wage Jobs: Evidence from Job Vacancy Data. SSRN Electronic Journal.
Adams-Prassl, A., Boneva, T., Golin, M., and Rauh, C. (2022). Work that can be done
from home: Evidence on variation within and across occupations and industries. Labour
Economics, 74:102083.
Adrjan, P., Ciminelli, G., Judes, A., Koelle, M., Schwellnus, C., and Sinclair, T. (2021). Will
it stay or will it go? Analysing developments in telework during COVID-19 using online
job postings data.
Aksoy, C. G., Barrero, J. M., Bloom, N., Davis, S. J., Dolls, M., and Zarate, P. (2022).
Working From Home Around the World.
Alipour, J. V., Falck, O., and Schüller, S. (2020). Germany’s Capacities to Work from Home.
Ash, E. and Hansen, S. (2023). Text Algorithms in Economics. Unpublished Manuscript.
Bai, J. J., Brynjolfsson, E., Jin, W., Steffen, S., and Wan, C. (2021). Digital Resilience:
How Work-From-Home Feasibility Affects Firm Performance.
Bajari, P., Cen, Z., Chernozhukov, V., Manukonda, M., Wang, J., Huerta, R., Li, J., Leng,
L., Monokroussos, G., Vijaykunar, S., and Wan, S. (2021). Hedonic prices and quality
adjusted price indices powered by AI. Working Paper CWP04/21, Cemmap.
Bamieh, O. and Ziegler, L. (2022). Are remote work options the new standard? Evidence
from vacancy postings during the COVID-19 crisis. Labour Economics, 76:102179.
Bana, S. H. (2022). Work2vec: Using Language Models to Understand Wage Premia. Unpublished Manuscript.
Barrero, J. M., Bloom, N., and Davis, S. J. (2020). COVID-19 Is Also a Reallocation Shock.
Barrero, J. M., Bloom, N., and Davis, S. J. (2021). Why Working from Home Will Stick.
Bartik, A. W., Cullen, Z. B., Glaeser, E. L., Luca, M., and Stanton, C. T. (2020). What Jobs
are Being Done at Home During the Covid-19 Crisis? Evidence from Firm-Level Surveys.
Bick, A., Blandin, A., and Mertens, K. (2022). Work from Home Before and After the
COVID-19 Outbreak.
Bloom, N., Liang, J., Roberts, J., and Ying, Z. J. (2015). Does Working from Home Work?
Evidence from a Chinese Experiment. The Quarterly Journal of Economics, 130(1):165–
24

218.
Brown, T., Mann, B., Ryder, N., et al. (2020). Language Models are Few-Shot Learners. In
Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran
Associates, Inc.
Brynjolfsson, E., Horton, J. J., Ozimek, A., Rock, D., Sharma, G., and TuYe, H.-Y. (2020).
COVID-19 and Remote Work: An Early Look at US Data.
Burke, M., Sasser Modestino, A., Sadighi, S., Sederberg, R., and Taska, B. (2020). No
Longer Qualified? Changes in the Supply and Demand for Skills within Occupations.
Federal Reserve Bank of Boston Research Department Working Papers, Federal Reserve
Bank of Boston.
Criscuolo, C., Gal, P., Leidecker, T., Losma, F., and Nicoletti, G. (2021). The role of teleworkfor productivity during and post-COVID-19. (31).
Davis, S. J., Faberman, R. J., and Haltiwanger, J. C. (2012). Recruiting Intensity during andafter the Great Recession: National and Industry Evidence. American Economic Review,
102(3):584–588.
Davis, S. J. and Samaniego de la Parra, B. (2020). Application Flows.
del Rio-Chanona, R. M., Mealy, P., Pichler, A., Lafond, F., and Farmer, J. D. (2020). Supply
and demand shocks in the COVID-19 pandemic: An industry and occupation perspective.
Oxford Review of Economic Policy, 36(Supplement 1):S94–S137.
Deming, D. and Kahn, L. B. (2018). Skill Requirements across Firms and Labor Markets:
Evidence from Job Postings for Professionals. Journal of Labor Economics, 36(S1):S337–
S369.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep
Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:
Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186,
Minneapolis, Minnesota. Association for Computational Linguistics.
Dingel, J. I. and Neiman, B. (2020). How many jobs can be done at home? Journal of
Public Economics, 189:104235.
Draca, M., Duchini, E., Rathelot, R., Turrell, A., and Vattuone, G. (2022). Revolution in
Progress? The Rise of Remote Work in the UK.
Forsythe, E., Kahn, L. B., Lange, F., and Wiczer, D. (2020). Labor demand in the time of
COVID-19: Evidence from vacancy postings and UI claims. Journal of Public Economics,

25

189:104238.
Hershbein, B. and Kahn, L. B. (2018). Do Recessions Accelerate Routine-Biased Technological Change? Evidence from Vacancy Postings. American Economic Review, 108(7):1737–
1772.
Luktevich, B. (2022). BERT Language Model. https://www.techtarget.com/searchenterpriseai/
definition/BERT-language-model.
Marinescu, I. and Wolthoff, R. (2020). Opening the Black Box of the Matching Function:
The Power of Words. Journal of Labor Economics, 38(2):535–568.
Mas, A. and Pallais, A. (2017). Valuing Alternative Work Arrangements. American Economic Review, 107(12):3722–3759.
Modestino, A. S., Shoag, D., and Ballance, J. (2016). Downskilling: Changes in employer
skill requirements over the business cycle. Labour Economics, 41:333–347.
Mongey, S., Pilossoph, L., and Weinberg, A. (2021). Which workers bear the burden of social
distancing? The Journal of Economic Inequality, 19(3):509–526.
Ozimek, A. (2020). The Future of Remote Work.
Phuong, M. and Hutter, M. (2022). Formal Algorithms for Transformers.
Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2020). DistilBERT, a distilled version of
BERT: Smaller, faster, cheaper and lighter. arXiv:1910.01108 [cs].
Shapiro, A. H., Sudhof, M., and Wilson, D. J. (2022). Measuring news sentiment. Journalof Econometrics, 228(2):221–243.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L.,
and Polosukhin, I. (2017). Attention is All you Need. In Advances in Neural Information
Processing Systems, volume 30. Curran Associates, Inc.
Wiswall, M. and Zafar, B. (2018). Preference for the Workplace, Investment in Human
Capital, and Gender*. The Quarterly Journal of Economics, 133(1):457–507.

26

TABLES AND FIGURESTable 1: Counts of Vacancy Postings, Employers, and Cities, January 2014 toJanuary 2023
(1)

(2)

(3)

Vacancies

Employers

CitiesNew Zealand

1,700,523

36,201

67

Australia

8,607,160

197,87059

Canada11,711,357

712,577

3,691

United Kingdom

74,576,747

876,103

2,268

United States

161,872,915

3,485,630

31,635

Total

258,468,702

5,308,381

37,720Country(4)

Note: Reported counts pertain to the universe of online postings from January 2019 onwards and a 5% random sample from 2014 to 2018, after
we drop about 6% of the postings in the data-cleaning steps described in Appendix A. We rely on Lightcast’s proprietary algorithm to identify
employers and cities.

Table 2: Examples of Classification Errors in Dictionary Methods

Note: The left column provides examples of how a dictionary method falsely classifies a vacancy posting as saying the job allows remote work.
The right column shows examples of how it falsely classifies a vacancy posting as not saying that the job allows remote work. Bold fontdesignates dictionary keywords, and yellow shading highlights text that helps determine a correct classification. These examples are based on
actual vacancy postings in our dataset and the dictionary used in Adrjan et al. (2021).

Table 3: WHAM Attention Weights Compared to Dictionary Keywords

Note: The left column illustrates the role of attention weights in WHAM classifications of vacancy postings, where darker shadings pertain tohigher weights. The right column illustrates the application of dictionary methods to the same text passages, where highlight text pertains to
keywords.

Table 4: WHAM Outperforms Other Classification Methods
Audit Sample

Approximate Random Sample

(1)

(2)

(3)

(4)

(5)

(6)

Error Rate

Precision

F1 Score

Error Rate

Precision

F1 Score

All Zero

.28

.00

.00

.03

.00

.00

Dictionary

.16

.68

.74

.14

.15

.25

Dictionary w/ Negation

.12

.82

.78

.07

.28

.40

Logistic Regression

.11

.81

.81

.07

.26

.40

Logistic Regression w/ Negation

.08

.87

.85

.05

.36

.50

GPT-3

.06

.87

.89

.05

.36

.52

WHAM (Generic English)

.03

.95

.95

.02

.66

.78

WHAM (Baseline)

.02

.97

.97

.01

.75

.85

Note: This table reports classification performance metrics, which we calculate using a hold-out sample of human-classified text sequences.
“Error Rate” is the overall rate of misclassifications (relative to humans). “Precision” is the ratio of true-positive classifications to the sum of true
positives and false positives. “F1 score” is the harmonic mean of Precision and “Recall”, where Recall is the fraction of true positives divided by
the sum of true positives and false negatives – i.e., the denominator is the true number of positives, according to human classifications. Columns(1)-(3) uses a 40% random subset of our audit sample, and Columns (4)-(6) uses a sample that approximates a random sample of our fulluniverse of postings. See Appendix B for details, including a description of each algorithm.

Figure 1: WHAM and Dictionary Methods Applied to U.S. Vacancy Postings
Postings Classified as Job Allows Hybrid or Fully Remote Work30

3 large job
boards phasein changes toad structure

Big increase inboth WFH and
negated WFH text

25

Percent

20
15
10
5

2022

2021

2020

2019

0

2023

WHAM
Dictionary

Note: This figure shows the percent of postings that say the job allows one or more remote workdays per week, as classified by WHAM (blue)
and a dictionary-based approach (black) using the keywords in Adrjan et al. (2021). For both methods, we reweight the data to match the U.S.
occupational distribution of vacancies in 2019 at the six-digit SOC level.

Growth Rate from 2019 to 2022, Dictionary

Figure 2: Share of U.S. Postings that Allow Some Remote Work, Growth Rate
by Two-Digit Occupations, WHAM Compared to Dictionary Method
1.5

Food Preparation and Serving

Building Cleaning
and Maintenance1

Healthcare
SupportArchitectureandEngineering

Transportation

Construction

0.5

Life, Physical,
and Social Science

ComputerandMathematical0
Farming, Fishing,and Forestry

-0.5
-0.5

0

0.5

1

1.5

Growth Rate from 2019 to 2022, WHAM
Note: We sort postings into Standard Occupational Classifications (SOC) at the two-digit level and calculate the share of postings that say the
job allows for one or more days per week of remote work in 2019 and 2022. We then calculate the DHS growth rate from 2019 to 2022 as(X_2022 - X_2019) / 0.5 * (X_2019 + X_2022). For the dictionary method, we use the keywords in Adrjan et al. (2021). The blue-dashed line
shows a 45 degree line.

Figure 3: Vacancy Postings that Explicitly Offer Hybrid or Fully Remote Work
Rose Sharply in All Five Countries from 2020
18

UK

Percent of All Postings

16
14

US
Australia
Canada
NZ

12
10
8
6
4
2

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

0

Note: This figure shows the percent of vacancy postings that say the job allows one or more remote workdays per week, encompassing bothhybrid and fully-remote working arrangements). We compute these monthly, country-level shares as the weighted mean of the own-country
occupation-level shares, with weights given by the U.S. vacancy distribution in 2019. Our occupation-level granularity is roughly equivalent to sixdigit SOC codes. See Appendix B for the corresponding raw series and series based on alternative weighting schemes.

Figure 4: Professional, Scientific and Computer-Related Occupations Have theHighest Shares of Postings that Offer Hybrid or Fully-Remote Work, U.S. Data
Computer & MathematicalBusiness & Financial Operations
Legal
Management
Architecture & Engineering
Life, Physical, & Social Science
Arts, Design, Entertainment
Community & Social Service
Office & Administrative Support
Sales & Related
Military SpecificProtective Service
Education, Training, & Library
Healthcare Practitioners
Installation, MaintenanceFarming, Fishing, & Forestry
Personal Care & Service
Production
Construction & Extraction
Healthcare Support
Transportation & Material Moving
Building & Grounds MaintenanceFood Preparation

3.4X
1.8X
3.4X
1.9X
1.5X
1.3X
1.8X
1.9X
3.5X
3.1X
1.5X
1.4X
2X
1.7X
2.5X
0

5

10

3.2X

3.5X
5.1X
3X
3.4X
2.7X

5.4X

3.9X

2019
2022

15

20

25

30

35

Percent
Note: Each bar reports the percent of vacancy postings that say the job allows one or more remote workdays per week in the indicated period
and occupation group (two-digit SOC).

Figure 5: The Share of Vacancy Postings that Explicitly Offer Hybrid or Fully
Remote Work Rose in Almost Every Occupation, U.S. Data
SoftwareDevelopers
SoftwareDevelopers
SoftwareDevelopers
SoftwareDevelopers
SoftwareDevelopers
SoftwareDevelopers

64

Loan
Officers
Loan
Officers
Loan
Officers
Loan
Officers
Loan
Officers
Loan
Officers

32

Percent (2022)

16
8
4Advertising
Advertising
Sales
AgentsAdvertising Sales
Sales Agents
AgentsTravel AgentsTravelAgentsTravel Agents
AgentsTravelTravelAgentsTravelAgentsTelemarketers

MentalHealthWorkers
Mental Health
Health Workers
Workers
MentalMentalHealthWorkers
MentalHealthWorkers
Gas
Plant
Operators
Gas
Plant
Operators
Legal Secretaries
Secretaries
Legal
Legal
Secretaries
Legal
Secretaries

Interpreters && Translators
Translators
Interpreters
Interpreters&
Translators
Interpreters&
Translators

Pharmacy Technicians
Registered
NursesRegistered
NursesRegistered
Registered Nurses
NursesRegistered
NursesRegistered
Nurses2

MedicalSecretaries
MedicalSecretaries
Medical
Medical Secretaries
Secretaries
MedicalSecretaries

1

Roofers
Roofers
Roofers
Roofers

0.5

Kindergarten Teachers
Teachers
Kindergarten
Kindergarten
Teachers
Kindergarten
Teachers

0.25
0.125
0.125

0.25

0.5

1

2

4

8

16

32

64

Percent (2019)
D&N Classification:

Not Teleworkable

Teleworkable

Note: This figure plots the percent of postings that say the job allows one or more remote workdays per week for 875 occupations in 2019 and2022. We define occupations by ONET codes, dropping those with fewer than 250 posting in 2019. The line shows the unweighted OLS fit:log(y) = 1.22 + 0.76 log(x), which has an R2 value of 0.63. The color and shape denote whether Dingle & Neiman (2020) classify the occupation
as feasible for fully remote working.

Figure 6: The Share of Vacancy Postings that Explicitly Offer Hybrid or Fully
Remote Work Varies Widely across Major Cities
3X
4.3X
6.7X

Washington, D.C. (US)San Francisco (US)London (UK)
Glasgow (UK)
Chicago (US)Atlanta (US)Phoenix (US)New York (US)
Sydney (AUS)
Toronto (CAN)
Philadelphia (US)Los Angeles (US)
Dallas (US)Melbourne (AUS)
Montreal (CAN)
Miami (US)Brisbane (AUS)
Houston (US)
Auckland (NZ)
Perth (AUS)
Adelaide (AUS)

5.2X
3.1X
3.2X
3.1X
3.7X
10.5X
7.6X
2.8X
2.7X
2.6X
9.2X
7.9X
2.7X
10.1X
3X

2019
2022

7.1X
8.3X
7.7X
0

5

10

15

20

25

30

Percent
Note: Each bar reports the percent of vacancy postings that say the job allows one or more remote workdays per week in the indicated period
and city. City refers to the location of the establishment or firm that is hiring.

Figure 7: The Share of Vacancy Postings that Explicitly Offer Hybrid or Fully
Remote Work Grew at Different Rates across Cities since the Pandemic
32

BostonLondonChicagoToronto16

Percent (2022)

Washington, D.C.

San Francisco

Phoenix

SydneyMelbourne

Montreal
Jacksonville

8

MemphisAuckland
Savannah

4

2
0.5

1

2

4

8

16

Percent (2019)
Australia

CanadaNZ

UK

US

Note: This figure plots the city-level percent of postings that say the job allows one or more remote workdays per week in 2019 and 2022. “City”
refers to the location of the establishment or firm that is hiring. The line shows the unweighted OLS fit: log(y) = 1.61 + 0.46 log(x), which has an
R2 value of 0.28.

Figure 8: Share of Postings Offering Hybrid or Fully Remote Work vary across
US cities
30

San Francisco, California

25

Boston, Massachusetts

Percentage

20

Phoenix, Arizona
Denver, Colorado
New York, New York

15
Houston, Texas
Jacksonville, Florida
US National
Louisville, Kentucky
Memphis, Tennessee

10

Savannah, Georgia

5

Miami Beach, Florida

20

19

20 01
19
20 04
19
20 07
19
20 10
20
20 01
20
20 04
20
20 07
20
20 10
21
20 01
21
20 04
21
20 07
21
20 10
22
20 01
22
20 04
22
20 07
22
20 10
23
-0
1

0

Note: We calculate the monthly share of all new job vacancy postings which explicitly advertise remote working arrangements (i.e. both hybrid
and fully-remote), by selected cities. Prior to aggregation at the monthly level, we employ a jackknife filter to remove a small number of outlier
days (see Appendix A: Data for further details). This figure shows the 3-month moving average. Cities chosen above are selected examples toillustrate the wide cross-city spread.

Figure 9: Share of Vacancy Postings Offering Hybrid or Fully Remote Work Comparedto Share of Employed that Designate “Worked from home” as commute method, U.S.
Metropolitan Statistical Areas

Percent of Vacancies AdvertisingHybrid or Remote Work (2022)

20

Washington
Arlington
Alexandria

15

Little Rock

ChicagoNaperville
Elgin

BostonCambridge
NewtonHouston10
Corpus Christi

5Jacksonville

New York
NewarkJersey CitySan Jose
SunnyvaleSanta Clara

Durham-Chapel Hill

Memphis

Chattanooga

0
0
5
10
15
20
25
30
35
Percent of Employed who selected “Worked from home” as commute method (2021 ACS)
Note: The vertical scale is the percent of postings in 2022 that say the job allows one or more remote workdays per week (i.e. both hybrid andfully-remote). The horizontal scale is the percent of employees who select “Worked from home” as their commute method in 2021 in theAmerican Communities Survey (ACS). ACS respondents are instructed to “Mark (X) ONE box for the method of transportation used for most of
the distance,” which suggests that only those who work in a fully-remote capacity should select this box. (Persons with 1+ days of commute per
week have more mileage from that commute mode.) The line shows the unweighted OLS fit: log(y) = 3.12 + 0.36 log(x), which has an R2 value of0.55. The regression includes one observation that is outside the plotted axes.

Figure 10: The Prevalence of Postings that Allow Hybrid or Fully-Remote Work Varies
Greatly, even among Same-Industry Firms Recruiting in the Same Occupational
Category
A. Selected Aerospace Firms (NAICS
3364), Management Occupations(SOC 11)
BoeingB. Selected Insurance Firms (NAICS
5241), Mathematical Science
Occupations (SOC 15-20)

Mutual of
Omaha
Company

Lockheed
MartinHonda

GeneralMotorsUnitedHealthGroup

Northrop
GrummanC. Selected Auto Manufacturing
Firms, Engineering Occupations(SOC 17-2)

Ford
HumanaSpaceX
0

25

Percent
2019
2022

50

Tesla
0

25

50

Percent

75

2019
2022

100

0

20

Percent

40
2019
2022

Note: For each firm, year and indicated occupation, we report the percent of U.S. postings that say the job allows one or more remote workdays
per week.

ONLINE APPENDIXA

Data Appendix

In this Appendix we provide further commentary on the corpus of online job vacancy postings.

A.1

Data Provider

Our corpus of online job vacancy postings is provided by the labour market and analytics
company ‘Lightcast’ (formerly Emsi Burning Glass). Lightcast has been scraping online job
vacancy postings in the USA since 2007, and has continued to expand to other countries.

A.2

Web Sources

Each job vacancy posting is scraped by Lightcast from the internet. Specifically, the company
scrapes over 50,000 web sources. These sources include private online job vacancy aggregators(e.g. Indeed.com, Monster.com), public online job boards (e.g. New York City Department ofLabour’s ‘JobZone’), and employers’ own recruitment web pages (e.g. careers.microsoft.com,
usajobs.gov). Lightcast actively audits their list of web sources to ensure data from new
websites is on-boarded in a timely manor.32 One of the main competitive advantages ofLightcast’s data product is the breadth of their sources. These data are often referred to in
the literature as the ‘near universe’ of online job vacancy postings.

A.3

What’s in the job vacancy posting data?

Once an online job vacancy posting is scraped, Lightcast processes this data to produce
three categories of information: (i) plain text, (ii) meta data, and (iii) structured data. A
description of each of these categories follows presently:
A.3.1

Plain Text

The plain text of each job ad contains the full textual description of the job, as written by
employers. To construct this, Lightcast takes the HTML file scraped from a given website
and does two further processing steps. First, it parses out portions of the HTML file which
do not contain information about the vacancy (e.g. removing website headers, footers, andside-menu bars). Second, Lightcast takes this portion of HTML which (ideally) contains
only information about the job vacancy, and turns it HTML into plain-text.
A.3.2

Meta Data

Each vacancy posting also contains a number of meta-data items. These are immutableproperties of each web scraped vacancy. The most important of these is the date the page
was scraped. Another important piece of meta-data is the URL from which the posting was
scraped.
32

One reason we eschew analysis of the count of postings and instead focus on shares is that the underlying
donor pool of online sources is constantly changing.

43

A.3.3Structured Data

The most commonly used data product that Lightcast creates is the set of structured data.
This dataset contains one row for each job vacancy posting, and a large number of additional
information such as the job title, occupation, salary, educational requirements, location, andemployer name. These variables are extracted using Lightcast’s own proprietary algorithms.
These fields differ from meta data because they may contain missing values and/or measurement error due to imperfect algorithmic extraction.

A.4

Errors and Missing Information

Overall, the data product is a highly informative and accurate product. We view the incidence of errors as very minute, but acknowledge that any dataset with hundreds of millions
of observations scraped from over 50,000 sources will never be perfect. Both the structured
data and the plain text data require a number of pre-processing steps and the use of algorithmic feature extraction, which in a very small number of cases produce errors (e.g.
misclassification of occupations, truncation of plain text, presence of erroneous text). In thissubsection we highlight some of the errors we have encountered, and discuss the strategies
we employed to ensure our results remain robust to such issues.
A.4.1

Missing Values

A specific value (e.g. the educational requirement for a job) might be missing for at least
two reasons: (i) the employer does not mention this explicitly in the text of the job ad,
and (ii) the algorithm used to extract this feature from the text failed. The former issue
is especially problematic in the context of educational requirements (e.g. we see that very
few vacancies for Medical Doctors explicitly mention a requirement to have gone to medical
school). This is because certain features of the job will likely be taken as given (for example,
specialized degrees for medical doctors). We also see that a large share of vacancy postings
do not list the salary (this is almost entirely due to lack of information, and not poor feature
extraction). One could employ imputation methods to address these missing values (seeBana (2022), who predict the salary with a very high degree of accuracy from the text).
The main strategy employed in this paper was to only utilise covariates which contain fewer
missing values, such as occupation classifications and location information.
A.4.2

Erroneous Plain Text

In a very small number of cases we observe that the plain text includes some parts of thewebsite other than the job description. For example, the plain text from one job board in
New Zealand included a number of vacancy posting text from ads that were being crosspromoted to the browser, essentially turning each document into a compilation of six job
ads.

44

A.4.3

Truncated Plain Text

In a small number of other cases, the plain text is truncated. For example, we found one
employer who listed each jobs location using an interactive link which must be clicked toappear. Since the web scraper only parses static information, this portion of the job ad
was missing from the plain text. We conducted extensive tests, and stress that in the vast
majority of cases the plain text provides an accurate representation of the job vacancy
posting.

A.5

Checking for Correlated Measurement Error

As discussed above, since our measurement of remote working relies on the underlying plain
text, some measurement error is inevitable. One concern we took seriously is the possibility that this noise may be correlated within online sources. We discuss our approach toaddressing this below.
A.5.1

Validation of Large Job Boards

In some instances the pre-processing of job posting websites includes additional erroneous
text. When this occurs, it is very likely to be true for all job postings scraped from the same
web source. To ensure our results are not overly sensitive to such issues, we first identify thetwenty largest web sources for each country. We then create twenty versions of country-leveltime series of monthly remote work vacancy shares, leaving one job board out at a time.
This process revealed one problematic source from each of Canada, USA, NZ and UK. We
found two problematic job boards in Canada. Table A.1 reports the fraction of total job ads
that were removed after dropping postings from these sources.
A.5.2

Outlier Detection and the Jack-Knife Filter

When we present monthly time series data, we apply an algorithm which filters outlier days
whose contribution to the over-all monthly share of vacancy postings offering remote work
is at odds with other days in a given month. This filter has a very minimal impact on the
results (e.g. we drop less than a quarter of one percent of job postings from the US basedon this filter). The few outlier days we do filter out occur when a large number of vacancies
get posted on a single day which are concentrated by employer/occupation/web source. Our
extensive audits of the data reveal that outlier days are due to compositional discontinuities
at the daily frequency, and not caused by measurement error in our algorithm. Our filter isbased on the Jack-Knife resampling procedure, and works as follows:• For a given calendar month M denote SM as the share of vacancy postings which offer
remote work
• For each day t ∈ M , compute the share of remote work postings excluding all postings
on this focal day t from the calculation. Define this share as SM \{t}45• If the absolute level deviation between SM and SM \{t} is greater than 2 percentage
points, or else if the absolute ratio of their natural logarithms is greater than 0.1, thenwe classify focal day t as an outlier• Recalculate the share of vacancy postings for month M excluding all postings on outlier
daysThis filter alters the data minimally. For example, in the United States, it removes 0.2% of
the total number of vacancy postings. The number of postings which are filtered is shownin the below table:

A.6

Representativeness of Online Job Vacancy Postings

Lightcast frequently reviews the representativeness of the job vacancy postings it scrapes, toensure the information renders an accurate picture of the overall labour market. Both ouranalysis and that of our data provider, as well as many other papers in the literature who
utilise these data, all find a high degree of fidelity between the share of job vacancies acrossoccupations and industries, and other official Government data products which measure
similar phenomena.
In our baseline results, we also re-weight the data to reduce sensitivity to shifts in theoverall composition of the labour market. The next section discusses this further, but wenote that this provides additional robustness to concerns of representativeness

B

Supplementary Results

See figures below.

46

C

Supplementary Information on Measurement

C.1

Estimation details for WHAM

WHAM builds from DistilBERT (Sanh et al., 2020), which has a Transformer architecturewith six layers and 66 million parameters. It was originally estimated to predict randomly
deleted words in a corpus of unpublished books and all English Wikipedia. We use theuncased version of the model.
The first estimation step in WHAM is to pre-train off-the-shelf DistilBERT (Sanh et al.,2020) to predict randomly deleted words in a random sample of 900,000 job posting sequences
which is balanced across all years and countries. The total fraction of deleted words is15%. We use guidelines from the original BERT paper (Devlin et al., 2019) to select thehyperparameters for estimation: a batch size of 8, three training epochs, and a learning rateof 5e-5.33
The second estimation step in WHAM is to fine-tune the model to predict human labels.
To select the estimation hyperparameters, we use three-fold cross validation and the training
data used for the benchmark exercises reported in section 3. We perform an exhaustive searchover learning rates {2 ∗ 10−5 , 3 ∗ 10−5 , 5 ∗ 10−5 }, epochs {2, 3, 5} and batch sizes {16, 32},
and select the set of hyperparameters with the highest average F1 score across training
data splits. The resulting choices are 5e-5, 2, and 16, respectively. The model estimatedwith these choices solely on the training data is used to determine the test-set performance
reported in section 3. To produce output on the entire dataset, we re-estimate the modelusing all human labels (from both training and test sets) using the same hyperparameters
and use this model to predicted remote work on all sequences in the Lightcast data.

C.2

Details for other classification approaches

Section 3 compares various alternatives to WHAM for classifying remote work, and here we
provide additional details on these.
C.2.1

Dictionary

We implement the dictionary approach with the following steps:
1. Preprocessing: We lowercase all text, remove punctuation symbols (except for hyphens and apostrophes), remove numbers, and replace all sequences of white spaces
with a single white space.
2. Tagging: We search for the appearance of any of the keyword phrases from Table
Table C.1. For phrases containing multiple words (e.g. ‘work from home’) we allow
for any arbitrary combination of white spaces and hyphens separating the words thatcompose the dictionary keyword (e.g. ‘work-from-home’, ‘work- from- home’).
33

The batch size determines how many text sequences are processed at each step in estimation. The numberof epochs determines the total number of times the entire data set is passed through in estimation. The
learning rate determines the speed at which the model parameters update in gradient descent.

47

3. Binary classification: Any job posting that contains a match to any of the dictionary
keywords is classified as positive.
C.2.2

Negation adjustment

Our strategy for negation adjustment follows that proposed by Shapiro et al. (2022) tocapture negation in the context of sentiment analysis. For every keyword match from thedictionary within a job posting, we consider it to be negated if any of the following is true:
1. There is a negation term in any of the three words before the keyword match. The setof negation terms is displayed in Table C.2, and comes from the VADER Sentiment
Analysis toolkit.
2. “no” or “not” appear in the two words after the keyword match
3. A word that contains “n’t” is the immediate word after the keyword match
If a job posting is negated we then change its binary label from positive to negative.
C.2.3

Logistic regression

Our approach to logistic regression follows the approach in Adams-Prassl et al. (2020). We
start by applying the same pre-processing steps used for the dictionary approach to thejob postings: i) lowercase text, ii) remove punctuation (except for the hyphen), iii) remove
numbers, and iv) clean white spaces. Next we split the text into individual tokens and build
the document-term frequency matrix by using the 5,000 most common tokens. For eachkeyword in our dictionary (the phrases in Table Table C.1) that is not part of the 5,000 mostcommon tokens, we add a column in the document-term matrix with its counts. Finally, wetransform the matrix into its binary form; every entry above one is replaced with a one. Thismatrix then becomes the set of covariates used to predict human labels via logistic regression
with L1 regularization (LASSO). To determine the LASSO penalty, we use five-fold crossvalidation on the training data, and select the regularization parameter that achieves thehighest average F 1 score across the five splits.
C.2.4

Logistic regression with negation

We follow an identical procedure to the one described for logistic regression but we further
extend the document-term matrix with one extra column per keyword in the dictionary thatindicates that the keyword was negated (according to our negation adjustment described
before).
C.2.5

GPT-3

We use OpenAI’s GPT-3 model to generate predictions on the presence of remote work in
our job postings. To do this, we craft a simple prompt that instructs the model to “Identify if
the text offers the possibility of remote work at least one day per week” and ask the model to48

generate an answer. Figure C.1 illustrates a particular example using OpenAI’s Playground.
In most cases, the text generated by GPT-3 is a Yes/No answer. Sometimes, however,
the model generates longer answers (e.g. “temporarily due to covid”). In order to transformthese answers into a binary prediction we do the following: i) lowercase the answer of GPT-3
and clean any additional white spaces and ii) if the answer contains “no” as part of its three
first characters we assign a zero (no remote work) to the sequence, else we give it a one(remote work).
We test both the ‘text-davinci-02’ model and the ‘text-davinci-03’ model using the same
prompt and report performance of the former given its lower error rate with respect to ourhuman labels.

49

Figure A.1: Fraction of job vacancy postings we drop from sample due toproblematic job boards

(1)

(2)

(3)

SourceCountry

Percent of Raw Data Dropped

A

USA

6.7

B

UK

3.6

C

NZ

28.9

D

Canada3.9

E

Canada3.5

Note: We do not identify these job boards, and instead refer to each by a letter e.g. “A”. This is to avoid any potential conflict with the commercial
interests of these websites, though it should be noted that this is not a failure of the job board but rather an issue with our web scraped data.
Researchers should contact us if they would like to know the names of each source which we drop. Column (3) reports the fraction of all postingsfrom Jan 2014 to Jan 2023, within the relevant country, that is removed from the raw data after we dropped the corresponding source.

Figure A.2: Jack-Knife Time Series Filter Removes a very small amount of job
postings from monthly time series results

(1)

(2)

Country

Percent of Raw Data Dropped

NZ

7.74

Australia

0.78

Canada1.43

United Kingdom

0.06

United States

0.21

Note: This table records the total fraction of all online job vacancy postings which are dropped from our sample after applying our jackknife filter.
This is only imposed when we present monthly frequency time series plots, and works by identifying outliers at a daily frequency prior to monthly
aggregation.

Figure B.1: Most Sequences are Assigned a Predicted Probability by WHAM at
Extreme Values

WHAM Probability Distribution (Sequence Level)

Share of Sequences in Each Bin(Exponential Scale)

Mass (Exponential Scale)

0.980000
0.100000
0.010000
0.001000
0.000100
0.000010
0.000001

0.000000
0.00

0.25

0.50

0.75

1.00

Probability a Sequenceis Offering
Hybrid or Fully Remote Work
WHAM
Probability
assigned by WHAM Model

Note: WHAM assigns a predicted probability to each sequence in the full job posting dataset using our trained neural network model. This figure
presents a histogram of the share of sequences that fall in different bins according to these predictions.

Table B.1: Most Job Postings Either Have Zero or One Sequence that gets
Classified as Offering Hybrid or Fully Remote Work Arrangements(1)
RemoteWork
Sequences(2)
Number of
Vacancy
Postings(3)
Share Of
Total
(%)

0

40,006,052

90.4

1

2,682,844

6.1

2

989,084

2.2

3

365,9700.8More than 3

201,5230.5

Note: This table tabulates how many text sequences in each US job posting from 2021 are classified as offering remote work (either hybrid or
fully remote) according to WHAM. A typical job ad is split into six sequences. Most postings (90.42%) have no positive sequences. Of theremaining fraction, most have only one positive sequence.

Table B.2: Computing Hardware and Costs for the Three Stages of WHAM

Computational setup

(1)

(2)

(3)

Pretraining

Fine-Tuning

Full Sample
Prediction

GCP Virtual Machine GCP Virtual Machine GCP Virtual Machinewith 1 NVIDIA
with 1 NVIDIA
with 8 NVIDIA
V100 GPU
V100 GPU
A100 GPUs

Total time (hours)

12

3

36

Job postings (per
hour)

NA

NA

7,000,000

Cost per hour (USD)

$3

$3

$40

Total Cost (USD)

$36

$9

$1,440Note: This table details the computational setup and time/money costs associated with the different stages of WHAM. All computations were
performed on the Google Cloud Platform.

Figure B.2: The raw unweighted share of new job ads offering hybrid or fully
remote work is highest in the UK, as UK has very high proportion of ‘whitecollar’ jobs being advertised
22
20
18

UK

16

Percent

14
US
Australia
Canada
NZ

12
10
8
6
4
2

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

0

Note: This figure shows the share of vacancy postings that say the job allows one or more remote workdays per week. We compute thesemonthly, country-level shares as the raw mean from the universe of new job vacancy postings in each country from each month. Our baseline
approach presented in Figure 3 uses vacancy shares from the US to control for occupational composition across countries.

Table C.1: Keywords Used to Implement the Dictionary Approach to RemoteWork Classification

working remotely

working from home

work remotely

work from home

work at hometeleworking

telework

telecommuting

telecommute

smartworking

smart working

remote work
teleworking

remote work

remoteremotely

homeofficehome officehome based

homebased

Note: These are the keywords that appear in Table A.2 of Adrian et al. (2021) for detecting the presence of remote work in the text of job
postings. The three exceptions are `homebased', `home based', and `remotely' which we add to the original terms to improve accuracy.

Table C.2: Terms Used for Negation in the Dictionary Approach

aint

arent

cannotcant

couldnt

darentdidnt

doesntain't

aren'tcan't

couldn't

daren'tdidn'tdoesn't

dont

hadnt

hasnt

haventisnt

mightnt

mustntneither

don't

hadn'thasn'thaven't

isn't

mightn't

mustn'tneedntneedn't

never

none

nope

nor

not

shant

shouldnt

uhuh

wasnt

werentoughtn't

shan'tshouldn't

uh-uh

wasn'tweren't

without

wont

wouldnt

won't

wouldn't

rarelyseldomdespitenonothing nowhere oughtnt

Note: This is a set of terms that the VADER sentiment analysis tool uses for negation, and which Shapiro et al. (2022) adopt. We add the term`no' to the baseline negation set.

Figure C.1: GPT-3 Example Prompt

Note: To compare our WHAM model to recent advances in generative AI, we analysed our audit sample of text sequences drawn
from job vacancy postings using GPT-3. The above is an illustration of the prompt we used. We report the performance of thisapproach to classification in Table 4 which compares this measurement algorithm to WHAM. The example shown above highlights
that, unlike some other widely used methods, this technology is similar to WHAM in its ability to process context.