Page 1 GAO-19-273R Contingent Workforce

441 G St. N.W.

Washington, DC 20548

January 29, 2019 

The Honorable Robert C. Scott

Chairman

Committee on Education and Labor

House of Representatives

CONTINGENT WORKFORCE: BLS is Reassessing Measurement of Nontraditional 

Workers

Dear Mr. Scott:

The nature of employment for many Americans is changing. The Bureau of Labor Statistics

(BLS) data show millions of workers are no longer in traditional full-time, year-round jobs. 

Several companies that have been established since 2008 rely on nontraditional types of 

employment arrangements, with workers who typically use electronically-mediated platforms 

such as websites and mobile applications (apps) to secure work.

1

 

Comprehensive, nationally representative data on nontraditional workers were first collected in 

1995 when BLS introduced the Contingent Worker Supplement (CWS) to its Current Population 

Survey (CPS). 2 The CWS asks a series of questions about workers’ employment, including 

whether their jobs are contingent. CWS has been administered six times: 1995, 1997, 1999, 

2001, 2005, and the focus of our report, 2017. For the purposes of this report, we refer to 

contingent, alternative, and electronically-mediated employment comprehensively as 

nontraditional employment.

BLS defines contingent workers as those with temporary employment. In our prior work, we 

found that the discourse on contingent employment, which BLS defines as people who do not 

expect their jobs to last or who reported that their jobs are temporary, is shaped by limitations in 

available data.3 We also found researchers and analysts were mining a number of alternative 

datasets to provide insights about segments of the nontraditional workforce. We reported that

the data limitations at that time could make identifying emerging trends difficult. 

In response to GAO, stakeholder, and researchers’ concerns, BLS again fielded the CWS in 

May 2017. BLS worked with stakeholders to draft four additional questions for the 2017 CWS. 

The purpose was to obtain data about work arrangements that have emerged since 2005, the 

1 BLS lists some businesses that commonly facilitate electronically-mediated work, such as Uber, Lyft, TaskRabbit, 

Handy, Amazon mTurk, and Crowdflower, in their September 2018 Monthly Labor Review on electronically-mediated 

work.

2 For the purposes of this study, the term nontraditional is used by GAO to cover workers classified as contingent, 

alternative, and electronically-mediated workers.

3 GAO, Contingent Workforce, Size, Characteristics, Earnings, and Benefits, GAO-15-168R (Washington, D.C.: April 

20, 2015). 
Page 2 GAO-19-273R Contingent Workforce

last time CWS was administered. The four questions were to measure electronically-mediated 

employment, or employment found through websites and mobile apps.4

The May 2017 CWS estimated, by its broadest measure, that 3.8 percent of all employed 

individuals are contingent workers, compared to 4.1 percent in the 2005 CWS. This decrease in 

the contingent worker population amid a rise in electronically-mediated platforms for locating 

employment raised concerns for researchers, stakeholders, and policymakers about whether 

this segment of the workforce is being accurately measured.

You asked us to review BLS’s measurements of nontraditional workers and whether they 

appropriately represent the workers. This report describes (1) how BLS defines and measures 

contingent, alternative, and electronically-mediated employment workers in the CWS and other 

BLS data sets and (2) what limitations of the CWS BLS has identified and what steps BLS has

taken to address them. On November 30, 2018, we briefed members of your staff on the results 

of our review. This report formally transmits the briefing slides (see Enclosure 1).

To determine how BLS defines and measures nontraditional workers, we reviewed technical

documentation and interviewed agency officials. We reviewed documentation from 2017 and 

2018 on which populations BLS data sets cover to determine how BLS defines different 

populations of nontraditional workers. Additionally, we reviewed the technical documentation on 

how the CWS is administered to better understand how BLS solicits responses and measures 

the populations of nontraditional workers as reported by those surveyed. Finally, we interviewed 

officials at BLS and the U.S. Census Bureau about information found in the technical 

documentation and to clarify the purposes of the data sets.

To determine limitations BLS has identified in measuring nontraditional work and steps BLS has 

taken to address them, we reviewed prior BLS reports and technical documentation.

Specifically, we reviewed prior BLS reports from fiscal years 2016 through 2018, to identify the 

limitations of data sets as identified by BLS and reviewed BLS’s technical documentation on 

how the data sets are created to determine what steps BLS has taken to address the limitations. 

Additionally, we interviewed BLS officials to better understand the limitations and steps BLS has

taken, or steps planned to address them.

We conducted this performance audit from May 2018 through January 2019 in accordance with 

generally accepted government auditing standards. Those standards require that we plan and 

perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on our audit objectives.

In summary, we found the following:

BLS periodically collects data on and measures contingent, alternative, and electronicallymediated workers in the CWS. BLS broadly defines 1) contingent workers as those with 

temporary employment; 2) alternative workers as those that are independent contractors, oncall workers, temporary help agency workers, and workers provided by contract firm; and 3) 

electronically-mediated workers as those who obtain employment through mobile apps and 

websites that both connect workers with customers and arrange payment for tasks. According to 

BLS reports, there is overlap in BLS’s three groups identified. For example, a worker that 

temporarily drives for a ride-share company while waiting to start a new job would be 

 4 These four questions are included in Enclosure 1. 
Page 3 GAO-19-273R Contingent Workforce

considered both a contingent worker and an electronically-mediated worker. BLS measures 

populations of contingent, alternative employment, and electronically-mediated workers across 

its datasets – CWS, Occupational Employment Statistics, and Current Employment Statistics.

According to BLS, there are several possible explanations for why there was a decline in the 

estimated number of contingent workers, from 4.1 percent in 2005 to 3.8 percent in 2017.

However, they offered no definitive explanation for the decline but said this is an area for 

additional research.

BLS has identified several limitations that require additional research regarding the May 2017 

CWS’s measurement of nontraditional workers. According to officials we interviewed and 

documentation we reviewed:

• CWS measures only respondents’ main jobs for contingent and alternative employment, 

thus potentially missing workers with nontraditional second or supplementary income 

jobs.

5

• CWS only asks respondents about their work in the past week and may fail to capture

seasonal workers or workers that supplement their income with occasional work.

• BLS officials said in evaluating the gathered data, they found the added questions 

regarding electronically-mediated employment resulted in a large number of false 

positive answers.

6 False positives are when people responded "yes" to the new 

questions when the description of job duties and employers was incompatible with 

electronically-mediated work.7 BLS officials said after manually adjusting the data to 

correct for false positives, the questions regarding electronically-mediated employment

provide an accurate measure.8 BLS officials said they plan to perform additional 

research before administering the questions again.

BLS officials stated they are implementing changes to address limitations in the CWS:

• According to these officials, BLS is currently working with stakeholders on CWS 

questions that will better capture nontraditional employment, including:

• contracting with the Committee on National Statistics of the National Academies 

of Science, Engineering, and Medicine (CNSTAT) to evaluate the CWS and other 

existing measures, and consider the methodological issues and factors that could

improve the design of a new supplement; and

• planning, as part of the contract, a CNSTAT-hosted panel of experts, a public 

workshop, and a report with objectives for achieving a more reliable 

measurement of contingent and alternative employment workers as well as 

insights on the electronically-mediated economy.

 5 BLS included questions regarding second and supplemental jobs for electronically-mediated employment, only.

6 The four electronically-mediated employment questions are included in Enclosure 1.

7 For example, a fast food manager and a vice president of a major bank said they had used a website or mobile app 

to connect with customers and were paid through that same website or app. 

8 However, according to BLS officials, manually adjusting the data to correct for false positives answers regarding 

secondary and supplemental income jobs was not as accurate because BLS did not have enough data collected on 

those jobs.
Page 4 GAO-19-273R Contingent Workforce

• BLS expects the contractor to finish its work by March 2020. 

Agency Comments and Our Evaluation

We provided a draft of this report to the Department of Labor (DOL) and BLS for review and 

comment. DOL and BLS provided technical comments, which we incorporated into the report as 

appropriate. 

- - - - -

We are sending copies of this report to the Secretary of Labor, relevant congressional 

committees, and other interested parties. In addition, this report will be available at no charge on 

GAO’s website at www.gao.gov.

If you or your staff members have any questions concerning this report, please contact me at 

(202) 512-9345 or brownbarnesc@gao.gov or Oliver Richard at (202) 512-8424 or 

richardo@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs 

may be found on the last page of this report. Key contributors to this report were Kimberley

Granger (Assistant Director), Benjamin Bolitzer (Assistant Director), Michael Naretta (Analyst-inCharge), Susan Aschoff, David Dornisch, Sarah Gilliland, Michael Kniss, Dan Luo, Mimi 

Nguyen, James Rebbe, and Almeta Spencer.

Sincerely yours,

Cindy S. Brown Barnes

Director, Education, Workforce, and Income Security

Oliver M. Richard

Director, Applied Research and Methods

Enclosure
Contingent Workforce

BLS is Reassessing Measurement of 

Nontraditional Workers

November 30, 2018

Page 1

Enclosure
Contents

• Objectives

• Methodology

• Background

• Findings

Page 2
Objectives

• What is known about how the Bureau of Labor Statistics (BLS) 

defines and measures contingent, alternative, and 

electronically-mediated workers in the Contingent Worker 

Supplement (CWS) of the Current Population Survey (CPS) 

and other BLS data sets? 

• What limitations of the CWS has BLS identified and what 

steps has BLS taken to address them?

Page 3
Methodology

• We reviewed BLS data set documentation for CWS, Current 

Employment Statistics (CES), and Occupational Employment 

Statistics (OES), including: 

• how the data sets are gathered and reported,

• worker populations covered and their definition, and

• supplementary questions and data.

• We reviewed the contract between BLS and Committee on National 

Statistics (CNSTAT) of the National Academies of Sciences, 

Engineering, and Medicine for an evaluation of the CWS. 

• We interviewed BLS and U.S. Census Bureau officials.

• We reviewed prior GAO reports and existing literature on the 

nontraditional workforce and its measurement. For the purposes of 

this report, we refer to contingent, alternative, and electronically- mediated employment comprehensively as nontraditional 

employment.

Page 4
Background

• Comprehensive, nationally representative data on nontraditional workers were 

first collected in 1995 when BLS introduced the CWS to its CPS, a monthly 

survey of about 60,000 households that collects data on the U.S. labor force. 

• CWS asked a series of additional questions about workers’ employment, 

including whether their jobs were contingent. CWS has been administered six 

times: 1995, 1997, 1999, 2001, 2005, and the focus of our report, 2017. 

• In a 2015 report, GAO stated limitations of the current CWS data may prompt 

interest among stakeholders to weigh the advantages versus cost of collecting 

better information about contingent workers.1

• For the purposes of this report, we refer to contingent, alternative, and 

electronically-mediated employment comprehensively as nontraditional 

employment. 

1 GAO, Contingent Workforce, Size, Characteristics, Earnings, and Benefits, GAO-15-168R (Washington, D.C.: April 20, 2015). Page 5
Background (cont.)

• On September 28, 2018, BLS released data obtained through four 

questions added to the May 2017 CWS regarding employment 

obtained through mobile applications and websites that both 

connect workers with customers and arrange payment for tasks.2

BLS calls this type of work electronically-mediated employment; 

others more broadly call it gig employment.

• In developing the four questions, BLS sought feedback through: 

• presentations and briefings to stakeholders outside BLS on the 

new questions;

• outreach to members of Congress and their staff;

• circulation of the draft questions to academics, industry experts, special interest groups, and other data users for comment;

• discussions in the Department of Labor’s Structure of Work 

Policy Working Group; and

• soliciting and reviewing public comments.

2 The four new questions are included in the last two slides below. Page 6
Objective 1: BLS Collects Data On and Measures 

Nontraditional Workers

• BLS collects data on nontraditional workers in three key categories: contingent, 

alternative employment, and electronically-meditated workers (see table 1).

• In addition, BLS uses three measures to estimate contingent workers ranging 

from a narrow definition to a broad definition, as shown in Table 1; all three 

definitions are included in BLS’s report on the CWS.

• According to BLS, there are several possible explanations for why there is a 

decline in BLS’s estimate of contingent workers from 2005 to 2017. However, they 

offered no definitive explanation for the decline but said that this is an area for 

additional research. 

• BLS estimated a decline in contingent workers from 4.1 percent of the total 

working population in 2005 to 3.8 percent in 2017, a decline in independent 

contractors from 7.4 percent in 2005 to 6.9 percent in 2017, and a decline in on- call workers from 1.8 percent in 2005 to 1.7 percent in 2017. Also in 2017, after 

manually adjusting the data to improve accuracy by removing false positive 

responses, BLS estimated 1 percent of the total working population worked as 

electronically-mediated employment. Since these populations overlap, the net 

change in total nontraditional workers is unclear.

Page 7
Table 1: Bureau of Labor Statistics Has Multiple 

Definitions for Nontraditional Worker Populations 

Nontraditional Worker 

Type

Definition 1 Definition 2 Definition 3

Contingent workers –

includes temporary help 

agency workers

Narrowest definition:

• Wage and salary 

workers only, 

excluding the selfemployed and 

independent 

contractors

• Worked at their 

current job 1 year or 

less

• Do not expect to 

continue at current 

job for another year 

or more

Middle definition:

• All workers, including 

the self-employed 

and independent 

contractors

• Worked at their 

current job 1 year or 

less

• Do not expect to 

continue at current 

job for another year 

or more

Broadest definition:

• Wage and salary workers 

who do not expect their 

job to last (even if they 

have held it for more than 

1 year and expect to 

continue at the job for 

longer than 1 year)

• Self-employed workers 

and independent 

contractors who:

• have been selfemployed for 1 year 

or less

• do not expect to be 

self-employed for 

another year or 

more

Page 8
Table 1: Bureau of Labor Statistics Has Multiple Definitions 

for Nontraditional Worker Populations (cont.)

Nontraditional Worker 

Type

Definition

Alternative employment 

workers

Four different alternative employment arrangements—independent contractors, on-call 

workers, temporary help agency workers, and workers provided by contract firms—are 

considered alternative employment workers.

Electronically-mediated

workers

Workers who obtain employment through mobile applications and websites that both 

connect workers with customers and arrange payment for tasks. This definition is similar 

to that used by many stakeholders to define gig employment.a

Source: GAO analysis of BLS documentation, reports, and interviews | GAO-19-273R

a In a previous report, GAO identified gig workers as self-employed individuals providing labor services and completing single projects or tasks on 

demand for pay, such as performing tasks through digital platforms, ride-hailing services, or in offline work, such as house painting, house cleaning, 

and babysitting or child care services. GAO, WORKFORCE TRAINING: DOL Can Better Share Information on Services for On-Demand, or Gig, 

Workers, GAO-17-561, (Washington, D.C.: September 26, 2017).

Page 9
Objective 1: BLS Collects Data On and Measures 

Nontraditional Workers (cont.)

Hypothetical examples of nontraditional workers

• Albert started a job with a local landscaping company at the 

beginning of summer. He expects the job to only last for the 

summer. Albert is a contingent worker.

• Brenda works as an independent contractor. She is an 

alternative employment worker.

• Carl works as a ride-share driver and gets work through an 

online platform. Carl is an electronically-mediated worker.

Page 10
Objective 1: BLS Collects Data On and Measures 

Nontraditional Workers (cont.)

• BLS’s three nontraditional worker groups—contingent, 

alternative employment, and electronically-mediated—share 

some employment characteristics (see figure 1). 

• One example of the overlap of definitions is a worker retired 

from their main job temporarily drives for a ride-share 

company to earn supplementary income. This worker would 

be considered both a contingent worker and an electronicallymediated worker. 

Page 11
Figure 1: Populations of Nontraditional Workers 

Overlap Under Bureau of Labor Statistics Definitions

Page 12
Objective 1: BLS Collects Data On and Measures 

Nontraditional Workers (cont.)

• BLS measures the overlapping populations of contingent, 

alternative employment, and electronically-mediated workers 

to varying degrees across its multiple data sets (see table 2).

Page 13
Table 2: Bureau of Labor Statistics (BLS) Measures 

for Nontraditional Worker Populations 

Data set Electronically-mediated 

Workers

Contingent Workers Alternative Employment 

Workers

Contingent Worker 

Supplement (CWS)

Measures electronicallymediated workers through 

four questions added to 

CWS in May 2017; 

however, BLS officials said 

the data required 

significant recoding, i.e. 

manually adjusting the 

data, to correct for false 

positives. This is discussed

in more detail below.

Measures contingent 

workers with three 

estimates (from narrowest 

to broadest). The broadest 

definition of contingent 

workers showed a decline 

from 4.1 percent of the 

total working population in 

2005 to 3.8 percent in 

2017.a

Measures alternative 

employment workers by 

giving estimates for 

independent contractors, 

on-call workers, temporary 

help agency workers, and 

workers provided by 

contract firms. BLS 

reported that each of these 

populations declined from 

2005 to 2017, or remained 

the same for each year.b

a The narrowest definition of contingent workers declined from 1.8 percent of the total working population in 2005 to 1.3 percent in 2017 

and the middle definition, as shown in Table 1, of contingent workers declined from 2.3 percent in 2005 to 1.6 percent in 2017.

b Independent contractors declined from 7.4 percent of the total working population in 2005 to 6.9 percent in 2017, on-call workers 

declined from 1.8 percent of the total working population in 2005 to 1.7 percent in 2017, temporary help agency workers were 0.9 

percent of the total working population in both 2005 and 2017, and workers provided by contract firms at 0.6 percent of the total working 

population in both 2005 and 2017.

Page 14
Table 2: Bureau of Labor Statistics (BLS) Measures 

for Nontraditional Worker Populations (cont.)

Data set Electronically-mediated 

Workers

Contingent Workers Alternative Employment 

Workers

Occupational Employment 

Statistics (OES)

Does not measure 

electronically-mediated 

workers

Does not directly measure 

contingent workers, but 

does measure temporary 

help services

Does not directly measure 

alternative employment 

workers and does not 

include self-employed 

individuals.

Current Employment 

Statistics (CES)

Does not measure 

electronically-mediated 

workers

Does not directly measure 

contingent workers, but 

does measure temporary 

help services

Does not directly measure 

alternative employment 

workers and does not 

include self-employed 

individuals.

Page 15

Source: GAO analysis of BLS documentation, reports, and interviews | GAO-19-273R
Objective 2: BLS Has Identified Several Areas for 

Additional Research in the CWS and Is Reassessing Its 

Measurements

BLS identified areas for additional research in the 2017 CWS

• CWS measures only respondents’ main jobs for contingent and alternative 

employment, thus potentially missing workers with nontraditional second or 

supplementary income jobs. 

• CWS only asks respondents about their work in the past week and may fail to 

capture seasonal workers or workers that supplement their income with 

occasional work.

• BLS said in evaluating the gathered data, they found the added questions 

regarding electronically-mediated employment resulted in a large number of 

false positive answers. False positives are when people responded "yes" to 

the new questions when the description of job duties and employers was 

incompatible with electronically-mediated work.3 BLS officials said after 

recoding, i.e. manually adjusting, the data, the questions regarding 

electronically-mediated employment provide an accurate measure for the 

respondents’ main job. Officials said they would perform additional research 

before administering the questions again.

Page 16 3 For example, a fast food manager and a vice president of a major bank said they had used a website or mobile app to connect with customers and were paid through 

that same website or app. BLS manually adjusted these false positives by cross-referencing them with individuals’ job description. BLS did not have enough data 

collected on the second or supplemental income jobs to accurately manually adjust the data.
Objective 2: BLS Has Identified Several Areas for 

Additional Research in the CWS and Is Reassessing Its 

Measurements (cont.)

BLS is taking steps to implement further changes to the CWS 

• BLS officials said they are currently working with stakeholders on developing 

CWS questions that will better capture nontraditional employment. Recent 

steps taken by BLS include: 

• contracting with the Committee on National Statistics (CNSTAT) of the 

National Academies of Sciences, Engineering, and Medicine to evaluate 

the CWS and other existing data sets, and to identify the methodological 

issues and factors that should improve the design of a new supplement; 

and

• planning, as part of the contract, a CNSTAT-hosted panel of experts, a 

public workshop, and a report with objectives to improve CWS’s 

measurements of contingent and alternative employment workers as well 

as insights on the electronically-mediated economy.

• The period of performance for BLS’s contract with CNSTAT is September 24, 

2018 to March 23, 2020. 

Page 17
BLS’s Four Electronically-Mediated Employment 

Questions

Q1. Some people find short, IN-PERSON tasks or jobs through companies 

that connect them directly with customers using a website or mobile app. 

These companies also coordinate payment for the service through the app or 

website. For example, using your own car to drive people from one place to 

another, delivering something, or doing someone’s household tasks or 

errands.

Does this describe ANY work you did LAST WEEK?

• Yes

• No

Q1a. Was that for your main job, your second job, or other additional work for 

pay?

• Main job

• Second job

• Additional work for pay

Page 18
BLS’s Four Electronically-Mediated Employment 

Questions (cont.)

Q2. Some people select short, ONLINE tasks or projects through companies 

that maintain lists that are accessed through an app or a website. These 

tasks are done entirely online and the companies coordinate payment for the 

work. For example, data entry, translating text, web or software development, 

or graphic design.

Does this describe ANY work you did LAST WEEK?

• Yes

• No

Q2a. Was that for your main job, your second job, or other additional work for 

pay?

• Main Job

• Second job

• Additional work for pay

(102839) Page 19
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to https://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, https://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at https://www.gao.gov.

Contact FraudNet:

Website: https://www.gao.gov/fraudnet/fraudnet.htm

Automated answering system: (800) 424-5454 or (202) 512-7700

Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.