1

Mary Dorinda Allard

allard.dorinda@bls.gov

Mary Dorinda Allard is a division chief in the 

Office of Employment and Unemployment 

Statistics, U.S. Bureau of Labor Statistics.

Adding questions on certifications and licenses to 

the Current Population Survey

While sources of federal data on educational attainment are 

plentiful, information on nondegree credentials, such as 

professional certifications and state or industry licenses, is 

much more limited. Including a small set of questions about 

certifications and licenses on the Current Population Survey 

(CPS)—the monthly source of labor force statistics 

published by the U.S Bureau of Labor Statistics—would 

add valuable information about the labor market and would 

enable researchers and policymakers to examine how 

these types of credentials affect labor market success. After 

a lengthy development process, three questions on 

certifications and licenses were added to the monthly CPS 

in January 2015. This article describes the process of 

developing the CPS questions, starting with the early work 

on question development done by the Interagency Working 

Group on Expanded Measures of Enrollment and 

Attainment (GEMEnA), which was formed late in 2009. It 

then describes the work done by the Bureau of Labor 

Statistics to adapt the questions developed by GEMEnA to 

the CPS, the development and acceptance of a proposal to 

add the questions, the fielding of the questions, and the first 

publication of results.

There are many sources of federal data on educational attainment. These measures generally use the traditional 

educational attainment categories, which range from less than a high school diploma through a bachelor’s degree 

or higher. However, many other credentials appear to be valuable in the labor market, such as professional 

certifications and state or industry licenses. Until recently, there were few federal sources of data on these types of 

credentials.

In response to the increased interest in data about alternative credentials, the Interagency Working Group on 

Expanded Measures of Enrollment and Attainment (GEMEnA) was formed late in 2009 and continued into 2016. 

GEMEnA was chaired by the Department of Education’s National Center for Education Statistics (NCES). Over the 

years, members included senior staff from the U.S. Bureau of Labor Statistics (BLS), the Census Bureau, the 

Council of Economic Advisors, the U.S. Department of Education’s Office of the Under Secretary, the National 

November 2016
 U.S. BUREAU OF LABOR STATISTICS

2

MONTHLY LABOR REVIEW 

1.

2.

Center for Science and Engineering Statistics, and the OMB. GEMEnA worked on a variety of education- and 

credential-related subjects, including certifications and licenses.1

Two of GEMEnA’s main goals were

to develop a limited set of questions to measure the prevalence and key characteristics of nondegree 

credentials, such as certifications and licenses—these questions could then be placed on existing federal 

household surveys;2 and

to develop a common vocabulary for nondegree credentials among federal statistical agencies, federal 

programs, and state data systems—this would allow people to better communicate about the data.

In 2009, President Obama called for Americans to commit to achieving at least 1 year of higher education or 

career training,3 and as a result, GEMEnA concentrated its initial research efforts on subbaccalaureate 

certifications and licenses. Over time, however, GEMEnA expanded its scope to include certifications and licenses 

at all educational levels.

This article describes the process of developing the questions on certifications and licenses for the Current 

Population Survey (CPS),4 starting with the early work on question development done by GEMEnA. It then 

describes the work done by BLS to adapt the questions to the CPS, the development and acceptance of a 

proposal to add the questions, the fielding of the questions, and the first publication of results.

Defining certifications and licenses

GEMEnA began its work on certifications and licenses by conducting a literature review, consulting with subject 

matter experts, and reviewing the credential-related questions that existed on a variety of nonfederal surveys. After 

these efforts, the group identified two distinct types of credentials that are commonly referred to as certifications or 

licenses: (1) those that are awarded by a certification body and reflect the acquisition of a level of skill in an 

occupation but do not convey a legal authority to work in that occupation and (2) those awarded by a governmental 

body that convey a legal authority to work in an occupation. Because there was some confusion about how the two 

terms were used, GEMEnA created working definitions for certifications and licenses. These definitions were a first 

step in developing the common language to better discuss these credentials. (See table 1.)

Characteristic Certification License

Awarded by 

whom

A credential awarded by a nongovernmental certification 

body on the basis of individuals demonstrating, through an 

examination process, that they have acquired the 

designated knowledge, skills, and abilities to perform a 

specific job; the examination can be written, oral, or 

performance based

A credential awarded by a governmental licensing 

agency on the basis of predetermined criteria; the 

criteria may include some combination of degree 

attainment, certifications, educational certificates, 

assessment, apprenticeship programs, or work 

experience

Legal 

authority Does not convey a legal authority to work in an occupation Conveys a legal authority to work in an occupation

Renewal Time limited, renewed through a recertification process(1) Time limited, must be renewed periodically

Examples Information technology certification and project management 

professional certification Cosmetology license and teacher’s license

Table 1. Definitions of certification and license as developed by the Interagency Working Group on 

Expanded Measures of Enrollment and Attainment

See footnotes at end of table.
 U.S. BUREAU OF LABOR STATISTICS

3

MONTHLY LABOR REVIEW 

Notes:

(1) Some certifications are time limited but do not have a recertification process. For example, many information technology certifications are for a specific 

software that becomes obsolete when it is replaced with a newer version.

Source: Interagency Working Group on Expanded Measures of Enrollment and Attainment and U.S. Bureau of Labor Statistics.

GEMEnA also investigated a third type of nondegree credential—educational certificates. According to GEMEnA 

definitions, educational certificates, which are awarded by an educational institution upon the recipient’s 

completion of all requirements for a program of study, are not the same as certifications and licenses. GEMEnA’s 

background research showed that certifications and licenses are quite distinct from educational certificates. 

Certifications and licenses are job or occupational requirements which demonstrate that the holder has the 

authority or skills that are needed to perform specific job duties. By contrast, educational certificates are awarded 

to show completion of a program of study; while this training may help in the performance of a specific job, 

educational certificates are not necessarily required or considered proof of qualification.5 Like educational 

certificates, certificates of attendance or participation in short-term training are also considered to be different from 

certifications and licenses.

GEMEnA focus groups and cognitive testing

After developing definitions of certifications and licenses, GEMEnA decided that some exploratory qualitative 

research would be helpful. In December 2009, NCES funded three focus groups in the Washington, DC, area on 

nondegree credentials. Reflecting GEMEnA’s early interest in subbaccalaureate certifications and licenses, the 

focus group participants were ages 21 to 40 and had at least a high school diploma or GED (a high school 

equivalency diploma) but did not have a bachelor’s degree. Two of the focus groups were composed of people 

who said they were certified, registered, or licensed in a health care, information technology, or business field. A 

third focus group was made up of individuals who said they had a certification in another field or who were 

interested in obtaining a certification.6

In 2010, NCES also funded three rounds of cognitive testing (60 interviews total) to determine whether people 

could answer questions about nondegree credentials. Participants included individuals with certifications or 

licenses as well as those without these credentials. Because of the concern that there might be regional 

differences, the interviews were conducted with people in three different areas: Washington, DC; Charlotte, NC; 

and Minneapolis, MN. Respondents were ages 21 to 40. The first round of testing concentrated on certifications 

and licenses, the second on educational certificates, and the third on both types of credentials.7

In general, the focus groups and cognitive testing confirmed that respondents were aware of their certifications and 

licenses and were able to answer questions about them. For many, these types of credentials were necessary for 

their jobs, either because the law or their employer required them. Particularly in some occupations, respondents 

often had more than one credential. Also, respondents did not confuse these credentials with the traditional 

educational attainment categories, and often closely associated certifications and licenses with their work.

Because of the interest in distinguishing between certifications and licenses, GEMEnA investigated the possibility 

of asking respondents to identify their credentials as either a certification or a license. However, in both the focus 

groups and the cognitive interviews, some participants found it difficult to distinguish between the two types of 

credentials, with a few saying that they thought the two terms were synonymous. Many factors contribute to this 
 U.S. BUREAU OF LABOR STATISTICS

4

MONTHLY LABOR REVIEW 

confusion. In some fields, certification is a prerequisite for applying for a license—such as a financial planner who 

must first become a certified financial planner before applying for a license through a state board of standards. The 

reverse is also true in some fields—that is, the individual must first obtain a license before becoming certified. For 

example, a real estate agent may need to get a license before being certified in a specialization area. In addition, 

in some fields, both “certification” and “license” may be used to refer to the same type of credential. For example, 

all states have credentialing requirements for public school teachers. These requirements are variously referred to 

as teaching licenses, certifications, credentials, or certificates. However, although state requirements vary, all of 

these credentials are issued by a governmental agency and convey a legal authority to teach school. Thus, 

according to GEMEnA definitions, these credentials are considered licenses.

Rather than asking about certifications and licenses separately, GEMEnA decided to test whether the two types of 

credentials could be asked about together. In the third round of cognitive testing, both the terms “certification” and 

“license” were used in the same question—that is, people were not asked to distinguish between the two but 

simply to indicate yes or no as to whether they had a “certification or license.” This approach was found to work 

well.8

Adult Training and Education Survey pilot study

The 2010 Adult Training and Education Survey (ATES) pilot study was an NCES-sponsored 15-minute mail and 

telephone survey that was fielded from September 2010 to January 2011. A major goal of this study was to 

evaluate proposed questions on certifications and licenses.9 The pilot study was designed to evaluate estimates 

generated from the questions, investigate the extent of measurement error, and determine whether proxy 

respondents—that is, respondents who provide information about other members of their households—could 

accurately answer the questions.10

Evaluating the estimates

The pilot contained a nationally representative sample of 3,730 adults age 18 and over. Although the pilot was 

never intended to provide official population estimates, weights were developed so that nationally representative 

estimates could be generated. Although no federal data on certifications and licenses existed at the time, there had 

been periodic attempts, using nonfederal surveys, to measure the numbers of people with certifications or 

licenses. Pilot-study estimates were compared with estimates from the Princeton Data Improvement Initiative 

(PDII), a survey that had been conducted in 2008 and included questions on occupational licensing. To create 

comparable estimates, the ATES and PDII samples were restricted to employed adults. According to the ATES, 38 

percent of employed adults had a certification or license, compared with 34 percent in the PDII. Although these 

estimates are not identical, the difference is relatively small, especially given that the two surveys were conducted 

2 years apart and had a number of differences in their questionnaires. GEMEnA concluded that estimates from the 

ATES questions were reasonable as compared with previous data collection efforts.

Measurement error

In addition to including the nationally representative sample, the ATES included a nonrandom sample of 340 adults 

who were known to hold nondegree credentials. Some of the adults in this seeded sample had been reported by a 

credentialing body to have received a certification or license.11 The seeded sample contained people in the 

following certified or licensed occupations: alcohol and drug counselor or aide; health technician, specialist, or 
 U.S. BUREAU OF LABOR STATISTICS

5

MONTHLY LABOR REVIEW 

sonographer; pharmacy technician; electrical aide; plumber; heating, ventilation, and air conditioning technician; 

and fuel technician. The seeded sample was included specifically to evaluate how well respondent answers lined 

up with the administrative data, allowing for the calculation of underreporting rates. Here are the underreporting 

rates (in percent) of certifications and licenses by age and employment status:

Although the underreporting rates vary for different groups, there are few statistically significant differences across 

different subgroups. A multivariate analysis showed that differences in underreporting rates are driven, in part, by 

employment status, with employed people being more likely than the not employed to accurately report that they 

have a certification or license. This difference may reflect the fact that a certification or license could be more 

meaningful for people who are employed and who, therefore, have these credentials for work-related reasons. 

Those who have credentials but are not employed—the majority of whom are not looking for work—may not view 

them as relevant and thus might be less likely to report them. Also, while considerable effort was made to find 

current and accurate administrative records for the seeded sample, the measurement error associated with these 

records is unknown. It is possible that some reporting differences could be due to errors in the seeded sample 

frame. After taking all of these factors into account, GEMEnA determined that the underreporting rates in the pilot 

were within acceptable ranges.12

Proxy responses

Many federal household surveys use proxy respondents. Thus, a key factor in developing questions to include on 

existing surveys was to determine whether respondents could answer the questions about others in their 

households. To evaluate the questions for proxy respondents, the 2010 ATES pilot study, in a subsample of about 

1,000 households with 2 or more adults, asked adults to report about other adults in their household. The study did 

not include multiple respondents from the same households, so direct comparisons between self-responses and 

proxy responses about individuals could not be done. Instead, GEMEnA compared the frequencies of selfresponses and proxy responses to determine whether there were large differences in the incidence of “don’t know” 

responses. High rates of nonresponse among proxies might lead to bias in the survey estimates and would mean 

the questions were unsuitable for surveys relying on proxy respondents. However, nonresponse rates for proxy 

respondents were fairly low for the main certification and license questions, and GEMEnA determined that proxy 

reporting was acceptable for these questions.13

Final recommendations

After all the ATES results had been examined, GEMEnA made several recommendations.14 Because of cost 

constraints and concerns about increasing respondent burden, most federal household surveys—particularly large 

                   Total 15.0                           

 18 to 34 years 19.0 

 35 to 64 years 11.6 

 65 years and over 29.2 

 Employed by a private company 15.4 

 Employed by government or nonprofit, or self-employed 6.9 

 Not employed 45.8 

 Certification 17.1 

 License 13.7 
 U.S. BUREAU OF LABOR STATISTICS

6

MONTHLY LABOR REVIEW 

•

•

•

•

surveys like the CPS—can add no more than a few questions, and so GEMEnA recommended only a small 

number of questions. The group suggested that surveys include the following:

The broad question about certifications and licenses used in ATES, which GEMEnA determined worked well.

A question to distinguish between certifications and licenses. Certifications are issued by industry or 

professional organizations, while licenses are issued by government agencies.

A question to determine the field of the certification or license. However, GEMEnA recognized that surveys 

with little available space for new questions might be unable to collect this information, particularly in the 

case of individuals with multiple certifications or licenses.

A question to determine whether the certification or license is related to an individual’s current work.15

Fielding the GEMEnA questions

A number of federal surveys have added GEMEnA questions on certifications and licenses. The first federal survey 

to include questions developed by GEMEnA was the Survey of Income and Program Participation (SIPP). In the 

autumn of 2012, nine questions on certifications and licenses were added to the 13th wave of the 2008 SIPP. 

These included the main questions recommended by GEMEnA as well as questions about whether coursework 

was required to earn the credential, the main reason for obtaining the credential, and whether the credential could 

be used to get a job with any employer in the field.16

After the SIPP had been fielded, the questions were proposed for a number of other federal household surveys, 

such as the National Science Foundation’s National Survey of College Graduates and several NCES surveys. The 

number of questions each survey could include varied according to the goals of the survey and how much of the 

questionnaire could be devoted to the topic.17

Because of the successful use of the questions in other surveys, BLS, the Census Bureau, NCES, and GEMEnA 

members began to consider whether the questions would be a valuable addition to the CPS.

Current Population Survey

The CPS is a survey of approximately 60,000 households that is jointly sponsored by BLS and the Census Bureau. 

The survey is probably best known for the national unemployment rate, which is published every month by BLS. 

Statistics from the CPS are among the country’s most timely economic indicators and are widely used by 

policymakers and researchers. The CPS provides extensive information about the employment and unemployment 

status of the population, and the survey data can be broken out by a variety of demographic characteristics, such 

as race and ethnicity, educational attainment, disability status, age, and gender. In addition, the CPS is a primary 

source of socioeconomic data about the labor force, including industry and occupation, hours of work, and 

earnings.

Including a small set of questions about certifications and licenses would add valuable information about the labor 

market and would enable researchers and policymakers to examine how these types of credentials affect labor 

market success. Unemployment rates and other key labor market indicators could be compared for those with and 

without certifications and licenses and could be broken out by the many demographic and labor force 

characteristics already collected in the CPS.
 U.S. BUREAU OF LABOR STATISTICS

7

MONTHLY LABOR REVIEW 

The core part of the survey is referred to as the monthly CPS (or the basic CPS). Households are in the survey for 

8 months total on what is referred to as a 4-8-4 rotation—that is, they are in the sample for 4 months, out of the 

sample for the next 8 months, and back in the sample for the following 4 months. For example, a household that 

was scheduled to be interviewed for the first time in April would also be contacted in May, June, and July. The 

household would then be out of the sample for the next 8 months and would return to the sample the following 

April, May, June, and July.

In addition to the monthly CPS, there are periodic CPS supplements, which are typically included in a particular 

month of the year. Supplements are sets of questions on a specific topic and generally take no longer than 10 

minutes to administer. The best known of the supplements is the Annual Social and Economic Survey (ASEC), 

also sometimes referred to as the “March supplement”; this supplement is the source of the poverty rate as well as 

other income and health insurance statistics published annually by the Census Bureau.

An immediate question of the working group was where to add questions to the CPS. Likely CPS supplement 

candidates were quickly narrowed down to two: the ASEC and the October school enrollment supplement.18

Another alternative was to add the questions to the monthly CPS. Each of the three candidates had advantages 

and disadvantages, as shown in table 2.

Source: U.S. Bureau of Labor Statistics.

Survey Advantages Disadvantages

Annual Social 

and Economic 

Survey (ASEC)

The ASEC contains detailed income 

data, and one key research question is 

how certifications and licenses affect 

income.

The ASEC is by far the longest of all CPS supplements. Adding 

questions would make it even longer and further increase the burden 

on respondents. This would severely limit the number of questions that 

could be included. Data users might not find a small set of questions 

sufficient for analytical needs.

Analysis of detailed subgroups would be limited because of the 

relatively small sample size.

School 

enrollment 

supplement

Because the supplement is fairly short, 

a more extensive set of questions about 

certifications and licenses could be 

included.

The supplement does not contain information about income and thus 

could not be used to answer key research questions on that topic.

Questions about certifications and licenses might not work well in the 

supplement because the cognitive task of reporting about enrollment 

might be too different from that of reporting about nondegree 

credentials.

Analysis of detailed subgroups would be limited because of the 

relatively small sample size.

Monthly 

Current 

Population 

Survey

Adding questions to the monthly CPS 

would create a monthly time series, 

allowing for an examination of cyclical 

trends, as well as long-term trends.

Since the survey is given every month, questions are added very rarely 

because of the increase in respondent burden. Only a very small set of 

questions could be added, and data users might not find such a small 

set sufficient for analytical needs.

Information on certifications and 

licenses could be linked to data 

collected in any CPS supplement, 

including income data from the ASEC 

and information from the school 

enrollment supplement.

Because the CPS is asked each month, 

the data can be pooled across months 

to increase sample size. This would 

allow for more detailed examination of 

subgroups.

Table 2. Advantages and disadvantages of adding questions about certifications and licenses to the 

monthly Current Population Survey (CPS) or CPS supplements
 U.S. BUREAU OF LABOR STATISTICS

8

MONTHLY LABOR REVIEW 

1.

2.

3.

1.

All agreed that the monthly CPS was the most desirable option for the questions. This choice was largely because 

the monthly CPS provided the greatest flexibility: it would provide monthly data suitable for cyclical analysis; would 

allow for analysis of the income data on the ASEC; and could be linked to any other supplement, including the 

school enrollment supplement.

However, there were considerable challenges to adding questions to the monthly CPS. Changes are made 

occasionally to CPS supplements but only rarely to the monthly CPS. In general, both BLS and the Census Bureau 

are very reluctant to increase the length of the monthly survey. Doing so not only increases respondent burden but 

also could make breakoffs in a given month or refusals in future months more common. In addition, there is always 

the possibility that a change in the monthly survey could adversely affect responses to other questions, including 

those related to unemployment. While such an effect may be unlikely, BLS and Census take great care to ensure 

that changes to the monthly CPS do not affect the unemployment rate and other key statistics. Also, BLS did not 

have funding available to pay the ongoing costs of permanently increasing the overall length of the survey. 

Therefore, both BLS and Census agreed that only a limited set questions could be added and that if questions 

were added, some existing questions would have to be removed to limit the increase in cost and respondent 

burden.

Identifying questions to add to the CPS

BLS, Census, and GEMEnA held a series of discussions about the questions. Three questions were proposed for 

addition:

A question to identify certifications or licenses. This is the main question recommended by GEMEnA and 

has been shown to reliably identify individuals who have these credentials. Because the monthly CPS is 

primarily a labor force survey, capturing this information for both the employed and the unemployed is 

important. Doing so would enable BLS to produce a variety of labor force statistics regarding certifications 

and licenses, including unemployment rates.

A question to identify whether the credential was issued by any level of government. The intent of this 

question is to determine the issuing authority of a credential and, thus, to distinguish between certifications 

and licenses. According to GEMEnA definitions, licenses are issued by a governmental body.

A question to determine whether the certification or license was needed for the current job. In this question, 

“required” can mean either legally required (such as a license to practice medicine) or required by the 

employer (such as an information technology certification). People may have credentials that they do not 

use for their current job, such as a computer technician who obtained a real estate license for personal 

interest or for a second or previous job. Also, people sometimes have certifications or licenses that may be 

helpful for their jobs but are not required by law or by their employer.

Two other questions were briefly considered:

A question about how long the individual had held a credential. GEMEnA had not yet tested and approved 

the wording of such a question. BLS survey methodologists were concerned about the possibility of recall 

bias, particularly among proxy respondents. Therefore, considerable work would have to be done to develop 

an acceptable question and would likely have delayed implementation.
 U.S. BUREAU OF LABOR STATISTICS

9

MONTHLY LABOR REVIEW 

2.

1.

2.

3.

A question about the field of the certification or license. Although GEMEnA had tested versions of this 

question, all were too long for the monthly CPS. Additionally, Census staff would have to review 

respondents’ answers and code them into a set of predetermined categories. This would be time consuming 

and labor intensive, making such a question prohibitively expensive. Also, for those certifications or licenses 

needed for individuals’ jobs, the broad topic of the credential could be inferred from the information on 

occupation and industry already collected in the monthly CPS.

Because of these concerns, as well as concerns about increasing the respondent burden and survey costs, BLS 

and Census determined that adding either of these questions was not practical.

Identifying questions to remove from the CPS

As previously mentioned, BLS and Census wished to limit the increase in the length of the survey and the 

corresponding increase in respondent burden. Therefore, both agencies agreed that if questions on certifications 

and licenses were added, some questions would have to be removed.

After an evaluation of the CPS questionnaire, three educational attainment questions were identified for possible 

removal. These educational attainment questions had been added to the CPS in 1996 and were designed to allow 

researchers to construct a measure of continuous years of education as an alternative to degree-based measures, 

but the information was little used. Data from these questions had never been published by BLS, and a literature 

review found very few publications using the questions, even though variables derived from the questions had 

been included on the CPS public-use microdata files for many years.

The educational attainment questions proposed for removal were as follows:

Since completing your bachelor’s degree, have you ever taken any graduate or professional school courses 

for credit? [Asked of people whose highest educational attainment was a bachelor’s degree]

Did you complete six or more graduate or professional school courses? [Asked of people who responded 

“yes” to the previous question]

Was your master’s degree program a 1-year, 2-year, or 3-year program? [Asked of people whose highest 

educational attainment was a master’s degree]

Developing the proposal

BLS formed a committee of CPS subject matter experts, survey methodologists, and research economists to 

develop a proposal to add questions on certifications and licenses to the monthly CPS. In addition to relying on 

internal expertise, BLS also consulted with GEMEnA and the GEMEnA expert panel. This proposal was to add 

three questions on certifications and licenses and remove the three little-used questions on educational 

attainment, but many details needed to be worked out, such as question wording, placement, and frequency.

Question wording

For all three questions, BLS started with the question wording recommended by GEMEnA. After the wording was 

reviewed by BLS survey methodologists, subject matter experts, and GEMEnA members, a few minor revisions 

were made to make the questions work in the context of the CPS. Specifically, because the CPS is collected 
 U.S. BUREAU OF LABOR STATISTICS

10

MONTHLY LABOR REVIEW 

through personal and telephone interviews, respondents answer for themselves and for other members of their 

households. Hence, a number of wording changes needed to be made to accommodate proxy respondents.

On the first question, the most notable change was that a sentence was added to the first question about business 

licenses. This was done because in the SIPP, people whose businesses required licenses (such as liquor licenses) 

sometimes answered that they had a license. Also, some clarifications were made to the text, which the interviewer 

could read if necessary. In addition, GEMEnA suggested some minor refinements to the question text. Final 

wording of the first question is as follows:

1.     Do (you/name) have a currently active professional certification or a state or industry license? Do not

include business licenses, such as a liquor license or vending license.

(Read if necessary: A professional certification or license shows you are qualified to perform a specific job. 

Examples include a real estate license, a medical assistant certification, a teacher’s license, or an IT certification. 

Only include certifications or licenses obtained by an individual.) [Asked of people age 16 and over]

For the second question, a change was made to accommodate respondents with more than one certification or 

license. The original question wording was to ask “was it issued by the federal, state, or local government?” For 

people with multiple certifications or licenses, such a question would be confusing, especially if the answer were 

different for different credentials. GEMEnA and the GEMEnA expert panel decided it was most important to find out 

if the respondent had at least one license. As they recommended, the second question was modified to allow for 

the situation in which a person has more than one certification or license. Instead of “was it issued by . . . ” the 

question wording was modified to “were any of your certifications or licenses issued by . . . ” to allow for the 

possibility of multiple credentials:

2.     Were any of (your/his/her) certifications or licenses issued by the federal, state, or local government? 

[Asked of people who responded “yes” to the previous question]

For the third question, modifications to the text were made for two reasons. First, like the second question, the 

third question required some modifications to wording for people who had multiple credentials. Second, CPS 

subject matter experts decided that it was important to ask the third question of the unemployed about their past 

jobs. This is consistent with other labor force questions, such as industry and occupation, and allows for an 

examination of the unemployed who have previously worked. The wording had to be modified for these individuals 

to clarify that it was their previous job that was being asked about. The final wording of the third question is as 

follows:

3.     Earlier you told me (you/name) had a currently active professional certification or license. Is (your/his/her) 

certification or license required for (your/his/her) (job/main job/job from which [you are/he is/she is] on layoff/job 

at which [you/he/she] last worked)? [Asked of people who were employed and those who were unemployed 

but had previously worked]

Note that the wording differs by individuals’ labor force status. For example, people who are employed at only one 

job are asked about their job, while those with multiple jobs are asked about their main job. Likewise, unemployed 

individuals who are on layoff are asked about the job from which they are on layoff, and unemployed individuals 

who previously worked are asked about their last job.
 U.S. BUREAU OF LABOR STATISTICS

11

MONTHLY LABOR REVIEW 

•

•

•

•

•

Question placement

The CPS has a long and complex questionnaire, and so another task for the BLS working group was to determine 

where best to place the questions. It was important that the questions not confuse respondents or interfere with 

responses to other questions. Because results from GEMEnA focus groups had shown that people did not 

necessarily regard certifications and licenses as related to educational attainment but rather as credentials for their 

jobs, the team evaluated whether they would be most appropriate next to other job-related questions.

After consideration, BLS decided that placing the first two questions after the educational attainment questions 

made the most sense. The primary reason behind this decision was the importance of collecting information from 

these two questions for all individuals, not just for those who were employed. Unlike the job-related questions, the 

educational attainment questions are asked of everyone. The third question, which asked specifically about 

whether the credential was for the respondent’s job, did belong among other job-related questions.

Question frequency

BLS also needed to determine how often the questions should be asked. Some questions on the CPS are asked 

only once because the requested information is unlikely to change, such as questions about race, ethnicity, and 

foreign- or native-born status. By contrast, questions on labor force status are asked every month because the 

primary purpose of the monthly CPS is to track the nation’s employment and unemployment characteristics, and it 

is vital that this information is as current as possible. Other questions are asked with varying periodicity; for 

example, educational attainment is asked of people in their first and fifth interviews and also in July, October, and 

February. Questions about disability are asked in the first and fifth interviews.

After consultation with GEMEnA and the GEMEnA expert panel, BLS determined that whether or not a person has 

a certification or license is not likely to change much from month to month. In order to limit the addition to 

respondent burden, BLS proposed that the questions be asked only in the first and fifth interviews. A further reason 

for not asking the questions more often was the desire to limit the increase in the length of the survey.

Evaluating the proposal

Evaluation of the proposal took several months and included cognitive testing by BLS survey methodologists. It 

also involved considerable stakeholder outreach by both GEMEnA and BLS.

Cognitive testing

The questions on certifications and licenses had been thoroughly tested through the GEMEnA work. However, BLS 

staff felt that it was important to test the questions within the CPS to ensure that they worked as expected. 

Specifically, the cognitive testing sought to

test the placement of the questions

test the minor wording changes that had been made to the questions

determine whether the new questions would likely have an impact on existing CPS questions

determine if introductory or transition language is necessary before the questions

determine if interviewer instructions or help screens are necessary to explain the key concepts.
 U.S. BUREAU OF LABOR STATISTICS

12

MONTHLY LABOR REVIEW 

About 25 cognitive interviews were conducted. Respondents included individuals who had certifications or licenses 

as well as those who did not. The testing did not reveal any problems with the proposed placement or wording of 

the questions. Also, the cognitive testing did not find evidence that the new questions had an impact on responses 

to existing questions. Since adding transition language would lengthen the survey and respondents did not appear 

to have any difficulty understanding the questions without such language, the cognitive testing report 

recommended against adding transition language. The report did recommend that interviewers be trained on the 

objectives of the new questions.19

Stakeholder outreach

A great deal of outreach on the CPS proposal was done through GEMEnA. The proposal was presented to the 

GEMEnA expert panel at their annual meeting, and GEMEnA members presented the proposal to a variety of 

governmental and nongovernmental groups. As an agency particularly concerned with measures of educational 

attainment, NCES was also consulted.

In addition, BLS reached out to key stakeholders. Specifically, BLS sought to learn whether stakeholders thought 

the questions would be a valuable addition to the survey, whether the removal of the three educational attainment 

questions was an acceptable price to pay, and whether they thought BLS should proceed with the proposal.

The proposal was also presented to the BLS Technical Advisory Committee. Informally, BLS contacted key 

stakeholders who might have an interest in the educational attainment questions proposed for removal. In addition, 

the new questions were reviewed and cleared by the Office of Management and Budget, and the clearance 

process included two periods of public comment. Feedback to the proposal was almost exclusively favorable.

Collection of data

Because stakeholder feedback was positive and the cognitive testing revealed no issues, BLS decided to accept 

the proposal to permanently add the three questions about certifications and licenses to the monthly CPS. In 

accordance with the proposal, the first two questions would be asked after the educational attainment questions, 

and the third would be asked in the labor force section. The three questions would be asked only in the first and 

fifth interviews, though the questions would be asked in other months of new household members. Also, the third 

question would be reasked in other months in certain circumstances, such as when an individual changed jobs or 

an unemployed person became employed.

As recommended in the cognitive testing report of the proposed CPS questions, the BLS working group developed 

training materials that were given to interviewers prior to the first collection of data. These materials covered 

common issues that had come up in cognitive testing at BLS and through the GEMEnA work. In addition to being 

reviewed by BLS staff, the training materials were reviewed by GEMEnA members. These materials were also 

incorporated into a section in the CPS interviewers’ manual.20

The training materials were provided to interviewers prior to the introduction of the new questions with the 

collection of data for January 2015. Had the questions been asked starting in January only in the first and fifth 

interviews, the full sample would not have received the questions until May 2016. Thus, annual averages would 

not have been available until 2017, and ASEC data about certification and licenses would likewise not have been 

available until 2017. To allow for the possibility of 2015 annual averages and 2015 ASEC data, the questions were 
 U.S. BUREAU OF LABOR STATISTICS

13

MONTHLY LABOR REVIEW 

asked in all interviews (not just the first and fifth) in January 2015. While responses to the questions asked in other 

than the first and fifth interviews might be different, the risk was thought to be minimal.

During the first year of data collection, BLS staff listened to many interviews in order to ensure that the questions 

were working as expected. BLS staff also evaluated the data each month to identify any possible issues. One 

significant data collection problem was encountered: because of a programming error, the third question was not 

asked in the first and fifth interviews in May and June. The lack of data in these 2 months resulted in BLS not 

including 2015 annual averages for the third question in the first publication of data on certifications and licenses. 

Moreover, a public-use variable derived from the third question was not produced for 2015.21

Nonresponse on the questions is quite low. However, as with most survey questions, there are respondents who 

either refuse to answer or do not know the answers to the questions. As is customary for nonresponse to questions 

on the CPS, BLS economists devised an imputation method to fill in missing data. This is often done with survey 

data because this type of nonresponse is unlikely to be random; analyses using only reported values implicitly 

assume that all respondents are equally likely or unlikely to respond and that the estimate is approximately 

unbiased. The imputation method used for the questions on certifications and licenses is similar to that used for 

other CPS variables.22

Publication

BLS published the first CPS data from the questions about certifications and licenses on its website in April 

2016.23 As mentioned earlier, only data from the first two questions were tabulated for this first publication. An 

extract file containing public-use variables for 2015 for the first two questions is available to researchers on the 

Census Bureau website. These variables can be linked to the 2015 CPS public-use microdata files, enabling 

researchers to conduct their own analyses. A similar extract file that includes variables for all three questions will 

be made available early in 2017. Also, starting with the release of data for January 2017, variables for all three 

questions will be included on the CPS public-use microdata files.24

SUGGESTED CITATION

Mary Dorinda Allard, "Adding questions on certifications and licenses to the Current Population Survey," Monthly 

Labor Review, U.S. Bureau of Labor Statistics, November 2016, https://doi.org/10.21916/mlr.2016.52

NOTES

1 For more information about GEMEnA, see http://nces.ed.gov/surveys/GEMEnA/.

2 Realizing that adding a limited set of questions to large national household surveys could not address all the many questions about 

certifications and licenses, GEMEnA also worked to develop a more extensive survey for more indepth analysis to be administered by 

NCES.

3 U.S. President Barack Obama, Address to Joint Session of Congress, February 24, 2009. See https://

obamawhitehouse.archives.gov/the-press-office/remarks-president-barack-obama-address-joint-session-congress/. 
 U.S. BUREAU OF LABOR STATISTICS

14

MONTHLY LABOR REVIEW 

4 For more information about the CPS, see the BLS website at www.bls.gov/cps.

5 Educational certificates are defined by GEMEnA as credentials awarded by an educational institution—such as a community 

college, another type of college or university, or a trade school—on the basis of completion of all requirements for a program of study, 

including coursework and test or other performance evaluations, and are not time limited. Examples of these types of credentials 

include a digital arts certificate from an online university or a motorcycle mechanics diploma from a community college. Most 

educational certificates are awarded at the subbaccalaureate level, and a small number are awarded after the completion of a 

postsecondary degree. Individuals may enroll in an educational certificate program to gain the knowledge needed to attain a 

certification, which may be required prior to applying for a license. For more information about educational certificates, see the Adult 

Training and Education Survey (ATES) pilot study technical report, U.S. Department of Education, April 2013, http://nces.ed.gov/

pubs2013/2013190.pdf.

6 For detailed information about the focus groups, see appendix F, ATES pilot study technical report.

7 For detailed information about the cognitive testing, see appendix G, ATES pilot study technical report.

8 Over the years, a number of additional focus groups and cognitive testing efforts have included questions about certifications and 

licenses. These have supported the findings of the original GEMEnA focus group and cognitive testing reports.

9 The ATES pilot study also included questions about educational certificates. For more information, see ATES pilot study technical 

report.

10 For more information about all of these analyses, see the ATES pilot study technical report.

11 All those in the seeded sample were known to have a nondegree credential. However, some of those in the seeded sample had 

educational certificates rather than certifications or licenses.

12 Further GEMEnA studies have also included seeded samples. Underreporting rates of the number of certifications and licenses 

from the later studies are consistent with the ATES.

13 More detailed questions about certifications and licenses, such as the credential name and the year the credential was awarded, 

were not found to work for proxy respondents.

14 More detailed discussion of these recommendations can be found in the ATES pilot study technical report.

15 Two other pilot studies were conducted that included certification and license questions: the 2013 National Training and Education 

Survey pilot study and the National Household Education Survey feasibility study in the 2014 ATES. These pilot studies had similar 

findings.

16 Stephanie Ewert and Robert Kominski, Measuring alternative educational credentials: 2012, U.S. Census Bureau, January 2014, 

https://www.census.gov/prod/2014pubs/p70-138.pdf.

17 A list of federal household surveys that include questions on certifications and licenses is available on the GEMEnA website at 

http://nces.ed.gov/surveys/gemena/surveys.asp.

18 The school enrollment supplement has been sponsored by NCES, BLS, and the Census Bureau since the late 1960s.

19 For more information on the cognitive testing, see appendix H of the 2014 Office of Management and Budget clearance for the 

CPS.

20 Interviewer instructions on the questions about certifications and licenses can be found on pages C3-22 and C3-23 of the Current 

Population Survey interviewing manual, U.S. Census Bureau, April 2015, http://www2.census.gov/programs-surveys/cps/

methodology/intman/CPS_Manual_April2015.pdf.

21 Three other minor data collection issues occurred. In January 2015, the first two questions were not asked of people who were 

known to be over age 15 but who had not provided an exact age. Also, for the first half of the year, the questions were not asked of 

people who had responses of “don’t know” or “refused” for the educational attainment questions. BLS investigated both of these 

issues and found that they affected a small number of respondents. After examining later months, BLS determined that responses for 
 U.S. BUREAU OF LABOR STATISTICS

15

MONTHLY LABOR REVIEW 

these groups do not appear to be systematically biased. A third issue was that the third question on certifications and licenses was not 

asked of certain people who should have been asked; for example, some people who were unemployed and had previously been 

employed were not asked the question.

22 Typically, CPS imputation methods rely on information about basic demographic characteristics, such as age and educational 

attainment. Unlike other CPS imputation methods, the imputation for certifications and licenses also uses information on labor force 

status and occupation. More information about imputation methods can be found in chapter 9 of Design and methodology: Current 

Population Survey, technical paper 66, U.S. Bureau of Labor Statistics and U.S. Census Bureau, October 2006, https://

www.census.gov/prod/2006pubs/tp-66.pdf.

23 Results are available on the BLS website at https://www.bls.gov/cps/certifications-and-licenses.htm.

24 To download the CPS public-use microdata and extract files and accompanying documentation, see the Census Bureau website at 

http://thedataweb.rm.census.gov/ftp/cps_ftp.html.

RELATED CONTENT

Related Articles

Will I need a license or certification for my job?, Career Outlook, September 2016.

The de-licensing of occupations in the United States, Monthly Labor Review, May 2015.

The development of questions on disability for the Current Population Survey, Monthly Labor Review, April 2014.

Life, limbs, and licensing: occupational regulation, wages, and workplace safety of electricians, 1992–2007, Monthly Labor Review, 

January 2014.

Employment projections through the lens of education and training, Monthly Labor Review, April 2012.

Related Subjects

Education and training Current population survey Statistical programs and methods Employment