U.S. Merit Systems Protection Board: 2021 Merit Principles Survey Methodology and Materials 1/5

U.S. Merit Systems Protection Board

2021 Merit Principles Survey Design and Methodology

This document describes the data (public use files and supporting documentation) and methodology 

for MSPB’s 2021 Merit Principles Survey (MPS or MPS 2021).

Background

The U.S. Merit Systems Protection Board (MSPB) has the statutory responsibility to assess the 

health of Federal merit systems and the authority to conduct special studies of the Federal civil 

service [see 5 U.S.C. 1204(a)(3) and 5 U.S.C. 1204(e)(3)]. MSPB administers a periodic Merit 

Principles Survey to help carry out those studies. Those studies, including summaries and analyses of 

data from the MPS, are officially submitted to the President and Congress and shared with Federal 

policymakers and agencies.

Survey Design

Typically, the MPS contains a combination of core items that MSPB tracks over time and specialpurpose items developed to support a particular study or provide information on a topic of research 

interest. This survey differs from the Federal Employee Viewpoint Survey administered by OPM in 

several respects, including: a focus on merit system principles and Governmentwide civil service 

issues; administration every few years instead of annually; and a smaller sample. Agency participation 

in the MPS was mandatory, but individual response to the survey was voluntary.

Survey Content

The MPS2021 supports several lines of research in MSPB’s research agenda. The survey comprised 

sections related to particular research topics and sections that support data validation and analysis. 

The major sections of the survey are listed below.

• Work Experience/Employee Engagement

• Adherence to Merit System Principles

• Experience of Prohibited Personnel Practices

• Experience of Sexual Harassment

• Respondent Demographics and Career Intentions

Survey Sampling

MSPB developed a sampling strategy, using a stratified random sample, to provide a reliable measure 

of Governmentwide opinion among permanent, full-time civilian Federal employees. The sample 

was stratified by Federal agency (and agency bureau or component for selected agencies) and 

supervisory status (nonsupervisor, supervisor, or executive). To provide useful measures across 

agency and supervisory status, MSPB oversampled (i.e., surveyed a higher proportion of) some 

groups, including supervisors, executives, and employees in smaller agencies.

MSPB drew the survey samples from rosters provided by participating agencies. Those rosters 

contained data needed to administer the survey (such as the email address), confirm eligibility for the 

survey, and categorize employees for survey sampling. Rosters were provided by agencies in 

November 2020 and the sample was drawn in December 2020. For the two small agencies 
U.S. Merit Systems Protection Board: 2021 Merit Principles Survey Methodology and Materials 2/5

participating, all eligible employees were surveyed. The final sample included just over 100,000 

employees from the 27 Federal agencies listed below.

Table: Federal Agencies Participating in the 2021 Merit Principles Survey

Departments

Department of Agriculture

Department of the Air Force

Department of the Army

Department of Commerce

Department of Defense

Department of Education

Department of Energy

Department of Health and Human Services

Department of Homeland Security

Department of Housing and Urban Development

Department of the Interior

Department of Justice

Department of Labor

Department of the Navy

Department of State

Department of Transportation

Department of the Treasury

Department of Veterans Affairs

Independent Agencies

Environmental Protection Agency

Federal Deposit Insurance Corporation

General Services Administration

Merit Systems Protection Board

National Aeronautics and Space Administration

Office of Government Ethics

Office of Personnel Management

Securities and Exchange Commission

Social Security Administration

The agencies that participated in the 2021 MPS account for approximately 98 percent of the 

permanent full-time Federal workforce covered by the Enterprise HR Integration–Statistical Data 

Mart, a Governmentwide workforce data repository maintained by the U.S. Office of Personnel 

Management (OPM). Accordingly, the MPS population is functionally almost identical to the 

corresponding Governmentwide population, and the MPS can provide a useful measure of 

employee opinions Governmentwide.

Survey Administration

MSPB notified participating Federal agencies and selected stakeholders (e.g., major Federal 

employee unions) of the survey to request their support. The survey was administered electronically 
U.S. Merit Systems Protection Board: 2021 Merit Principles Survey Methodology and Materials 3/5

(over the Internet) using a Federal Government-specific implementation of a survey development 

and administration application that is used in the private, public, and academic/nonprofit sectors.

Employees were invited to complete the survey via an email message that contained a link unique to 

the individual employee. Those invitations and other publicly available documents (such as MSPBdeveloped fact sheets) informed employees that their participation was voluntary and their responses 

confidential. MSPB further assured participants that it would not disclose data that could be used to 

identify an individual employee.

The MPS was launched in January 2021, with periodic email reminders to complete the survey, and 

closed in April 2021. The survey was open to invited employees for six to eight weeks, allowing for a 

rolling start date and extensions to accommodate IT-related delays in distribution.

Response Acceptance, Rate, and Weighting

After the survey closed, MSPB reviewed the response records to determine whether they would be 

accepted. Acceptance was based on completing a minimum number (approximately ten percent) of 

the non-demographic items asked of all respondents. MSPB then estimated a response rate, 

calculated by dividing the number of accepted responses by the estimated number of invited 

employees who were available to complete the survey.1

Table: Response Rate for the MPS2021

Selected 100,234

Available 99,369

Responded 33,138

Response Rate 33.3%

As discussed previously, sampling rates differed across survey strata. Response rates also varied.

Accordingly, MSPB calculated response weights to produce results that are representative of the 

population surveyed. The initial weights were based on both the data elements used for sampling 

(agency or agency/subelement and supervisory status) and age group. As discussed below, some of 

those initial weights were subsequently adjusted based on analysis of response rates and patterns.

Generally, tabulations and analysis should use the weighting variable (MPS_Weight) included in the 

survey datasets. Results presented in official MSPB publications are weighted unless stated 

otherwise.

Survey Representativeness

For selected dimensions, MSPB compared the proportion of respondents from a particular 

demographic group (based on survey responses) with the estimated proportion of employees in that 

group in the survey population (based on September 2020 records from OPM’s Enterprise HR 

Integration–Statistical Data Mart, a Governmentwide workforce data repository). That comparison 

indicated that younger employees had, on average, a lower response rate than older employees. 

1 An employee was recorded as “unavailable” for reasons such as an invalid email address and receipt of an automated 

reply indicating that the employee had separated from the Federal service, transferred to another agency, or was 

otherwise absent for all or most of the survey period.
U.S. Merit Systems Protection Board: 2021 Merit Principles Survey Methodology and Materials 4/5

Accordingly, MSPB made a post-stratification adjustment to the initial weights based on age group. 

Weights were adjusted only for respondents who self-reported an age group (reporting was 

voluntary, and some respondents declined to provide demographic information), for strata in which 

two or more age subgroups had 30 or more responses. With this adjustment, MSPB concluded that 

representation of major demographic groups was acceptable and that the survey results are 

reasonably representative of the population from which they were drawn.

Margin of Error

MSPB estimated margins of error at a specified confidence level for each item on the survey. For 

items that were asked of all respondents,2 the margin of error at a 95 percent confidence level was at 

or below 1 percentage point.

Table: Illustration of Applying the Margin of Error

Example: An item has a margin of error of 0.50 percent and the percentage of respondents who 

agreed was 42 percent.

Interpretation: We are 95 percent confident that the true percentage of agreement lies between 

of 41.5 and 42.5 percent (= 42 percent, plus or minus 0.50 percentage points).

3

Survey Access and Files 

Data from merit system study surveys is available to the public through one or more of the 

following: MSPB’s public website, data.gov, and the National Archives and Records Administration

(NARA). MSPB plans to submit the MPS 2021 data files and supporting materials to NARA. 

(NARA, rather than MSPB, determines whether to accept those records and whether the records are 

temporary or permanent.) If accepted, those files will be accessible at www.archives.gov as part of 

record group 479 (permanent records of MSPB) under the Merit Principles Survey file series. The 

public use files that MSPB has made available are listed below.

Table: Public Use Files for the 2021 MPS

Name(s) Description

MSPB_MPS2021_Methodology.pdf Overview of the MPS 2021 methodology and 

public use data (this document).

MSPB_MPS2021_ItemList.pdf

MSPB_MPS2021_Instrument.pdf

A list of items appearing on the MPS 2021 and a 

representation of the survey for references 

purposes that contains variable names, response 

codes and outlines the survey flow and logic.

MSPB_MPS2021_Release.sav

MSPB_MPS2021_Release.sas7bdat

MSPB_MPS2021_Release.csv

Datasets in SPSS, SAS, and comma-delimited 

formats and SAS format files (numeric and 

character.

2 Certain items were asked only of respondents who provided a particular response to a preceding item. For example, 

only employees who reported observing or experiencing a particular prohibited personnel practice were asked a question 

concerning the person(s) who had committed the practice. For such items, the number of respondents is smaller, and 

the margin of error correspondingly larger.

3 This interpretation is simplified but not strictly correct; more accurately (but less intuitively), an interval calculated 

using this method will, on average, contain the true percentage 95 percent of the time.
U.S. Merit Systems Protection Board: 2021 Merit Principles Survey Methodology and Materials 5/5

MSPB_MPS2021_Dictionary_Release.pdf Data dictionary. MPS2021_Formats.sas7bdat 

MPS2021_Formats_C.sas7bdat

SAS format files

MPS2021_FactSheet_Agencies.pdf

MPS2021_FactSheet_Agencies.pdf

Fact sheets provided to participating agencies and 

employees.

To protect the confidentiality of responses and individual privacy, in accordance with the 

Privacy Act of 1974 (Public Law 93-579), responses to some survey items have been eliminated or 

recoded (combined into larger categories or groups) in the public use dataset. Those items include 

certain demographic characteristics that are sensitive or, in combination, could be used to identify an 

individual and open-ended responses that might contain personally identifying or otherwise sensitive 

information.