U.S. Merit Systems Protection Board: Merit Principles Survey 2016 Methodology and Materials 1/5

U.S. Merit Systems Protection Board

Merit Principles Survey 2016 Data and Methodology

This document describes the data (public use files and supporting documentation) and 

methodology for MSPB’s 2016 Merit Principles Survey (MPS or MPS2016).

Background

The U.S. Merit Systems Protection Board (MSPB) has the statutory responsibility to assess the 

health of Federal merit systems and the authority to conduct special studies of the Federal 

civil service [see 5 U.S.C. 1204(a)(3) and 5 U.S.C. 1204(e)(3)]. MSPB administers a periodic 

Merit Principles Survey to help carry out those studies. Those studies, including summaries 

and analyses of data from the MPS, are officially submitted to the President and Congress and 

shared with Federal policymakers and agencies.

Survey Design and Sampling

Typically, the MPS contains a combination of core items that MSPB tracks over time and 

special-purpose items developed to support a particular special study. This survey differs from 

the Federal Employee Viewpoint Survey administered by OPM in several respects, including: 

a focus on merit system principles and Governmentwide civil service issues; administration 

every few years instead of annually; and a smaller sample. Agency participation in the MPS 

was mandatory, but individual response to the survey was voluntary.

Survey Content

The MPS2016 comprised three distinct surveys (“paths”), denoted Path 1, Path 2, and Path L. 

Path 1 and Path 2 were directed to both line employees and supervisors; Path L was directed 

to supervisors only. The major topics in each survey path are listed below. All the topics, 

with one exception, were selected and developed by MSPB. One topic (ethics, on Path 2) was 

included in the MPS at the request of the U.S. Office of Government Ethics (OGE). Items for 

that topic were developed by OGE and adapted by MSPB for integration into the MPS.

Path 1

Employee Engagement

Work Environment

Career Entry and Career Interests

Fair Treatment

Preventive Measures and Complaint Resolution

Workplace Violence

Workplace Aggression/Harassment

Sexual Harassment

Respondent Demographics
U.S. Merit Systems Protection Board: Merit Principles Survey 2016 Methodology and Materials 2/5

Path 2

Your Work and Workplace

Your Supervisor

Work Behaviors

Employee Engagement

Employee Performance

Work Tasks

Emotional Labor

Career Interests

Classification and Pay Factors

Prohibited Personnel Practices

Non-Disclosure Agreements and Policies

Workplace Conflict

Respondent Demographics

Ethics

Path L (Leadership)

Respondent Demographics

Being a Supervisor

Managing Performance

Performance Appraisal

Supervisory Tasks and Skills

Managing Poor Performance

Managing Conduct

Managing Supervisors

HR Responsibilities and Services

Survey Sampling

MSPB developed a sampling strategy, using a stratified random sample, to provide a reliable 

measure of Governmentwide opinion among permanent, full-time civilian Federal employees. 

The sample was stratified by Federal agency (and agency bureau or component for selected 

agencies) and supervisory status (nonsupervisor, supervisor, or executive).

MSPB drew the survey samples from the Central Personnel Data File (CPDF), a repository of 

information on Federal employees developed and maintained by the U.S. Office of Personnel 

Management. The sample was drawn in summer 2015, and included nearly 126,000

employees from 25 Federal agencies, representing all major Departments and independent 

agencies. However, ultimately the Department of Health and Human Services could not be 

surveyed, for technical reasons, reducing the sample to 24 agencies.
U.S. Merit Systems Protection Board: Merit Principles Survey 2016 Methodology and Materials 3/5

Survey Administration

MSPB notified participating Federal agencies and selected stakeholders (e.g., major Federal 

employee unions) of the survey to request their support. The survey was administered 

electronically (over the Internet) by a support contractor in an environment that complied 

with Federal Government information security and accessibility standards.

Employees were invited to complete the survey via an email message that contained a link 

unique to the individual employee. Invitations were sent to nearly 114,000 of the 126,000 

employees initially selected (an invitation was not sent if the employee had left the agency, or 

did not have a valid email address).

Employees were informed, through the invitation and other media, that their participation 

was voluntary and that their responses were confidential. MSPB also assured participants that 

it would not disclose data that could be used to identify an individual employee.

The MPS was launched in July 2016, with periodic reminders to complete the survey, and 

closed in September 2016.

Response Acceptance, Rate, and Weighting

After the survey closed and the support contractor transmitted survey data to MSPB, MSPB 

reviewed the response records to determine whether they would be accepted. For Path L, the 

initial criterion was holding a supervisory position, based on a screening item asking the 

respondent whether he or she supervised civilian Federal employees. For all paths, the final 

criterion was completing a minimum number (approximately ten percent) of the nondemographic items asked of all respondents in the designated survey path.

MSPB then estimated a response rate, calculated by dividing the number of accepted responses 

by the estimated number of invited employees who were eligible to complete the survey.

Table: Response Rates for the MPS2016

Version Eligible Accepted Response Rate

Path 1 37,452 14,515 38.8%

Path 2 37,397 14,473 38.7%

Path L 32,654 13,058 40.0%

(TOTAL) 107,053 42,046 39.1%

To provide useful measures across lines of agency and supervisory status, MSPB oversampled 

(i.e., surveyed a higher proportion of) some groups, including supervisors, executives, and 

employees in smaller agencies. Accordingly, MSPB calculated response weights to produce 

results that are representative of Government wide employee opinions. Generally, 

tabulations and analysis should use the weighting variable (STRAT_WEIGHT) included in 

the survey datasets. Results presented in official reports are weighted unless stated otherwise.
U.S. Merit Systems Protection Board: Merit Principles Survey 2016 Methodology and Materials 4/5

Survey Representativeness

For selected dimensions, MSPB compared the proportion of respondents from a particular 

demographic group (based on survey responses) with the estimated proportion of employees 

in that group in the survey population (based on OPM’s CPDF). MSPB concluded that 

representation of major demographic groups was acceptable and that the survey results are 

reasonably representative of the population from which they were drawn.

Margin of Error

MSPB estimated margins of error and the confidence levels for each item on each of the three 

paths. At a 95 percent confidence level, the margin of error for most items was at or below 1 

percentage point. As shown below, the margin of error varies, based on the structure of the 

survey item and the number of employees responding to the item (some items were asked 

only of a subset of participants, based on their reported experiences or opinions).

Table: Estimated Margins of Error for the MPS2016

Version Average Range

Path 1 1.1% 0.30%−8.80%

Path 2 1.0% 0.50%−4.40%

Path L 1.2% 0.30%−3.10%

Survey Access and Files 

Data from merit system study surveys is available to the public through the National 

Archives and Records Administration (NARA). MSPB submitted data files from the 2016 

MPS, in a variety of machine-readable formats, to NARA in April 2017. Following 

acceptance, those files will be accessible online at www.archives.gov, as part of record group 

479 (permanent records of MSPB) under the Merit Principles Survey file series. The files 

submitted to NARA for public use are listed below.

Table: Public Use Files for the 2016 MPS

Name(s) Description

MSPB_MPS2016_MethodologyMaterials.pdf Overview of the MPS 2016 methodology and 

public use data (this document).

MSPB_MPS2016_Path1_Images.pdf

MSPB_MPS2016_Path1_Representation.pdf

Screen shots and representation of the Path 1 

survey instrument (survey items, response 

format and options, and response values). MSPB_MPS2016_Path1_Rel.sav

MSPB_MPS2016_Path1_Rel.sas7bdat

MSPB_MPS2016_Path1_Rel.csv

Datasets for Path 1, in SPSS, SAS, and 

comma-delimited formats.

MSPB_MPS2016_Path1_Release_Dictionary.pdf Data dictionary for Path 1 (response 

frequencies, values, and codes). MSPB_MPS2016_Path2_Images.pdf Representation of the Path 2 survey 
U.S. Merit Systems Protection Board: Merit Principles Survey 2016 Methodology and Materials 5/5

MSPB_MPS2016_Path2_Representation.pdf instrument (survey items, response format 

and options, and response values).

MSPB_MPS2016_Path2_Rel.sav

MSPB_MPS2016_Path2_Rel.sas7bdat

MSPB_MPS2016_Path2_Rel.csv

Datasets for Path 2, in SPSS, SAS, and 

comma-delimited formats.

MSPB_MPS2016_Path2_Release_Dictionary.pdf Data dictionary for Path 2 (response 

frequencies, values, and codes).

MSPB_MPS2016_PathL_Images.pdf

MSPB_MPS2016_PathL_Representation.pdf Representation of the Path L survey 

instrument (survey items, response format 

and options, and response values).

MSPB_MPS2016_PathL_Rel.sav

MSPB_MPS2016_PathL_Rel.sas7bdat

MSPB_MPS2016_PathL_Rel.csv

Datasets for Path L, in SPSS, SAS, and 

comma-delimited formats.

MSPB_MPS2016_PathL_Release_Dictionary.pdf Data dictionary for Path L (response 

frequencies, values, and codes).

MPS2016_Formats.sas7bcat SAS format catalog (usable with all MPS 

2016 SAS datasets)

To protect the confidentiality of responses and individual privacy, in accordance with the 

Privacy Act of 1974 (Public Law 93-579), responses to survey items have been eliminated or 

recoded (combined into larger categories or groups) in the public use dataset. Those items 

include certain demographic characteristics that are sensitive or, in combination, could be 

used to identify an individual and open-ended responses that might contain personally 

identifying or otherwise sensitive information.