Federal Workers: Results of Studies on Federal Pay Varied Due to Differing Methodologies
Published: 2012-06-22T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: A careful consideration of federal pay is an essential part of fiscal stewardship and is necessary to support the recruitment and retention of a competent, successful workforce. Recent studies comparing the compensation of federal workers to workers in other sectors have produced varying findings. To improve understanding of federal pay setting, GAO was asked to examine (1) how annual pay adjustments for the GS system are determined; (2) the extent to which the pay increases and awards available to GS employees recognize individual performance, and how the Office of Personnel Management (OPM) provides oversight of pay increases and awards; and (3) how selected studies compare federal and private pay and total compensation and the factors that may account for the different findings.
GAO reviewed legislation, OPM regulations, executive orders, and federal agency documents; analyzed OPM data; and interviewed agency officials. GAO reviewed six studies that met three criteria: issuance since 2005, original analysis, and focus on federal and private sector compensation. GAO compared and contrasted the differences between their approaches, methodologies, and data sources, and interviewed the studies authors, people with expertise in compensation issues, and agency officials responsible for the data. GAO provided drafts to agencies and study authors for review and comment and made technical changes as appropriate in response to comments received. One study author provided written comments concurring with the findings. GAO is not making any recommendations in this report.
section_title: What GAO Found
paragraphs: Annual pay adjustments for the General Schedule (GS), the pay system covering the majority of federal workers, are either determined through the process specified in the Federal Employees Pay Comparability Act of 1990 (FEPCA) or set based on percent increases authorized directly by Congress. GS employees receive an across-the-board increase (ranging from 0 to 3.8 percent since FEPCA was implemented) that has usually been made in accordance with a FEPCA formula linking increases to national private sector salary growth. This increase is the same for each employee. GS employees also receive a locality increase that varies based on their location; there were 34 pay localities in 2012. While FEPCA specifies a process designed to reduce federal-nonfederal pay gaps in each locality, in practice locality increases have usually been far less than the recommended amount, which has been over 15 percent in recent years. For 2012, when there was a freeze on annual pay adjustments, the FEPCA process had recommended a 1.1 percent across-the-board increase and an average 18.5 percent locality increase.
GS employees are eligible to receive three types of pay increases and monetary awards that are linked to individual performance appraisals: within-grade increases, ratings-based cash awards, and quality step increases. Within-grade increases are the least strongly linked to performance, ratings-based cash awards are more strongly linked to performance depending on the rating system the agency uses, and quality step increases are also more strongly linked to performance.
Findings of selected pay and total compensation (pay and benefit) comparison studies varied due to different approaches, methods, and data. Regarding their pay analysis, the studies conclusions varied on which sector had the higher pay and the size of pay disparities. However, the overall pay disparity number does not tell the whole story; each of the studies that examined whether differences in pay varied among categories of workers, such as highly or less educated workers or workers in different occupations, found such variations. Three approaches were used to compare pay:
human capital approach (3 studies)compares pay for individuals with various personal attributes (e.g., education, experience) and other attributes (e.g., occupation, firm size);
job-to-job approach (2 studies)compares pay for similar jobs of various types based on job-related attributes such as occupation, does not take into account the personal attributes of the workers currently filling them; and
trend analysis approach (1 study)illustrates broad trends in pay over time without controlling for attributes of the workers or jobs.
When looking within and across the studies, it is important to understand the studies differences in approach, methods, and data because they impact how the studies can be interpreted. The differences among the selected studies are such that comparing their results to help inform pay decisions is potentially problematic. Given the different approaches of the selected studies, their findings should not be taken in isolation as the answer to how federal pay and total compensation compares with other sectors.

section_title: Letter
paragraphs: Skilled federal workers are integral to the successful operation of every  government function. They are law enforcement officers, engineers,  program managers, scientists, clerks, and all the other workers who carry  out the federal government’s business. In 2011, there were approximately  2.2 million federal civilian workers comprising about 1.6 percent of the  U.S. workforce, and civilian personnel costs were about $220 billion,  comprising about 6 percent of the total federal budget. A careful  consideration of federal workers’ pay is an essential part of fiscal and  operational stewardship and is necessary to support the recruitment and  retention of a competent, successful workforce. It is the policy of  Congress that pay for federal workers in the General Schedule (GS), the  pay system covering the majority of federal workers,for comparable nonfederal workers.     be in line with pay  The composition of the federal workforce has changed over the past 30  years, with the need for clerical and blue collar roles diminishing and  professional, administrative, and technical roles increasing. As a result,  today’s federal jobs require more advanced skills at higher grade levels  than federal jobs 30 years ago. Additionally, federal jobs, on average,  require more advanced skills and degrees than private sector jobs. This is  because a higher proportion of federal jobs than nonfederal are in skilled  occupations such as science, engineering, and program management,  while a lower proportion of federal jobs than nonfederal are in  occupations such as manufacturing, construction, and service work. The  result is that the federal workforce is on average more highly educated  than the private sector workforce.
Given the changes in the federal workforce over the last 30 years, there  has been growing interest in reexamining the federal pay system—how  pay is determined and how comparisons with other sectors are made.  Specifically, we have reported on the importance of considering the skills,  knowledge, and performance of employees as well as the local labor  market in making pay decisions. The President’s Pay Agent, the entity  responsible for recommending federal locality pay adjustments to the  President, has recommended that the underlying model and methodology  for estimating pay gaps be reexamined to ensure that private sector and  federal sector pay comparisons are as accurate as possible.
The Subcommittee on Federal Workforce, U.S. Postal Service and Labor  Policy held a hearing in March 2011 on the issue of comparability in pay  and benefits between the federal and nonfederal workforces. The  research presented at the hearing showed varying findings on the gap in  pay between federal and private sector workers as estimated by outside  organizations as well as the gap between federal and nonfederal workers  as estimated by the President’s Pay Agent. The outside organizations’  studies also varied in their findings on the gap in total compensation (pay  plus benefits). The hearing raised questions about how the  methodologies used by the various authors differed and how these  differences affected their overall findings.
Accordingly, at your request, this report examines (1) how annual pay  adjustments for the GS system are determined; (2) the extent to which  the pay increases and awards available to GS employees recognize  individual performance, and how the Office of Personnel Management  (OPM) provides oversight of pay increases and awards; and (3) how  selected studies compare federal and private sector pay and total  compensation and the factors that may account for the different findings.
To examine how GS annual across-the-board and locality pay  adjustments are determined, we reviewed legislation, OPM regulations,  executive orders, Presidents’ alternative pay plans, President’s Pay Agent  Reports, Federal Salary Council recommendations, OPM and Bureau of  Labor Statistics (BLS) documents and reports, and reports by the  Congressional Budget Office (CBO) and Congressional Research  Service. We also examined how the methodology for determining locality  pay has changed since the start of locality pay to the present. We  interviewed selected members of the Federal Salary Council and its  working group; the Council is to be made up of six representatives of  federal employee groups and three experts in labor relations, and makes  annual recommendations to the President’s Pay Agent. We interviewed  BLS officials, OPM officials who are knowledgeable about federal pay  policy and serve as staff to the President’s Pay Agent, and people with  expertise in compensation issues including former federal officials  experienced with pay and benefits issues.
To determine the extent to which pay increases and awards recognize  individual performance, we analyzed legislation and OPM regulations on  pay increases and awards available to employees in the GS pay system  and identified those pay increases and awards that are determined in part  by an individual’s performance rating as measured by the agency’s  performance appraisal system. We analyzed data for employees in the  GS pay plan in the aggregate on the number, percentage, and dollar  amount of selected pay increases and awards; the amount of these  increases and awards as a portion of GS payroll (total adjusted basic pay  for all employees in the GS pay plan); and the distribution of these  increases and awards by rating pattern and rating levels from OPM’s  Central Personnel Data File (CPDF) for fiscal year 2011. To help  determine the reliability and accuracy of the CPDF data elements used,  we checked the data for reasonableness and the presence of any obvious  or potential errors in accuracy and completeness; reviewed past GAO  analyses of the reliability of CPDF data; and interviewed OPM officials  knowledgeable about the data. We believe the CPDF is sufficiently  reliable for the purpose of this report. To describe how OPM provides  oversight of pay increases and awards, we collected and analyzed OPM  guidance to agencies including regulations, memoranda, reports, fact  sheets, and frequently asked questions. We also interviewed OPM  officials responsible for federal pay policies to discuss the implementation  of the guidance and monitoring of agencies’ use of increases and awards  through reports and other means; and interviewed OPM officials  responsible for conducting human capital management evaluations at  agencies on pay increases and awards and the overall GS pay system,  among other things.
To review selected studies that compare federal and private sector pay  and total compensation and describe factors that help account for the  different study findings, we conducted a detailed literature review of  academic journals, agency and organization publications, and other  sources and applied three criteria to the results, selecting five studies  that: (1) were published/issued since 2005; (2) include original analysis;  and (3) have the explicit and primary purpose of comparing federal and  private sector pay and total compensation. After our literature review was  completed, CBO issued a study comparing federal and private sector  compensation. We included this study in our review because it met our  criteria. That brought the total number of selected studies up to six. We  did not examine the reliability or the appropriateness of the approaches,  methods, and data used by the studies, and we did not exclude any study  on the basis of methodological quality. We reviewed the studies,  summarized each study’s methodologies and key findings, and confirmed  the accuracy of our summaries with the authors. We compared and  contrasted the differences between the approaches, methodologies, and  data sources of the selected studies. We interviewed the selected study  authors to obtain their views on the various methodologies and data  sources available, why they chose the ones they used, and their  conclusions based on their work. We interviewed a number of individuals  chosen for their expertise in compensation issues to obtain their views on  the data sources for analyzing compensation and to provide a general  context for the issues involved in comparing federal and private or  nonfederal pay and total compensation. The findings regarding the  selected studies are not based on input from these individuals. We  interviewed officials from the Bureau of Economic Analysis (BEA), BLS,  and the U.S. Census Bureau to discuss how these agencies’ data are  used to measure federal and private or nonfederal pay, compensation, or  benefits, and limitations of their data or surveys. We also interviewed  officials from OPM involved in federal pay policies. (See app. I for a more  detailed discussion of our objectives, scope, and methodology.)
We conducted this performance audit from July 2011 to June 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The GS pay system covered 69 percent of federal civilian workers in  2011, with compensation costing about $147 billion (about 67 percent of  total federal civilian compensation of about $220 billion). The GS  workforce is divided into 15 pay grades, with 10 rates of pay (referred to  as steps) within each grade. Agencies use a uniform set of classification  standards to determine grade levels for their positions organized within  five occupational categories—Professional, Administrative, Technical,  Clerical, and Other White-Collar (PATCO). The GS system of  classification was established by the Classification Act of 1949 in  response to calls for a modernized system to ensure equity in pay  setting. Until the late 1960s, general pay adjustments for federal  employees were made through acts of Congress. The Federal Pay  Comparability Act of 1970 permanently authorized the President to adjust  GS pay rates annually, and established a system for recommending  adjustments with the goal of increasing federal pay to be comparable with  the private sector; however, we previously found that the gap between  average federal and private sector salaries for similar jobs continued after  implementation of the act because the recommended adjustments were  not always made.
The Federal Employees Pay Comparability Act of 1990 (FEPCA) created  annual locality-based pay adjustments for GS employees to reduce  reported gaps between federal and nonfederal pay in metropolitan  areas.  In addition, FEPCA maintained an annual across-the-board pay  adjustment that is the same for each employee to keep the GS base pay  schedule in line with salary growth in the general labor market, similar to  what had already existed under the 1970 act.
Before FEPCA, federal employees doing the same job at the same level  anywhere in the country were paid the same amount. However, there  was a growing concern that it was difficult to recruit and retain skilled  federal employees in areas with higher nonfederal wages. We concluded  that locality-based pay adjustments were necessary. FEPCA  established locality pay, and the President’s Pay Agent designated pay  localities based on Office of Management and Budget (OMB)  Metropolitan Statistical Areas. FEPCA’s goal was to reduce the gap  between federal and nonfederal pay in each locality, as measured by BLS  data and reported by the President’s Pay Agent, to 5 percent over the  course of 9 years. This goal was not met, but locality pay increases have  been provided every year since locality pay was implemented in 1994,  except during the pay freeze in 2011 and 2012. According to OPM,  locality pay is now a broadly accepted practice in federal pay  administration. See app. II for more information on the implementation of  locality pay.
Figure 1 illustrates the extent to which locality pay has been implemented  for a representative employee. The annual pay (base plus locality) in  2012 for an employee at GS-11 (approximately the midpoint grade level),  step 1 is shown for selected pay localities. Examples of positions that a  GS-11 employee might hold are Administrative Officer, Scientist,  Paralegal Specialist, Accountant, Engineer, Medical Records  Administrator, Nurse Specialist, and Information Technology Specialist.  There were 34 pay localities in the United States in 2012, composed of  the states of Alaska and Hawaii, 31 metropolitan areas, and a residual  locality called “Rest of U.S.” that includes all other areas in the United  States and its territories and possessions. Rest of U.S. was the lowest- paying locality in 2012, with a GS-11, step 1 earning $57,408, and San  Francisco was the highest, with a GS-11, step 1 earning $67,963. We  selected additional localities with various pay rates and population sizes  and from various regions to create figure 1.
section_title: GS Pay Adjustments Can Be Determined through FEPCA Process or Set Directly by Congress
subsections: section_title: Across-the-Board Adjustments Are Usually Based on Private Sector Salary Growth
paragraphs: Across-the-board adjustments are designed to keep the GS base pay  schedule in line with salary growth in the general labor market. FEPCA  specifies that unless the President provides for alternative pay  adjustments, across-the-board pay adjustments are to be determined  using a simple formula: Pay rates are to be increased by the 12-month  percentage increase in the wage and salary component of the  Employment Cost Index (ECI) for private sector workers, minus one-half  of one percentage point. For example, the ECI reference period for the  January 2013 increase is the 12-month period ending September 2011.  The ECI shows that during that period, pay for private sector workers rose   Therefore, the across-the-board increase for 2013 would  by 1.7 percent.be 1.2 percent. The ECI, an index compiled by the BLS and published  quarterly, measures percentage changes in wages and salaries for  private sector employees.
As specified in FEPCA, the President may decide to either provide  across-the-board pay adjustments based on this calculation, or provide  alternative pay adjustments based on national emergency or serious  economic conditions affecting the general welfare. Congress may legislate an increase that is different from the formula  result or the President’s alternative plan; this is not part of the process  specified by FEPCA. The FEPCA formula increase has gone into effect in   An  12 of the 19 years since 1994; the largest increase was 3.8 percent.
In evaluating economic conditions, the President is to consider a range of economic  measures, including (but not limited to) Gross National Product, the unemployment rate,  the budget deficit, and the Consumer Price Index. amount lower than the formula amount went into effect in the other 7  years due to President’s alternative pay plans and laws passed by  Congress. The smallest increase was 0 percent, during the freeze on  annual pay adjustments in 2011 and 2012.
section_title: Locality Adjustments Are Usually Set by Congress
paragraphs: Locality adjustments are designed to reduce the gap between federal and  nonfederal pay in each locality to no more than 5 percent based on  surveys to be conducted by BLS. FEPCA specifies that locality pay  adjustments are to be recommended by a Pay Agent designated by the  President, which is to consider the views of employee organizations:
The President’s Pay Agent recommends annual comparability  payment amounts, establishes and modifies pay localities as it  considers appropriate, and submits an annual report to the President  on these items. The Secretary of Labor and the Directors of OMB and  OPM serve as the Pay Agent. In making its recommendations, the  President’s Pay Agent considers the views and recommendations of a  Federal Salary Council and other employee organizations.
The Federal Salary Council makes annual recommendations to the  President’s Pay Agent on locality pay adjustments, including the  establishment or modification of pay localities, the coverage of salary  surveys used to set locality pay, the process for making pay  comparisons, and the level of comparability payments that should be  made. The Council is to be comprised of three experts in labor  relations and pay policy and six representatives of employee  organizations representing large numbers of GS employees.
To recommend locality pay adjustments, the President’s Pay Agent  compares the annual GS base pay rates of federal workers in each area  to the annual pay rates of nonfederal workers in the same area for the  same levels and types of work. The sidebar provides details on this  process. The target locality pay is the amount that reduces these  differences to 5 percent.
The surveys and models used for making these pay comparisons have  changed somewhat between the passage of FEPCA in 1990 and the  2011 President’s Pay Agent Report (which recommends pay adjustments  for 2013 and is the most current report available). Some changes were  initiated by BLS, and some changes were made in response to concerns  expressed by the Federal Salary Council or President’s Pay Agent. For  example, BLS changed the survey used to measure nonfederal pay in  1996; the Federal Salary Council and President’s Pay Agent expressed  concerns, and BLS worked together with OPM and OMB to improve the  suitability of the new survey for recommending locality payments.  Improvements were phased in from 2002 to 2011. Changes are  summarized in app. II.
As specified in FEPCA and similar to the process for across-the-board  adjustments, the President may decide to either provide locality pay  adjustments based on the Pay Agent’s recommendation, or provide for  alternative pay adjustments based on national emergency or serious  economic conditions affecting the general welfare. Additionally, Congress  may legislate an average percent increase that is different from the Pay  Agent’s recommendation or the President’s alternative plan; this is not  part of the process specified by FEPCA.
For 1994, the first year that locality payments were made, FEPCA  specified that the locality increase should be not less than one fifth of the  amount needed to reduce the pay disparity to 5 percent. This amount,  providing a 3.95 percent average locality pay rate for the average GS  employee as recommended by the Pay Agent, went into effect. In  subsequent years through 2012, the effective increase has usually been  far less than the one recommended by the Pay Agent, either due to a  President’s alternative pay adjustment or to a law passed by Congress.
Nonetheless, some locality pay increase has been provided every year  since locality pay was implemented in 1994 (except during the pay freeze  in 2011 and 2012), and reported disparities between federal and  nonfederal pay by locality have been reduced. The President’s Pay Agent  reported that pay disparities were lower in 2011 than in 1994 in 16 of the  21 pay localities that existed in both of those years. Federal Salary  Council members and OPM officials we spoke with said that FEPCA was  successful in its goal of improving federal pay setting for large  metropolitan areas by more closely aligning pay to local labor markets.
Figure 2 summarizes pay adjustments during the past 6 years, illustrating  the differences between the President’s Pay Agent recommendations and  the final effective amounts. These differences were driven primarily by  locality pay, since the across-the-board adjustments required under the  FEPCA formula were smaller and were provided in some years, while the  recommended locality adjustments were larger and were not provided.  For example, for 2007, the President’s Pay Agent recommended a 1.7  percent across-the-board increase to comply with the formula in FEPCA,  and a 7 percent average locality increase based on BLS salary survey  data. The President provided for the 1.7 percent across-the-board  increase but limited the average locality increase to the alternative  amount of 0.5 percent.
As another example, for 2012, the FEPCA process specified a 1.1  percent across-the-board increase and an average 18.5 percent locality  increase, but annual pay adjustments were frozen instead.Agent had reported that in 2010 (the reference year for setting 2012 pay),  taking both across-the-board and locality pay into account, the average  federal-nonfederal pay gap was 24 percent. The approximately 20  percent overall average increase recommended by the Pay Agent for   The Pay  2012 would have lowered the pay disparity to FEPCA’s target of 5  percent.
section_title: Pay Increases and Awards for GS Employees Vary in their Links to Individual Performance
paragraphs: The pay increases and awards available to GS employees are designed  to recognize individual performance to varying degrees. Across-the-board  and locality pay increases, which are given to all covered employees  nearly every year, are not linked to performance at all. Awards such as  suggestion/invention awards and superior accomplishment awards are  designed to recognize performance without being linked specifically to  performance ratings.
Three pay increases and monetary awards available to GS employees  are linked to performance ratings as determined by agencies’  performance appraisal systems:
Within-grade increases are periodic increases in a permanent  employee’s rate of basic pay from one step of a grade to the next  higher step within the grade.  Ratings-based cash awards are lump sum cash payments that are  designed to recognize performance.
Quality step increases are faster-than-normal step increases that  are designed to recognize excellence in performance.
Agencies are permitted to provide quality step increases to eligible employees under 5  U.S.C. § 5336.
Factors that are to affect GS employee eligibility for these pay increases and awards are  specified in legislation and regulations and clarified in OPM guidance. two summary rating levels to a system with five summary rating levels.All the systems used by the agencies include level 3, “fully successful,”  which is the pass level for a pass/fail system.
In practice, based on our analysis of CPDF data from fiscal year 2011,  the degree to which individual performance drove receipt of these pay  increases and awards for employees in the GS pay plan varied. Of the  three pay increases and awards we analyzed, within-grade increases  were the least strongly linked to performance. Ratings-based cash  awards were more strongly linked to performance depending on the rating  system the agency used, and quality step increases were also more  strongly linked to performance.
Within-grade increases were the least strongly linked to performance of  the three pay increases and awards we analyzed, in accordance with  their design. As noted in table 1, agencies are required to provide within- grade increases to employees whose performance is at least “fully  successful” and who have finished their waiting period. Over 99 percent  of employees in the GS pay plan received performance ratings at or  above “fully successful” in fiscal year 2011. Thirty-nine percent received  within-grade increases, comprising nearly all the employees who  completed their waiting period.
Ratings-based cash awards were more strongly linked than within- grade increases to performance. All GS pay plan employees may receive  ratings-based cash awards every year (unlike within-grade increases), so  frequency limits are not a primary determinant of who receives them. In  fiscal year 2011, the degree of linkage of awards with performance  ratings varied by the type of appraisal system used by the agency.
In fiscal year 2011, 81 percent of employees in the GS pay plan were  covered by 5-level rating systems and other systems that allowed for  distinctions between “fully successful” and higher levels of performance.Ratings-based cash awards for these employees were given at higher  rates to employees with better performance. For example, for the 5-level  system, which covered 63 percent of GS employees, awards were given  to 65 percent of employees with “outstanding” ratings, 58 percent of  employees with ratings “between outstanding and fully successful,” and  24 percent of employees with “fully successful” ratings. Along with the  performance rating received, agency criteria were used to determine who  received awards. As noted in table 1, an agency should identify any other  criteria to be considered when making award recommendations and  decisions, including any other awards or personnel actions that should be  taken into consideration such as time off, a quality step increase, or a  recent promotion.
In accordance with OPM regulations, employees with higher ratings are to  receive larger ratings-based awards, and award patterns reflected this  distinction in 2011, as shown in figure 3. Employees who received  “outstanding” ratings within the 5-level system received the largest  awards.
In fiscal year 2011, about 19 percent of employees in the GS pay plan  were covered by a pass/fail rating system or another system that did not  allow for distinctions in performance above the “fully successful” level.  Over 99 percent of employees in these systems received a “fully  successful” rating in fiscal year 2011, while only 31 percent received a  ratings-based award, meaning that most decisions not to provide awards  were made based on other criteria than ratings. Performance ratings and  agency criteria, including performance-related criteria, were used to  determine who received awards.
Quality step increases were also more strongly linked to performance  than within-grade increases. As shown in table 1, GS employees must  perform at their agency’s highest possible level to be eligible to receive a  quality step increase. About 49 percent of employees received the  highest possible rating their agency’s system allowed in fiscal year  2011. Of those employees, about 7 percent received a quality step  increase. Unlike within-grade increases, the waiting period for quality step  increases is 1 year for all employees, eliminating the waiting period as a  primary determinant for receiving quality step increases; rather, decisions  were made based on performance rating and agency criteria, including  performance-related criteria.
Figure 4 illustrates the percentage of employees receiving each type of  increase and award, the average amounts of the increases and awards in  dollars and as a percent of the recipient’s pay, and the cost to the  government of ratings-based pay increases and awards for GS  employees for fiscal year 2011.
OPM’s role with respect to awards and increases includes providing policy direction to agencies, including regulations, reporting on agencies’ use of awards and increases, and evaluating agencies’ linkage of awards and increases with results.
Agencies, in turn, must ensure they have met statutory and regulatory  requirements and may develop agency-specific criteria for providing  quality step increases and cash awards. According to OPM officials,  awards regulations are highly decentralized because the statutes provide  agency heads, not OPM, with the authority to grant awards.agency heads may grant quality step increases within the limits of  available appropriations and regulatory requirements.
Policy direction. To help agencies understand how to administer pay  increases and awards, OPM issues regulations and supporting  memoranda and posts fact sheets, frequently asked questions, and other  resource documents on its website. Topics have included approaches to  calculating ratings-based cash awards, tax issues for awards, how the  timing of quality step increases affects within-grade increases, and recent  limitations on awards given budgetary constraints. According to OPM  officials, OPM responds to agency questions about guidance as needed.
Reporting. OPM provides agencies with an annual Federal Award  Statistics report on cash awards, time-off awards, quality step increases,  and other awards received by GS and other employees.OPM officials, OPM uses the report to show trends and compare usage of  awards between agencies and across the government. OPM also uses  the report data to help inform its decisions about awards policy and  monitor agency compliance with the policy, such as limitations on awards  usage.
Evaluation. OPM evaluates selected agencies’ human capital  management systems as part of its broader strategy for maintaining  human capital accountability. As part of these evaluations, OPM  determines whether an agency’s human capital system provides and  clearly communicates linkages between employee performance  expectations, performance recognition through increases and awards,  and the agency’s mission. OPM also reviews a sample of case files to  check that the awards granted meet the requirements of the law and   and assists agencies in leading their own evaluations.  regulations,OPM officials said that they have identified the following issues in regard  to pay increases and awards:
Some agencies tried to circumvent limitations on award amounts by  issuing several incremental awards within a short time period.
Some agencies granted quality step increases to compensate for low  award budgets.
Some agencies’ human capital management systems did not link  individual performance expectations and recognition through pay  increases and awards to the accomplishment of specific mission- related goals or milestones.
When OPM determines that an agency violated the law or regulations,  such as circumventing award limitations by issuing several incremental  awards within short periods of time, it requires the agency to take  corrective action and respond to OPM with evidence of how it addressed  or plans to address the violation within 60 days. For example, according  to an OPM official, corrective action may result in the agency recovering  the award from the recipient and correcting the documentation for the  award.
When OPM observes an issue with an agency’s award implementation  that does not violate regulations, OPM may recommend to the agency  improvements that could be made. For example, when OPM determines  that an agency has granted quality step increases to compensate for a  low award budget, it recommends that the agency review its policies for  granting pay increases and awards to ensure the policies comply with the  intent of the laws and regulations. According to an OPM official, OPM  requires an agency to respond to the recommendations made, but the  agency is not required to take action on addressing the issue.
section_title: Findings of Selected Pay and Total Compensation Comparison Studies Varied Due to Different Approaches, Methods, and Data
subsections: section_title: Selected Studies Differed in Their Conclusions and Basic Approaches to Analyzing Pay
paragraphs: The different study designs used by the authors of six studies resulted in  varying conclusions on how federal pay differed from private sector or  nonfederal pay. As shown in table 2, conclusions varied on which sector  had the higher pay (which does not include benefits) and the size of pay  disparities. All but one of the studies estimated the difference in pay after  controlling for some personal and job-related attributes that can affect pay  levels such as education and locality. This remaining difference is  sometimes called the unexplained difference because it persists after  controlling for attributes that can affect pay.
However, the overall pay disparity number does not tell the whole story;  each of the studies that examined whether differences in pay varied  among categories of workers, found such variations (see table 2). For  example, CBO found that federal workers with graduate and professional  degrees were paid less in comparison to the private sector, while workers  without college degrees were paid more.
Importantly, all of the study authors acknowledged that the data they used  in their analyses had limitations which could affect their findings. Any  comparison of the studies needs to take these data limitations into  account. For example, studies that used the Census Bureau’s Current  Population Survey (CPS) were unable to directly control for years of work  experience given this measure is not available in the CPS; some of the  authors said that work experience is an attribute that affects how much a  person is paid. Also, it was acknowledged that many federal jobs may not  have equivalents in the private sector.
The studies used three basic approaches to analyze differences in pay,  as shown in table 3. Each author chose the approach they thought would  best describe differences in pay. The Pay Agent is mandated by law to  compare the rates of pay under the GS system with the rates of pay  generally paid to nonfederal workers for the same levels of work within  each pay locality, as determined on the basis of appropriate BLS  surveys.
The studies’ differing conclusions on the overall pay disparity between  federal and private or nonfederal workers were affected by their basic  approaches—human capital, job-to-job, and trend analysis. Across these  approaches, data sources and types of attributes controlled for differed.  Within each approach, conclusions differed due to studies’ specific  methodologies—specific attributes controlled for and statistical methods  used.
Basic approaches: Across the three basic approaches, the  differences in the data sources and types of attributes controlled for  (personal or job-related) contributed to the differing conclusions.
Data sources: The type of approach the study authors chose  influenced the data sources they used. Studies using the human  capital approach used data from the CPS to determine the pay  for federal and private sector workers. Studies using the job-to-job  approach used data from BLS’s National Compensation Survey  (NCS) to determine pay for nonfederal (Pay Agent) and private  sector (POGO) workers and data from OPM to determine pay for  federal workers. For the trend analysis approach, Edwards used  data from BEA’s national income and product accounts (NIPA)tables to determine pay for federal and private sector workers.  Types of attributes difference in pay, accounting for the fact that employees earn  different amounts based on education, locality, and other personal  and job-related attributes. However, studies using different basic  approaches controlled for different types of attributes. Studies  using the human capital approach controlled for attributes related  to both the individual worker and the job the person occupied.  Studies using the job-to-job approach controlled for only job- related attributes. The trend analysis approach did not control for  attributes.  : Most of the studies estimated the unexplained
Specific methodologies: Within the human capital and job-to-job  approaches, the studies controlled for different specific attributes and  used different statistical methods, as shown in table 4. These  differences led to differing conclusions.
The study authors and people with expertise in compensation issues that  we interviewed differed in their views on which type of approach is most  informative in comparing pay of workers across sectors. According to  study authors who used the human capital approach, this approach is the  standard method in the field of economics to compare workers’ pay  across sectors. The overall unexplained difference between federal and  private sector pay is a way to measure the extent to which the federal  government may be paying more or less for the services it receives from  its workers relative to what those workers could earn in the private sector.
These findings could help inform policy decisions regarding the pay of  federal workers. However, study authors (including those who used the  human capital model) and people with expertise in compensation issues  did not suggest that the human capital approach be used for setting an  individual’s rate of pay. They explained that some of the personal  attributes that are associated with analyzing differences in pay using a  human capital approach are demographic in nature (e.g., race, gender)  and not work-related. OPM officials added that they are not aware of any  employers that use the human capital approach to set pay for their  employees.
The President’s Pay Agent and POGO used the job to job approach in  their analyses of pay differences, not the human capital approach.  According to OPM officials who serve as staff to the President’s Pay  Agent, employees with the same human capital characteristics can  choose to work in markedly different jobs with large variations in pay.  POGO and some people with expertise in compensation issues said that  the fundamental concept of setting pay based on the job, without taking  account of the personal characteristics of individuals in similar jobs, is the  most appropriate approach. They said it is not appropriate to pay  individuals differently according to personal attributes, such as education  or job experience, if they hold the same job. However, others said that  matching individuals by occupation and level of work involved some  subjective judgment and lacks transparency, which makes it difficult for  other interested parties to understand the analysis.
The President’s Pay Agent has stated that it has serious concerns about  a process that requires a single percentage adjustment in the pay of all  white-collar civilian federal employees in each locality pay area without  regard to the differing labor markets for major occupational groups, and it  believes that reforms of the GS system should be considered.  Specifically, the Pay Agent stated that the underlying model and  methodology for estimating pay gaps should be reexamined to ensure  that private sector and federal sector pay comparisons are as accurate as  possible.
section_title: Selected Studies Used Different Methods to Include Benefits in Estimates of Total Compensation
paragraphs: Five studies found a wide range of disparities in benefits as part of total  compensation (pay and benefits) between the federal and private sector  workforces, as shown in table 5. (The President’s Pay Agent Report did  not include an analysis of benefits as part of total compensation.) Most  studies presented the disparity in terms of total compensation, not just the  benefits portion, because the levels of some benefits—for example, most  retirement benefits—are a function of pay rates, years of service, and  type of plan. The five studies included benefit comparisons in an effort to  capture the cost of benefits to the federal government.
As with their analyses of pay, the study authors acknowledged that  limitations in data affected their analyses of total compensation and could  affect their findings, as discussed below table 5. These limitations need to  be taken into account when comparing the studies. Additionally, the  studies do not all analyze the same group of federal workers; for  example, POGO analyzed workers in 35 selected occupations.
The wide range of estimates between the studies is due to the different  data sources, types of benefits analyzed, and specific methodologies  used.
Data sources. Study authors agreed that available data were less  adequate for comparing federal to private sector benefits than pay.  Benefits data at the individual level are not available from a single source  so the studies used multiple sources. This makes it challenging to  compare across the sectors. For example, some data sources, such as  the CPS, ask workers questions about their pay, but do not ask about the  cost of their benefits because workers generally do not know the  monetary value of their benefits. As a result, study authors used data  sources such as the NCS that ask employers questions about the cost of  their workers’ benefits. Additionally, different studies drew from different  data sources, contributing to the range of different results.
Biggs/Richwine used BLS’s NCS data, specifically the Employer  Costs for Employee Compensation portion for private sector worker  data. For federal workers, they used the OPM/OMB civilian position  full fringe benefit cost factor—a percent factor describing the cost of  benefits relative to salaries. To capture benefits the OPM/OMB  source did not cover, Biggs/Richwine used OPM’s Federal Civilian  Work Force Statistics: Work Years and Personnel Costs Report to  determine paid leave, and the Annual Social and Economic  Supplement of the CPS to estimate job security.
POGO used NCS data on private sector workers. For federal workers,  it used the OPM/OMB civilian position full fringe benefit cost factor as  Biggs/Richwine did.
CBO used more detailed data from the NCS and OPM for private  sector and federal workers, respectively. These data were not  publically available.
Edwards and Sherk both used BEA’s NIPA data. According to BEA,  this data source includes annual intra-governmental payments to  amortize the accumulated unfunded liability of the Civil Service  Retirement and Disability trust fund. This reduces the data’s accuracy  for measuring compensation for current workers, according to the  study authors that used the data. Sherk used OPM data to correct  for this issue of federal retiree benefits.
Benefits analyzed. The studies included different types of benefits in  their analyses, contributing to the range of different results. In addition,  the study authors made assumptions in determining the value of benefits.
All of the studies included health insurance, retirement benefits, and  the employer portion of mandatory government benefits such as  Social Security.
Biggs/Richwine, CBO, and POGO (for private sector workers only)  included paid leave, while Sherk did not.
Biggs/Richwine included job security, asserting that federal workers  are less likely to experience periods of unemployment than private  sector workers and so can expect a higher income for a given salary  over the course of a year.
All of the studies relied on estimates of future benefits, which requires  assumptions to be made about the present value of the benefit, which  may introduce uncertainty in the estimates. According to BEA,  estimates of the present value of future benefits are inherently  dependent on assumptions about the discount rate, participant  separation rates, retirement ages, mortality, and even future pay  increases and future inflation. As a result, the amount of money that  has to be set aside today to pay for tomorrow’s benefits could be  different.
Specific methodologies. It was not possible to estimate the cost of  benefits directly while controlling for differences between the federal and  private workforces, so most authors used various indirect methodologies.
The indirectness increased uncertainty, and the wide range of  methodologies led to different results.
CBO developed a model to estimate the relationship between federal  workers’ pay and the cost of the benefits they received, and an  analogous model for private sector workers. CBO imputed employee  benefits using those models, then compared benefits for federal and  private sector workers controlling for personal and job-related  attributes, just as they did for pay, to estimate the portion of the  difference in total compensation unexplained by attributes. CBO was  the only study to use a model that allowed for varying benefits-to-pay  ratios for different pay levels.
Sherk calculated the difference in average total compensation for  federal and private sector workers. He used his estimates of the  unexplained difference in pay from the human capital model and  applied this to the difference in average total compensation. He  assumed the unexplained difference in total compensation was the  same as the unexplained difference in pay.
Biggs/Richwine used different benefits-to-pay ratios for federal  workers and private sector workers. They applied these ratios to the  unexplained differences in pay from their human capital model to  obtain the unexplained difference in total compensation.  Biggs/Richwine assumed the unexplained difference in total  compensation was the same as the unexplained difference in pay.
POGO used different benefits-to-pay ratios for federal workers and  private sector workers. It applied these ratios to differences in pay for  the selected occupations in each sector to obtain the percent  difference in total compensation for these occupations.
Edwards calculated the difference in average total compensation for  federal and private sector workers. He did not control for attributes  between the workers.
section_title: Concluding Observations
paragraphs: The findings of the selected studies comparing federal and private sector  pay and total compensation varied because they used different  approaches, methods, and data. When looking within and across the  studies, it is important to understand these differences because they  impact how the studies can be interpreted. On the one hand, the human  capital approach compares pay for individuals taking into account  personal attributes such as education and job experience. Study authors  who used this approach said that analyzing federal and private sector  workers’ pay was a way to measure the extent to which the federal  government may be overpaying or underpaying its employees compared  to what they could earn in the private sector. On the other hand, the job- to-job approach compares pay for similar jobs on such job-related  attributes as occupation and level of work rather than personal attributes.  The President’s Pay Agent, which used this approach, examined how pay  for GS and nonfederal jobs compared for the same occupations and  levels of work within the same locality pay areas with the goal of reducing  existing pay disparities. Simply put, the differences among the selected  studies are such that comparing their results to help inform pay decisions  is potentially problematic. Given the different approaches of the selected  studies, their findings should not be taken in isolation as the answer to  how federal pay and total compensation compares with other sectors.
As stated earlier, we have reported on the importance of considering the  skills, knowledge, and performance of federal employees as well as the  local labor market in making pay decisions. The President’s Pay Agent  has recommended that the underlying model and methodology for  estimating the pay gaps be reexamined to ensure that private sector and  federal sector pay comparisons are as accurate as possible. As a step in  this direction, the administration recommended in its September 2011  deficit reduction proposal that Congress establish a Commission on  Federal Public Service Reform composed of members of Congress,  representatives from the President’s Labor-Management Council,  members of the private sector, and academic experts to identify  fundamental reforms for the federal government’s human capital systems  including compensation reform. As of June 2012, such a commission has  not been established.
section_title: Agency and Third- Party Comments
paragraphs: We provided a draft of this report to the Secretary of Commerce (for  Census), the Commissioner of BLS, and the Directors of BEA and OPM  for their review and comment. The Census Bureau had a technical  comment on the draft report, which we incorporated into the final report.  BEA and BLS had no comments on the draft report. OPM provided  technical comments on the draft report, which we incorporated as  appropriate.
We provided applicable sections of the draft report to the authors of the  selected compensation comparison studies for their review and comment.  Biggs/Richwine and CBO provided technical comments, which we  incorporated as appropriate. Edwards and Sherk did not have any  comments on the draft section. POGO provided written comments (see  app. IV). In its letter, POGO stated it concurred with our draft finding that  many factors hinder public and private sector pay comparisons, such as a  lack of detailed data. POGO also suggested that we analyze OPM  federal-nonfederal salary comparisons as part of our final report. We  believe this information is already addressed in other sections of the  report, which POGO did not receive for comment. In these sections, we  discuss in detail how annual pay adjustments are determined including  the President’s Pay Agent process, which uses the comparisons referred  to by POGO.
As we agreed with your office, unless you publicly announce the contents  of this report earlier, we plan no further distribution of it until 30 days from  the date of this letter. At that time, we will send copies of this report to the  appropriate congressional committees; the Secretary of Commerce; the  Commissioner of BLS; the Directors of BEA, Census, and OPM; and  other interested parties. In addition, the report will be available at no  charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-6806 or goldenkoffr@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in app. V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines (1) how annual pay adjustments for the General  Schedule (GS) system are determined; (2) the extent to which the pay  increases and awards available to GS employees recognize individual  performance, and how the Office of Personnel Management (OPM)  provides oversight of pay increases and awards; and (3) how selected  studies compare federal and private sector pay and total compensation  and the factors that may account for the different findings.
To examine how the GS annual across-the-board and locality pay  adjustments are determined, we reviewed legislation, OPM regulations,  executive orders, Presidents’ alternative pay plans, President’s Pay Agent  Reports, Federal Salary Council recommendations, OPM and Bureau of  Labor Statistics (BLS) documents and reports, and reports by the  Congressional Budget Office (CBO) and Congressional Research  Service. We also examined how the methodology for determining locality  pay has changed since the start of locality pay to the present. We  interviewed selected members of the Federal Salary Council and its  working group; the Council is to be made up of six representatives of  federal employee groups and three experts in labor relations, and makes  annual recommendations to the President’s Pay Agent. We interviewed  BLS officials, OPM officials who are knowledgeable about federal pay  policy and serve as staff to the President’s Pay Agent, and people with  expertise in compensation issues including former federal officials  experienced with pay and benefits issues. To provide background  information illustrating a range of pay areas, we selected localities  including the lowest paid locality, highest paid locality, and other localities  to include a range of pay rates, population sizes, and geographic regions.
To determine the extent to which pay increases and awards recognize  individual performance, we analyzed legislation and OPM regulations on  pay increases and awards available to employees in the GS pay system  and identified those pay increases and awards that are determined in part  by an individual’s performance rating as measured by the agency’s  performance appraisal system. These pay increases and awards are:  within-grade increases, quality step increases, and ratings-based cash  awards. We recognize that there are other types of pay increases and  awards that reflect an individual’s contributions, such as  suggestion/invention and superior accomplishment awards, and pay  increases that do not reflect an individual’s performance at all including  across-the-board and locality pay adjustments. We identified eligibility  requirements outlined in the legislation and regulations and clarified in  OPM guidance that can affect a GS employee’s eligibility for the increase  or award, such as a waiting period given the individual’s position in the  pay grade, frequency of receiving an increase or award, and agency- specific criteria.
To provide statistics on how the pay increases and awards were  distributed among GS employees, we analyzed data from OPM’s Central   The data we examined  Personnel Data File (CPDF) for fiscal year 2011.included only federal employees in the GS pay plan. The GS  classification and pay system includes several pay plan codes: GS  (covered by pay system established under 5 U.S.C. chapter 53,  subchapter III); GM (covers employees covered by the Performance  Management and Recognition System termination provisions of Pub. L.  No. 103-89); GL (covers law enforcement officers who receive special  base rates at grades 3-10 under section 403 of FEPCA); GP (covers GS  physicians and dentists paid market pay under 38 U.S.C. § 7431(c)); and  GR (covers physicians and dentists covered by the Performance  Management and Recognition System termination provisions who are  paid market pay under 38 U.S.C. § 7431(c)). In addition to the GS pay  plan, the GM and GL pay plans are used in federal-nonfederal pay  comparisons to set locality pay. For the purposes of this analysis, we  excluded the GM and GL pay plans because the GS pay plan covers the  majority of the individuals in the GS, GM, and GL pay plans. We also  excluded the GP and GR pay plans since individuals in these pay plans  are no longer limited to GS rates of pay and they receive market pay  under a different pay system.
We analyzed CPDF data for employees in the GS pay plan in the  aggregate on the number, percentage and dollar amount of quality step  increases, within-grade increases, and ratings-based cash awards; the  amount of these increases and awards as a portion of the GS payroll  (total adjusted basic pay for all employees in the GS pay plan); and the  distribution of these increases and awards by rating pattern and rating  levels. For the award/increase amounts as percentages of recipients’ pay,  we excluded employees whose adjusted basic pay amount was missing.  For the calculations based on ratings, we excluded employees who were  coded in CPDF as “not rated”. The not rated code applies to an employee  who has not yet received a rating of record under the agency  performance appraisal system (e.g., someone newly hired). We also  excluded employees whose ratings were missing due to data errors. For  calculations based on rating levels or patterns (e.g., 5-level system), we  excluded employees who were coded as not being covered by a  performance appraisal system and generally do not have their  performance appraised. We also excluded employees whose rating  patterns were missing from the data due to data errors.
To help determine the reliability and accuracy of the CPDF data elements  used, we checked the data for reasonableness and the presence of any  obvious or potential errors in accuracy and completeness. For example,  we excluded employees who were coded as receiving an increase or  award in error (e.g., individuals who received a level 1 or 2 rating and a  within-grade increase or ratings-based cash award) from our data. We  also reviewed past GAO analyses of the reliability of CPDF data and  interviewed OPM officials knowledgeable about the data to discuss the  data’s accuracy and steps OPM takes to ensure they are reliable. For  example, in its checks of the data, OPM excludes data where the dollar  value is zero for ratings-based cash awards and within-grade and quality  step increases. Also, for within-grade and quality step increases, OPM  checks to make sure values for current and prior adjusted basic pay exist  and the difference is greater than zero. On the basis of these procedures,  we believe the data we used from the CPDF are sufficiently reliable for  the purpose of this report.
To describe how OPM provides oversight of pay increases and awards,  we collected and analyzed OPM guidance to agencies on administering  relevant pay increases and awards including regulations, memoranda,  reports, fact sheets, and frequently asked questions. We interviewed  OPM officials responsible for federal pay policies to discuss the  implementation of the guidance and monitoring of agencies’ use of  increases and awards through reports and other means, and we  interviewed OPM officials responsible for conducting human capital  management evaluations at agencies on pay increases and awards to  determine how they evaluate agencies’ linkage of pay increases and  awards with organizational results and monitor the overall GS system,  among other things.
To review selected studies that compare federal and private sector pay  and total compensation and describe factors that help account for the  different study findings, we reviewed the studies, summarized each  study’s methodologies and key findings, and confirmed the accuracy of  our summaries with the authors. We compared and contrasted the  differences between the approaches, methodologies, and data sources of  the selected studies. We interviewed the selected study authors to obtain  their views on the various methodologies and data sources available, why  they chose the ones they used, and their conclusions based on their  work. From July through December 2011, we conducted a detailed  literature review of academic journals, agency and organization  publications, and grey literature to identify the selected studies.applied three criteria for study selection to the results—(1) studies that  were published/issued since 2005; (2) studies that include original  analysis; and (3) studies that have the explicit and primary purpose of   We  comparing federal and private sector pay and total compensation. Using  these criteria, we identified at that time the following five studies as our  proposed set to review (with the option to add other studies that may be  issued during the course of our engagement and meet our criteria), see  below:
Comparing Federal and Private Sector Compensation, Andrew Biggs  and Jason Richwine, American Enterprise Institute for Public Policy  Research, June 2011. (Co-author Richwine is from The Heritage  Foundation.)
Federal Pay Continues Rapid Ascent, Chris Edwards, The Cato  Institute, August 2009.
Report on Locality-Based Comparability Payments for the General  Schedule, Annual Report of the President’s Pay Agent 2010, The  President’s Pay Agent, March 2011.
Bad Business: Billions of Taxpayer Dollars Wasted on Hiring  Contractors, The Project On Government Oversight, September 2011.  Inflated Federal Pay: How Americans Are Overtaxed to Overpay the  Civil Service, James Sherk, The Heritage Foundation, July 2010.
All of the selected studies except for the President’s Pay Agent compared  federal to private sector pay and total compensation. The President’s Pay  Agent compared federal to nonfederal pay (not benefits) and defined  nonfederal as private sector, state government, and local government.  We decided to include the President’s Pay Agent Report as one of our  selected studies given that it plays a major role in the overall discussion  of federal pay comparability. The President’s Pay Agent encompasses  the Secretary of Labor and Directors of OPM and the Office of  Management and Budget (OMB). To inform our understanding of the Pay  Agent’s report and process, we interviewed OPM officials who are staff to  the Pay Agent, members of the Federal Salary Council and its working  group including officials from the National Treasury Employees Union and  the American Federation of Government Employees, and officials at BLS,  which provides the nonfederal data used for the Pay Agent’s analysis.
Through our literature review, we also identified articles and papers that  compare compensation in other sectors (state and local government to  private sector, or industry to industry). Additionally, we identified  discussions of the selected studies’ findings and methodologies and of  the issues of federal and private sector pay and total compensation  comparison in general to further inform our review of the studies. We  interviewed a number of individuals chosen for their expertise in  compensation issues to obtain their views on the data sources for  analyzing compensation and to provide a general context for the issues  involved in comparing federal and private or nonfederal pay and total  compensation. The findings regarding the selected studies are not based  on input from these individuals. Representing a wide range of  perspectives and experiences related to compensation issues, we  identified these individuals through our literature review, background  research on the topic, and recommendations from the study authors and  other individuals knowledgeable about compensation issues. The  selected individuals, some of whom were selected authors of the  discussions noted above, included a university professor who has done  research on compensation issues across sectors, a private sector  compensation consultant, a staff member who researches compensation  at an organization with a policy focus, and former senior federal officials  who are experienced in federal pay and benefits issues. We interviewed  officials from the Bureau of Economic Analysis (BEA), BLS, and Census  Bureau to discuss how these agencies’ data are used to measure federal  and private or nonfederal pay, compensation, or benefits, and limitations  of their data or surveys. We also interviewed officials from OPM involved  in federal pay policies. We asked everyone we interviewed about their  views on the strengths and limitations of the data sources used in the  studies.
We also asked everyone we interviewed, as applicable, to identify any  additional studies that address our criteria for study selection. They did  not identify any additional studies that met our criteria, but provided  additional information, such as background articles. However, in January  2012, after our literature review was concluded, CBO issued a report:
Comparing the Compensation of Federal and Private-Sector  Employees, Congressional Budget Office, January 2012.
We included this study in our review because it met our criteria. This  brought the total number of studies up to six. We interviewed the authors  of the CBO study to obtain their views on the various methodologies and  data sources available, why they chose the ones they used, their  conclusions based on their work, and our understanding of their work.
We did not examine the reliability or the appropriateness of the  approaches, methods, and data used by the six selected studies in our  scope, and we did not exclude any study on the basis of methodological  quality.
We conducted this performance audit from July 2011 to June 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Implementation of Locality Pay
subsections: section_title: Locality Payments from 1994 to 2011
paragraphs: Even though the full locality payments recommended by the President’s  Pay Agent have not been provided after locality pay was implemented in  1994, some locality increase has been provided each year since that time  except during the pay freeze in 2011 and 2012. The President’s Pay  Agent reported that pay disparities were lower in 2011 than in 1994 in 16  of the 21 pay localities that existed in both of those years.
Figure 5 shows the relative pay rates for a GS-11 employee  (approximately the midpoint grade level) in San Francisco and in the Rest  of U.S. (the residual locality for areas not included in one of the other pay  localities) and nonfederal equivalents based on the President’s Pay Agent  Reports. In 1994, the pay disparity between federal and nonfederal  workers in San Francisco at the GS-11 level was 30 percent, which  decreased to 26 percent by 2011 (the most recent year for which disparity  data is available). In 1994, the pay disparity between federal and  nonfederal workers in the Rest of U.S. locality at the GS-11 level was 19  percent, which increased to 22 percent by 2011.
There have been several changes to the surveys and models used for  locality pay setting between the passage of the Federal Employees Pay  Comparability Act (FEPCA) in 1990 and the Pay Agent process and  report for 2011, the most current report available. Changes are illustrated  in figure 6 and additional information is below.
From 1991 to 1996, BLS conducted the Occupational Compensation  Survey Program (OCSP) to collect data on pay of nonfederal workers.  OCSP used a fixed list of 3 to 8 positions in each of the five PATCO  categories (Professional, Administrative, Technical, Clerical, and Other  White-Collar) to represent the range of different white collar jobs. In 1996,  there were 26 different positions - for example, Scientist (a professional  position) and Key Entry Operator (a clerical position). Each position had  one or more levels - for example, Scientist I to Scientist VIII; Key Entry  Operator I and Key Entry Operator II. BLS referred to a particular position  at a particular level (e.g., Scientist I) as a “job.” BLS asked surveyed  establishments to identify positions they had that corresponded to one of  the representative jobs. BLS and OPM worked together to write, test, and  maintain survey job descriptions tied to a single GS grade level.
In 1996, BLS stopped conducting the OCSP and started conducting the  National Compensation Survey (NCS), which uses probability sampling of  jobs. BLS randomly selected positions at surveyed nonfederal  establishments and determined which Standard Occupational  Classification System job, PATCO category, and GS grade corresponded  to the selected jobs. The Employment Cost Index (ECI) and a benefits  survey were also merged into NCS. These changes were made to reduce  costs and respondent burden and expand occupational coverage.
The President’s Pay Agent began reviewing the NCS in 1996, with input  from the Federal Salary Council. During the time of their review, they  used OCSP data, aged to a common reference date based on the ECI, to  calculate pay disparities and recommend locality pay. In 1998, they  determined that the NCS was not suitable for use without improvements,  and a working group with representatives of OPM, BLS, and OMB was  formed to recommend improvements. The working group made  recommendations in 1999 that led to five improvements in the NCS data.  The improvements were implemented starting in 2002, at which point the  Pay Agent began to phase in use of NCS data. The recommendations are  outlined in figure 6 above.
In 2008, the Federal Salary Council asked BLS to explore the use of  additional sources of pay data so the Council could better evaluate the  need for establishing additional locality pay areas, especially in areas  where the NCS could not provide estimates of nonfederal pay. BLS  developed a model to combine data from the Occupational Employment  Statistics (OES) survey, another BLS survey, with NCS data in order to  increase locality coverage. In 2010, due to budget cuts, BLS announced a  reduction in the size of the NCS sample, and said that the model results  from the combined surveys could still be used to calculate pay gaps.  According to BLS officials, only the size of the NCS sample has changed,  not the substance of what is collected, and the reduction should not affect  the ability to determine levels of work.
The Federal Salary Council wrote in its 2011 memo to the President’s  Pay Agent that it had concerns about the reduction. For 2011, the final  year when the larger NCS data set was available, the Federal Salary  Council reviewed modeled results both with and without the reduction,  and found concerning discrepancies (about a 5 point average difference  in computed pay gaps). In its memo, the Council recommended that the  Pay Agent use only NCS data for setting pay until the new model is better  understood, and that the full NCS survey be reinstated. The Council wrote  that it plans to continue working with OPM and BLS to study the  NCS/OES model.
The President’s Pay Agent wrote in its 2011 report dated March 2012 that  it does not consider more funding for NCS to be feasible before exploring  other options. The Pay Agent supported the Council’s plan to continue its  review of the new model and to focus on the impact of dropping roughly  half of the NCS sample on the volatility of the model. The Pay Agent also  noted that the administration recommended Congress establish a  Commission on Federal Public Service Reform composed of members of  Congress, representatives from the President’s Labor-Management  Council, members of the private sector, and academic experts to identify  fundamental reforms for the federal government’s human capital systems  including compensation reform. As of June 2012, such a commission has  not been established.
section_title: Appendix III: Information on the Selected Studies’ Data Sources and Methodologies for Analyzing Pay and Total Compensation
subsections: section_title: Pay Analysis
paragraphs: The six selected studies used different data sources and methodologies  to analyze differences in pay between the federal and private sector or  nonfederal workforces, as shown in table 6. They also varied slightly in  how they defined the federal workforce and restricted their analysis of  workers.
Studies could control for many attributes—personal or job-related—to  help explain the differences between federal and private sector pay, as  shown in the previous table. The types of attributes the selected study  authors controlled for depended on the type of approach used to analyze  pay—human capital or job-to-job. For example, the human capital  approach controls for personal attributes (e.g., education, job experience)  and job-related attributes (e.g., occupation, firm size). The job-to-job  approach involves controlling for job-related attributes (e.g., occupation,  level of work) without considering the personal attributes of the workers.  The trend analysis approach does not control for any attributes. Attributes  such as occupation, level of work, firm size, locality, education, and job  experience were considered relevant by several of the studies’ authors  and people with expertise in compensation issues that we interviewed.
Controlling for occupation allows a study to account for  different pay rates for different types of jobs. The distribution of  occupations in the federal government is different from the private or  nonfederal sector, which may be a factor that explains differences in  pay. For example, according to the CBO study, 33 percent of the  federal workforce compared with 18 percent of the private sector  workforce was in a professional occupation. A job-to-job approach, as  demonstrated by the study authors who used the approach, involves  matching federal workers to equivalent positions in another sector.  POGO limited its comparison to 35 selected occupations, while the  Pay Agent used over 200 occupations. According to one of the people  with expertise that we interviewed, one challenge with this approach is  the difficulty of finding nonfederal equivalents for certain positions,  such as the Federal Bureau of Investigation agents, that only exist in  the government. Another person with expertise said matching  occupations across sectors is a subjective process. In contrast, study  authors using the human capital approach used fewer and much  broader occupational groups. For example, Biggs/Richwine used 10  categories, while Sherk and CBO used 22 and 24, respectively. In  addition to his analysis of the overall pay disparity, Sherk analyzed  pay data with and without occupation controls, and reported that less  was explained when occupation was not included in the analysis.
Level of Work: Controlling for level of work (or grade level) allows a  study to account for different pay rates for different levels of job  complexity and responsibility (e.g., entry-level, mid-level, senior level  or finer distinctions by level). Level of work encompasses types of  duties performed, the scope and effect of the work, the level of  difficulty and responsibility, and the level of supervision received. It  can be difficult to measure level of work since levels are defined  differently in different workplace settings. Of the studies we examined,  only the President’s Pay Agent Report controls for level of work. For  federal employment in the GS pay system, there are 15 grade levels.  To compare these with levels in nonfederal workplaces, BLS  economists ranked nonfederal positions based on four factors:  knowledge, job controls and complexity, contacts (nature and  purpose), and physical environment.observed that the human capital approach does not recognize that  there are many different levels within an occupation such as  accountant or lawyer.
Firm size: Controlling for firm size allows a study to account for the  effect of the number of workers in a firm. Some of the study authors  asserted that large firms tend to offer higher salaries and greater  benefits than smaller firms, but they differed on the decision to control  for this attribute. CBO and Biggs/Richwine felt that federal workers  should be compared to private sector workers at similarly sized  institutions (e.g., firms with at least 1,000 workers) and included a  measure of firm size in their analyses. The reasons the authors cited  included large firms requiring more occupational specialization or  higher levels of skill than smaller firms. Sherk said he chose not to  control for firm size because he views it as a proxy for individual ability  in the private sector—the larger firms pay a premium to hire more  capable individuals and the associated pay reflects that. He said this  is not the case in the federal government; the federal government  does not selectively hire employees from large corporations but  competes for hiring with all sizes of firms in the private sector. Sherk  felt that including firm size could bias results if more productive  workers tend to work in larger firms in the private, but not the federal  sector. A person with expertise we interviewed agreed that a larger  firm would pay more and have better benefits and noted that large  firms are in head-to-head hiring competition with the federal  government. In 2008, the President’s Pay Agent decided to include  data from all establishments in its locality pay recommendations to  increase the amount of data available for jobs. Since locality pay  began in 1994, the Pay Agent had used only data from large  establishments in its calculations.expertise that we interviewed, the larger sample of data helps improve  the quality of the job matching.
See app. II for more information on the implementation of locality pay since 1994. organization. An individual with a master’s degree and a PhD may be  paid the same pay rate in the market if they are producing the same  output.
Job experience: Controlling for job experience allows a study to  account for the length of time an individual has spent working.  Experience both at a specific job and in general can affect pay,  presumably because it can affect productivity, which can be  accounted for in the human capital approach. Biggs/Richwine, CBO,  and Sherk considered job experience a relevant attribute. However,  the CPS does not include a direct measure of job experience. As a  result, the studies use proxies to measure experience. For example,  Biggs/Richwine and CBO used a common approach for measuring  experience, “age minus years of education minus 6,” while Sherk  included age in his model. According to a university professor we  interviewed who has done research on compensation issues across  sectors, there is no data set that measures how long a private sector  worker has been out of the workforce or how long a worker has been  working for a given employer. Age can be used as a proxy, but age  does not reflect time out of the workforce for child-rearing or other  reasons.
section_title: Total Compensation Analysis
paragraphs: The selected studies varied in the data sources used, benefits included,  and methodologies chosen in analyzing benefits as a part of total  compensation, as shown in table 7. The President’s Pay Agent is  mandated to analyze pay, not benefits, so its study is not included in the  following table.
The study authors had a variety of data sources to choose from in  analyzing pay and total compensation. They chose the data sources for  their studies based on their overall approach and data needs. The study  authors and people with expertise in compensation issues that we  interviewed identified strengths and limitations of two common data  sources the studies used in analyzing pay or total compensation—the  CPS and NCS. Agency officials who oversee these data sources also  weighed in on the use of the data in analyzing compensation.
Current Population Survey. The CPS—and in particular, the  monthly CPS—has a large sample size relative to other data sources  enabling analyses that would not have been possible in data sets with  a smaller sample size. According to Sherk, he used the monthly CPS  because he needed at least 30 valid observations of occupations in  both the public and private sectors for his analysis comparing detailed  occupations. The Annual Social and Economic Supplement of the  CPS has questions that are more indepth than the monthly CPS and it  contains measures of job tenure, educational degree, and firm size.  Individuals interviewed for the monthly or Annual Social and  Economic Supplement of the CPS are self-reporting in their  responses, which can result in reporting errors. As an example of an  error that could occur, individuals who work for a contractor employed  by the federal government could identify themselves as federal  employees, which would be incorrect. Census officials said that there  are CPS interviewer manuals to assist interviewers in helping  respondents with their answers.
National Compensation Survey. BLS conducts the NCS by  interviewing employers, which allows for cost data on pay and  benefits to be directly collected from employers as opposed to  individuals self-reporting the information. While the survey covers all  sectors, it does not collect data on federal workers, which—according  to the study authors who used the NCS—results in the need to piece  together different sources of benefits information in order to get  comparable data. The NCS also provides detailed pay information by  occupational work level that is based on the duties and responsibilities  of a job, which is a key source of information for the President’s Pay  Agent when determining locality pay adjustment amounts. Recently,  the sample size for the NCS was reduced, and BLS has developed a  model to determine locality pay using a combination of the NCS and  the Occupational Employment Statistics (OES) survey. The OES is a  larger survey with broader coverage of locality areas than the NCS,  but it does not contain information on levels of work. (See app. II for  additional information on locality pay and the use of these surveys.)
The following table provides additional details on the data sources  relevant for analyzing compensation across sectors including a  description of the data source and supporting methodology.
section_title: Appendix IV: Comments from the Project On Government Oversight
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Trina Lewis (Assistant Director),  Laurel Beedon, Benjamin Bolitzer, Sara Daleski, Karin Fangman, Robert  Gebhart, Janice Latimer, Rebecca Shea, and Meredith Trauner made key  contributions to this report.