Results-Oriented Management: OPM Needs to Do More to Ensure Meaningful Distinctions Are Made in SES Ratings and Performance Awards
Published: 2015-01-22T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The career SES, a cadre of senior leaders, has a pay-for-performance compensation system, which includes annual cash performance awards. OPM has a key leadership and oversight role in the implementation of the SES pay-for-performance system, including the certification of SES performance appraisal systems.
GAO was asked to examine SES performance awards. Specifically, this report (1) describes key characteristics of executive branch agency ratings and performance awards for fiscal years 2010 through 2013, and (2) provides a more in-depth look at five departments' fiscal year 2013 ratings and awards.
GAO analyzed data from OPM on the 24 CFO Act agencies for fiscal years 2010 through 2013. GAO also selected five case study departments: Defense, Energy, Health and Human Services, Justice, and Treasury and examined how they factored organizational and individual performance into their fiscal year 2013 SES performance awards.
section_title: What GAO Found
paragraphs: In 2012, the Office of Personnel Management (OPM) facilitated development of a new Senior Executive Service (SES) performance appraisal system with a more uniform framework to communicate expectations and evaluate the performance of executive branch agency SES members. The new system is expected to promote consistency, clarity, and transferability of performance standards and ratings across agencies. To obtain SES appraisal system certification for agencies seeking access to higher levels of pay, agencies are required to make meaningful distinctions based on the relative performance of their executives as measured through the performance and pay criteria. Further, if the modal rating is at the highest level of “outstanding,” agencies must provide an acceptable justification to OPM for the high level. (The modal rating is the rating level assigned most frequently among the actual ratings.)
More than 85 percent of career Chief Financial Officers (CFO) Act agency SES were rated in the top two of five categories for fiscal years 2010 through 2013, and career SES received approximately $42 million in awards for fiscal year 2013. The average award amount was higher for executives with higher ratings.
In a closer examination of five departments for fiscal year 2013, GAO found that they used or planned to use OPM's new SES performance system. The departments also had performance plans with links between individual SES responsibilities and organizational goals. Similar to the government-wide results, departments rated SES primarily in the top two categories. Four out of five departments awarded the same or higher performance awards to some SES with lower ratings. Department officials gave several reasons for giving lower-rated SES higher performance awards, including that they considered relative contributions, and that the awards were consistent within subcomponents of the department.
OPM plans to convene a cross-agency working group in 2015 to revisit the SES certification process. It will be important for OPM and the working group to consider whether, given the continued high SES performance ratings, the new SES appraisal system is contributing to making meaningful distinctions in performance ratings and awards without creating forced distributions, and if not, what refinements are needed.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Director of OPM consider various refinements to better ensure the SES performance appraisal system certification guidelines promote making meaningful distinctions in performance. Options could include not certifying appraisal systems where the modal rating is “outstanding.” OPM disagreed with the recommendation stating that, among other things, it could result in forced distributions in ratings. GAO maintains that additional action should be considered to ensure equity in ratings and performance awards across departments.

section_title: Letter
paragraphs: Compensation for members of the career Senior Executive Service  (SES), the government’s cadre of senior leaders, is grounded in the  notion of pay for performance, which has as its cornerstone creating  incentives and rewarding excellence and achievement. Career SES  employees receive a base salary and benefits, but pay increases—as  well as performance awards—are to be performance driven, based on  annual ratings of executives’ performance following reviews within their  agencies. A central element of SES pay-for-performance systems  encourages executive branch agencies to have an Office of Personnel  Management (OPM) certified performance appraisal system to gain  access to higher levels of pay for their senior executives. OPM, with the  Office of Management and Budget (OMB) concurrence, certifies these  systems, which include elements such as factoring organizational  performance into appraisal decisions and making meaningful distinctions  in performance and pay. In addition, a November 2013 memo from OPM  and OMB stated that it is critical that agencies’ use of performance  awards be managed in a way that is cost effective and leads to increased  employee performance and organizational results.
In 2003, Congress refined the pay systems for members of the SES by  requiring a clearer link between performance and pay. This legislative  provision came several years after OPM had advised federal agencies  that they needed to provide more rigorous and realistic ratings of their  senior executives. Additionally, the President’s Management Agenda  emphasized pay for performance for senior executives and criticized  agency performance management systems that failed to make  meaningful distinctions between senior executives’ job performance. Key  features of the new pay system, which took effect on January 11, 2004,  included the elimination of locality pay and across-the-board annual pay  adjustments; the replacement of six pay rates with one broad pay range;  an increase in the cap on base pay; and the addition of a second higher  cap for those covered under an SES appraisal system that had been  certified by OPM with the concurrence of OMB.
Performance awards are part of SES compensation and recognize  excellence in SES performance during a one-year period. Executive  branch career SES employees received approximately $42 million in  performance awards for fiscal year 2013, the latest year for which data  were available, and according to OPM regulations on certification of SES  appraisal systems for executive branch agencies, agencies seeking  certification are to recognize the highest performing executives with the  highest ratings and the largest performance awards. Maintaining an SES  compensation system that rewards and helps retain top performers is  critical to achieving the goal of developing a world-class executive team in  the federal government.
You requested that we examine SES performance awards. This report (1)  describes key characteristics of the awards, such as rating and award  distributions, award amounts, and percentage of executives receiving  awards, from fiscal years 2010 through 2013, and (2) describes and  assesses the extent to which selected departments’ SES performance  appraisal systems factored in organizational and individual performance  and made meaningful distinctions in their fiscal year 2013 performance  awards. For an additional perspective, the second objective provides an  in-depth view of five selected departments’ SES ratings and performance  awards’ processes for the last fiscal year of rating and awards data.
To address our first objective, we reviewed agency data provided to OPM  on the number of SES performance awards within the 24 Chief Financial  Officers (CFO) Act agencies and other variables from fiscal year 2010— the year prior to the implementation of policies that limited awards— through fiscal year 2013. Only career members of the SES who received  a performance rating for a given fiscal year are eligible for performance  awards, so we limited our analysis to those executives. We reviewed  OPM’s SES data for the presence of any obvious or potential errors in  accuracy and completeness and interviewed OPM officials about the  accuracy of the data. On the basis of these procedures, we believe the  data are sufficiently reliable for use in the analyses presented in this  report.
To address the second objective, we selected five departments based on  criteria such as varied distributions of SES performance awards at  different rating levels and those with the highest numbers of SES. Case  studies included the Departments of Defense (DOD), Energy, Health and  Human Services (HHS), Justice (DOJ), and Treasury. We reviewed these  departments’ SES performance appraisal systems and their most recent  certification documentation submitted to OPM. We also examined  whether individual SES performance metrics were linked to agency  performance and whether performance awards were tied to outcomes by  looking at examples of individual SES performance appraisals from our  case studies. We did not verify the validity of the measures used or the  performance of the senior executives. We also interviewed agency  officials from the selected departments with responsibility for their  department’s SES program as well as selected members or  representatives of the Performance Review Boards (PRB) charged with  ensuring consistency, stability, and objectivity in SES performance  appraisals.
We conducted this performance audit from June 2014 to December 2014  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. More detailed information on  our scope and methodology appears in appendix I.
subsections: section_title: Background
paragraphs: The SES is relatively small—about 7,900 members in 2013—and  represents less than one percent of the over two million federal civilian  employees. As a corps of executives selected for their leadership  qualifications, members serve in the key positions just below the top  presidential appointees. SES members are the major link between these  appointees and the rest of the federal work force. They operate and  oversee nearly every government activity in approximately 75 federal  agencies. OPM manages the overall federal executive personnel  program, and OPM staff provides the day-to-day oversight of and  assistance to executive branch agencies as they develop, select, and  manage their federal executives.
OPM has a key leadership and oversight role in the design and  implementation of executive branch agencies’ SES performance-based  pay systems by certifying that the agencies’ systems meet certain criteria.  Specifically, agencies are allowed to raise SES basic pay and total  compensation caps if OPM certifies, with the concurrence of OMB, that  agencies’ performance appraisal systems make—in design and  application—meaningful distinctions based on relative performance.  Agencies’ performance appraisal systems are evaluated against  certification criteria, including linking performance for senior executives to  the organization’s goals. Barring any compliance problems that might  arise after certification has been awarded, full certification is for about 24  months. Provisional certification for about 12 months is awarded when an  appraisal system meets design requirements, but there is insufficient  documentation to determine whether implementation meets certification  requirements. Certifying SES performance appraisal systems is also  OPM’s opportunity to ensure that these systems meet statutory  requirements.
From July 2011 through January 2012, OPM, SES members, and other  agency representatives from various agencies and organizations  developed a new SES performance appraisal system to try to meet the  needs of executive branch agencies and their SES members. Under the  new system, agencies were intended to have a more consistent and  uniform framework to communicate expectations and evaluate the  performance of SES members. While promoting greater consistency, the  new system was also designed to enhance clarity, transferability, and  equity in the development of performance requirements, the delivery of  feedback, the development of ratings, and the link to compensation. OPM  stressed that a major improvement of the new system included dealing  with the wide disparity in distribution of ratings by agency through the  provision of clear, descriptive performance standards and rating score  ranges that establish mid-level ratings as the norm and top-level ratings  as truly exceptional. While agencies are not required to adopt the new  system, OPM encourages agencies to do so. Table 1 shows the criteria  and documentation needed for distinctions in performance and  differentiation in pay and a description of the guidelines used for those  criteria. (Appendix II has a complete copy of OPM’s certification report  form.)
In our 2008 report on SES performance management systems, we noted  that while OPM certified that the selected agencies were making  meaningful distinctions based on relative performance as measured  through the pay and performance differentiation certification criteria,  performance ratings at the selected agencies raised questions about the  extent to which meaningful distinctions based on relative performance  were being made and how OPM applied these criteria. For example, we  reported that fiscal year 2007 SES ratings were concentrated at the top  two levels. As part of making meaningful distinctions in performance,  OPM has emphasized to agencies through its certification guidance that  its regulations prohibit forced distribution of performance ratings and that  agencies must avoid policies or practices that would lead to forced  distributions or even the appearance of it. We recommended that OPM  strengthen communication with agencies and executives regarding the  importance of using a range of rating levels when assessing performance  while avoiding the use of forced distributions. We also noted that  communicating this information to agencies will help them begin to  transform their cultures to ones where a “fully successful” rating is valued  and rewarded. OPM implemented the recommendation, and the agency  has been communicating the importance of using a range of rating levels  through the new SES performance management system.
section_title: Most Senior Executives Continued to Receive High Performance Ratings and Performance Awards for Fiscal Years 2010 through 2013
subsections: section_title: For Fiscal Years 2010 through 2013, Approximately 85 Percent of Career SES Received the Two Highest Ratings
paragraphs: In 2003, when Congress refined the pay systems for members of the SES  by requiring a clearer link between performance and pay, many senior  executives were receiving the top rating. Under its regulations, OPM  requires agencies to write performance requirements for each senior  executive at the “fully successful” level. In addition, under OPM’s new  SES performance appraisal system, a “fully successful” rating indicates a  “high level of performance” and “effective, solid, and dependable”  leadership. A rating of 5 is the highest, (labeled “outstanding”), followed  by 4 (“exceeds fully successful”), 3 (“fully successful”), 2 (“minimally  satisfactory”), and the lowest rating of 1 (“unsatisfactory”). Executives with  a rating of 2 or 1 are ineligible for performance awards, and a rating of 1  also triggers immediate additional performance actions. An executive  receiving a rating of 1 must either be reassigned or transferred within, or  removed from, the SES.
For fiscal years 2010 through 2013, all of the CFO Act agencies had four  or five rating levels in place for assessing SES performance. Figure 1  shows SES performance rating distributions for the CFO Act agencies for  those years. As the figure shows, more than 85 percent of career SES  were given a rating of either 5 or 4 each year.
For the same four years, approximately 46 percent of career SES  members received the highest possible rating. At a few agencies, the  proportion of senior executives who received a rating of 5 was larger than  70 percent. Table 2 shows the number of career SES rated and the  percentage at each rating level for the 24 CFO Act agencies for fiscal  year 2013. (Appendix III shows career SES ratings and performance  awards for the 24 CFO Act agencies for fiscal year 2013.)
A small proportion of senior executives received a rating of 3 or lower. For  fiscal years 2010 through 2012, about 13 percent of career executives  were given a rating of 3. For fiscal year 2013, 641 (or 10.3 percent) of  executives received a rating of 3, and at a third of the agencies, less than  5 percent were given a rating of 3 or lower. Twenty-one (or 0.3 percent)  senior executives were rated less than fully successful. Across all of the  CFO Act agencies, 17 executives received a rating of 2 for fiscal year  2013, and 4 executives received a rating of 1.
section_title: Because of Budgetary Constraints since Fiscal Year 2010, SES Are Receiving Fewer and Smaller Awards
paragraphs: Budget constraints have affected SES performance awards in recent  years, and the number of executives receiving performance awards and  the size of awards has decreased since fiscal year 2010. While legal  requirements have not changed—that SES performance awards be  between 5 and 20 percent of an individual executive’s rate of basic pay— OPM and OMB issued guidance in June 2011 that capped spending on  SES performance awards at no more than 5 percent of aggregate SES  salaries at a given agency, rather than the normal cap of 10 percent. This  cap was further reduced in February 2014 to 4.8 percent of aggregate  salaries. Additionally, senior executives did not receive pay adjustments  for three years (from January 1, 2011 through December 31, 2013) due to  federal pay-freeze legislation, which included a prohibition on SES pay  increases. Figure 2 shows a timeline of selected events affecting SES  performance awards from 2003 through 2014.
Table 3 shows the average SES performance awards for the four fiscal  years in inflation-adjusted dollar amounts and as a percentage of base  salary. To deal with the effects of sequestration, DOD chose to limit  funding for SES performance awards to 1 percent of aggregate career  SES salaries for fiscal year 2013. Several other agencies also limited  funding for performance awards to around 1 percent of aggregate career  SES salaries. This is one reason for the sharp decrease in the  percentage of SES receiving performance awards and the decrease in  average award amounts across the 24 CFO Act agencies for fiscal year  2013.
section_title: Distinctions between Award Amounts at Different Rating Levels Have Narrowed in Recent Years
paragraphs: Officials at several agencies said that in recent years, budget constraints  have forced them to make difficult decisions about how to allocate limited  award money and still make distinctions in performance pay. Figure 3  shows the average SES performance awards by rating level for the 24  CFO Act agencies for fiscal years 2010 through 2013. Since 2010,  agencies have made smaller distinctions in performance award amounts  between senior executives rated at different levels of performance. For  example, for fiscal year 2010, the average performance award for an  executive with a rating of 5 was $4,991 more than the average award for  an executive with a rating of 4. By fiscal year 2013, the average  performance award for a rating of 5 was $2,604 more than the average  award for a rating of 4. In a report on federal performance management,  we noted that it was frequently perceived that ratings were inflated by  supervisors because, among other things, they were used for multiple  decisions involving pay and awards, which can create a situation in which  a significant number of employees are rated in the “outstanding” and  “exceeds fully successful” levels. To the extent that employees with such  high ratings do not receive a monetary award, the perception that rewards  are not directly linked to performance is reinforced. On the other hand, as  the number of individuals receiving monetary awards increases, the  average dollar award will be reduced, resulting in the perception that the  awards are less motivating.
Since fiscal year 2010, the percentage of eligible SES receiving a  performance award at each rating level has decreased. Figure 4 shows  the percentage of SES receiving a performance award by rating for fiscal  years 2010 through 2013.
To help assess how agencies are meeting the certification requirement to  make distinctions in pay based on performance, OPM uses Pearson  correlation coefficients as a metric to analyze the strength of the  relationship between executives’ pay adjustments and performance  awards and their ratings. Correlation coefficients measure the linear  association between SES performance pay and senior executives’  ratings, and the value of the coefficient will be lower if the actual  relationship between the two is not a straight line. For this reason,  differences between years or agencies in the value of the correlation  coefficient may not be meaningful. To meet the certification guidelines for  pay differentiation, agencies are generally expected to have a correlation  coefficient of 0.5 or greater; if the correlation coefficient is lower than 0.5,  the guidelines state that an agency’s SES appraisal system can still  receive full certification if pay and awards data show the system makes  distinctions in pay. OPM reported that fiscal year 2013 correlations of  SES ratings and performance pay ranged from .19 to .99 for the 24 CFO  Act agencies: the National Aeronautics and Space Administration had the  lowest coefficient and the Nuclear Regulatory Commission had the  highest. Although correlation coefficients are useful for measuring the  strength of the linear relationship between ratings and performance pay,  they do not measure whether there are meaningful distinctions in pay  based on performance. For example, if an agency makes small  distinctions in pay across different rating levels (such as 5 percent of  salary for executives rated 4 and 5.05 percent of salary for executives  rated 5), the value of the correlation coefficient may be high, even though  the difference in pay between rating levels is not meaningful.
section_title: Selected Departments’ SES Performance Management Systems Are Becoming Standardized, but Disparities in Rating Distributions and Pay Remain for Fiscal Year 2013
subsections: section_title: Selected Departments’ SES Performance Systems Used or Were Converting to the New SES System
paragraphs: According to a 2012 OPM document on the new SES performance  appraisal system, with a different SES system in each agency,  inconsistency among the executive branch agencies was a problem  because of different definitions for rating levels across government, a mix  of four- and five-level rating systems, and variable application of rating  levels in evaluating SES—which led to a disparity in the ratings  distribution across government. OPM officials said that when the new  SES performance appraisal system was first available in fiscal year 2012,  seven agencies used the system because they were already closely  aligned with it. Since then, OPM officials said that about 90 percent of  agencies have started to use the new system.
As more agencies adopt the new system, the system’s intent of promoting  greater consistency may also result in greater uniformity in the  development of ratings with their link to compensation. As mentioned  previously, under the new system, OPM stated that agencies will be able  to rely upon a more consistent and uniform framework to communicate  expectations and evaluate the performance of SES members. While  promoting greater consistency, the new system was also intended to  enhance clarity, transferability, equity in the development of performance  requirements, and the delivery of feedback. In addition, OPM noted that  SES mobility is complicated by inconsistency—for executives moving  between agencies or considering moving, there has been uncertainty  regarding performance evaluations. The new SES appraisal system was  intended to help address all of these areas.
Of the five systems we reviewed—DOD, Energy, HHS, DOJ, and  Treasury— DOD, Energy, and HHS used the new SES performance  appraisal system. DOJ and Treasury were in the process of converting to  the new system. For example, a human capital official from Treasury said  the department is transitioning toward using government-wide  performance requirements; the official said Treasury’s fiscal year 2013  system had three critical elements rather than the five in OPM’s standard  version. Four out of the five departments had automated SES appraisal  systems or had plans to convert to one within the next fiscal year; DOJ  did not. In addition, all of the five selected departments had performance  appraisal systems that were certified by OPM and OMB. DOD, Energy,  and DOJ had full certification; HHS and Treasury had provisional  certification. As mentioned previously, for an agency’s SES performance  appraisal system to be certified, agencies’ appraisal systems must meet  criteria including that there is alignment between organizational and  individual performance and that distinctions in pay are being made based  on performance.
The selected departments were all on a fiscal year appraisal period with  ratings decisions for the previous fiscal year made in the first few months  of the next fiscal year. Department officials told us that the process of  issuing ratings and making awards decisions happens in a fairly short  timeframe.
section_title: Selected Departments Aligned Organizational and Individual SES Performance in the Performance Plans Provided to OPM for Certification
paragraphs: We previously identified the alignment of individual performance  expectations with organizational goals as a key practice for an effective  performance management system. It is important for individuals to see a  connection between their daily operations and results to help them  understand how individual performance can contribute to organizational  success. Leading organizations have recognized that effective  performance management systems create a “line of sight” showing how  unit and individual performance can contribute to overall organizational  goals and can help them drive internal change and achieve external  results.
The five selected departments linked individual metrics or competencies  to either component- or department-wide goals in the performance plans  provided by the departments. The plans included specific links between  individual SES competencies or responsibilities and a specific  organizational goal. Additionally, OPM’s certification criteria require that  agencies align SES individual performance expectations with  organizational goals, and OPM has issued relevant guidance.
Agencies must establish one or more SES Performance Review Boards  (PRB), which is a higher level of review within the SES performance  management system. The PRBs review and evaluate the senior  executive’s initial summary rating and, if applicable, the executive’s  response and a higher level official’s comments on the initial rating. The  boards also make written recommendations to the appointing official on  annual summary ratings and performance awards. PRBs serve to ensure  consistency, stability, and objectivity in performance appraisals. Boards  also take into account organizational performance when making  recommendations. OPM guidance states that, for agencies seeking  access to higher levels of pay through certification, PRBs are to ensure  meaningful distinctions in executive performance and that pay increases  and performance awards are made based on individual and  organizational performance. When appraising a career appointee’s  performance or recommending a career appointee for a performance  award, more than one-half of the PRB’s members must be SES career  appointees.
The selected departments varied somewhat in their PRB structures as  well as in who provided the final approval of the appraisal decisions;  some departments have additional steps or guidance as part of their  processes. For example, in addition to the PRB that evaluates ratings,  Treasury has a front-end PRB that meets to discuss SES employees’  commitments early in the performance planning cycle. A Treasury official  said this helps to ensure consistency in the performance plans as well as  the rating and award. Once PRBs have reviewed ratings and awards,  they make recommendations to the appointing authority—such as the  head of an agency—for final ratings and awards decisions. PRB  representatives we interviewed said that awards decisions are based on  ratings within individual pay pools.
Although the selected departments linked SES performance plans with  agency strategic objectives, the departments varied in their requirements  that the PRBs compare the performance ratings to the outcomes of  department goals and objectives. Most of the selected departments’ PRB  representatives said that the PRB explicitly helps to ensure that there is  alignment between individual performance and organizational  performance by having PRBs consider the organizational assessment— an assessment of the agency’s overall performance—when reviewing  proposed ratings and performance awards. For example, Energy provides  the PRB with the organizational assessment, as well as guidance on how  to use the assessment when reviewing ratings. An Energy official said  that both rating officials and PRBs consider the organization’s  performance when determining senior executives’ ratings. However, a  Treasury official told us that the PRB for some bureaus within Treasury  does not have access to an organizational assessment when reviewing  ratings. The official said that tracking and reviewing organizational  performance is done by the final rating official.
section_title: Consistent with Government-wide Data, Selected Departments Rated SES Primarily in the Top Two Categories for Fiscal Year 2013
paragraphs: All five departments rated the majority of SES in the top two categories,  indicating little differentiation between executives in their ratings. As figure  5 shows, the five selected departments gave outstanding ratings to  executives ranging from 30.6 percent in DOD to 73.6 percent in DOJ.
In 2008, we reported that senior executives for fiscal year 2007 were  concentrated in the top two rating levels. As figure 6 shows, at the three  departments (DOD, Energy, and Treasury) that we looked at in both fiscal  years 2007 and 2013, the proportions of ratings were nearly the same,  although Energy had switched from a four-rating system to a five-rating  system during that time.
Agency officials offered varied explanations for the high concentration of  performance ratings at the top two rating levels, ranging from stating that  the ratings are justified to stating that the ratings may be too high, but  they are reinforced by an agency culture where executives may not view  a rating of 3 as acknowledgement of a fully successful performance. For  example, human capital officials at DOJ pointed out that the individuals  chosen for the SES are already high performers and continue to perform  well as SES, earning high ratings. An HHS human capital official,  however, noted that competencies are written so that a rating level of 3  represents a fully successful performance, but it is difficult to convince  executives who have traditionally received higher ratings that this rating  reflects successful performance. Similarly, an Energy official noted that  the department communicates the message to rating officials (both  verbally and in writing) that a “fully successful” rating is not average or  ordinary; it demonstrates a significant level of accomplishment. One of  the purposes of the new SES appraisal system is to help ensure  standardized ratings with the understanding that a rating level of  “outstanding” should always be a difficult goal to reach.
section_title: Four of Five Selected Departments Awarded the Same or Higher Performance Awards to SES with Lower Ratings
paragraphs: The departments had several layers of review, including both component- level and department-level review to ensure consistency across the  department. For example, in addition to the PRBs, DOJ has an additional  review level (the Senior Executive Resources Board) that analyzes the  performance awards and attempts to identify trends and anomalies.  However, for fiscal year 2013, four out of five selected departments  awarded the same or higher performance awards as a percentage of  base salary to SES with lower ratings as was awarded to those SES with  higher ratings. While OPM has certified that the selected departments’  appraisal systems make meaningful distinctions based on relative  performance, actual awards at some departments do not seem to support  that meaningful distinctions are being made. Figure 7 shows the range of  performance awards given to eligible SES at the five selected  departments for fiscal year 2013.
PRB representatives from the selected departments indicated that the  variation in performance awards by ratings was caused by a number of  different factors. For example, a former PRB Chair said that in fiscal year  2012, they had 9 different pay pools within certain DOD non-combat  entities; each was given the latitude to determine how to distribute  performance awards within the pay pool. Although the distribution of  performance awards across ratings looked inconsistent when aggregated,  no one with a lower rating received a larger award than anyone with a  higher rating in the same pay pool. A Treasury PRB representative said  the range of performance awards is based on ratings as well as several  other variables, such as relative contributions to the organization. The  Treasury PRB has the flexibility to review those factors when determining  performance award amounts, and this sometimes results in an SES with  a lower rating getting an equal (or larger) performance award as a  percentage of salary than an SES with a higher rating. The PRB  representative from DOJ said components were forced to prioritize; when  a large percentage of executives are outstanding and only 55 percent can  receive a performance award, there are some difficult decisions. DOJ  also noted that different executives may get performance awards from  year to year, based on their contributions to overall mission achievement.
section_title: OPM Needs to Ensure That Its New SES Appraisal System Is Fulfilling Its Intended Purposes
paragraphs: One of the primary purposes for establishing the new SES appraisal  system included increasing equity in ratings across agencies and their  link to compensation. The new system provides for the uniform  administration of SES executive branch performance management  systems by promoting consistency, clarity, and transferability of  performance standards and ratings across agencies. Additionally,  effective performance management systems recognize that merit-based  pay increases should make meaningful distinctions in relative  performance; this principle is central to the SES performance  management system, where under the law, to be certified and thereby  able to access the higher levels of pay, the appraisal system must make  meaningful distinctions based on relative performance. OPM’s guidelines  state that the modal rating should be below “outstanding” and that  multiple rating levels should be used. However, OPM’s guidelines also  state that if an agency’s modal rating level is “outstanding,” the appraisal  system can still be certified if accompanied with a full, acceptable  justification. Nonetheless, the continued concentration of senior  executives at the top two rating levels indicates that this principle is not  being met across government.
While making meaningful distinctions in SES performance continues to be  a challenge for many agencies, some others have made progress. For  example, our 2008 report noted that according to a DOD official, DOD  was communicating the message that the SES performance-based pay  system recalibrates performance appraisals as a way to help change the  culture and to make meaningful distinctions in performance, with a “fully  successful” or equivalent rating as a high standard as well as a valued  and quality rating. According to DOD, levels above “fully successful”  require extraordinary results. Of the five selected departments that we  examined in fiscal year 2013, DOD had the lowest percentage of senior  executives receiving the highest rating—almost 31 percent.
According to OPM officials, in 2015 OPM plans to convene a cross- agency working group that is to revisit the SES certification process. As  part of this effort, it will be important for OPM and the working group to  consider whether—given the continued high SES performance ratings— the new system is contributing to making meaningful distinctions in  performance ratings and awards, and if not, what refinements are  needed. The goal of having a uniform system would appear compromised  if an “outstanding” rating in one agency does not have the same meaning  in another agency. Some options might include revisiting and perhaps  eliminating the guideline that allows OPM to certify agencies’  performance management systems with an SES modal rating of  “outstanding,” so long as the agency provides acceptable justification.  This guideline could serve to work against encouraging agencies to make  meaningful distinctions in SES performance. Alternatively, enhancing the  transparency of OPM’s approval of agencies’ justification for a modal  rating of “outstanding” could shed light on whether the individual agency’s  high ratings seem justified. These justifications are not on OPM’s website  and OPM does not report them to Congress, nor does the Chief Human  Capital Officers Council review them for any consistency.
section_title: Conclusions
paragraphs: Understandably, it could take more time for OPM’s 2012 efforts to  standardize SES performance management to fully materialize. However,  data for fiscal year 2013 (both government-wide as well as at our case  study departments) showed that a large majority of SES employees are  still receiving one of the top two ratings. Coupled with evidence of overlap  in performance awards across rating levels, this indicates that the link  between performance ratings and awards is not being consistently  applied.
By convening a cross-agency working group to review the SES  certification process, OPM is in a position to evaluate whether the new  SES appraisal system actually helps agencies use SES compensation  and performance awards in ways that are cost effective and lead to  increased employee performance and organizational results. If the  performance definitions cannot be consistently applied across the  government, creating a uniform framework to communicate expectations  and evaluate the performance of SES members will be difficult to attain.
section_title: Recommendation for Executive Action
paragraphs: As OPM convenes the cross-agency working group, we recommend that  the Director of OPM, as the head of the agency that certifies—with OMB  concurrence—SES performance appraisal systems, consider the need for  refinements to the performance certification guidelines addressing  distinctions in performance and pay differentiation. Options could include
Revisiting and perhaps eliminating the guideline that allows OPM to  certify agencies’ performance management systems with an SES  modal rating of “outstanding,” or
Strengthening the accountability and transparency of this guideline by
Reporting agencies’ justifications for high ratings to OPM on its  website.
Reporting agencies’ justifications for high ratings to Congress.
Obtaining third party input on agencies’ justifications for high  ratings, such as by the Chief Human Capital Officers Council.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Director of OPM and to the Acting  Secretary of Defense, the Secretary of Energy, the Secretary of Health  and Human Services, the Assistant Attorney General for Administration at  the Department of Justice, and the Secretary of the Treasury for review  and comment. OPM’s comments are reprinted in appendix IV. OPM also  provided technical comments, which we incorporated as appropriate.  DOD, Energy, DOJ, and Treasury responded saying they did not have  comments on the report. HHS did not respond to our request for  comments.
In its written comments, OPM generally agreed with the information in the  report but did not agree with our recommendation. In disagreeing with our  recommendation to consider not certifying agencies with modal ratings of  “outstanding,” OPM expressed concerns that imposing such a criterion  would lead to arbitrary manipulation of the final ratings rather than an  appropriate comparison of performance to standards. OPM asserted that  this situation would be ripe for forced distribution of the ratings, which is  explicitly prohibited by regulation. OPM also stated that the more  appropriate action is to continue emphasizing the importance of setting  appropriate, rigorous performance requirements and standards that  logically support meaningful distinctions in performance.
As recognized in our report, OPM’s regulations contemplate that it is  possible to apply standards that make meaningful performance  distinctions and to use a range of ratings while avoiding the use of forced  distributions. As we also note, since our 2008 report on SES performance  management systems—continuing through the career SES performance  ratings for fiscal year 2013—questions persist about the extent to which  meaningful distinctions based on relative SES performance are being  made. Although OPM has emphasized that an “outstanding” rating  represents a level of rare, high-quality performance, it appears from  examining fiscal year 2013 SES ratings data that some agencies are not  appropriately applying these performance standards to their SES ratings.  This undercuts one of the primary purposes for establishing the new SES  appraisal system, which includes increasing equity in ratings across  agencies and their link to compensation. Certifying agencies that are not  adhering to the agreed-upon performance standards provides little  incentive to those agencies that are adhering to the standards and could  lead to ratings that are more skewed toward “outstanding.” As recently as  2008, OPM agreed on the importance of communicating to agencies the  value of using a range of rating levels and transforming their cultures to  those in which a “fully successful” rating is valued and rewarded.
OPM also did not support the second part of our recommendation  regarding three suggestions for increasing transparency for those  agencies that are certified with a modal rating of “outstanding.” Although  we suggested that OPM report high rating justifications to Congress  through its Annual Performance Report, we understand that this may not  be the most appropriate vehicle to use; another avenue of reporting to  Congress would certainly be acceptable, and we have adjusted the text  accordingly. In addition, by suggesting that the Chief Human Capital  Officers Council have input on agencies’ justifications for high ratings, we  were in no way suggesting that this Council role impact OPM’s ultimate  authority over the regulation and oversight of the SES performance  appraisal system (including certification of agencies’ systems). We  maintain, however, that—as an alternative action to more direct  enforcement of the performance standards—transparency regarding  OPM’s approval of justification for a modal rating of “outstanding” could  shed light on whether the individual agency’s high ratings seem justified.   Given the recent data on SES performance ratings and awards, we  remain concerned that meaningful distinctions in relative SES  performance are not being made in a uniform fashion.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretaries of  Defense, Energy, Health and Human Services, and the Treasury, to the  Assistant Attorney General of Administration at the Department of Justice,  and to the Director of the U.S. Office of Personnel Management, as well  as to the appropriate congressional committees and other interested  parties. In addition, this report is available at no charge on the GAO  website at http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-6806 or goldenkoffr@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff members making key  contributions to this report are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines the distribution of performance awards to career  Senior Executive Service (SES) employees in executive branch agencies.  The objectives of this report were to (1) describe key characteristics of the  awards, such as rating and awards distributions, award amounts, and  percentage of executives receiving awards, from fiscal years 2010  through 2013, and (2) describe and assess the extent to which selected  departments’ SES performance appraisal systems factored in  organizational and individual performance and made meaningful  distinctions in their fiscal year 2013 performance awards. For an  additional perspective, the second objective provides an in-depth view of  five selected departments’ SES ratings and performance awards’  processes for the last fiscal year of ratings and award data.
For this report, we reviewed applicable legislation and regulations, as well  as the Office of Personnel Management (OPM) and the Office of  Management and Budget (OMB) guidance and government-wide reports,  such as OPM’s annual reports for fiscal years 2010-2013, Report on  Senior Executive Pay and Performance. We also interviewed officials at  OPM and reviewed applicable reports from non-governmental  organizations, such as the Senior Executive Association. We reviewed  data from OPM on agency performance rating levels, as well as the  number and amount of SES performance awards. We defined our  universe of analysis as career senior executives who received ratings.  We also excluded SES employees (where identifiable) from agency  Inspector General Offices because their inclusion in the data was  inconsistent. We reviewed OPM’s SES data for reasonableness and the  presence of any obvious or potential errors in accuracy and  completeness, and OPM officials confirmed the correctness of the data.  On the basis of these procedures, we believe the data are sufficiently  reliable for use in the analyses presented in this report.
To address our first objective, we reviewed data from OPM on the amount  of SES performance awards given to career SES within the 24 Chief  Financial Officers (CFO) Act agencies and other variables since fiscal  year 2010 (the year prior to the limitation of performance award policies)  through fiscal year 2013. We analyzed aggregate SES basic pay and  performance ratings as provided by OPM for fiscal years 2010 through  2013. In calculating the percentage of eligible senior executives who  received performance awards, we excluded executives who did not  receive a performance rating.
To address the second objective, we selected five case study  departments from the CFO Act agencies based on several criteria. Using  2012 Enterprise Human Resources Integration data, we identified  departments that had the largest number of SES employees and varying  performance award distributions, including departments with both a large  and small percent of SES employees who received an award, and we  identified departments that had both a small and large range of award  amounts, as both dollars and percent of salary. Selected case studies  included the Departments of Defense (DOD), Energy, Health and Human  Services (HHS), Justice (DOJ), and Treasury. We reviewed these  departments’ SES performance appraisal systems and their most recent  certification documentation submitted to OPM. We also examined  whether individual SES performance metrics were linked to agency  performance and whether performance awards were tied to outcomes by  looking at examples of individual SES performance appraisals. We did  not verify the validity of the measures used or the performance of the  senior executives. We also interviewed agency officials from the selected  departments with responsibility for their department’s SES program as  well as selected members or representatives of the Performance Review  Boards charged with ensuring consistency, stability, and objectivity in  SES performance appraisals.
Additionally, we analyzed OPM data for each case study department to  identify the percentage of ratings distributed across each rating category  in fiscal year 2013. We compared results of three of the case study  departments that were also reviewed in 2007, using data from a previous  report. We also analyzed the data to determine the amount of  performance awards—as a percentage of salary—given to SES  employees for each performance rating level.
We conducted this performance audit from June 2014 to December 2014  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Office of Personnel Management (OPM) Certification Report Form
paragraphs: OPM Report on the (Agency Name)
Senior Executive Service (SES) Performance Appraisal System  Certification  Date of Report: MM/DD/YYYY  OPM (Enter Decision Here) certification of the (Agency Name) SES  performance appraisal system. The table below indicates the certification  criteria fully met, minimally met, or not met during OPM’s review of the  system.
Description of “meets” criteria   The agency is using the basic SES  appraisal system description and it  has been approved by OPM. (This  means the system includes all the  required system language, and the  performance plans include all the  required language for Consultation,  Employee Perspective, Customer  Perspective, and Accountability.)
All performance plans include specific  organizational goals in the Strategic  Alignment cell for each performance  requirement under the Results Driven  element in Part 5 of the appraisal  form.
Each performance requirement under  the Results Driven element for each  performance plan contains adequate  measurable results. If the agency also  includes a list of activities in the plans,  the plan clearly identifies the  measurable result(s) and denotes that  it is the results that are to be rated for  the Results Driven element.
subsections: section_title: Documentation required/criteria addressed Organizational Assessment and Guidelines
paragraphs: Description of “meets” criteria  The agency has provided a copy of its memo  issuing guidelines to executives, rating officials  and PRB members that includes the results of  the organizational assessment, and guidelines  for how to use the results when determining  ratings, pay, and awards; OR, for small  agencies, a description for how the guidelines  and results of the organizational assessment  were given to rating officials and PRB  members and the content of the guidelines  (e.g., if communicated by email, a copy of the  email is included).
The agency has provided a description of  training or communications given that should  include a briefing on the agency’s SES  performance management system, OR the  agency provides evidence it has conducted the  training, but cannot verify how many  executives attended the training, OR the  system has not yet been implemented and the  agency provides training plans; AND  communication of the average rating, pay, and  awards given the previous year.
The SES rating distribution indicates the  agency is clearly making distinctions in  performance (i.e. the modal rating is below  “outstanding”) AND the distribution appears to  reflect organizational performance as  explained by the agency, OR, the modal rating  is “outstanding” but the agency gave a full,  acceptable justification; AND the percent of  SES members not rated is less than 5 percent.
Differentiation in Pay Based on Performance   a) The agency's correlation coefficient of the  rating and performance compensation (that is,  pay adjustments and awards) is 0.500 or more,  OR the pay and awards data show the agency  makes distinctions in pay b) AND the average  performance compensation is higher for  executives rated Outstanding than for those  rated Exceeds, and Exceeds is higher than  Fully Successful; c) AND the data does not  include any violations of pay and awards limits.
section_title: Documentation required/criteria addressed
paragraphs: Description of “meets” criteria  Pay Policy  The agency provides a written, official pay  policy and the policy describes clear  differentiations in performance compensation  (that is, pay adjustments and awards) based  on the annual summary rating.
section_title: Appendix III: Career Senior Executive Service (SES) Ratings and Performance Awards at the 24 Chief Financial Officers (CFO) Act Agencies for Fiscal Year 2013
subsections: section_title: Percent of career SES at each rating level
paragraphs: Office of Personnel Management (OPM), Small Business Administration (SBA), and Social Security  Administration (SSA).
section_title: Appendix IV: Comments from the Office of Personnel Management
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Robert Goldenkoff, (202) 512-6806 or goldenkoffr@gao.gov.
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Thomas Gilbert, Assistant  Director, and Judith Kordahl, Analyst-in-Charge, supervised the  development of this report. Charles Culverwell and Mary Diop made  significant contributions to all aspects of this report. Other important  contributors included Sara Daleski, Karin Fangman, Donna Miller, and  Rebecca Shea.