Results-Oriented Management: OPM Needs to Do More to Ensure Meaningful Distinctions Are Made in SES Ratings and Performance Awards
Released: 2015-02-23T00:00:00

Letter

Background

The SES is relatively small—about 7,900 members in 2013—and represents less than one percent of the over two million federal civilian employees. As a corps of executives selected for their leadership qualifications, members serve in the key positions just below the top presidential appointees. SES members are the major link between these appointees and the rest of the federal work force. They operate and oversee nearly every government activity in approximately 75 federal agencies. OPM manages the overall federal executive personnel program, and OPM staff provides the day-to-day oversight of and assistance to executive branch agencies as they develop, select, and manage their federal executives.

OPM has a key leadership and oversight role in the design and implementation of executive branch agencies’ SES performance-based pay systems by certifying that the agencies’ systems meet certain criteria. Specifically, agencies are allowed to raise SES basic pay and total compensation caps if OPM certifies, with the concurrence of OMB, that agencies’ performance appraisal systems make—in design and application—meaningful distinctions based on relative performance. Agencies’ performance appraisal systems are evaluated against certification criteria, including linking performance for senior executives to the organization’s goals. Barring any compliance problems that might arise after certification has been awarded, full certification is for about 24 months. Provisional certification for about 12 months is awarded when an appraisal system meets design requirements, but there is insufficient documentation to determine whether implementation meets certification requirements. Certifying SES performance appraisal systems is also OPM’s opportunity to ensure that these systems meet statutory requirements.

From July 2011 through January 2012, OPM, SES members, and other agency representatives from various agencies and organizations developed a new SES performance appraisal system to try to meet the needs of executive branch agencies and their SES members. Under the new system, agencies were intended to have a more consistent and uniform framework to communicate expectations and evaluate the performance of SES members. While promoting greater consistency, the new system was also designed to enhance clarity, transferability, and equity in the development of performance requirements, the delivery of feedback, the development of ratings, and the link to compensation. OPM stressed that a major improvement of the new system included dealing with the wide disparity in distribution of ratings by agency through the provision of clear, descriptive performance standards and rating score ranges that establish mid-level ratings as the norm and top-level ratings as truly exceptional. While agencies are not required to adopt the new system, OPM encourages agencies to do so. Table 1 shows the criteria and documentation needed for distinctions in performance and differentiation in pay and a description of the guidelines used for those criteria. (Appendix II has a complete copy of OPM’s certification report form.)

In our 2008 report on SES performance management systems, we noted that while OPM certified that the selected agencies were making meaningful distinctions based on relative performance as measured through the pay and performance differentiation certification criteria, performance ratings at the selected agencies raised questions about the extent to which meaningful distinctions based on relative performance were being made and how OPM applied these criteria. For example, we reported that fiscal year 2007 SES ratings were concentrated at the top two levels. As part of making meaningful distinctions in performance, OPM has emphasized to agencies through its certification guidance that its regulations prohibit forced distribution of performance ratings and that agencies must avoid policies or practices that would lead to forced distributions or even the appearance of it. We recommended that OPM strengthen communication with agencies and executives regarding the importance of using a range of rating levels when assessing performance while avoiding the use of forced distributions. We also noted that communicating this information to agencies will help them begin to transform their cultures to ones where a “fully successful” rating is valued and rewarded. OPM implemented the recommendation, and the agency has been communicating the importance of using a range of rating levels through the new SES performance management system.

Most Senior Executives Continued to Receive High Performance Ratings and Performance Awards for Fiscal Years 2010 through 2013

For Fiscal Years 2010 through 2013, Approximately 85 Percent of Career SES Received the Two Highest Ratings

In 2003, when Congress refined the pay systems for members of the SES by requiring a clearer link between performance and pay, many senior executives were receiving the top rating. Under its regulations, OPM requires agencies to write performance requirements for each senior executive at the “fully successful” level. In addition, under OPM’s new SES performance appraisal system, a “fully successful” rating indicates a “high level of performance” and “effective, solid, and dependable” leadership. A rating of 5 is the highest, (labeled “outstanding”), followed by 4 (“exceeds fully successful”), 3 (“fully successful”), 2 (“minimally satisfactory”), and the lowest rating of 1 (“unsatisfactory”). Executives with a rating of 2 or 1 are ineligible for performance awards, and a rating of 1 also triggers immediate additional performance actions. An executive receiving a rating of 1 must either be reassigned or transferred within, or removed from, the SES.

For fiscal years 2010 through 2013, all of the CFO Act agencies had four or five rating levels in place for assessing SES performance. Figure 1 shows SES performance rating distributions for the CFO Act agencies for those years. As the figure shows, more than 85 percent of career SES were given a rating of either 5 or 4 each year.

For the same four years, approximately 46 percent of career SES members received the highest possible rating. At a few agencies, the proportion of senior executives who received a rating of 5 was larger than 70 percent. Table 2 shows the number of career SES rated and the percentage at each rating level for the 24 CFO Act agencies for fiscal year 2013. (Appendix III shows career SES ratings and performance awards for the 24 CFO Act agencies for fiscal year 2013.)

A small proportion of senior executives received a rating of 3 or lower. For fiscal years 2010 through 2012, about 13 percent of career executives were given a rating of 3. For fiscal year 2013, 641 (or 10.3 percent) of executives received a rating of 3, and at a third of the agencies, less than 5 percent were given a rating of 3 or lower. Twenty-one (or 0.3 percent) senior executives were rated less than fully successful. Across all of the CFO Act agencies, 17 executives received a rating of 2 for fiscal year 2013, and 4 executives received a rating of 1.

Because of Budgetary Constraints since Fiscal Year 2010, SES Are Receiving Fewer and Smaller Awards

Budget constraints have affected SES performance awards in recent years, and the number of executives receiving performance awards and the size of awards has decreased since fiscal year 2010. While legal requirements have not changed—that SES performance awards be between 5 and 20 percent of an individual executive’s rate of basic pay— OPM and OMB issued guidance in June 2011 that capped spending on SES performance awards at no more than 5 percent of aggregate SES salaries at a given agency, rather than the normal cap of 10 percent. This cap was further reduced in February 2014 to 4.8 percent of aggregate salaries. Additionally, senior executives did not receive pay adjustments for three years (from January 1, 2011 through December 31, 2013) due to federal pay-freeze legislation, which included a prohibition on SES pay increases. Figure 2 shows a timeline of selected events affecting SES performance awards from 2003 through 2014.

Table 3 shows the average SES performance awards for the four fiscal years in inflation-adjusted dollar amounts and as a percentage of base salary. To deal with the effects of sequestration, DOD chose to limit funding for SES performance awards to 1 percent of aggregate career SES salaries for fiscal year 2013. Several other agencies also limited funding for performance awards to around 1 percent of aggregate career SES salaries. This is one reason for the sharp decrease in the percentage of SES receiving performance awards and the decrease in average award amounts across the 24 CFO Act agencies for fiscal year 2013.

Distinctions between Award Amounts at Different Rating Levels Have Narrowed in Recent Years

Officials at several agencies said that in recent years, budget constraints have forced them to make difficult decisions about how to allocate limited award money and still make distinctions in performance pay. Figure 3 shows the average SES performance awards by rating level for the 24 CFO Act agencies for fiscal years 2010 through 2013. Since 2010, agencies have made smaller distinctions in performance award amounts between senior executives rated at different levels of performance. For example, for fiscal year 2010, the average performance award for an executive with a rating of 5 was $4,991 more than the average award for an executive with a rating of 4. By fiscal year 2013, the average performance award for a rating of 5 was $2,604 more than the average award for a rating of 4. In a report on federal performance management, we noted that it was frequently perceived that ratings were inflated by supervisors because, among other things, they were used for multiple decisions involving pay and awards, which can create a situation in which a significant number of employees are rated in the “outstanding” and “exceeds fully successful” levels. To the extent that employees with such high ratings do not receive a monetary award, the perception that rewards are not directly linked to performance is reinforced. On the other hand, as the number of individuals receiving monetary awards increases, the average dollar award will be reduced, resulting in the perception that the awards are less motivating.

Since fiscal year 2010, the percentage of eligible SES receiving a performance award at each rating level has decreased. Figure 4 shows the percentage of SES receiving a performance award by rating for fiscal years 2010 through 2013.

To help assess how agencies are meeting the certification requirement to make distinctions in pay based on performance, OPM uses Pearson correlation coefficients as a metric to analyze the strength of the relationship between executives’ pay adjustments and performance awards and their ratings. Correlation coefficients measure the linear association between SES performance pay and senior executives’ ratings, and the value of the coefficient will be lower if the actual relationship between the two is not a straight line. For this reason, differences between years or agencies in the value of the correlation coefficient may not be meaningful. To meet the certification guidelines for pay differentiation, agencies are generally expected to have a correlation coefficient of 0.5 or greater; if the correlation coefficient is lower than 0.5, the guidelines state that an agency’s SES appraisal system can still receive full certification if pay and awards data show the system makes distinctions in pay. OPM reported that fiscal year 2013 correlations of SES ratings and performance pay ranged from .19 to .99 for the 24 CFO Act agencies: the National Aeronautics and Space Administration had the lowest coefficient and the Nuclear Regulatory Commission had the highest. Although correlation coefficients are useful for measuring the strength of the linear relationship between ratings and performance pay, they do not measure whether there are meaningful distinctions in pay based on performance. For example, if an agency makes small distinctions in pay across different rating levels (such as 5 percent of salary for executives rated 4 and 5.05 percent of salary for executives rated 5), the value of the correlation coefficient may be high, even though the difference in pay between rating levels is not meaningful.

Selected Departments’ SES Performance Management Systems Are Becoming Standardized, but Disparities in Rating Distributions and Pay Remain for Fiscal Year 2013

Selected Departments’ SES Performance Systems Used or Were Converting to the New SES System

According to a 2012 OPM document on the new SES performance appraisal system, with a different SES system in each agency, inconsistency among the executive branch agencies was a problem because of different definitions for rating levels across government, a mix of four- and five-level rating systems, and variable application of rating levels in evaluating SES—which led to a disparity in the ratings distribution across government. OPM officials said that when the new SES performance appraisal system was first available in fiscal year 2012, seven agencies used the system because they were already closely aligned with it. Since then, OPM officials said that about 90 percent of agencies have started to use the new system.

As more agencies adopt the new system, the system’s intent of promoting greater consistency may also result in greater uniformity in the development of ratings with their link to compensation. As mentioned previously, under the new system, OPM stated that agencies will be able to rely upon a more consistent and uniform framework to communicate expectations and evaluate the performance of SES members. While promoting greater consistency, the new system was also intended to enhance clarity, transferability, equity in the development of performance requirements, and the delivery of feedback. In addition, OPM noted that SES mobility is complicated by inconsistency—for executives moving between agencies or considering moving, there has been uncertainty regarding performance evaluations. The new SES appraisal system was intended to help address all of these areas.

Of the five systems we reviewed—DOD, Energy, HHS, DOJ, and Treasury— DOD, Energy, and HHS used the new SES performance appraisal system. DOJ and Treasury were in the process of converting to the new system. For example, a human capital official from Treasury said the department is transitioning toward using government-wide performance requirements; the official said Treasury’s fiscal year 2013 system had three critical elements rather than the five in OPM’s standard version. Four out of the five departments had automated SES appraisal systems or had plans to convert to one within the next fiscal year; DOJ did not. In addition, all of the five selected departments had performance appraisal systems that were certified by OPM and OMB. DOD, Energy, and DOJ had full certification; HHS and Treasury had provisional certification. As mentioned previously, for an agency’s SES performance appraisal system to be certified, agencies’ appraisal systems must meet criteria including that there is alignment between organizational and individual performance and that distinctions in pay are being made based on performance.

The selected departments were all on a fiscal year appraisal period with ratings decisions for the previous fiscal year made in the first few months of the next fiscal year. Department officials told us that the process of issuing ratings and making awards decisions happens in a fairly short timeframe.

Selected Departments Aligned Organizational and Individual SES Performance in the Performance Plans Provided to OPM for Certification

We previously identified the alignment of individual performance expectations with organizational goals as a key practice for an effective performance management system. It is important for individuals to see a connection between their daily operations and results to help them understand how individual performance can contribute to organizational success. Leading organizations have recognized that effective performance management systems create a “line of sight” showing how unit and individual performance can contribute to overall organizational goals and can help them drive internal change and achieve external results.

The five selected departments linked individual metrics or competencies to either component- or department-wide goals in the performance plans provided by the departments. The plans included specific links between individual SES competencies or responsibilities and a specific organizational goal. Additionally, OPM’s certification criteria require that agencies align SES individual performance expectations with organizational goals, and OPM has issued relevant guidance.

Agencies must establish one or more SES Performance Review Boards (PRB), which is a higher level of review within the SES performance management system. The PRBs review and evaluate the senior executive’s initial summary rating and, if applicable, the executive’s response and a higher level official’s comments on the initial rating. The boards also make written recommendations to the appointing official on annual summary ratings and performance awards. PRBs serve to ensure consistency, stability, and objectivity in performance appraisals. Boards also take into account organizational performance when making recommendations. OPM guidance states that, for agencies seeking access to higher levels of pay through certification, PRBs are to ensure meaningful distinctions in executive performance and that pay increases and performance awards are made based on individual and organizational performance. When appraising a career appointee’s performance or recommending a career appointee for a performance award, more than one-half of the PRB’s members must be SES career appointees.

The selected departments varied somewhat in their PRB structures as well as in who provided the final approval of the appraisal decisions; some departments have additional steps or guidance as part of their processes. For example, in addition to the PRB that evaluates ratings, Treasury has a front-end PRB that meets to discuss SES employees’ commitments early in the performance planning cycle. A Treasury official said this helps to ensure consistency in the performance plans as well as the rating and award. Once PRBs have reviewed ratings and awards, they make recommendations to the appointing authority—such as the head of an agency—for final ratings and awards decisions. PRB representatives we interviewed said that awards decisions are based on ratings within individual pay pools.

Although the selected departments linked SES performance plans with agency strategic objectives, the departments varied in their requirements that the PRBs compare the performance ratings to the outcomes of department goals and objectives. Most of the selected departments’ PRB representatives said that the PRB explicitly helps to ensure that there is alignment between individual performance and organizational performance by having PRBs consider the organizational assessment— an assessment of the agency’s overall performance—when reviewing proposed ratings and performance awards. For example, Energy provides the PRB with the organizational assessment, as well as guidance on how to use the assessment when reviewing ratings. An Energy official said that both rating officials and PRBs consider the organization’s performance when determining senior executives’ ratings. However, a Treasury official told us that the PRB for some bureaus within Treasury does not have access to an organizational assessment when reviewing ratings. The official said that tracking and reviewing organizational performance is done by the final rating official.

Consistent with Government-wide Data, Selected Departments Rated SES Primarily in the Top Two Categories for Fiscal Year 2013

All five departments rated the majority of SES in the top two categories, indicating little differentiation between executives in their ratings. As figure 5 shows, the five selected departments gave outstanding ratings to executives ranging from 30.6 percent in DOD to 73.6 percent in DOJ.

In 2008, we reported that senior executives for fiscal year 2007 were concentrated in the top two rating levels. As figure 6 shows, at the three departments (DOD, Energy, and Treasury) that we looked at in both fiscal years 2007 and 2013, the proportions of ratings were nearly the same, although Energy had switched from a four-rating system to a five-rating system during that time.

Agency officials offered varied explanations for the high concentration of performance ratings at the top two rating levels, ranging from stating that the ratings are justified to stating that the ratings may be too high, but they are reinforced by an agency culture where executives may not view a rating of 3 as acknowledgement of a fully successful performance. For example, human capital officials at DOJ pointed out that the individuals chosen for the SES are already high performers and continue to perform well as SES, earning high ratings. An HHS human capital official, however, noted that competencies are written so that a rating level of 3 represents a fully successful performance, but it is difficult to convince executives who have traditionally received higher ratings that this rating reflects successful performance. Similarly, an Energy official noted that the department communicates the message to rating officials (both verbally and in writing) that a “fully successful” rating is not average or ordinary; it demonstrates a significant level of accomplishment. One of the purposes of the new SES appraisal system is to help ensure standardized ratings with the understanding that a rating level of “outstanding” should always be a difficult goal to reach.

Four of Five Selected Departments Awarded the Same or Higher Performance Awards to SES with Lower Ratings

The departments had several layers of review, including both component- level and department-level review to ensure consistency across the department. For example, in addition to the PRBs, DOJ has an additional review level (the Senior Executive Resources Board) that analyzes the performance awards and attempts to identify trends and anomalies. However, for fiscal year 2013, four out of five selected departments awarded the same or higher performance awards as a percentage of base salary to SES with lower ratings as was awarded to those SES with higher ratings. While OPM has certified that the selected departments’ appraisal systems make meaningful distinctions based on relative performance, actual awards at some departments do not seem to support that meaningful distinctions are being made. Figure 7 shows the range of performance awards given to eligible SES at the five selected departments for fiscal year 2013.

PRB representatives from the selected departments indicated that the variation in performance awards by ratings was caused by a number of different factors. For example, a former PRB Chair said that in fiscal year 2012, they had 9 different pay pools within certain DOD non-combat entities; each was given the latitude to determine how to distribute performance awards within the pay pool. Although the distribution of performance awards across ratings looked inconsistent when aggregated, no one with a lower rating received a larger award than anyone with a higher rating in the same pay pool. A Treasury PRB representative said the range of performance awards is based on ratings as well as several other variables, such as relative contributions to the organization. The Treasury PRB has the flexibility to review those factors when determining performance award amounts, and this sometimes results in an SES with a lower rating getting an equal (or larger) performance award as a percentage of salary than an SES with a higher rating. The PRB representative from DOJ said components were forced to prioritize; when a large percentage of executives are outstanding and only 55 percent can receive a performance award, there are some difficult decisions. DOJ also noted that different executives may get performance awards from year to year, based on their contributions to overall mission achievement.

OPM Needs to Ensure That Its New SES Appraisal System Is Fulfilling Its Intended Purposes

One of the primary purposes for establishing the new SES appraisal system included increasing equity in ratings across agencies and their link to compensation. The new system provides for the uniform administration of SES executive branch performance management systems by promoting consistency, clarity, and transferability of performance standards and ratings across agencies. Additionally, effective performance management systems recognize that merit-based pay increases should make meaningful distinctions in relative performance; this principle is central to the SES performance management system, where under the law, to be certified and thereby able to access the higher levels of pay, the appraisal system must make meaningful distinctions based on relative performance. OPM’s guidelines state that the modal rating should be below “outstanding” and that multiple rating levels should be used. However, OPM’s guidelines also state that if an agency’s modal rating level is “outstanding,” the appraisal system can still be certified if accompanied with a full, acceptable justification. Nonetheless, the continued concentration of senior executives at the top two rating levels indicates that this principle is not being met across government.

While making meaningful distinctions in SES performance continues to be a challenge for many agencies, some others have made progress. For example, our 2008 report noted that according to a DOD official, DOD was communicating the message that the SES performance-based pay system recalibrates performance appraisals as a way to help change the culture and to make meaningful distinctions in performance, with a “fully successful” or equivalent rating as a high standard as well as a valued and quality rating. According to DOD, levels above “fully successful” require extraordinary results. Of the five selected departments that we examined in fiscal year 2013, DOD had the lowest percentage of senior executives receiving the highest rating—almost 31 percent.

According to OPM officials, in 2015 OPM plans to convene a cross- agency working group that is to revisit the SES certification process. As part of this effort, it will be important for OPM and the working group to consider whether—given the continued high SES performance ratings— the new system is contributing to making meaningful distinctions in performance ratings and awards, and if not, what refinements are needed. The goal of having a uniform system would appear compromised if an “outstanding” rating in one agency does not have the same meaning in another agency. Some options might include revisiting and perhaps eliminating the guideline that allows OPM to certify agencies’ performance management systems with an SES modal rating of “outstanding,” so long as the agency provides acceptable justification. This guideline could serve to work against encouraging agencies to make meaningful distinctions in SES performance. Alternatively, enhancing the transparency of OPM’s approval of agencies’ justification for a modal rating of “outstanding” could shed light on whether the individual agency’s high ratings seem justified. These justifications are not on OPM’s website and OPM does not report them to Congress, nor does the Chief Human Capital Officers Council review them for any consistency.

Conclusions

Understandably, it could take more time for OPM’s 2012 efforts to standardize SES performance management to fully materialize. However, data for fiscal year 2013 (both government-wide as well as at our case study departments) showed that a large majority of SES employees are still receiving one of the top two ratings. Coupled with evidence of overlap in performance awards across rating levels, this indicates that the link between performance ratings and awards is not being consistently applied.

By convening a cross-agency working group to review the SES certification process, OPM is in a position to evaluate whether the new SES appraisal system actually helps agencies use SES compensation and performance awards in ways that are cost effective and lead to increased employee performance and organizational results. If the performance definitions cannot be consistently applied across the government, creating a uniform framework to communicate expectations and evaluate the performance of SES members will be difficult to attain.

Recommendation for Executive Action

As OPM convenes the cross-agency working group, we recommend that the Director of OPM, as the head of the agency that certifies—with OMB concurrence—SES performance appraisal systems, consider the need for refinements to the performance certification guidelines addressing distinctions in performance and pay differentiation. Options could include

Revisiting and perhaps eliminating the guideline that allows OPM to certify agencies’ performance management systems with an SES modal rating of “outstanding,” or

Strengthening the accountability and transparency of this guideline by

Reporting agencies’ justifications for high ratings to OPM on its website.

Reporting agencies’ justifications for high ratings to Congress.

Obtaining third party input on agencies’ justifications for high ratings, such as by the Chief Human Capital Officers Council.

Agency Comments and Our Evaluation

We provided a draft of this report to the Director of OPM and to the Acting Secretary of Defense, the Secretary of Energy, the Secretary of Health and Human Services, the Assistant Attorney General for Administration at the Department of Justice, and the Secretary of the Treasury for review and comment. OPM’s comments are reprinted in appendix IV. OPM also provided technical comments, which we incorporated as appropriate. DOD, Energy, DOJ, and Treasury responded saying they did not have comments on the report. HHS did not respond to our request for comments.

In its written comments, OPM generally agreed with the information in the report but did not agree with our recommendation. In disagreeing with our recommendation to consider not certifying agencies with modal ratings of “outstanding,” OPM expressed concerns that imposing such a criterion would lead to arbitrary manipulation of the final ratings rather than an appropriate comparison of performance to standards. OPM asserted that this situation would be ripe for forced distribution of the ratings, which is explicitly prohibited by regulation. OPM also stated that the more appropriate action is to continue emphasizing the importance of setting appropriate, rigorous performance requirements and standards that logically support meaningful distinctions in performance.

As recognized in our report, OPM’s regulations contemplate that it is possible to apply standards that make meaningful performance distinctions and to use a range of ratings while avoiding the use of forced distributions. As we also note, since our 2008 report on SES performance management systems—continuing through the career SES performance ratings for fiscal year 2013—questions persist about the extent to which meaningful distinctions based on relative SES performance are being made. Although OPM has emphasized that an “outstanding” rating represents a level of rare, high-quality performance, it appears from examining fiscal year 2013 SES ratings data that some agencies are not appropriately applying these performance standards to their SES ratings. This undercuts one of the primary purposes for establishing the new SES appraisal system, which includes increasing equity in ratings across agencies and their link to compensation. Certifying agencies that are not adhering to the agreed-upon performance standards provides little incentive to those agencies that are adhering to the standards and could lead to ratings that are more skewed toward “outstanding.” As recently as 2008, OPM agreed on the importance of communicating to agencies the value of using a range of rating levels and transforming their cultures to those in which a “fully successful” rating is valued and rewarded.

OPM also did not support the second part of our recommendation regarding three suggestions for increasing transparency for those agencies that are certified with a modal rating of “outstanding.” Although we suggested that OPM report high rating justifications to Congress through its Annual Performance Report, we understand that this may not be the most appropriate vehicle to use; another avenue of reporting to Congress would certainly be acceptable, and we have adjusted the text accordingly. In addition, by suggesting that the Chief Human Capital Officers Council have input on agencies’ justifications for high ratings, we were in no way suggesting that this Council role impact OPM’s ultimate authority over the regulation and oversight of the SES performance appraisal system (including certification of agencies’ systems). We maintain, however, that—as an alternative action to more direct enforcement of the performance standards—transparency regarding OPM’s approval of justification for a modal rating of “outstanding” could shed light on whether the individual agency’s high ratings seem justified. Given the recent data on SES performance ratings and awards, we remain concerned that meaningful distinctions in relative SES performance are not being made in a uniform fashion.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretaries of Defense, Energy, Health and Human Services, and the Treasury, to the Assistant Attorney General of Administration at the Department of Justice, and to the Director of the U.S. Office of Personnel Management, as well as to the appropriate congressional committees and other interested parties. In addition, this report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-6806 or goldenkoffr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff members making key contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

This report examines the distribution of performance awards to career Senior Executive Service (SES) employees in executive branch agencies. The objectives of this report were to (1) describe key characteristics of the awards, such as rating and awards distributions, award amounts, and percentage of executives receiving awards, from fiscal years 2010 through 2013, and (2) describe and assess the extent to which selected departments’ SES performance appraisal systems factored in organizational and individual performance and made meaningful distinctions in their fiscal year 2013 performance awards. For an additional perspective, the second objective provides an in-depth view of five selected departments’ SES ratings and performance awards’ processes for the last fiscal year of ratings and award data.

For this report, we reviewed applicable legislation and regulations, as well as the Office of Personnel Management (OPM) and the Office of Management and Budget (OMB) guidance and government-wide reports, such as OPM’s annual reports for fiscal years 2010-2013, Report on Senior Executive Pay and Performance. We also interviewed officials at OPM and reviewed applicable reports from non-governmental organizations, such as the Senior Executive Association. We reviewed data from OPM on agency performance rating levels, as well as the number and amount of SES performance awards. We defined our universe of analysis as career senior executives who received ratings. We also excluded SES employees (where identifiable) from agency Inspector General Offices because their inclusion in the data was inconsistent. We reviewed OPM’s SES data for reasonableness and the presence of any obvious or potential errors in accuracy and completeness, and OPM officials confirmed the correctness of the data. On the basis of these procedures, we believe the data are sufficiently reliable for use in the analyses presented in this report.

To address our first objective, we reviewed data from OPM on the amount of SES performance awards given to career SES within the 24 Chief Financial Officers (CFO) Act agencies and other variables since fiscal year 2010 (the year prior to the limitation of performance award policies) through fiscal year 2013. We analyzed aggregate SES basic pay and performance ratings as provided by OPM for fiscal years 2010 through 2013. In calculating the percentage of eligible senior executives who received performance awards, we excluded executives who did not receive a performance rating.

To address the second objective, we selected five case study departments from the CFO Act agencies based on several criteria. Using 2012 Enterprise Human Resources Integration data, we identified departments that had the largest number of SES employees and varying performance award distributions, including departments with both a large and small percent of SES employees who received an award, and we identified departments that had both a small and large range of award amounts, as both dollars and percent of salary. Selected case studies included the Departments of Defense (DOD), Energy, Health and Human Services (HHS), Justice (DOJ), and Treasury. We reviewed these departments’ SES performance appraisal systems and their most recent certification documentation submitted to OPM. We also examined whether individual SES performance metrics were linked to agency performance and whether performance awards were tied to outcomes by looking at examples of individual SES performance appraisals. We did not verify the validity of the measures used or the performance of the senior executives. We also interviewed agency officials from the selected departments with responsibility for their department’s SES program as well as selected members or representatives of the Performance Review Boards charged with ensuring consistency, stability, and objectivity in SES performance appraisals.

Additionally, we analyzed OPM data for each case study department to identify the percentage of ratings distributed across each rating category in fiscal year 2013. We compared results of three of the case study departments that were also reviewed in 2007, using data from a previous report. We also analyzed the data to determine the amount of performance awards—as a percentage of salary—given to SES employees for each performance rating level.

We conducted this performance audit from June 2014 to December 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Office of Personnel Management (OPM) Certification Report Form

OPM Report on the (Agency Name)

Senior Executive Service (SES) Performance Appraisal System Certification Date of Report: MM/DD/YYYY OPM (Enter Decision Here) certification of the (Agency Name) SES performance appraisal system. The table below indicates the certification criteria fully met, minimally met, or not met during OPM’s review of the system.

Description of “meets” criteria The agency is using the basic SES appraisal system description and it has been approved by OPM. (This means the system includes all the required system language, and the performance plans include all the required language for Consultation, Employee Perspective, Customer Perspective, and Accountability.)

All performance plans include specific organizational goals in the Strategic Alignment cell for each performance requirement under the Results Driven element in Part 5 of the appraisal form.

Each performance requirement under the Results Driven element for each performance plan contains adequate measurable results. If the agency also includes a list of activities in the plans, the plan clearly identifies the measurable result(s) and denotes that it is the results that are to be rated for the Results Driven element.

Documentation required/criteria addressed Organizational Assessment and Guidelines

Description of “meets” criteria The agency has provided a copy of its memo issuing guidelines to executives, rating officials and PRB members that includes the results of the organizational assessment, and guidelines for how to use the results when determining ratings, pay, and awards; OR, for small agencies, a description for how the guidelines and results of the organizational assessment were given to rating officials and PRB members and the content of the guidelines (e.g., if communicated by email, a copy of the email is included).

The agency has provided a description of training or communications given that should include a briefing on the agency’s SES performance management system, OR the agency provides evidence it has conducted the training, but cannot verify how many executives attended the training, OR the system has not yet been implemented and the agency provides training plans; AND communication of the average rating, pay, and awards given the previous year.

The SES rating distribution indicates the agency is clearly making distinctions in performance (i.e. the modal rating is below “outstanding”) AND the distribution appears to reflect organizational performance as explained by the agency, OR, the modal rating is “outstanding” but the agency gave a full, acceptable justification; AND the percent of SES members not rated is less than 5 percent.

Differentiation in Pay Based on Performance a) The agency's correlation coefficient of the rating and performance compensation (that is, pay adjustments and awards) is 0.500 or more, OR the pay and awards data show the agency makes distinctions in pay b) AND the average performance compensation is higher for executives rated Outstanding than for those rated Exceeds, and Exceeds is higher than Fully Successful; c) AND the data does not include any violations of pay and awards limits.

Documentation required/criteria addressed

Description of “meets” criteria Pay Policy The agency provides a written, official pay policy and the policy describes clear differentiations in performance compensation (that is, pay adjustments and awards) based on the annual summary rating.

Appendix III: Career Senior Executive Service (SES) Ratings and Performance Awards at the 24 Chief Financial Officers (CFO) Act Agencies for Fiscal Year 2013

Percent of career SES at each rating level

Office of Personnel Management (OPM), Small Business Administration (SBA), and Social Security Administration (SSA).

Appendix IV: Comments from the Office of Personnel Management

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Robert Goldenkoff, (202) 512-6806 or goldenkoffr@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Thomas Gilbert, Assistant Director, and Judith Kordahl, Analyst-in-Charge, supervised the development of this report. Charles Culverwell and Mary Diop made significant contributions to all aspects of this report. Other important contributors included Sara Daleski, Karin Fangman, Donna Miller, and Rebecca Shea.