Results Oriented Management: Opportunities Exist for Refining the Oversight and Implementation of the Senior Executive Performance-Based Pay System
Published: 2008-11-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Agencies are allowed to raise pay caps for their Senior Executive Service (SES) members if the Office of Personnel Management (OPM) certifies and the Office of Management and Budget (OMB) concurs that their appraisal systems meet applicable criteria. As requested, this report examines selected agencies' policies and procedures for (1) factoring organizational performance into SES appraisal decisions, (2) making meaningful distinctions in SES performance and (3) building safeguards into SES systems. Also, this report examines OPM and OMB oversight in certifying the pay systems through their statutory roles. GAO selected six agencies based on mission, structure, and number of career SES variations. GAO analyzed the agencies' policies and fiscal year 2007 aggregate SES appraisal data and OPM guidance.
section_title: What GAO Found
paragraphs: All of the selected agencies--the U.S. Departments of Defense, Energy, State, and the Treasury; U.S. Nuclear Regulatory Commission; and USAID--have policies in place that require senior executives' performance expectations to be aligned with organizational results and organizational performance to be factored into appraisal decisions. While almost all of the agencies provided and communicated the importance of considering organizational performance, USAID did not provide its performance review board members (PRB) and other reviewing officials with any specific information on organizational performance to help inform their executive appraisal recommendations. All of the selected agencies have multiple rating levels in place for assessing senior executive performance. For the fiscal year 2007 appraisal cycle, senior executives were concentrated at the top two rating levels, which raises questions about the extent to which meaningful distinctions based on relative performance are being made and how OPM applies this criterion. OPM has an opportunity to strengthen its communication with agencies and executives on the importance of using a range of rating levels when assessing performance while avoiding the use of forced distributions. All of the selected agencies have safeguards, including higher level reviews of performance appraisal recommendations, PRBs, and transparency in communicating the aggregate results, although agencies varied in how they implemented such safeguards. While generally satisfied with OPM's and OMB's oversight, officials at the selected agencies said OPM could strengthen its communication with agencies and executives on how it uses the SES performance appraisal data and correlation between ratings and performance pay in determining whether agencies are making meaningful distinctions based on relative performance. Further communication from OPM is important in order for agencies to have a better understanding of how they are being held accountable for these certification criteria and make the necessary improvements to their systems to maintain certification. Further, senior-level officials at the selected agencies suggested options--such as moving to an electronic submission process and lengthening the certification coverage beyond 2 years once their systems are operating at the fully certified level--to increase the efficiency of the process. Moving forward, it will be important for OPM and OMB to identify ways to improve the certification process and make it more streamlined while ensuring that agencies have the guidance, tools, and training they need to implement effective performance appraisal and pay systems for their senior executives.

section_title: Letter
paragraphs: To successfully respond to the array of complex challenges facing the  nation—including long-term fiscal imbalance, evolving national and  homeland security threats, and turmoil in global financial markets—the  federal government must build the capacity to plan more strategically,  react more expeditiously, and focus on achieving results. Strategic human  capital management is the centerpiece of federal agencies’ efforts to  transform to meet these challenges. Senior executives need to lead the  way in transforming their agencies to become more results-oriented,  collaborative in nature, and customer-focused. In 2003, Congress and the  administration modernized the performance appraisal and pay systems for  members of the Senior Executive Service (SES) by requiring a clearer link  between individual performance and pay. Specifically, agencies are  allowed to raise SES basic pay and total compensation caps if the Office of  Personnel Management (OPM) certifies, with the concurrence of the  Office of Management and Budget (OMB), that their performance  appraisal systems, among other things, link performance for senior  executives to the organization’s goals and make meaningful distinctions  based on relative performance.
In our past work on performance management and pay issues, we have  reported that performance-based pay cannot be simply overlaid on most  organizations’ existing performance management systems. Rather, as a  precondition to effective pay reform, individual expectations must be  clearly aligned with organizational results, communication on individual  contributions to annual goals must be ongoing and two-way, meaningful  distinctions in employee performance must be made, and cultural changes  must be undertaken. Most important, leading organizations have  recognized that effective performance management systems create a “line  of sight” showing how unit and individual performance can contribute to  overall organizational goals and can help them drive internal change and  achieve external results. Effective performance management systems that  hold executives accountable for results can help provide continuity during  times of leadership transition, such as the upcoming change in  administration, by maintaining a consistent focus on organizational  priorities.
OPM’s recently released governmentwide SES survey results found that  senior executives across the government recognize the importance of  linking pay to performance, with about 93 percent of the respondents  strongly agreeing or agreeing that pay should be based on performance. In  addition, the majority of senior executives reported that their salary  increases and bonuses were linked to their performance ratings to a very  great or great extent. However, senior executives recognized the challenge  of making meaningful distinctions in performance—a key criterion for  agencies’ certification of their SES appraisal systems. Specifically, less  than a third of senior executives governmentwide strongly agreed or  agreed that bonuses or pay distinctions were meaningfully different among  executives.
At your request, this report examines selected agencies’ policies and  procedures for their career SES performance appraisal and pay systems in  three key areas: (1) factoring organizational performance into senior  executive performance appraisal decisions, (2) making meaningful  distinctions in senior executive performance, and (3) building safeguards  into senior executive performance appraisal and pay systems. Collectively,  these three areas cover half of the nine criteria that are critical to  appraising and rewarding senior executive performance that agencies  must meet to be certified by OPM and OMB. In addition, this report  examines how OPM and OMB are providing oversight to the certification  of the senior executive performance-based pay system through their  statutory roles. In July 2008, we testified before the Senate Subcommittee  on Oversight of Government Management, the Federal Workforce, and the  District of Columbia on our preliminary results of the selected agencies’  policies and procedures for their SES performance-based pay systems.  This report supplements the information provided in our July 2008  testimony.
For this review, we selected the U.S. Departments of Defense (DOD),  Energy (DOE), State, and the Treasury; the U.S. Nuclear Regulatory  Commission (NRC); and the United States Agency for International  Development (USAID) based on variations in agency mission;  organizational structure; size of their career SES workforces to reflect  agencies with a large, average, and small number of executives; and results  of their SES performance appraisal systems in terms of the percentage of  SES rated at the highest rating levels and the percentage that received  performance awards or bonuses from fiscal years 2004 to 2006, according  to OPM’s governmentwide data reports. To meet our objectives, we  analyzed these agencies’ SES performance management documents;  analyzed OPM’s guidance; interviewed cognizant senior-level agency  officials regarding their SES performance-based pay systems and their  views on OPM’s and OMB’s oversight of the certification process;  interviewed senior-level OPM and OMB officials who oversee the  certification review process on their oversight of the SES performance- based pay system and certification process; and analyzed aggregate SES  performance appraisal and compensation data as provided by the agencies  and comparable governmentwide data as reported by OPM for fiscal year  2007. Appendix I contains a detailed discussion of our objectives, scope,  and methodology.
We checked the agency SES performance appraisal data for  reasonableness and the presence of any obvious or potential errors in  accuracy and completeness. We also reviewed related agency  documentation, interviewed agency officials knowledgeable about the  data, and brought to the attention of these officials any concerns or  discrepancies we found with the data for correction or updating. The  agency officials confirmed the correctness of the data or in some cases  provided corrections to the data, which we used in our analysis. On the  basis of these procedures, we believe the data are sufficiently reliable for  use in the analyses presented in this report.
The examples of the selected agencies’ policies and procedures for their  SES performance-based pay systems are not generalizable to the  governmentwide SES population and all executive branch agencies. We  did not assess how the selected agencies are implementing all the policies  and procedures for their SES performance-based pay systems. We  recognize that focusing on 1 year of appraisal data may not show the  complete picture of how agencies are making meaningful distinctions in  senior executive performance. An agency may have implemented a policy  related to the study objectives even if it is not specifically highlighted in  this report. We conducted our work from October 2007 through November  2008 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Since 2000, Congress and OPM have gradually shifted to performance- based pay for senior executives through legislative and regulatory  changes.
In October 2000, OPM amended its senior executive performance  management regulations requiring agencies to (1) hold senior  executives accountable for their individual and organizational  performance by linking performance management with the results- oriented goals of the Government Performance and Results Act of  1993; (2) evaluate senior executive performance using measures that  balance organizational results with customer satisfaction, employee  perspectives, and any other measures agencies decide are appropriate;  and (3) use performance results as a basis for pay, awards, and other  personnel decisions. While emphasizing the use of performance  results as the basis for pay and other awards, members of the SES still  received the annual across-the-board and locality pay adjustments.
In 2002, Congress raised the total annual compensation limit—from  Executive Schedule (EX) level I to the total annual compensation  payable to the Vice President—for senior executives and other senior  professionals in agencies with systems that have been certified by  OPM with OMB concurrence as having performance appraisal systems  which as designed and applied make meaningful distinctions based on  relative performance. The act instructed OPM and OMB to promulgate  regulations regarding certification that, if met by an agency, would  allow it to access the higher total compensation cap, which includes  bonuses and other forms of compensation.
In 2003, Congress changed the basis for how agencies pay their senior  executives and the overall SES pay structure. Beginning in January  2004, senior executives no longer received annual across-the-board or  locality pay adjustments. Agencies are to base pay adjustments for  senior executives on individual performance and contributions to the  agency’s performance by considering the individual’s accomplishments  and such things as unique skills, qualifications, or competencies of the  individual and the individual’s significance to the agency’s mission and  performance, as well as the individual’s current responsibilities. In  addition, the SES pay structure changed from six pay levels to a single,  open-range pay band with a higher basic pay cap—EX-level III for  agencies without certified appraisal systems and EX-level II for  agencies with such systems. For calendar year 2008, the pay caps are  $158,500 for basic pay (EX-level III) with a senior executive’s total  compensation not to exceed $191,300 (EX-level I). If an agency’s senior  executive performance appraisal system is certified by OPM and OMB  concurs, the caps are increased to $172,200 for basic pay (EX-level II)  and $221,100 for total compensation (the total annual compensation  payable to the Vice President).
To qualify for senior executive pay flexibilities, agencies’ performance  appraisal systems are evaluated against nine certification criteria and  any additional information that OPM and OMB may require to make  determinations regarding certification. OPM’s and OMB’s certification  criteria are broad principles that position agencies to use their pay  systems strategically to support the development of a stronger  performance culture and the attainment of their mission, goals, and  objectives. (See app. II for additional information on the certification  criteria). Two levels of performance appraisal system certification are  available to agencies—full and provisional. Through a law passed in  October 2008, an agency’s certification now lasts for up to 24 months,  with the possibility of a 6-month extension by the OPM Director, rather  than a calendar-year-based coverage. Previously, an agency’s  certification lasted for 2 calendar years for full certification and 1  calendar year for provisional certification.
In addition to SES employees, many agencies use senior employees  with scientific, technical, and professional expertise, commonly  known as senior-level (SL) and scientific or professional (ST)  positions. An agency may apply to OPM and OMB for certification of  its SL/ST performance management system, and if its system is  certified as making meaningful distinctions in relative performance, an  agency may raise the total annual compensation maximum for SL/ST  employees to the salary of the Vice President. Beginning in April 2009,  the recently passed law allows certified agencies to raise the basic pay  cap for SL/ST employees to EX-level II—the same maximum rate of  basic pay as SES members, and also exempts SL/ST employees from  receiving locality pay. Previously, SL/ST employees under certified  appraisal systems had a maximum rate of basic pay equal to EX-level  IV plus locality pay up to EX-level III. However, unlike the SES, their  individual rate of pay does not necessarily have to be based on  individual or agency performance.
OPM has a key leadership and oversight role in the design and  implementation of agencies’ SES performance-based pay systems by  certifying that the agencies’ systems meet the certification criteria  before they can receive the pay flexibilities. In our January 2007  report examining the senior executive performance-based pay system,  we made a series of recommendations to OPM designed to address  issues specific to the performance-based pay system, such as sharing  best practices, tracking progress towards goals, and developing a  timeline for issuance of certification guidance. We are following up on  the status of these recommendations through this report.
section_title: Selected Agencies Are Generally Addressing Key Areas Related to the Certification Criteria through Their SES Performance- Based Pay Systems, but an Opportunity for Refinement Exists
paragraphs: The selected agencies are generally addressing three key areas related to  OPM’s and OMB’s certification criteria through their SES performance- based pay systems— factoring organizational performance into senior  executive performance appraisal systems, making meaningful distinctions  in senior executive performance, and building safeguards into senior  executive performance appraisal and pay systems. However, USAID did  not provide its PRB members and other reviewing officials with any  specific information on organizational performance to help inform their  senior executive appraisal recommendations.
subsections: section_title: Selected Agencies Have Policies for Factoring Organizational Performance into Appraisal Decisions, but USAID Did Not Provide Organizational Performance Information to Reviewing Officials
paragraphs: In our past work on performance management, we identified the  alignment of individual performance expectations with organizational  goals as a key practice for effective performance management systems.  Having a performance management system that creates a “line of sight”  showing how unit and individual performance can contribute to overall  organizational goals helps individuals understand the connection between  their daily activities and the organization’s success. To receive  certification of their systems, agencies are to align senior executive  performance expectations with the agency’s mission, strategic goals,  program and policy objectives, or annual performance plan and budget  priorities. While many agencies are doing a good job overall of aligning  executive performance plans with agency mission and goals, according to  OPM some of the plans do not fully identify the measures used to  determine whether the executive is achieving the necessary results, which  can affect the executive’s overall performance appraisal. This challenge of  explicitly linking senior executive expectations to results-oriented  organizational goals is consistent with findings from our past work on  performance management.
To help hold senior executives accountable for organizational results,  beginning in 2007, OPM required agencies to demonstrate that at least 60  percent of each senior executive’s performance plan is focused on  achieving results and has clear measures associated with those results to  show whether the goals have been achieved to be certified. The selected  agencies have designed their appraisal systems to address OPM’s  requirement of aligning individual expectations with organizational goals.  For example, in setting expectations for individual performance plans,  DOE requires senior executives and supervisors to identify three to five  key performance requirements with metrics that the executive must  accomplish in order for the agency to achieve its strategic goals. Weighted  at 60 percent of the summary rating, the performance requirements are to  be specific to the executive’s position and described in terms of specific  results with clear, credible measures (e.g., quality, quantity, timeliness,  cost-effectiveness) of performance, rather than activities. For each  performance requirement, the executive is to identify the applicable  strategic goal in the performance plan. To ensure that agencies are  implementing their policies for alignment of performance expectations  with organizational goals, OPM requires agencies as part of their  certification submissions to provide a sample of executive performance  plans, the strategic plan or other organizational performance documents  for establishing alignment, and a description of the appraisal system  outlining the linkage of executive performance with organizational goals.
Further, OPM requires agencies to factor organizational performance into  senior executive performance appraisals to receive certification of their  SES appraisal systems. According to OPM and OMB officials overseeing  the certification review process, the main sources of organizational  performance that agencies use are the performance and accountability  reports (PAR); program assessment rating tool (PART) summaries, which  capture agencywide as well as program- or office-specific performance;  and the President’s Management Agenda (PMA) scorecards, as applicable.  However, agencies have the flexibility to determine the format and type of  organizational performance information for the performance appraisal  process and certification submissions, according to OMB’s lead official for  the certification review process.
All of the selected agencies have policies in place for factoring  organizational performance into senior executive appraisal decisions and  have identified common organizational assessments—such as the PMA,  PAR, or PART results—for highlighting organizational performance  results. As a next step, a few of the agencies, such as NRC and Treasury,  have developed customized tools summarizing organizational performance  at different levels of the organization, such as the bureau, office, or  program levels to help ensure that senior executive appraisal decisions are  consistent with organizational performance. For example, NRC provides  summary reports capturing office-level performance to rating and  reviewing officials to ensure that these officials have the information they  need to make consistent assessments between senior executive and  organizational performance. At the midpoint and end of the appraisal  cycle, NRC’s senior performance officials (SPO)—two top-level executives  responsible for assessing organizational performance—conduct  assessments for each office that take into account quarterly office  performance reports on their operating plans, an interoffice survey on the  office’s performance completed by the other directors as identified by  NRC, as well as the office director’s self-assessment of the office’s  performance.
To assess bureau-level performance, Treasury uses a departmentwide  organizational assessment tool that provides a “snapshot” of each bureau’s  performance across various indicators of organizational performance,  such as the PAR, PART results, PMA areas, OPM’s Federal Human Capital  Survey results, budget data, and information on material weaknesses. PRB  members and reviewing officials receive copies of the organizational  performance assessments, which serve as a basic framework for reviewing  and recommending senior executive ratings, pay, and bonuses to help  ensure ratings and pay are consistent with the organization’s performance.  According to Treasury’s Deputy Assistant Secretary for Human Resources  and Chief Human Capital Officer (CHCO), the indicators of organizational  performance are updated throughout the year as organizational  performance is always changing and the senior executives need to have a  sense of the organization’s performance at all times.
Prior to the completion of individual performance ratings, agencies are to  communicate organizational performance to senior executives, PRB  members, and other reviewing officials—including supervisors who  complete the ratings—involved in appraisal decisions to ensure they  understand the effect organizational performance can have on rating  distributions. Almost all of the selected agencies provided organizational  performance assessments and communicated the importance of  considering organizational performance in individual appraisals through  briefings, training, or document packages for the PRB meetings. One  agency, however, did not provide any specific information regarding  organizational performance to PRB members and other reviewing officials.
DOD provided the heads of its components with a departmentwide  organizational assessment against its overall priorities for fiscal year  2007 that was to be used in appraising senior executive performance  and, as a check across the components, asked for copies of the  training given to PRB members and other reviewing officials on  factoring organizational performance into senior executive appraisal  recommendations. According to the Principal Director to the Deputy  Under Secretary of Defense for Civilian Personnel Policy, the  components had the flexibility to use the departmentwide assessment  and to develop their own organizational assessments. Component  organizational assessments were required to be linked to the  departmentwide priorities and assessment. Component organizational  assessments can provide a level of specificity that enables a clearer  connection or “line of sight” between individual executive and  organizational performance. Having the components provide the  department with their communications of organizational performance  and how it was used to inform executive rating decisions provides  accountability across the components for the departmental  performance management policies, according to this official.
DOE provides its PRB members with snapshots of the Consolidated  Quarterly Performance Reports relevant to the senior executives that  measure how each departmental element performed respective to the  goals and targets in its annual performance plan. According to the  Director of the Office of Human Capital Management, the Deputy  Secretary also verbally briefed PRB members on the importance of  considering organizational performance in appraising executive  performance.
For its most recently completed appraisal cycle, State for the first time  provided PRB members an organizational assessment composed of  various indicators from the most recent PART, PMA scorecard, and  PAR. For the previous appraisal cycle, PRB members received  various documents, such as senior executives’ performance plans and  appraisals and the performance management policy, but did not  receive any specific assessments of organizational performance.  According to a senior human resources official at State, based on  OPM’s and OMB’s feedback for its 2008 certification submission, the  agency has committed to providing organizational performance results  in its guidance to the PRB members on how to consider organizational  performance in making individual senior executive appraisal  recommendations, among other things.
In contrast, USAID did not provide its PRB members and other  reviewing officials with any specific information on organizational  performance to help inform their senior executive appraisal  recommendations for the fiscal year 2007 appraisal cycle. According to  a senior human resources official at USAID, the agency does not  provide PRB members and reviewing officials with these  organizational performance assessments because they know where to  find the relevant information applicable for each senior executive’s  performance appraisal given the small size of the agency. Nevertheless,  providing and communicating uniform organizational performance  assessments can help ensure consistency and clarity in how  organizational performance is considered in appraising executive  performance among PRB members, rating officials, and other  reviewers. According to USAID’s Deputy Director for Human  Resources, USAID has developed various indicators of organizational  performance—such as individual operating unit reports, the Agency  Financial Report, the PMA, PART results, and the Congressional  Budget Justification outlining agency performance and other  information—which are readily available for use by PRB members and  other reviewing officials responsible for appraising senior executive  performance.
section_title: Selected Agencies Use Various Mechanisms to Help Make Meaningful Distinctions in Performance, but Senior Executives Were Concentrated at the Top Two Rating Levels and Received Varying Performance Payout Amounts
paragraphs: Effective performance management systems make meaningful distinctions  between acceptable and outstanding performance of individuals and  appropriately reward those who perform at the highest level. In order to  receive OPM certification and OMB concurrence, agencies are to design  and administer performance appraisal systems that make meaningful  distinctions based on relative performance through performance rating  and resulting performance payouts (e.g., bonuses and pay adjustments).  To address the certification criteria of performance and pay  differentiation, agencies are to use multiple rating levels—four or five  levels including a level for outstanding performance—and recognize the  highest performing executives with the highest ratings and largest pay  adjustments and bonuses, among other things.
subsections: section_title: Selected Agencies Designed Their Appraisal Systems to Help Make Meaningful Distinctions in Senior Executive Performance
paragraphs: Five of the selected agencies designed their appraisal systems to help  allow for differentiations when assessing and rewarding executive  performance by establishing tier structures or prescribed performance  payout ranges based on the resulting performance rating.
For example, NRC uses three tiers called position groups to  differentiate its senior executives’ basic pay and the resulting bonus  amounts based on ratings received at the end of the appraisal cycle.
NRC divides its executives into three groups (A, B, and C) based on the  position’s difficulty of assignment and the scope of responsibilities and  annually sets basic pay ceilings for each of the groups tied to the EX  pay levels. NRC uses the position groups and resulting performance  ratings as the basis for its bonus structure to help ensure that  executives in the higher position groups with the higher performance  ratings receive the larger bonuses, as shown in table 1. In fiscal year  2007, an executive in the highest position group A who received an  outstanding rating was to receive a $30,000 bonus, while an executive  in the lowest group C with the same rating was to receive a $20,000  bonus. According to a senior human resources official at NRC, the  bonus range for executives in group C with excellent ratings was  intended to help allow for meaningful distinctions in performance to  be made within that group, as well as to give the agency flexibility in  the monetary amounts of the bonuses awarded.
State uses a six-tier structure to help differentiate executive  performance based on the ratings and bonuses and allocate pay  adjustment amounts for its senior executives, with senior executives  who are placed in the highest tier (I) receiving a larger percentage pay  adjustment than those in a lower tier (V), who received the annual  percentage adjustment to the EX pay schedule—2.5 percent in 2008.
In 2008, DOD implemented a departmentwide tier structure to help  ensure comparability and transparency in SES position and  compensation management with pay ceilings for each of the tiers tied  to EX-level II and III pay rates. Specifically, DOD assigned SES  positions to three tiers based on the position’s impact on mission, level  of complexity, span of control, and influence in joint, national security  matters, among other things. According to the Principal Director, DOD  is now using the tier structure to differentiate executive performance  payouts to recognize that high-level performance in some positions has  more impact than comparable performance in other positions. Further,  DOD uses a mathematical formula to differentiate the performance  payout amounts among its senior executives based on the  recommended performance rating, performance score, and  performance payout shares, as shown in table 2. In determining the  number of performance payout shares to recommend, rating officials  are to consider areas such as the executive’s level of responsibility,  mission impact, current basic pay, and performance against the  relative performance of other executives, if applicable. The formula for  computing the actual amount of the performance payout takes into  account various indicators, such as the budget for bonuses and pay  increases, annual adjustment to the EX pay rates, and total salaries and  number of performance shares for all the senior executives in the pay  pool.
DOE sets prescribed ranges tied to performance ratings for its senior  executives prior to finalizing ratings to help create a greater distinction  between bonus amounts for top and middle performers and  differentiate pay adjustment caps. Specifically, for fiscal year 2007,  DOE required that all executives receiving an outstanding rating  receive a bonus of 12 to 20 percent of basic pay, while executives  receiving a meets expectations rating were eligible to receive a bonus  of 5 to 9 percent at management’s discretion. For pay adjustments,  executives were eligible to receive a discretionary increase of up to 5  or 7 percent of basic pay if rated at meets expectations or outstanding,  respectively. Executives who received needs improvement or  unsatisfactory ratings were not eligible for any bonuses or pay  increases.
section_title: Selected Agencies Rated Their Senior Executives at the Top Two Rating Levels with Varying Bonus and Pay Adjustment Amounts
paragraphs: We have reported that using multiple rating levels provides a useful  framework for making distinctions in performance by allowing an agency  to differentiate among individuals’ performance. As required for  certification, all of the selected agencies have four or five rating levels in  place for assessing senior executive performance. For the fiscal year 2007  appraisal cycle, senior executives were concentrated at the top two rating  levels, as shown in figure 1. At State and USAID, about 69 percent and 60  percent of senior executives, respectively, received the top performance  rating. At the other four agencies, the largest percentage of executives  received the second highest rating—ranging from about 65 percent at NRC  to 45 percent at Treasury. Conversely, less than 1 percent of senior  executives across the selected agencies received a rating below fully  successful (level 3). As a point of comparison, about 47 percent of career  SES governmentwide received the top performance rating for fiscal year  2007, according to governmentwide data as reported by OPM. Similar to  the selected agencies, less than 1 percent of career senior executives  governmentwide received ratings below fully successful for fiscal year  2007.
While OPM officials have certified that the selected agencies’ systems are  making meaningful distinctions, performance ratings at the selected  agencies raise questions about the extent to which meaningful distinctions  based on relative performance are being made and how OPM applies this  criterion, as indicated in figure 1. As part of making meaningful  distinctions in performance, OPM has emphasized to agencies through its  certification guidance that its regulations prohibit forced distribution of  performance ratings and that agencies must avoid policies or practices  that would lead to forced distributions or even the appearance of it. A  senior OPM official acknowledged that it is difficult for OPM to determine  if an agency is using forced distributions through its review of agencies’  aggregate appraisal results and policy documents. The official indicated  that OPM looks at trends in the data across different components of  agencies for statistical improbabilities, such as a certain percentage of SES  members receiving an outstanding rating each year within an office that  could be explained by a quota system. OPM has not provided specific  guidance to agencies on how to make meaningful distinctions in senior  executive performance while avoiding the perception of forced  distributions of performance ratings. OPM has an opportunity to  strengthen its communication with agencies and executives regarding the  importance of using a range of rating levels when assessing performance  while avoiding the use of forced distributions. Communicating this  information to agencies will help them begin to transform their cultures to  one where a fully successful rating is valued and rewarded.
Senior-level officials at three of the selected agencies recognized the  challenge in using a range of rating levels when appraising senior  executive performance. In a memo to all SES members, DOE’s Deputy  Secretary stated his concern with the negligible difference in bonuses and  pay adjustments among executives receiving the top two rating levels and  stressed the importance of making meaningful distinctions in the  allocation of compensation tied to performance ratings in the upcoming  appraisal cycle. According to State’s Deputy Assistant Secretary for the  Bureau of Human Resources, historically the vast majority of senior  executives have received the highest rating of outstanding, including for  fiscal year 2007. Since the implementation of performance-based pay, this  official said State has struggled with changing the culture and general  perception among senior executives that any rating less than outstanding  is a failure. According to DOD’s Principal Director, DOD is communicating  the message that the SES performance-based pay system recalibrates  performance appraisals as a way to help change the culture and make  meaningful distinctions in performance. A fully successful or equivalent  rating is a high standard as well as a valued and quality rating. Levels  above fully successful require extraordinary results. Part of this  communication is developing common benchmark descriptors for the  performance elements at the 5, 4, and 3 rating levels. The Principal  Director said she hopes that developing common definitions for the  performance elements at all three levels will aid the development of a  common understanding and in turn make more meaningful distinctions in  ratings. The agency official recognizes that this shift will require a  significant cultural change, and that such cultural transformation takes  time.
The percentage of eligible executives who received bonuses or pay  adjustments varied across the selected agencies for fiscal year 2007, as  shown in table 3. The percentage of eligible senior executives who  received bonuses ranged from about 92 percent at DOD to about 30  percent at USAID, with the average dollar amount of bonuses ranging from  $11,034 at State to about $17,917 at NRC. All eligible executives at State  received pay adjustments, while about 88 percent of eligible executives at  DOE received adjustments, with the average dollar amount of such  adjustments ranging from about $5,414 at NRC to about $6,243 at DOE. As  a point of comparison, about 75 percent of career senior executives  received bonuses with an average dollar amount of $14,221 for fiscal year  2007, according to OPM’s governmentwide data report. The  governmentwide percentage of career senior executives receiving pay  adjustments and the average dollar amount of such adjustments in the  aggregate are not available from OPM’s governmentwide data report for  fiscal year 2007.
The selected agencies have policies in place where only senior executives  who receive a rating of fully successful (level 3) or higher are eligible to  receive bonuses or pay increases. Also affecting executives’ bonus  eligibility are the agencies’ policies on awarding bonuses to executives  who also received Presidential Rank Awards that year, which varied  among the selected agencies. NRC, State, and Treasury do not allow  executives to receive both awards in the same year, while DOD, DOE, and  USAID allow the practice.
According to OPM regulations, agencies are to recognize the highest  performing executives with the highest ratings and largest bonuses and  pay adjustments. At five of the selected agencies, the highest performing  executives (rated at level 5) made up the greatest percentage of eligible  executives receiving bonuses. At NRC, all eligible executives rated at the  top two levels received a bonus. At all the agencies, the executives rated at  the highest level received the largest bonuses on average—about $23,333  at NRC compared to about $11,034 at State. State only awarded bonuses to  executives receiving outstanding ratings for fiscal year 2007. According to  State’s senior human resources official, State does not have an official  policy prohibiting those receiving ratings of exceeds expectations or fully  successful from receiving a bonus. Rather, the agency official stated that  State’s decision to only award bonuses to executives who received  outstanding ratings was due to budget constraints and an effort to keep  the SES parallel with the SFS in the allocation of bonuses and pay  adjustments. In addition, senior executives at NRC and USAID rated at  fully successful (level 3) did not receive bonuses. (see fig. 2).
In a memo to agencies on the certification process, OPM has stated that it  expects senior executives who receive a fully successful or higher rating  and are paid at a level consistent with their current responsibilities will  receive a performance-based pay increase. According to a senior OPM  official, agencies are not required to give these executives pay increases,  but OPM considers fully successful to be a good rating and encourages  agencies to recognize and reward executives performing at this level. At  the selected agencies, the majority of eligible senior executives rated at  fully successful received pay adjustments for fiscal year 2007, as shown in  figure 3.
The highest-performing executives (rated at level 5) did not make up the  greatest percentage of executives receiving pay adjustments with the  largest increases on average at some of the selected agencies. Specifically,  at Treasury, about 95 percent of eligible executives rated at level 4  received a pay adjustment, compared with about 91 percent of eligible  executives rated at level 5 and about 90 percent rated a level 3. At NRC, all  of the eligible executives rated at level 5 and level 3 received pay  adjustments compared with about 92 percent of eligible executives rated  at level 4. For all the agencies except Treasury, the executives rated at  the highest level received the largest pay adjustments on average—about  $7,473 at USAID compared to about $6,133 at NRC. At Treasury,  executives rated at levels 5, 4, and 3 on average received about the same  pay adjustment amounts, primarily due to pay cap issues.
We have reported that the federal government as a whole may face  challenges in offering competitive compensation to its senior leaders. In  2003, about 70 percent of senior executives received the same basic pay  due to compression—which occurred when their pay reached the  statutory cap of EX-level III. In 2004, the SES performance-based pay  system and certification process provided an interim solution to this issue  of pay compression by creating a single, open-range pay band and allowing  agencies to increase the basic pay cap for their senior executives to EX- level II upon certification of their performance appraisal systems by OPM  with OMB concurrence.
While the pay cap was raised for certified agencies, agencies are to reserve  the pay rates above EX-level III for truly outstanding performers only,  which in effect slows the growth of senior executives’ pay within the pay  band. According to OPM regulations, the rates of basic pay higher than  EX-level III but less than or equal to EX-level II are generally reserved for  senior executives who have demonstrated the highest levels of individual  performance and/or made the greatest contributions to the agency’s  performance, or newly appointed senior executives who possess superior  leadership or other competencies.
The basic pay for senior executives at the selected agencies shows that  pay compression may be a problem in the future for some agencies.  Overall, however, only a small percentage of senior executives at the  selected agencies have their basic pay capped out at EX-level II ($172,200)  in 2008. Specifically, about half to three-quarters of senior executives at  the selected agencies are paid at or above EX-level III ($158,500) in 2008,  after performance-based pay adjustments were made for the fiscal year  2007 performance appraisal cycle, as shown in figure 4. For example, at  State, about 76 percent of senior executives are paid at or above EX-level  III, while about 50 percent of senior executives at DOD and USAID are  paid at these rates. Of the senior executives paid at or above $158,500, the  percentage of senior executives who are paid at the governmentwide pay  cap for 2008—$172,200—varies across the agencies. Specifically, at State  and DOE, about 26 percent of senior executives are paid at the pay cap for  2008, while only 1 percent of senior executives at DOD and none at USAID  are paid at this cap. OPM has found that about 13 percent of SES members  governmentwide are paid at the pay cap based on the fiscal year 2007  performance appraisal data they received from agencies.
According to a senior OPM official, the SES performance-based pay  system was never intended to fix the problem of pay compression that  occurred prior to 2004 or be the answer to future pay compression issues.  OPM recognizes that pay compression is a problem in the SES. While the  majority of senior executives at the selected agencies have yet to reach the  governmentwide basic pay cap, officials from two of the selected agencies  recognized the challenge of making distinctions in executive performance  given potential pay compression issues. Specifically, Treasury’s Deputy  Assistant Secretary for Human Resources and CHCO said an agency does  not have enough room in the governmentwide pay band to fully recognize  the outstanding performers through the appraisal system since the best  performers are already near the top of the pay range and their  performance payouts in the form of basic pay increases are limited. DOD’s  Principal Director said when pay increases are not possible given salary  cap issues, bonuses are a tool for components to reward their executives’  performance for achieving results.
NRC’s three position groups and the associated pay ceilings are intended,  in part, to help reserve pay above EX-III for those executives who  demonstrate the highest levels of performance, including the greatest  contribution to organizational performance as determined through the  appraisal system, according to a senior human resources official at NRC. A  senior executive would not receive a pay increase if the executive had  already reached the pay ceiling for the applicable position group. While  there is little room for pay increases within the pay bands for each  position group, the agency official indicated that NRC tries to give pay  adjustments and generous bonus amounts when possible to acknowledge  that their senior executives are high-performing individuals. To identify  possible areas and options for improvement to its performance appraisal  and pay system including the tier structure, NRC convened an executive  working group of PRB members and senior executives from different  areas and position groups. As part of its June 2008 findings, the working  group recommended retaining the three position groups and existing pay  ceilings with slight revisions to the positions that fall within each group.  NRC management accepted this recommendation, according to the agency  official.
section_title: Selected Agencies Have Built Safeguards into Their Senior Executive Performance Appraisal and Pay Systems to Help Ensure Fairness and Transparency
paragraphs: We have reported that agencies need to have modern, effective, credible,  and validated performance management systems in place with adequate  safeguards to ensure fairness and prevent politicization and abuse. All of  the selected agencies have safeguards including higher-level reviews of  performance appraisal recommendations, PRBs, and transparency in  communicating the aggregate results, although agencies varied in how  they implemented such safeguards.
Higher-level reviews. By law, as part of their SES appraisal systems, all  agencies must provide their senior executives with an opportunity to view  their performance appraisals and to request a review of the recommended  performance ratings by higher-level officials, before the ratings become  final. The higher-level reviewer cannot change the initial rating given by  the supervisor, but may recommend a different rating in writing to the  PRB that is shared with the senior executive and the supervisor. For  example, according to State’s policy, an executive may request a higher- level review of the initial rating in writing prior to the PRB convening, at  which time the initial summary rating, the executive’s request, and the  higher-level reviewer’s written findings and recommendations are  considered. The PRB is to provide a written recommendation on the  executive’s summary rating to State’s Director General of the Foreign  Service and Director of Human Resources, who makes the final appraisal  decisions.
Performance review boards. All agencies must establish one or more  PRBs to help ensure that performance appraisals reflect both individual  and organizational performance and rating, bonus, and pay adjustment  recommendations are consistently made. The PRB is to review senior  executives’ initial summary performance ratings and other relevant  documents and make written recommendations on the performance of the  senior executives to the agency head or appointing authority. When  appraising a career appointee’s performance or recommending a career  appointee for a bonus, more than one-half of the PRB’s members must be  SES career appointees. The selected agencies varied in their PRB  structures and who provided the final approval of the appraisal decisions.  On the one hand, given its small number of senior executives, USAID has  one PRB that is responsible for making recommendations to the  Administrator for his/her final approval on all rated career executives’  annual summary ratings, bonuses, performance-based pay adjustments,  and Presidential Rank Award nominations. On the other hand, DOD has  multiple PRBs within and across its components and agencies with  separate authorizing officials who give the final approval of rating and  performance payout recommendations. As another level of review after  the PRB, DOE convenes a Senior Review Board—comprised mainly of  political appointees—to review and approve the PRB recommendations  for ratings, pay adjustments, and bonuses, and look for consistency in  recommendations across the senior executives in headquarters, the field,  and the various organizations within DOE. The Director of the Office of  Human Capital Management said the Deputy Secretary, who serves as the  chair, ultimately makes the final decisions on senior executives’ ratings,  pay adjustments, and bonuses.
Transparency in communicating aggregate appraisal results.  Agencies should communicate the overall aggregate results of the  performance appraisal decisions—ratings, bonuses, and pay adjustment  distributions—to senior executives while protecting individual  confidentiality, and as a result, provide a clear picture of how the  executive’s performance compares with that of other executives in the  agency. Further, as part of its certification decisions, OPM requires  agencies to brief their SES members on the results of the completed  appraisal process to make sure that the dynamics of the general  distribution of ratings and accompanying rewards are fully understood. All  the selected agencies communicated the aggregate appraisal results to  senior executives, although their methods of communication and the types  of information provided varied.
Treasury and DOD posted the aggregate rating, bonus, and pay  adjustment distributions for senior executives on their Web sites with  comparison of data across previous fiscal years.
NRC sent an e-mail to all senior executives providing the percentage of  executives at each rating level and the percentage who received  bonuses and pay adjustments, as well as the average dollar amounts.  According to a senior human resources official at NRC, the agency  periodically holds agencywide “all hands” SES meetings where the  results of the appraisal cycle, among other topics, are communicated  to executives.
The Deputy Secretary of DOE provides a memo to all senior executives  summarizing the percentage of executives at the top two rating levels  and the average bonus and pay adjustment amounts, as well as OPM’s  governmentwide results as a point of comparison.
USAID communicated the aggregate SES appraisal results to SES  members throughout the appraisal cycle. In a February 2008 notice,  USAID communicated to all SES members the pay adjustment  distributions in ranges by rating level for the fiscal year 2007 appraisal  cycle. In a September 2008 e-mail to all SES members and rating  officials at the end of the appraisal cycle, USAID communicated the  aggregate performance rating distributions for the past two appraisal  cycles for fiscal years 2006 and 2007.
While the selected agencies all shared aggregate appraisal results with  their senior executives, the results of the OPM SES survey show that the  communication of overall performance appraisal results is not widely  practiced throughout the government. Specifically, 65 percent of  respondents said that they were not given a summary of their agency’s SES  performance ratings, bonuses, and pay adjustments. At the June 2008  forum with agency executive resources staff where it shared the survey  results, OPM officials emphasized the importance of communicating  aggregate appraisal results to all senior executives. According to a senior  OPM official, agencies need to figure out how best to communicate  aggregate appraisal results in a way that supports their different cultures  and practices. The official said OPM plans to continually monitor how well  the agencies are communicating aggregate appraisal results through the  certification review process.
section_title: OPM and OMB Provide Oversight through Various Formats; Selected Agency Officials Suggested Opportunities for Refinements
paragraphs: To ensure agencies’ senior executive appraisal systems are designed and  implemented to address the certification criteria, OPM and OMB, as  applicable, provide continuing oversight by issuing guidance to agencies  on revisions to the certification process, using tools and other initiatives to  help assess how agencies are implementing their SES performance-based  pay systems, providing training and forums, and interacting with agencies  on the review of their certification submissions. While generally satisfied  with OPM’s and OMB’s oversight, officials at the selected agencies said  OPM could strengthen its communication with agencies and executives on  how it uses the SES performance appraisal data and the correlation  between ratings and performance pay in determining whether agencies are  making meaningful distinctions based on relative performance. In  addition, senior-level officials at the selected agencies identified a need for  increased efficiency in the certification submission process.
subsections: section_title: Issuing Guidance to Agencies on Revisions to the Certification Process
paragraphs: Providing agencies with clear and timely guidance is one way for OPM and  OMB to effectively communicate with agencies upcoming revisions to the  certification process. OMB does not issue its own guidance to agencies,  but reviews OPM’s guidance to agencies, according to OMB’s lead official  in the certification review process. Officials at five of the selected agencies  said that, in the past, OPM has revised its guidance midway through the  appraisal cycle, which did not allow agencies sufficient time to change  their systems in order to receive certification for that calendar year.  Recognizing that it was late in issuing the guidance in 2006, OPM since has  issued guidance in the fall via memos to agency heads. In the future, OPM  plans to continue issuing any changes to the guidance in the fall, according  to a senior OPM official, since this is when agencies are finishing up the  performance appraisal cycle and starting to set expectations for the next  cycle. This should also provide agencies with adequate time to revise their  appraisal systems to reflect any new requirements before the certification  submission deadline at the end of June. In light of changes to the law in  October 2008, in the near future OPM plans to issue regulations and  revised certification guidance to agencies reflecting the modifications to  SL/ST basic pay rates for certified appraisal systems and changing the  certification cycle coverage to up to 24 months from a calendar-year-based  coverage.
section_title: Using Tools and Other Initiatives to Help Assess How Agencies Are Implementing Their SES Performance-Based Pay Systems
paragraphs: OPM and OMB use tools and other initiatives—such as the SES  Performance Appraisal Assessment Tool (SES-PAAT), the correlation  coefficient, and the governmentwide survey to all SES members on  performance-based pay—to help assess how agencies are implementing  their performance-based pay systems and addressing the certification  criteria. Overall, selected agency officials were in favor of OPM and OMB  using these tools and initiatives, although officials from three of the  selected agencies expressed concern about how OPM calculated the  correlation coefficient and the effect it had on the resulting score.
In 2007, OPM developed the SES-PAAT to help streamline the certification  process, improve the efficiency of its oversight process, and offer a more  transparent and organized way for agencies, OPM, and OMB to examine  SES appraisal systems, among other things. OPM first required agencies  with fully certified SES appraisal systems to use this tool when requesting  full certification for 2009 and 2010. Based on a set of questions that relate  to the certification criteria, the SES-PAAT helps clarify the certification  criteria and quantifies aspects of the certification package that agencies  had previously supported through narrative form. In making an agency’s  certification decision, OPM and OMB consider the SES-PAAT score and  the quality of the provided supporting documentation, such as the sample  of individual performance plans.
OPM is hopeful that the SES-PAAT will improve the efficiency of its  oversight process and the feedback provided to the agencies, but officials  from our selected agencies that completed the SES-PAAT have mixed  views on using this tool. OPM removed one office from the review process  for SES-PAAT submissions with the intention of spending less time  overseeing the continuation of full certifications and more time focusing  on provisional certifications or those agencies that were in danger of  dropping from full to provisional certification, according to a senior OPM  official. Officials from the two selected agencies that completed the SES- PAAT for the first time said the certification submission process was more  efficient with less documentation submitted overall; however, the process  was still labor intensive and time consuming, specifically with regard to  the sample of performance plans required. Even though the SES-PAAT  requires agencies to submit a smaller sample of performance plans, a  senior human resources official said her agency needed to do a complete  review of its SES performance plans to ensure that all the plans were  adequate for certification approval.
subsections: section_title: Correlation Coefficient
paragraphs: To help assess in part how agencies are meeting the pay differentiation  certification criterion, OPM is using a metric based on a correlation  coefficient that summarizes the strength of the relationship between SES  members’ ratings and their performance-based pay adjustments and  bonuses as part of the Human Capital Assessment and Accountability  Framework’s systems, standard, and metrics. Given that at least 60  percent of executives’ performance ratings are to be based on  organizational results, a senior OPM official said calculating the  relationship between executive ratings and performance pay provides an  indication of how well an agency is recognizing its executives based on  organizational results achieved.
Officials from three of the selected agencies expressed concern about how  OPM calculated the correlation coefficient and the effect it had on the  resulting score. Specifically, OPM decided to include in its calculations  those senior executives who received Presidential Rank Awards, but  because of their agencies’ policies were not eligible for and did not receive  bonuses. As a result, the coefficients for those agencies may show a  weaker connection between SES ratings and performance pay because  highly rated executives did not receive bonuses. A senior OPM official said  OPM recognizes that the decision to include all executives regardless of  their bonus eligibility in the correlation coefficient may have a negative  effect on an agency’s coefficient, especially in the case of smaller agencies  with few SES members, but it is a policy decision that OPM has made to  ensure that the coefficients were calculated consistently across the  government. For small agencies, OPM said that a correlation coefficient  may not be appropriate for determining how the agency is addressing the  pay differentiation criteria; rather, OPM will review the mean, median, and  mode of the agencies’ total compensation including pay adjustments and  bonuses, as applicable to determine whether higher-rated executives were  rewarded appropriately.
section_title: SES Survey
paragraphs: In January 2008, OPM conducted a governmentwide survey of all SES  members to evaluate the performance-based pay system. While OPM  found considerable variability in the executives’ responses across the  different agencies, according to OPM the overall results show that the vast  majority of executives believe pay should be based on performance and  that areas for improvement exist, for example, in communicating  aggregate appraisal results to senior executives. According to selected  agency officials, the SES survey results were very helpful and useful to  their agency.
We previously recommended that OPM develop a strategy to allow it,  other executive agencies, and Congress to monitor the progress of  implementation of the senior executive performance-based pay system.  The SES survey could be a vehicle for regularly monitoring progress in the  future. OPM has not committed to administering the survey on an ongoing  basis, in part due to the concern of over-surveying agency officials, but  plans to revisit the idea of administering the survey again in the next  several years. According to a senior OPM official, the Federal Human  Capital Survey is administered every 2 years and provides OPM with the  opportunity to monitor senior executives’ satisfaction with the appraisal  process, including whether they consider their appraisals to be a fair  reflection of their performance, based on the senior executives who  responded. Rather, when OPM decides to administer the survey again,  according to the OPM official, it plans to target the relevant issues of the  day with some of the original questions in order to track trends over time.
section_title: Providing Training and Forums for Agency Officials to Obtain Information from OPM and Discuss Relevant Issues Regarding the SES Performance-Based Pay System
paragraphs: To help facilitate its communications and interactions with agency  officials and the executive resources community, OPM periodically  provides training and holds forums for agency officials to discuss different  aspects of the SES performance-based pay system and the certification  process, among other topics. Selected agency officials found the forums to  be useful and helpful in understanding the certification process and  requirements while allowing agencies to share lessons learned from and  experiences with the certification process. OPM also finds these forums  helpful in gathering agency feedback, which it considers in future  revisions to the certification process and other human capital initiatives,  according to a senior OPM official. For example, in December 2007 OPM  conducted four training workshops for all interested agency executive  resources officials on how to complete the SES-PAAT and plans to hold  sessions in December 2008 and January 2009. OPM also holds forums five  times a year for executive resources staff from all agencies. At these  forums, OPM and agency officials have the opportunity to discuss  common concerns, obtain status updates on various OPM initiatives, and  learn about future plans for the certification process and other human  capital areas.
In addition, the CHCO Council chaired by the OPM Director works with  agencies to develop and share leading practices in implementing human  capital initiatives. For example, the CHCO Council periodically holds  training academy sessions that are open to agency officials other than  CHCOs to highlight and showcase human capital practices related to  senior executive pay and certification issues. Specifically, over the last 2  years, the CHCO Council has held several training academy sessions  related to SES performance management and pay systems, the SES-PAAT,  and lessons learned from the governmentwide SES survey results. In our  past work we recommended that OPM work with the CHCO Council to  develop a formal mechanism for sharing leading practices for  implementing human capital initiatives, such as the SES certification  process. OPM has addressed this recommendation by inviting all levels of  agency officials to attend CHCO Council training academy sessions when  relevant topics, such as the SES performance management and SES survey  results, were featured. Moving forward, the CHCO subcommittee on  performance management plans to partner with OMB’s Performance  Improvement Council to make improvements on SES certification and  other human capital efforts.
section_title: Interacting with Agencies on the Review of Their Certification Submissions
paragraphs: OPM provides oversight to the certification process by communicating and  working directly with agencies to help them improve their systems.  According to OMB’s lead official in the certification review process, OPM  takes the lead on the certification review process and OMB has a  concurrence role focusing most of its review on specific aspects  of agencies’ certification submissions including aligning executive  performance expectations with organizational and program goals,  ensuring executives’ goals are sufficiently results-oriented and challenging  to drive improved performance; measuring organizational performance;  and linking organizational results to the performance rating distribution.  Overall, the selected agencies have positive working relationships with  OPM on executive resources issues. For example, officials at five of the  selected agencies found that working with OPM on the individual  performance plans prior to submitting the certification package was  helpful and that OPM provided useful feedback at this step in the  certification process. Through its executive resources forums, OPM has  also communicated directly with agencies on SES performance-based pay  and the certification process, including sharing key results from the SES  survey with agencies.
However, OPM could strengthen its communication with agencies and  executives on how it uses the SES performance appraisal data and  correlation coefficient in determining whether agencies are making  meaningful distinctions based on relative performance as measured  through the performance and pay differentiation certification criteria.  Further communication from OPM is important in order for agencies to  have a better understanding of how they are being held accountable for  these certification criteria and make the necessary improvements to their  systems to maintain certification. Officials at four of the selected agencies  said they are unclear about how OPM uses the SES appraisal data to  assess whether agencies are meeting these criteria and making meaningful  distinctions in performance overall. In addition, officials at four of the  selected agencies said that the communication they have received from  OPM individually and through broader forums, such as OPM’s executive  resources forums and certification guidance, has not provided them with a  clear sense of how OPM is using the correlation coefficients to determine  how agencies are addressing the pay differentiation criterion. For the  coefficients based on fiscal year 2006 appraisal data, OPM provided each  agency with their coefficients and technical information explaining the  concept of a correlation coefficient, but did not communicate to agencies  how the scores were used for certification decisions. In addition, OPM  gave this information only to PMA-scored agencies. OPM provided all  agencies with 10 or more senior executives their correlation coefficient  based on the fiscal year 2007 appraisal data along with some contextual  information, but OPM does not address the concerns expressed by  officials at the selected agencies regarding how the correlation coefficient  is being used in certification decisions.
With respect to agencies’ working relationships with OMB, officials at five  of the selected agencies had little to no direct contact with OMB through  past reviews of their certification submissions and did not have a clear  understanding of OMB’s role in the certification review process. However,  while their interaction with OMB and understanding of its role was  limited, the selected agency officials were satisfied overall with how they  received OMB’s feedback through OPM. OMB’s feedback on agencies’  systems is most commonly communicated to agencies via the letter that it  sends to OPM stating its concurrence or nonconcurrence with OPM’s  certification recommendation, according to OMB’s lead official in the  certification review process. For the 2008 certification review process,  OMB is working with agencies to identify areas of improvement for their  appraisal systems and asking agencies to commit to working with OPM  and OMB to address these areas. This communication and feedback to  agencies is taking place prior to receiving OPM’s and OMB’s certification  decisions, according to the OMB official.
section_title: Future Plans and Potential Refinements for the SES Performance-Based Pay System and Certification Process
paragraphs: OPM is considering phasing out the distinction between full and  provisional certification once all agencies have received full certification.  Provisionally certified agencies receive the same pay flexibilities—access  to higher basic pay and total compensation—as those with fully certified  systems. Currently, agencies’ systems are required to meet the same  criteria for both types of certification, with the only distinction being fairly  subtle differences in the degree to which the agencies meet the criteria,  according to OPM. When certification began in 2004, an agency needed to  only meet four of the nine criteria and demonstrate that its system in  design would meet the remaining certification criteria to receive  provisional certification. We reported that it would be important for OPM  to continue to monitor the certification process, especially for those  agencies’ systems with provisional certification, to help ensure that  provisional certifications do not become the norm. After that initial year,  agencies’ systems were required to meet all nine certification criteria to  receive provisional certification, which according to a senior OPM official  was done in part because agencies would have had the opportunity to  produce performance data showing that meaningful distinctions were  made by the second year. OPM expects that all applicant SES appraisal  systems will meet full certification criteria in the future. As of October 28,  2008, 76 percent of the agency certification submissions reviewed by OPM  and OMB had received full certification for calendar year 2008. According  to a senior OPM official, OPM is currently revisiting the certification  regulations to determine what revisions should be made.
Regarding potential refinements to the certification process, officials at  the selected agencies identified a need for increased efficiency in the  certification submission process given the large quantity of documents  required and the time it currently takes to gather the certification package.
For example, officials at five of the selected agencies are in favor of  moving to an electronic submission process, if it reduces the amount  of paper documentation that is required and streamlines the process of  compiling the necessary documentation for agencies. Selected agency  officials acknowledged that the certification process is moving in that  direction with the SES-PAAT, which is an electronic tool. For agencies  completing the SES-PAAT, OPM requested five copies of the  completed tool and supporting documentation, such as the sample of  performance plans and organizational performance assessments. A  senior human resources official who is not using the SES-PAAT said  her agency submitted to OPM seven copies of its certification package  for 2007. While OPM has informally encouraged agencies through its  executive resources forums or e-mail communications to submit their  certification submissions electronically, according to a senior OPM  official OPM has not directly discussed electronic certification  submissions in its guidance and submission instructions to the  agencies. In addition, officials at three of the selected agencies said  that OPM could lessen agencies’ reporting burden by only requiring  them to submit documentation to OPM and OMB on those areas of the  system where substantive changes have occurred since the previous  certification submission. While OPM and OMB should receive  completed certification submissions as part of their oversight  responsibilities, moving to electronic documentation may help  agencies more easily gather and submit the certification documents  that change minimally from year to year. OPM and OMB officials both  said they support electronic certification submissions and would not  rule out a more electronic certification process in the future, starting  with the SES-PAAT.
Overall, selected agency officials supported lengthening the  certification coverage beyond 2 years once agencies have proved their  systems are addressing the certification criteria. Currently, full  certification lasts for 24 months. Specifically, the process of gathering  the necessary documentation for certification submissions would be  less burdensome to agencies if they had more time between  recertification deadlines, according to a senior human resources  official. Prior to receiving an extended certification coverage period,  the agency officials acknowledged that their systems would have to be  operating at the fully certified level. The lead OMB official overseeing  the certification review process said OMB could potentially be  supportive of lengthening the certification coverage, once agencies  have their systems up and running and all agencies have full  certification. Recognizing extending coverage would require a  statutory change, OPM is reviewing the recently passed law that  changes the certification coverage to up to 24 months from a calendar- year-based coverage, among other things, and OPM is not taking a  position on extending certification coverage at this time.
section_title: Conclusions
paragraphs: OPM and OMB have taken important steps in overseeing the design and  implementation of agencies’ senior executive performance-based pay  systems. The selected agencies’ experiences with implementing their SES  systems can help inform other agencies’ efforts to hold executives  accountable for organizational results and link pay for performance.  USAID has an opportunity to strengthen the link between organizational  and individual performance by providing rating and reviewing officials  with specific information on organizational performance. For senior  executives to lead the way in transforming their organizations to meet the  complex challenges facing the nation, it will be important for them to have  confidence that the SES performance-based pay system is operating as  intended and that they will be rewarded according to their performance. In  this regard, making meaningful distinctions among executives when  assessing performance is important to the overall credibility of the SES  performance-based pay system. Less than a third of senior executives  governmentwide strongly agreed or agreed that bonuses or pay  distinctions were meaningfully different among senior executives,  according to OPM’s SES survey. OPM has an opportunity to help ensure  that agencies are making meaningful distinctions in executive  performance by strengthening its communication with agencies and  executives on the importance of using a range of rating levels when  assessing performance, while avoiding forced distributions. Additionally,  communicating how OPM uses data and other tools in making certification  decisions will be important so that agencies can make continuous  improvements to their systems to support the development of a stronger  performance culture and the attainment of their missions, goals, and  objectives. Moving forward, it will also be important for OPM and OMB to  identify ways to improve the certification process and make it more  streamlined while ensuring that agencies have the guidance, tools, and  training they need to implement effective performance appraisal and pay  systems for their senior executives.
section_title: Recommendations for Executive Action
paragraphs: To help ensure consistency and clarity in how organizational performance  is considered in appraising executive performance, we recommend that  the Administrator of USAID provide uniform organizational performance  assessments to PRB members and other reviewing officials to help inform  their appraisal recommendations for senior executives at the end of the  performance appraisal cycle.
To help improve agencies’ understanding of certain aspects of the  certification decisions, we recommend two areas for the Acting Director  of OPM to take action to strengthen OPM’s communication with agencies  and executives on  the importance of making meaningful distinctions in performance  while avoiding the use of forced distributions and that a fully  successful rating is valued and rewarded and    how it uses the SES performance appraisal data and the correlation  between ratings and performance pay in determining whether agencies  are making meaningful distinctions based on relative performance as  measured though the pay and performance differentiation certification  criteria.
In addition, to help improve the efficiency of the certification submission  process for agencies, we recommend that the Acting Director of OPM and  Director of OMB explore opportunities for streamlining the certification  process, such as electronic submissions or lengthening the full  certification coverage beyond 2 years for agencies that received full  certification.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a copy of the draft report to the Secretaries of Defense,  Energy, State, and the Treasury; the Commissioners of NRC; the  Administrator of USAID; the Acting Director of OPM; and the Director of  OMB for their review and comment.
DOE had no comments on the draft report. We received written comments  from DOD and OPM, which are included in appendixes III and IV. NRC,  OMB, State, Treasury, and USAID provided clarifying and technical  comments, which we incorporated as appropriate. With respect to making  meaningful distinctions in performance, Treasury officials provided broad  comments on the challenge agencies face in using the full range of rating  levels given that executives are high achievers and had to exhibit  exceptional performance to enter the SES. The officials suggested that  meaningful distinctions can not be defined simply by the distribution of  performance ratings. Our findings and recommendations about making  meaningful distinctions acknowledged challenges, but also the need for  additional communication from OPM in this area.
Regarding our recommendations, USAID, OPM, and OMB expressed  general agreement. The Acting Director stated that OPM looks forward to  working with agencies and OMB to find ways to further improve  communications with the agencies concerning the certification process.   OMB generally agreed with our assessment and recommendation  regarding the possibilities of streamlining the certification process to  improve efficiency and potentially extending full certification coverage  beyond 2 years. OMB stated that it agrees with OPM that careful review of  the newly passed law and its effect will be necessary before considering  such an extension. Regarding our discussion of pay compression, OPM  stated that it is not comfortable with the identification of tiers as a means  to address SES pay compression. The Acting Director commented that  using tiers results in salaries clustering around points in the salary range  rather than only at the top of the range. While we recognize OPM’s  concern about agencies’ use of tiers, we are not recommending the use of  tiers as a way for agencies to address future problems with pay  compression and have revised the language in the report to clarify this  point.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution of it until 30 days from  the date of this letter. At that time, we will send copies of this report to the  appropriate congressional committees; the Secretaries of Defense, Energy,  State, and the Treasury; the Commissioners of NRC; the Administrator of  USAID; the Acting Director of OPM; the Director of OMB; and other  interested parties. The report will also be available at no charge on the  GAO Web site at http://www.gao.gov.
If you or your staff have any questions regarding this report, please  contact me at (202) 512-6806 or goldenkoffr@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines selected agencies’ policies and procedures for their  career Senior Executive Service (SES) performance appraisal and pay  systems in (1) factoring organizational performance into senior executive  performance appraisal decisions, (2) making meaningful distinctions in  assessing and rewarding senior executive performance, and (3) building  safeguards into senior executive performance appraisal and pay systems.  In addition, this report examines how the Office of Personnel Management  (OPM) and the Office of Management and Budget (OMB) are providing  oversight in the certification of the senior executive performance-based  pay system through their statutory roles.
We selected the U.S. Departments of Defense, Energy, State, and the  Treasury; the U.S. Nuclear Regulatory Commission (NRC); and the United  States Agency for International Development based on variations in  agency mission, organizational structure, size of their career SES  workforces to reflect agencies with a large, average, and small number of  executives; and results of their SES performance appraisal systems in  terms of the percentage of SES rated at the highest rating levels and the  percentage that received performance awards or bonuses from fiscal years  2004 to 2006, according to OPM’s governmentwide data reports. We  focused on career members of the SES because they represent the  majority of all SES appointments governmentwide.
To address our first and third objectives, we analyzed selected agencies’  documents on their SES performance appraisal and pay systems including  performance management policies, directives, guidance, and other related  information; performance planning and appraisal templates; briefings,  memos, and other materials used to communicate aggregate appraisal  results to senior executives; and the submission documents to OPM and  OMB for certification in 2007. We also interviewed cognizant senior-level  officials in the human capital offices at the selected agencies regarding  their agencies’ SES systems and senior-level OPM and OMB officials on  how organizational performance and safeguards are integrated into the  certification process.
To address our second objective, we analyzed aggregate SES basic pay,  performance rating, bonus, and pay adjustment data as provided by the  agencies for fiscal year 2007. We defined our universe of analysis as career  senior executives who received ratings. In calculating the percentage of  eligible senior executives who received bonuses (cash awards) or pay  adjustments (increases to basic pay) and average (or mean) amounts, we  excluded executives who received a rating less than “fully successful”  (level 3), as applicable, from the eligible population since those executives  are not eligible to receive bonuses, according to the selected agencies’  policies, or pay increases, according to OPM regulation. We also excluded  senior executives at NRC, Treasury, and State who received Presidential  Rank Awards from our calculations of percentages of eligible SES  members receiving bonuses and average amounts because those  individuals were not considered for bonuses that year, according to the  agencies’ policies. In order to have consistency in our analysis across  selected agencies, we included senior executives who were rated but left  their positions—because of retirement, attrition, or assignment to a lower  grade—prior to performance payouts being made in our analysis. The  agencies’ policies and practices varied in whether or not senior executives  who retired were eligible for performance payouts.
In analyzing the basic pay rates, we used the rates of basic pay for rated,  career senior executives after pay adjustments were made for the fiscal  year 2007 appraisal cycle. We compared these basic pay amounts to the  governmentwide SES basic pay range and the Executive Schedule (EX)  pay levels, specifically EX-level II (the SES pay cap) and EX-level III  (within the SES pay band) for calendar year 2008. EX pay rates are set at  the beginning of each calendar year and because performance payouts  from the completed appraisal cycle are usually paid out in January,  agencies are able to use the next calendar year SES pay rates for their pay  ranges. In calculating the percentage of senior executives paid at or above  EX-level III, we included those SES paid at EX-level II in our calculations.  For this objective, we also analyzed relevant agency policies and guidance  regarding performance ratings, bonuses, and pay adjustments, and  interviewed selected senior-level agency officials from the human capital  offices. For the governmentwide perspective, we reviewed OPM’s Report  on Senior Executive Pay for Performance for Fiscal Year 2007 to identify  relevant governmentwide performance data as a comparison point for the  selected agencies. We analyzed OPM memos and guidance on the  certification process and criteria, and interviewed senior-level OPM and  OMB officials who oversee the certification review process on how the  SES performance data are used as part of the certification process.
We checked the agency data for reasonableness and the presence of any  obvious or potential errors in accuracy and completeness. We also  reviewed related agency documentation, interviewed agency officials  knowledgeable about the data, and brought to the attention of these  officials any concerns or discrepancies we found with the data for  correction or updating. The agency officials confirmed the correctness of  the data or in some cases provided corrections to the data, which we used  in our analysis. On the basis of these procedures, we believe the data are  sufficiently reliable for use in the analyses presented in this report.
For our fourth objective, we analyzed OPM’s and OMB’s guidance, memos,  and other documents regarding the certification process and criteria;  interviewed senior-level OPM and OMB officials involved in the  certification process regarding their oversight of the certification process;  and interviewed senior-level agency officials from the selected agencies’  human capital offices to obtain their perspectives on OPM and OMB  oversight. As part of this objective, we are reporting on the status of  OPM’s progress toward addressing recommendations from our past work  on the SES performance-based pay system and certification process.
We conducted our work from October 2007 through November 2008 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Certification Criteria for the Senior Executive Performance-Based Pay System
paragraphs: In November 2003, Congress authorized a new performance-based pay  system for members of the Senior Executive Service. Agencies are to base  pay adjustments for senior executives on individual performance and  contributions to the agency’s performance by considering the individual’s  accomplishments and such things as unique skills, qualifications, or  competencies of the individual and the individual’s significance to the  agency’s mission and performance, as well as the individual’s current  responsibilities. If an agency’s senior executive performance appraisal  system is certified by the Office of Personnel Management (OPM) and the  Office of Management and Budget (OMB) concurs, an agency can raise the  pay cap for its senior executives to $172,200 for basic pay (Level II of the  Executive Schedule) and $221,100 for total compensation (the total annual  compensation payable to the Vice President).
To qualify for senior executive pay flexibilities, agencies’ performance  appraisal systems are evaluated against nine certification criteria and any  additional information that OPM and OMB may require to make  determinations regarding certification. As shown in table 4, the  certification criteria jointly developed by OPM and OMB are broad  principles that position agencies to use their pay systems strategically to  support the development of a stronger performance culture and the  attainment of their mission, goals, and objectives.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: Comments from the Office of Personnel Management
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Belva Martin, Assistant  Director; Amber Edwards; Janice Latimer; Donna Miller; Meredith Moore;  Mary Robison; Sabrina Streagle; and Greg Wilmoth made major  contributions.