The General Schedule (GS) is the federal government’s primary pay and classification system for white-collar employees. Under this system, federal employees are paid according to governmentwide rules and procedures, and federal positions are classified according to the difficulty and responsibility of the work performed. The GS system was created in 1949, when most federal positions involved clerical work or revolved around the execution of established, stable processes—for example, posting census figures in ledgers or retrieving taxpayer records from vast file rooms. Over time, however, federal positions have become increasingly specialized and more highly skilled. In light of this change, a number of federal agencies have attempted to provide managers with greater flexibility in hiring and awarding pay raises to employees by implementing human capital initiatives, such as performance management systems, that reward employees’ performance and contribution to the agency’s mission. The need for human capital reform regarding these systems has been the subject of a number of previous GAO reviews. These reviews have noted, for example, that federal agencies must have modern, effective, credible, and validated performance management systems that are capable of supporting pay and other personnel decisions, and that pay for performance works only with adequate safeguards and appropriate accountability mechanisms in place to ensure that the safeguards are implemented in a fair, effective, and credible manner. In November 2003, the Congress included a provision in the National Defense Authorization Act for Fiscal Year 2004 providing DOD with the authority to establish a pay-for-performance management system as part of NSPS. In April 2004, the Secretary of Defense appointed an NSPS Senior Executive to, among other things, design, develop, and implement NSPS. Under the Senior Executive’s authority, the PEO was established as the central policy and program office for NSPS. The PEO’s responsibilities include designing the human resource/pay-for-performance systems, developing communication and training strategies, modifying personnel information technology, and preparing joint enabling regulations and internal DOD implementing regulations, called implementing issuances. As the central DOD-wide program office, the PEO also directs and oversees the four components’ NSPS program managers, who report to their parent components and the PEO. These program managers serve as their components’ NSPS action officers and also participate in the development, planning, implementation, and deployment of NSPS. Beginning in April 2006, DOD began phasing (or spiraling) civilian employees into NSPS; however, in January 2008 the National Defense Authorization Act for Fiscal Year 2008 prohibited the Secretary of Defense from converting more than 100,000 employees to NSPS in any calendar year and excluded Federal Wage System employees from coverage under NSPS. Further, in March 2009, DOD announced that it would delay the conversion of additional organizations to NSPS pending the outcome of a joint review of the system by DOD and OPM. According to DOD, this decision affected roughly 2,000 employees in organizations scheduled to convert to NSPS during the spring of 2009. As a result of these and other legislative changes that resulted in revisions to the NSPS regulations, the PEO has not developed new timelines for phasing any additional civilian employees into NSPS. As table 1 shows, according to DOD, almost 220,000 civilian employees have been phased into NSPS as of September 2009. The NSPS performance management process is ongoing and consists of several phases that are repeated during each annual performance cycle. The process begins with a planning phase that involves supervisors (or rating officials) and employees working together to establish performance plans. This includes developing job objectives—the critical work employees perform that is aligned with their organizational goals and focused on results—and identifying contributing factors—the attributes and behaviors that identify how the critical work established in the job objectives is going to be accomplished (e.g., cooperation and teamwork). After the planning phase comes the monitoring and developing phase, during which ongoing communication between supervisors and employees occurs to ensure that work is accomplished; attention is given to areas that need to be addressed; and managers, supervisors, and employees have a continued and shared understanding of expectations and results. In the rating phase, the supervisor prepares a written assessment that captures the employee’s accomplishments during the appraisal period. In the final—or reward—phase, employees should be appropriately rewarded or compensated for their performance with performance payouts. During this phase, employee assessments are reviewed by multiple parties to determine employees’ ratings and, ultimately, performance payouts. The performance management process under NSPS is organized by pay pools. A pay pool is a group of employees who share in the distribution of a common pay-for-performance fund. The key participants that make up pay pools are the employee, supervisor, higher-level reviewer, pay pool panel, pay pool manager, performance review authority, and, in some instances, the sub-pay pool, as shown in figure 1. Within a pay pool, each participant has defined responsibilities under the performance management process: Employees are encouraged to be involved throughout the performance management cycle. This includes initially working with their supervisors to develop job objectives and identify associated contributing factors; identifying and recording accomplishments and results throughout the appraisal period; and participating in interim reviews and end-of-year assessments, for example, by preparing a self- assessment of their performance during the annual appraisal period. Supervisors (or rating officials) are responsible for effectively managing the performance of their employees. This includes clearly communicating performance expectations; aligning performance expectations and employee development with organization mission and goals; working with employees to develop written job objectives that reflect expected accomplishments and contributions for the appraisal period and identifying applicable contributing factors; providing employees meaningful, constructive, and candid feedback relative to performance expectations, including at least one documented interim review; making meaningful distinctions among employees based on performance and contribution; and providing recommended ratings of record, share assignments, and payout distributions to the pay pool. The higher-level reviewer, typically the rating official’s supervisor, is responsible for reviewing and approving job objectives and recommended employee assessments. The higher-level reviewer is the first step in ensuring consistency of ratings because this individual looks across multiple ratings. The pay pool panel (or, in some cases, the sub-pay pool panel) is a board of management officials who are usually in positions of line authority or in senior staff positions with resource oversight for the organizations, groups, or categories of employees making up the pay pool membership. The primary function of the pay pool panel is the reconciliation of ratings of record, share distribution, and payout allocation decisions. For example, the pay pool panel may adjust a supervisor’s recommended rating of record in order to help ensure the equity and consistency of ratings across the pay pool. Each pay pool has a manager who is responsible for providing oversight of the pay pool panel. The pay pool manager is the final approving official of the rating of record. Performance payout determinations may be subject to higher management review by the performance review authority or equivalent review process. Finally, the performance review authority provides oversight of several pay pools and addresses the consistency of performance management policies within a component, major command, field activity, or other organization as determined by the component. DOD continues to take some steps to implement each of the safeguards we reported on in September 2008. However, opportunities exist to continually involve employees in the system’s implementation, and implementation of some safeguards—for example, providing effective training—could be improved. Also, we previously reported that continued monitoring of the safeguards was needed to help ensure that DOD’s efforts were effective as implementation of NSPS proceeded. However, we found that while DOD monitors some aspects of the implementation of NSPS, it does not monitor how the safeguards specifically are implemented across the department. Because DOD does not monitor the safeguards’ implementation, decision makers in DOD lack information that could be used to determine whether the department’s actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. DOD has taken a number of steps to involve employees and stakeholders in the design and implementation of NSPS. In our September 2008 report, we noted that DOD solicited comments from employees and unions representing DOD employees during the design of NSPS. We also noted that DOD involved employees during the implementation of the system in workshops and other efforts aimed at refining the system’s performance factors. DOD continues to take such steps. For example: DOD solicited comments from employees and unions on the system’s final rule, which was published in the Federal Register in September 2008. According to the Federal Register, the final regulations, which became effective in November 2008, include revisions based on 526 comments submitted during the public comment period and on comments from 9 of the department’s 10 unions with national consultation rights. DOD involved employees in efforts to improve the usability of the automated tools that support the NSPS performance and pay pool management processes. Specifically, the PEO and the department’s Civilian Personnel Management Service held a series of meetings with employees, rating officials, pay pool managers, and human resource practitioners in early 2008 to address concerns regarding the usability of the automated tools. These meetings allowed the department to gather requirements for the next version of the NSPS automated tools based on lessons learned and user input. Subsequently, DOD established six separate working groups to develop and evaluate the requirements for each of its automated tools. In addition, DOD initiated separate efforts to enhance the usability of the Performance Appraisal Application—the DOD-wide tool for employee self- assessments and appraisals. Specifically, the contractor that developed the Performance Appraisal Application enlisted the assistance of software usability experts to evaluate the tool and recommend changes that would enhance users’ experience with it. As a part of this effort, the contractor observed and worked with employees and rating officials to identify changes that could be made to the Performance Appraisal Application to make it more user-friendly. DOD also tested the functionality and usability of the enhancements that were made to the Performance Appraisal Application with over 300 users. DOD has taken steps to involve the components in the implementation of NSPS through biweekly conference calls held at key phases of the performance management process. According to the PEO, during these calls, PEO and Civilian Personnel Management Service representatives discuss topics submitted by the components, respond to questions regarding such things as NSPS policy and the system’s automated tools, and share lessons learned with participants. Further, according to the PEO, these conference calls allow participants to address systemic problems through feedback shared between different levels of the organization. At the locations we visited outside the continental United States, we found that some steps were generally being taken to involve employees in the implementation of NSPS as well. For example, officials at each of the eight locations reported that organizations identified lessons learned that were generally based upon employee feedback gathered following the mock pay pool. According to these officials, lessons learned were used to make changes to, among other things, training materials, business rules, and the use of job objectives and contributing factors. For example, two locations limited the number of contributing factors employees should use in their performance plans based upon lessons learned, while one location responded to employee feedback regarding a lack of time to devote to NSPS by mandating that employees be allowed to take time to complete NSPS training. While DOD has taken a number of steps to involve employees in the design and implementation of NSPS thus far, as stated above, we note that one way the department could continue to involve employees as implementation of the system proceeds is to develop and implement an action plan to address employees’ perceptions of NSPS, as we recommended in our first assessment of NSPS. However, DOD has not yet done so, which we discuss further in the second objective of this report. DOD continues to take steps to link employee job objectives to the agency’s strategic goals, mission, and desired outcomes. As we noted in our 2008 report, DOD’s automated tool for employee self-assessments and appraisals—the Performance Appraisal Application—provides a designated area for the mission of the employee’s command to be inserted as a guide while employees compose their job objectives and self- assessments. In May 2009, DOD published its evaluation of NSPS for 2008, entitled National Security Personnel System (NSPS) – 2008 Evaluation Report, which included an evaluation of employee performance plans to determine the extent to which employee job objectives are aligned with higher-level organizational goals and thus ensure that employee performance contributes to the achievement of organizational success. The evaluation included 240 employee performance plans encompassing a range of job series, pay schedules, pay bands, and organizations within each of the four components. The evaluation found that 95 percent of the objectives were strongly aligned to higher-level goals and demonstrated a clear, direct, and strong linkage to the organizational mission or relevant strategic goals. During our site visits, we found that the organizations were taking steps to ensure that employees understood how their job objectives link to the organization’s strategic goals. This was generally accomplished through documentation requirements in the Performance Appraisal Application and reinforced during employees’ discussions with their supervisors. Some organizations have taken additional steps to ensure that employee job objectives link to the organization’s strategic goals. For example, at one location we visited the commanding general issued a memorandum to managers emphasizing the importance of ensuring that employee objectives are linked to the command’s mission and objectives and that employees understand their roles in achieving those objectives. However, officials at five locations also reported that employees do not always understand this linkage. DOD continues to take steps to provide employees with required and other training on the implementation and operation of the NSPS performance management system, but has not yet evaluated the effectiveness of the training that it provides. In our September 2008 report, we noted that DOD encouraged employees who were transitioning to NSPS to receive training that covered skills and behaviors necessary to implement and sustain NSPS; foster support and confidence in the system; and facilitate the transition to a performance-based, results-oriented culture. DOD and each of the components continue to take such steps to provide employees with required and other training on the system, including introductory training for employees converting to NSPS and sustainment training for employees already under the system. While the components are responsible for providing employees with training on the NSPS performance management system, the PEO supports their efforts by offering a variety of departmentwide training courses and other materials. For example, Web- based training modules that the PEO has developed, such as its NSPS 101 and iSuccess courses (see fig. 2 for sample screen shots), provide employees with basic knowledge about NSPS and performance management principles in general, and are used by employees to develop their job objectives. As another example, the PEO developed training guides to educate employees on changes to the NSPS classroom materials resulting from the revised NSPS regulations and implementing issuances. The PEO also developed a Web site for accessing NSPS learning materials, resources, and other tools. In addition, we found that the Air Force has begun incorporating training on NSPS as a normal part of its operations and is working to embed NSPS topics in the regular training provided to Air Force civilians and servicemembers. Although DOD and the components continue to take steps to provide employees with training on NSPS, the department has not yet evaluated the effectiveness of the training provided. We previously reported that it is increasingly important for agencies to measure the real impact of training and thus evaluate their training—for example, by establishing clear goals about what the training is expected to achieve along with agreed-upon measures or performance indicators to ascertain progress toward achieving those goals. DOD has outlined the fundamental requirements, or goals, of the training that it provides to employees on NSPS. Specifically, DOD has noted that for NSPS, a training program must be implemented that enables employees to understand better how to succeed, and enables supervisors to communicate performance expectations to their employees, provide feedback to them on their performance against these expectations, and tell them what steps they can take to improve their performance and competencies and manage their careers. However, while DOD has undertaken efforts to understand employees’ perceptions of its training, the department has not yet evaluated the effectiveness of the training that it provides. For example, in early 2009 the PEO conducted what PEO officials describe as an ad hoc study of training needs. The PEO’s study consisted of conducting sensing sessions with 120 human resource practitioners at 11 locations across the department. According to the PEO, the purpose of these sessions included obtaining feedback on existing NSPS learning products and support and exploring options for next generation products. While the PEO’s study identified some needed improvements, it does not constitute an evaluation of the department’s training—for example, because it did not assess the department’s progress toward achieving the goals specified for the training. As another example, DOD’s 2008 evaluation report also highlighted deficiencies with the department’s training on NSPS. Specifically, the report notes that without effective communication and training, NSPS cannot achieve its goal of being a credible and trusted system. Further, three of the report’s six key recommendations focus on the need to enhance the effectiveness of the training provided to employees on NSPS: (1) provide more training on the performance management system; (2) provide high-level training for employees and supervisors that explains the pay pool process; and (3) hold mock pay pool panels, which serve as refreshers for continuing panel members and as training for new members. However, like the PEO’s study, DOD’s 2008 evaluation report does not constitute an evaluation of the department’s training—for example, because it did not include an in-depth assessment of DOD’s training and also did not assess the department’s progress toward achieving the goals for the training, per agreed-upon measures or performance indicators. Of the components, we found that only the Army has taken some steps to evaluate the training it provides to employees on the system. Specifically, the Army assesses the adequacy of NSPS training during on-site reviews that it conducts as part of its implementation of the system. The Army conducted three such assessments during 2008 and an additional four such assessments during 2009. With respect to our site visits, although we found that each of the eight locations provided training on NSPS to employees, officials at each location also expressed concerns over the effectiveness of the training provided. For example, officials at seven locations told us that additional training was needed on writing job objectives or employee self- assessments under the system, while other officials noted that additional training was needed on the pay pool process, use of the system’s automated tools, and how the reconsideration process works. Similarly, officials at two locations raised concerns that the training they received did not provide them with the skills they needed to use the Performance Appraisal Application. For example, officials told us that they received training too early and had either forgotten it or the training had become outdated by the time they actually used the Performance Appraisal Application. Further, some program officials raised concerns about their organizations’ ability to provide employees with adequate training on the system when the employees are located outside the continental United States. For example, program officials at one location in Germany reported challenges providing employees located in the field with training on NSPS because they lack the resources to send NSPS trainers to those locations. However, until DOD evaluates its training, it will not be able to determine whether the training meets the needs of its employees or the department is making progress toward achieving the goals it specified for the training. DOD continues to take steps to ensure that employees receive timely performance feedback that is meaningful, constructive, and in accordance with the department’s requirements. In our first assessment of NSPS we noted that DOD’s implementing issuances required at least one documented interim performance review and an annual performance appraisal and that the Performance Appraisal Application allowed supervisors and employees to document these feedback sessions. Since then, DOD has taken additional steps to enhance the Performance Appraisal Application by modifying the tool to allow supervisors and employees to identify where they are in the performance appraisal process and help them accomplish required actions in a timely manner. During our site visits, officials at all eight locations told us that NSPS helps ensure the occurrence of performance feedback between employees and supervisors. For example, officials noted that use of the Performance Appraisal Application encourages employee feedback by allowing employees to document and track feedback sessions, and that NSPS encourages direct discussions about performance-related issues, such as developing effective job objectives and establishing performance expectations. DOD continues to take steps to better link individual pay to performance as well. As we noted in our 2008 report, the NSPS performance management system uses a multirating system of five rating categories—of which the lowest rating is “1” (unacceptable performance) and the highest rating is a “5” (role model performance)—that allows distinctions to be made in employee performance and therefore compensation. Since then, DOD added details to the NSPS regulations to facilitate uniform, equitable practices across the department that accord with merit system principles. These include specifying specific share assignment ranges, rounding rules for converting raw performance scores to ratings, and formulas for determining share value and the calculation of performance payouts under NSPS. DOD also clarified that a common share value should apply throughout an entire pay pool. According to the PEO, these changes, in addition to the higher-level review of performance expectations, recommendations for ratings of record, share assignment, and payout distribution have helped to promote a more equitable method for appraising and compensating employees. However, during our site visits, officials at seven of the eight locations told us that they saw the potential for factors other than performance to influence employees’ ratings, such as the quality of employees’ and supervisors’ writing skills. For example, rating officials at one location commented that NSPS does not reward employees based on their performance but rather on how well employees and supervisors can communicate in writing what the employee achieved during the performance management cycle. Similarly, at another location, a pay pool panel member told us that the paperwork submitted to the panel by both the employee and the supervisor must be of very high quality because it can be difficult to defend a high rating recommended for an employee if the assessments are poorly written. DOD’s 2008 evaluation report also highlighted concerns from employees and managers over the department’s success in linking pay to performance under NSPS. For example, DOD’s report noted that while some employees believed that they saw some level of pay for performance under NSPS, others were uncertain. Further, DOD’s report also noted that some managers observed that the quality of written assessments contributed significantly to the way in which ratings were substantiated. We found that DOD also continues to take steps to ensure that adequate agency resources are allocated to NSPS. According to 5 U.S.C. § 9902(e)(4), to the maximum extent practicable, for fiscal years 2004 through 2012, the overall amount of money allocated for compensation of civilian employees in organizations under NSPS shall not be less than the amount that would have been allocated under the GS system. To meet the requirements of 5 U.S.C. § 9902(e)(4), DOD’s implementing issuances require that the components certify in writing to the Deputy Secretary of Defense through the Under Secretary of Defense for Personnel and Readiness that the amount expended for NSPS performance-based pay increases is no less than what would have been expended had these positions not been converted into NSPS. Following the 2008 NSPS performance management cycle, each of the components certified that it met this requirement. DOD also continues to capture NSPS implementation costs, and for fiscal year 2008, the PEO reported that NSPS implementation costs were about $61.8 million. According to the PEO, continuing implementation of NSPS will result in some additional program implementation costs, such as for training for NSPS, conducting outreach to employees and others, and improving the system’s automated tools. However, the PEO estimates that once the conversion of all non-bargaining unit employees is complete, the system’s implementation costs will decrease significantly unless there is a decision to convert bargaining unit employees. Thereafter, according to the PEO, the cost of administering NSPS will be no different than that of any other personnel system. In our 2008 report, we recommended that DOD take steps to better ensure the consistency and equity of the performance management process by requiring a third party to perform predecisional demographic and other analysis of the pay pool results. DOD did not concur with our recommendation, stating that its postdecisional analysis of the rating results was useful for identifying barriers and any needed corrective action. DOD also stated that if the information obtained from a postdecisional demographic analysis demonstrated that the results were not fair or equitable, for any reason, the process used to achieve those results could be examined with the intent to identify and eliminate barriers to a fair and equitable outcome. However, our review of the postdecisional analyses that the PEO and each of the components completed following the 2007 NSPS performance management cycle and the analyses that each of the eight organizations we visited for our review completed following the most recent performance management cycle in 2008 found these analyses were inconsistent, did not always include an analysis of the rating results by demographics, and were generally conducted at too high a level to provide decision makers with sufficient information to identify potential barriers or corrective actions. For example, we found that following the 2007 performance management cycle, the PEO analyzed the rating results for more than 100,000 employees by select demographic groups, such as race, gender, ethnicity, age, veteran status, and targeted disability, but limited its analysis to the aggregate data from its pay pools. That is, the PEO did not analyze the rating results at the level where decisions are made—in the case of NSPS ratings and payouts, the pay pool level. Similarly, in analyzing the postdecisional analyses that the components conducted following the 2007 performance management cycle, we found inconsistencies in their approaches, primarily because the components were allowed to develop their own approaches for conducting this analysis. For example, only the Army and Fourth Estate included an analysis of the rating results by demographics as part of their respective postdecisional analyses. However, we also found that neither the Army’s nor the Fourth Estate’s demographic analysis of the ratings provided decision makers with sufficient information to identify possible barriers or corrective actions that could be taken to address such barriers. As with the PEO, this problem results because neither the Army nor the Fourth Estate conducted its analysis at the pay pool level. Of the eight locations we visited for our review, we found that only one organization’s postdecisional analysis following the 2008 performance management cycle included an analysis of its ratings results by demographics. Since we issued our 2008 report, DOD has taken steps to promote a degree of consistency in its postdecisional analysis of NSPS ratings and payout data. Specifically, in December 2008, DOD revised its implementing issuance to require the heads of DOD components to conduct an annual analysis of NSPS performance ratings and payouts for subordinate elements, and issue guidance to lower echelons and otherwise act to identify, examine, and remove barriers to similar rating and payout potential for demographic and other groups in the workforce. Further, in May 2009, the PEO issued guidance, entitled Guidance for Conducting Annual Analysis of NSPS Performance Ratings and Payouts, in order to provide the components with a framework and suggested approaches for conducting their annual analysis and to serve as a starting point for identifying and examining barriers. For example, the guidance notes that the NSPS performance management system’s rating and payout process has four main outcomes—the rating of record, number of shares awarded, payout, and the distribution of the payout—and that each outcome should be reviewed. The guidance also notes that analysis is best done at the level where decisions are made—in the case of NSPS ratings and payouts, the pay pool level. Further, the guidance expresses the expectation that as the components conduct their analyses, changes and improvements to the guidance will be made; is careful to ensure that the components understand base parameters for conducting the analysis so it is conducted in a manner that is methodologically sound; encourages consultation with experts, such as statisticians and human resources researchers, to assist with determining the most suitable analytical models to employ, the statistical tools to utilize, and the standards to adopt in relation to understanding, measuring, and reporting significant findings; and makes responsibility for conducting the analysis a shared responsibility between various offices, including the components’ legal, equal employment opportunity, and human resources offices, but notes that the components should consider tasking their Office of General Counsel or Office of the Judge Advocate General, whose staff are well positioned to ensure that the components are in compliance with applicable statutes, regulations, and policies, with primary responsibility for conducting the analysis and ensuring that adequate resources are provided in support of the function. While issuance of the May 2009 guidance represents a noteworthy step, the guidance does not address all steps suggested by the Equal Employment Opportunity Commission for identifying and addressing potential barriers to fair, consistent, and equitable ratings. The Equal Employment Opportunity Commission’s Instructions to Federal Agencies for EEO Management Directive 715 provides guidance that agencies can use in identifying and addressing potential barriers. The instructions state that “barrier identification and elimination is the process by which agencies uncover, examine, and remove barriers to equal participation at all levels of the workforce.” Management Directive 715 further states that “where it is determined that an identified barrier serves no legitimate purpose with respect to the operation of an agency, this Directive requires that agencies take immediate steps to eliminate the barrier.” In conducting their analysis, the components’ data may uncover barriers or other potential problems. However, understanding why the barrier or problem exists, or what to do to address it, may require that the components take additional steps. To identify and eliminate potential barriers, the directive outlines a four-step process: (1) analyzing workforce data to identify potential barriers; (2) investigating actual barriers and their causes; (3) eliminating barriers, which includes devising a plan for improvement and developing overall objectives for barrier elimination, with corresponding action items, responsible personnel, and target dates; and (4) assessing the success of the plan. The PEO’s guidance aims to promote a degree of uniformity and standardization in conducting postdecisional analyses. However, the PEO’s guidance does not specify what process the components should follow to investigate potential barriers and their causes, nor does it specify a process for eliminating barriers that are found. By not specifying such steps in its guidance, the components may not follow a consistent approach when investigating barriers, which could hinder their efforts to eliminate them. While not predecisional, we recognize that DOD’s approach does provide some benefits, some of which are similar to those of a predecisional analysis. For example, DOD’s approach lays out a method of analyzing ratings that would address some of the merit principles in 5 U.S.C. § 2301—for example, that employees should receive fair and equitable treatment in all aspects of personnel management and that employees should be protected against arbitrary action, personal favoritism, or coercion for partisan political purposes. However, as stated previously, because DOD does not specify what process the components should follow to investigate and eliminate potential barriers, the components may not follow a consistent approach, which could hinder their efforts to ensure fair, consistent, and equitable ratings. While DOD continues to take steps to ensure a reasonable amount of transparency in its implementation of NSPS, concerns about the overall transparency of the system continue to exist. To improve the transparency of the NSPS performance management system, our September 2008 report recommended that DOD require commands to publish the final rating results to employees. DOD concurred with our recommendation and, in November 2008, amended its NSPS regulations and implementing issuances to require commands to publish the final rating results to employees. Under DOD’s revised guidance, performance review authorities are required to communicate the general pay pool results to the NSPS workforce in writing. At a minimum, this includes the number of pay pools (if aggregate pay pool results are necessary), the number of employees rated, the rating and share distributions, the average rating, the average share assignment, the share value (or average share value), and the average payout expressed as a percentage of base salary. At the eight locations we visited, we found that each of pay pools shared this information with employees following the 2008 NSPS performance management cycle. DOD continues to take other steps to ensure a reasonable amount of transparency of the NSPS performance management system. In May 2009, the PEO launched “NSPS Connect,” a centralized Web portal for employees, managers, and others to access NSPS products, such as online training courses, fact sheets, tips sheets, and information on the system’s automated tools. The PEO also continues to take steps to collect and share lessons learned on the department’s experiences implementing NSPS and facilitate lessons learned briefings with the components following each performance management cycle. Further, the PEO continues to report periodically on internal assessments and employee survey results relating to the NSPS performance management system. For example, in May 2009, the PEO published the results of its 2008 evaluation of NSPS. Similarly, as we previously reported, DOD posts the results of its survey of civilian employees on a Web site that is accessible to DOD employees, supervisors, and managers. Officials at each of the eight locations we visited told us that efforts were being made to help ensure transparency of the NSPS performance management system and the rating process. According to officials, among the steps being taken to help ensure transparency, for example, were establishing multiple communities of practice, disseminating business rules and other guidance on NSPS to employees and managers under the system, and publishing monthly newsletters on NSPS. At seven of the locations, officials told us that town hall meetings were held to keep employees informed of NSPS-related developments, and officials at six locations told us that mock pay pool panels were held to show employees how the pay pool process works. However, our site visits revealed some concerns about the overall transparency of the system. For example, at three locations officials expressed concerns over a lack of transparency with regard to their pay pools’ business rules, indicating that their business rules either had not been published or were published well after the performance management cycle had begun. At one location, pay pool panel members told us that though it was 6 months into the current performance management cycle, they did not yet have copies of their business rules. Similarly, rating officials at four locations told us that they did not understand what constituted a rating of “4” because neither their pay pools nor business rules provided clear criteria. Although our site visits revealed concerns over the extent to which meaningful distinctions in individual employee performance are being made under NSPS, DOD has taken some steps that include addressing a recommendation we made in our 2008 report aimed at encouraging the use of all available rating categories. Specifically, at all eight locations we visited, officials told us that they did not believe NSPS was being implemented in a manner that encouraged meaningful distinctions in individual employee performance. For example, officials at four locations told us that they were hesitant to give ratings lower than a “3,” and officials at six locations told us that they believed that there was a forced distribution of the ratings or manipulation of the ratings in order to achieve a predetermined quota. Further, at one location we found that management stressed the importance of maintaining employee share value at above 2.0, which would result in a higher payout, regardless of the employee’s rating. According to the PEO, any effort to limit share value is a roundabout way to establish preset limits on ratings and would constitute forced distribution, which the NSPS regulations prohibit. However, in response to concerns about the potential for the forced distribution of performance ratings under NSPS, in April 2009 the department posted to its NSPS Web site a fact sheet emphasizing that the forced distribution of ratings is prohibited under NSPS and describing how meaningful distinctions in performance are made under the system. DOD’s fact sheet provides guidance specifying what constitutes the forced distribution of ratings, why the forced distribution of ratings is prohibited, how use of standard performance indicators minimizes the potential for individual bias or favoritism, and how organizations can best apply this information when rating and rewarding employee performance under NSPS. DOD has also noted that if employees believe their rating did not result from meaningful distinctions or are unfair, the system affords them the opportunity to challenge their ratings through a formal process known as reconsideration. See appendix II for further information on the NSPS reconsideration process. Within DOD, both the PEO and the components are responsible for monitoring the implementation of NSPS. As part of its efforts to monitor the implementation of NSPS, the PEO conducts broad, annual evaluations of NSPS to determine whether the system is on track to achieve certain goals, or key performance parameters, by, among other things, monitoring employee perceptions of the system using DOD’s survey of civilian employees. In May 2009, the PEO published its first evaluation of NSPS, which focused on determining whether NSPS, as implemented in spiral 1 organizations, was on track to achieve some of the goals specified by DOD and if any improvements were needed. While some of DOD’s goals for the system lend themselves to comparisons to the safeguards— for example, one of its goals is ensuring a credible and trusted system (which could align with the transparency safeguard)—PEO officials stated that to date, their focus has been limited to understanding how the components have generally implemented NSPS and not on monitoring or assessing the components’ implementation of the safeguards. With respect to the components, DOD’s implementing issuances state that the heads of DOD components are accountable for the manner in which management in their organizations carries out NSPS policy, procedures, and guidance. However, we found that only the Army has taken steps similar to the PEO to assess whether it is on track to achieve DOD’s goals for the system—the other components have not done so. Furthermore, none of the components monitors how the safeguards specifically are implemented within their organizations because there is no requirement to do so. We have previously reported that transitioning to a more performance- oriented pay system is a huge undertaking that requires constant monitoring. Further, in our 2008 assessment of NSPS, we noted that continued monitoring of the safeguards was needed to ensure that DOD’s actions were effective as implementation of NSPS proceeded. While DOD’s efforts to date have helped provide decision makers with some information on how the department is implementing NSPS, including potential areas for changes or improvements, they do not provide decision makers in DOD and the Congress with information to determine whether the safeguards specifically have been implemented effectively. Without monitoring the safeguards’ implementation, decision makers in DOD and the Congress lack information that could be used to determine whether the department’s actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. For example, in conducting our review, we identified some issues related to the implementation of the safeguards that illustrate the need to monitor their implementation. These include the following: Ensuring that adequate agency resources are allocated for the system’s design, implementation, and administration. We found that each of the components generally lacks visibility over the reasons why organizations have supplemented their pay pool funds. For example, Air Force NSPS program officials told us that for the 2009 payout, while they knew that 8 of the Air Force’s 18 major commands supplemented their pay pool funds, they did not know the specific reasons why. According to PEO officials, organizations might elect to supplement their pay pool funds for a variety of reasons—for example, to recruit or retain employees or to compete with other organizations for talent. However, because they do not understand the reasons why pay pools supplement their pay pool funding—which could help DOD and the components understand the extent to which adequate resources have been allocated to the system—decision makers cannot identify or assess any trends in these practices. Indeed, DOD’s 2008 evaluation report notes that some employees in organizations that supplemented their pay pools’ funding questioned whether the higher funding levels could be sustained over the long term. Ensuring reasonable transparency of the system and its operation. We found evidence that three of the pay pools we visited deviated from their business rules during the last performance management cycle, indicating a lack of transparency of the performance management process in some instances. DOD’s guidance states that a pay pool’s business rules are the guiding principles or ground rules that are used throughout the pay pool process, that pay pool panels should establish these principles and hold one another accountable for following them, and that a pay pool’s policies—which would include its business rules—will be made available to employees before the end of the performance cycle. However, at three of the locations we visited, we found evidence that the pay pool had deviated from its business rules during the course of the last performance management cycle. For example, at one location the pay pool’s business rules required all recommended ratings be reviewed, noting that the pay pool panel will ensure that all employees receive appropriate consideration and that ratings are fair and consistent. However, officials we spoke with at that location told us that they did not review all recommended ratings in accordance with their business rules. Rather, only the recommended ratings of “4” or “5” were reviewed. As another example, we found evidence that a pay pool at another location used different criteria than what was specified in its business rules for allocating the number of shares to employees. According to component-level NSPS program officials, in order to ensure transparency of the system, pay pools should not deviate from their business rules once those rules are published. However, none of the components requires its pay pools to notify it when such an event occurs, or of the reasons why, though doing so could help provide decision makers with information on the extent to which pay pools are implementing the system in a manner that is transparent to employees. DOD civilian personnel have mixed perceptions about NSPS, and although DOD has taken some steps toward addressing employees’ concerns, it has not yet developed and implemented an action plan to address areas where employees express negative perceptions of the system. DOD’s most recent survey of civilian employees reveals that NSPS employees have mixed perceptions about NSPS. The responses to questions specifically asking about NSPS show positive perceptions about some aspects of performance management under NSPS, including connecting pay to performance, but show negative perceptions about other aspects of performance management, such as the appraisal process. Further, the most recent data indicate that the perceptions of those employees who have worked under NSPS the longest appear to have remained largely unchanged from the negative perceptions we reported in 2008. Moreover, during discussion groups we held at eight locations outside the continental United States, civilian employees and supervisors expressed consistent concerns and negative perceptions of NSPS that are similar to those identified in our 2008 report, although they also identified positive aspects of the system. DOD has taken some steps to address employees’ negative perceptions of the system; however, the department has yet to develop and implement an action plan that meets the intent of our prior recommendation because it does not specify such things as the actions DOD intends to take, who will be responsible for taking the action, and the timelines for doing so. We continue to believe that implementing such an action plan is important and note that doing so would be a step that DOD could take to involve employees in the system’s implementation—which is one of the safeguards we previously discussed. According to DOD’s most recent survey data, some NSPS employees recognize that positive aspects of performance management, such as connecting pay to performance, exist under the system. For example, as shown in table 2, DOD’s survey data for 2008 indicate that an estimated 38 percent of NSPS employees agree that differences in their performance are recognized in meaningful ways, as compared with an estimated 33 percent of non-NSPS employees. Further, an estimated 42 percent of NSPS employees agree that pay raises depend on how well employees perform their jobs, as compared with an estimated 25 percent of non-NSPS employees. When asked about how poor performers are dealt with, an estimated 34 percent of NSPS employees, versus an estimated 27 percent of non-NSPS employees, agreed that steps are taken to deal with poor performers. In comparison, an estimated 47 percent of non-NSPS employees, as compared with an estimated 44 percent of NSPS employees, agreed that their current performance appraisal system motivates them to perform well. Further, an estimated 34 percent of non-NSPS employees, as compared with an estimated 29 percent of NSPS employees, agreed that their performance appraisal system improves organizational performance. Table 3 shows additional comparisons between NSPS and non-NSPS employee responses to questions about performance appraisals. In our first assessment of NSPS, we reported that the results of DOD’s Status of Forces Survey of Civilian Employees indicated that the perceptions of employees who had been under the system the longest had become more negative on questions related to performance management. However, the results of DOD’s most recent administration of the survey in 2008 indicate that spiral 1.1 employee perceptions are about the same as the May 2007 survey, as shown in table 4. For example, from the November 2006 through February 2008 administrations of DOD’s survey, the percentage of spiral 1.1 employees that agreed that they understood what they had to do to be rated at a different performance level declined from an estimated 59 percent in November 2006 to an estimated 53 percent in May 2007, then remained largely unchanged in February 2008 at an estimated 54 percent. In addition, when asked about the overall impact that NSPS will have on personnel practices in DOD, spiral 1.1 employees’ perceptions have become significantly more negative since first converting to NSPS in 2006, but showed little change between the May 2007 and February 2008 surveys. Specifically, the results of the 2008 survey indicate that that an estimated 22 percent of spiral 1.1 employees responded that the overall impact of NSPS on the department’s personnel practices would be positive, as compared to an estimated 23 percent in May 2007 and an estimated 25 percent in November 2006. Table 5 shows a comparison of spiral 1.1 employee responses over time about the overall impact of NSPS on personnel practices in DOD. As with our first review of NSPS, DOD civilians in our discussion groups at locations outside the continental United States continue to express wide- ranging but consistent concerns about the NSPS performance management system. Although the results of our discussion groups are not generalizable to the entire population of DOD civilian employees, the themes that emerged provide valuable insight into employees’ perceptions about the implementation of NSPS thus far. Specifically, during these discussion groups, participants at six locations told us that they were initially optimistic about the intent of NSPS and the concept of pay for performance. Further, some participants indicated that they remain positive about the amount of performance pay and flexibilities afforded to supervisors to rate their employees under the system. However, as with our first review, discussion group participants at all eight locations we visited primarily expressed frustration with and concern about certain aspects of NSPS implementation and the system. The prevalent themes that emerged during our discussion groups include concerns over (1) the negative impact of NSPS on employee motivation and morale, (2) the excessive amount of time spent navigating the performance management process, (3) challenges with job objectives, (4) factors undermining employee confidence in the system, and (5) factors unrelated to job performance affecting employees’ final performance ratings. As we noted in 2008, the themes that emerged during our discussion group sessions are not surprising. Our prior work, as well as that of OPM, has recognized that organizational transformations, such as the adoption of a new performance management system, often entail fundamental and radical changes that require an adjustment period to gain employees’ trust and acceptance. As a result, we expect major change management initiatives in large-scale organizations to take several years to be fully successful. A prevalent theme from our discussions with both employees and supervisors was that several aspects of NSPS have had a negative impact on employee motivation and morale—consistent with our first assessment of NSPS. Specifically, discussion group participants at all eight locations discussed how various aspects of NSPS—for example, their perception that a rating of “3” is only average—have negatively affected their motivation and morale. Discussion group participants at six of the eight locations also told us they have negative perceptions of what a rating of “3” means. At five of those locations, discussion group participants told us that they continue to believe that a rating of “3” means only “average,” in contrast to “valued performer,” as it was initially defined to the workforce by DOD. Discussion group participants at five locations also discussed how achieving a rating higher than a “3” seemed to be an unattainable goal. For example, employees at four locations told us that they felt NSPS either does not provide incentives for high performance or encourages only mediocre performance from employees under the system because of the high number of employees receiving “3”-level ratings each year. As another example, supervisors at one location noted that across the installation there is a general feeling that everyone receives a rating of “3,” and therefore such a rating is considered average, no matter how DOD defines it. Similarly, discussion group participants at seven locations told us that they felt it was difficult for employees in certain positions to receive a rating of “5” because of the nature of their work or the perceived value their management placed on those positions. At one of those locations, supervisors told us that they felt such things as how the pay pool’s business rules were structured affected whether an employee could receive a high rating. At that location, the pay pool’s business rules specified that an employee must receive a higher-level award, such as a command or agencywide award, to receive a rating of “5.” However, discussion group participants told us that they felt some employees were not in a position to receive such awards because of their positions or the type of work they did. In addition, discussion group participants at all eight locations we visited expressed frustration over the amount of the annual performance payout provided under NSPS. For example, discussion group participants noted that they felt the payout was not significant enough to encourage anything other than average performance. Discussion group participants at six of the eight locations also told us that they felt NSPS discourages employees from seeking additional responsibilities and opportunities that fall outside the scope of their objectives because their payout may not reflect their additional work. In addition, discussion group participants at six locations told us that because supervisory positions under NSPS require such a significant increase in responsibility and effort, and because the maximum allowable pay increase for reassignments is capped at 5 percent, some employees may not seek promotion opportunities. Similarly, a discussion group participant at another location expressed frustration that some employees only received their payout in the form of a bonus and not an increase in salary. Discussion group participants at three of the eight locations also expressed concerns that they felt performance payouts under NSPS tended to benefit higher-paid employees at the expense of lower-paid employees. For example, employees at one location expressed concerns that in their pay pool, the higher payouts under NSPS seemed to go to employees at the top of the pay bands. Another prevalent theme at seven of the eight locations and also highlighted in our first assessment was that employees spend an excessive amount of time navigating the performance management process. While the discussion group participants’ complaints about the time- and labor- intensive nature of the system were not limited to any one part of the process, discussion group participants at seven locations pointed out that the time and effort required to complete the steps of the NSPS performance management process were significantly greater than what was required of them under previous systems. For example, one supervisor we spoke with speculated that his supervisory duties under NSPS took him six times as long to perform as they had under the GS system, while another supervisor told us that he may have spent from 45 to 50 hours assessing the performance of three employees, a task he could have completed in 10 hours under the GS system. At five of the locations we visited, employees expressed concerns about NSPS potentially affecting their ability to complete their jobs or affecting the mission because of what they perceived as an excessive amount of time required of employees and supervisors in navigating the NSPS performance management process. In some instances, employees spoke of impacts NSPS was having on their supervisors, while others spoke of their own experiences navigating the NSPS performance management process. At three locations, discussion group participants described how what they perceived as an excessive amount of time navigating the NSPS performance management process affected their ability to complete their job-related duties, requiring completion of some NSPS tasks, such as self- assessments and employee ratings, after work hours or on weekends. One employee described feeling inundated with information on NSPS and that it was difficult to stay on top of things while simultaneously performing his job, while another employee estimated that she spent about 2 hours per week on NSPS-related tasks. In some instances, discussion group participants told us that they saw the potential for the excessive time commitment required by NSPS to affect the missions of their organizations. According to one supervisor, any task that takes employees away from their daily work affects the mission, and any task that takes the time and patience of the command’s leadership detracts from the mission. Further, in discussing during a site visit the potential for NSPS to impact the organization’s mission, one general officer we spoke with described NSPS as “mission ineffective.” Another prevalent theme that emerged from our discussions with both supervisors and employees at all eight locations was that there are challenges with employee job objectives under NSPS. According to DOD, the NSPS performance management system is designed to provide a fair and equitable method for appraising and evaluating performance. As part of the system, DOD established the concept of “job objectives,” which are the required tasks of a given job as determined by managers and supervisors, and directed that job objectives be developed and used as the standards for evaluating employee performance. However, supervisors and employees at each of the eight locations discussed challenges they experienced developing their job objectives under NSPS. Specifically: Although DOD guidance encourages employees to develop job objectives that are specific, measurable, aligned, realistic, and timed— an approach summarized by the acronym S.M.A.R.T.—employees and supervisors we met at six of eight locations discussed how they found it challenging to develop job objectives that are measurable or that follow the S.M.A.R.T. approach. Supervisors at one location objected to the S.M.A.R.T. approach, particularly the “specific” portion, because they felt that job objectives needed to be broad enough to allow employees to discuss any accomplishments they make if they complete additional job activities or other tasks that might arise during the year. Supervisors at two locations discussed how the work they did was nebulous and unpredictable, which made it challenging to develop job objectives that not only reflected the nature of their work but that they could exceed. Similarly, supervisors at another location expressed concerns that employees’ job objectives may not reflect the work they do by the end of the performance management cycle because of constant changes within their organization. According to discussion group participants at four locations, guidance for developing job objectives is either limited or nonexistent, which may result in different approaches to developing job objectives across an organization. At one of these locations, employees told us that their management had not established consistent ground rules for developing job objectives and that as a result some employees’ job objectives were based on out-of-date position descriptions. One organization we visited used a mixture of mandatory and employee- specific job objectives; but, according to one employee, little guidance exists to help employees and supervisors when they need to develop personalized job objectives. Employees at another location told us that there were significant differences in the amount of involvement they had in developing their job objectives. For example, one individual told us that employees in her office develop their own objectives, while another said employees in her office are assigned mandatory objectives and were thus unable to provide input into their objectives. Discussion group participants at six locations expressed concerns that it can be difficult to achieve a high rating for some job objectives. Some locations we visited used mandatory job objectives, which left employees concerned that their job objectives did not accurately capture the full responsibilities of the work they performed. For example, at one location, a uniform, mandatory supervisory objective accounted for half of supervisors’ ratings, which, according to one supervisor, diminished the value of the other responsibilities they had. The supervisor expressed further concern that some mandatory job objectives, such as those assigned to government purchase card holders, require a pass-fail evaluation, making it difficult, if not impossible, for the employee to receive a high rating. In one instance, a location we visited required all employees to be rated against a mandatory safety objective. However, according to some supervisors, it did not make sense for everyone to have the mandatory safety objective because for many employees, safety issues were out of their control. During our discussion groups, participants at all eight locations also discussed how various factors undermine employees’ confidence in the system and its implementation thus far. For example, discussion group participants at six locations commented that they do not believe that the NSPS performance management system has the ability to rate employees fairly. At the locations we visited, discussion of these concerns centered on such things as the perception of subjectivity and the potential for favoritism under NSPS; a lack of transparency surrounding the pay pool panel process, including a lack of understanding of what employees needed to do to receive higher ratings; and the perception of inconsistencies in interpretations of the standards used to determine employee ratings. One prevalent theme at all eight locations involved perceptions of subjectivity, such as the potential for favoritism under NSPS during the rating and pay pool panel processes. At five locations, participants discussed their frustration with how NSPS takes the responsibility for rating employees out of the hands of supervisors and places it in the hands of the pay pool panel members, who may or may not have any direct knowledge of individual employees’ performance. One supervisor told us that NSPS may inadvertently favor employees who work closely or are in direct contact with members of the pay pool panel because those individuals have direct knowledge of the employees and, sometimes, their performance. Similarly, supervisors at another location told us that they did not feel that their pay pool panel understood their jobs and what they do and expressed frustration that the pay pool panel did not seem to be reaching out to their supervisors and higher-level reviewers for additional input on their performance. At five of the eight locations, discussion group participants also told us that they saw the potential for the employee- supervisor relationship to affect an employee’s rating—either to the benefit or detriment of the employee. Another prevalent theme at six of the eight locations—a theme also highlighted in our first assessment of NSPS—was a lack of transparency and understanding of the pay pool panel process. Specifically, supervisors at two locations commented that their organizations’ pay pool panel processes were not transparent. A supervisor at one location commented that everything “goes into a black vacuum,” while another likened the process to a “black box.” Employees at that same location referred to the organization’s pay pool panel process as a “star chamber,” where decisions are made but are not explained to employees. Employees and supervisors at five locations expressed concerns about the amount of information they received from their pay pools and about the process itself; some desired further information to help them better understand the pay pool panel process. In addition, at six of the eight locations, discussion group participants told us that they did not understand what they needed to do to receive a higher rating. For example, an employee at one location told us that she was told by her supervisor that all employees had to receive a rating of “3” because they would have had to “walk on water” to receive a higher rating. Discussion group participants at two other locations also discussed how “walking on water” was a perceived standard for receiving a high rating under NSPS. At three locations, supervisors commented that they were unclear about what they could do to help their employees receive better ratings, while employees at four locations were unclear about what they could do to achieve higher ratings. Discussion group participants at six locations also raised concerns about inconsistent interpretation of the standards used when evaluating civilian employees under NSPS. Discussion group participants reported concerns that military supervisors may rate employees using more stringent standards than their civilian counterparts. Discussion group participants also reported concerns that some military supervisors may not value the NSPS performance management process and sometimes devote less time and effort to the process, which could affect employees’ ratings. One civilian supervisor told us that some military supervisors with whom he attended NSPS training had a much harsher perspective of employee performance than their civilian counterparts. For example, he noted that the military supervisors indicated that giving a rating of “1” or “2” was acceptable, whereas he believed civilian supervisors would be more inclined to give an employee a rating of “3.” Employees also told us that they do not believe some military supervisors value the work of employees who perform certain job functions, such as providing child care on an installation. A prevalent theme expressed by discussion group participants at all eight locations we visited is that factors unrelated to performance may affect employees’ final performance ratings. Such factors include the existence of a forced distribution or quota of ratings, the writing ability of employees and supervisors, and pay pool panel members’ knowledge of employees. For example: Discussion group participants at all eight locations expressed concerns that their pay pool panels used a forced distribution or quota for ratings, which dictated the number of ratings in each category that could be awarded. Employees at one location told us that they were aware of their management’s attempts to artificially preserve a higher share value for employees by primarily awarding ratings of “3,” regardless of the employees’ performance. Further, at three locations discussion group participants told us that their management told them that all employees should expect to receive a rating of “3.” Moreover, some discussion group participants told us that they doubted that their actual performance had the bearing it was supposed to have on their final ratings, while others felt the use of a forced distribution or quotas was in direct conflict with the principles of pay for performance under NSPS. While no discussion group participants we met with were aware of any explicit guidance provided to pay pool panels or supervisors that limited the number of certain ratings they assigned employees, employees and supervisors from at least three locations believed that informal guidelines existed or that pay pool panels or supervisors were encouraged to limit the number of certain ratings they could assign. Discussion group participants at all eight locations also expressed concerns that the writing ability of employees and supervisors may affect ratings—a theme also highlighted in our first assessment of NSPS. Supervisors at one location likened the process of developing employees’ assessments under NSPS to a writing contest. Moreover, supervisors told us that they felt their writing ability could unintentionally affect their employees’ ratings, noting, for example, that a supervisor’s ability to articulate an employee’s achievements in writing plays a significant role in supporting a higher rating for that employee. Employees shared the supervisors’ concerns, noting that they believed that succeeding under NSPS depended on the quality of their written assessments, rather than their job performance, and that their ratings could suffer if their supervisors did not provide the pay pool panel with well-written assessments. In discussing the potential influence that employees’ and supervisors’ writing skills may have on a pay pool panel’s assessment of an employee, officials at seven of the eight locations acknowledged that in some instances writing skills had affected employees’ ratings and could overshadow employees’ performance. Discussion group participants at seven locations also expressed frustration that employee ratings were potentially affected by the extent to which pay pool panel members have personal knowledge of employees or understand the nature of their work in general. Some discussion group participants felt that pay pool panel members’ personal knowledge of employees helped some employees receive higher ratings, while others told us that they felt that members of the pay pool panel were too far removed from them and lacked direct knowledge of the work they performed. One employee believed that individuals who were involved in implementing NSPS worked closely with pay pool panel members, or were senior managers who were more likely to receive higher ratings under NSPS than others. Other employees told us that they were concerned about the potential for pay pool panel members to advocate in some way for employees they personally know—for example, by encouraging the pay pool panel to contact a specific employee’s supervisor to seek additional information or justification for a rating. As a result, they felt that pay pool panel members’ personal knowledge of employees could benefit some employees, but not others. In our first assessment of NSPS, we recommended that DOD develop and implement a specific action plan to address employees’ perceptions of NSPS, based on guidance published by OPM for conducting annual employee surveys and providing feedback to employees on the results. The guidance suggests that after an agency’s survey results have been reviewed, the agency has a responsibility to provide feedback to employees on the results of the survey, as well as to let employees know the intended actions to address the results and the progress made on these actions. Further, the guidance suggests that agencies consider the following when developing action plans: who will be responsible for taking action; who will be responsible for providing oversight; if the individuals taking the action have the necessary authority to make things happen; what coordination, if any, is required, and how it will be accomplished; how agencies will adjust given any changes or delays in their actions. Since then, in June 2009, the PEO issued a departmentwide memorandum entitled “Addressing Key NSPS Workforce Concerns”; however, issuance of this memorandum does not fully meet the intent of our 2008 recommendation. Specifically, the PEO’s June 2009 memorandum summarizes key concerns from the department’s 2008 evaluation of NSPS, summarizes departmentwide actions that had been taken to date to address employees’ concerns about the system, and suggests approaches to enhance local efforts to address workforce concerns. The PEO identified five key areas of concern, which are similar to those identified in our own discussion group sessions with DOD employees and supervisors: (1) performance communication and feedback, (2) understanding of performance management and the pay pool process, (3) trust in the system and its processes, (4) training and information, and (5) the amount of time needed to fulfill performance management responsibilities. The PEO’s memorandum urged the components to leverage information from the department’s 2008 evaluation of NSPS and focus on the five areas discussed above as they plan their own actions. Further, the PEO’s memorandum noted that DOD has taken some steps to address employees’ concerns about NSPS—for example, developing and fielding a pay pool training course for employees and rating officials, modifying its implementing issuances to require all performance review authorities to review pay pool panel results on an annual basis, and providing guidance to employees on the prohibition against the forced distribution of ratings. Issuance of the PEO’s memorandum represents an important first step. However, because the memorandum does not specify actions the department intends to take, who will be responsible for taking the action, and timelines for addressing areas where employees express negative perceptions of the system, it does not fully meet the intent of our 2008 recommendation. In developing an action plan, we note that OPM recently issued guidance that agencies can use in developing action plans for improving employee satisfaction. According to OPM, action plans should clearly (1) state the objectives, (2) identify actions to be taken, (3) provide outcome measures and improvement targets, and (4) describe how progress will be tracked. In addition to identifying the specific actions that will be taken to achieve improvements, OPM’s guidance also suggests that agencies specify time frames for accomplishing the actions, who will be responsible for implementing the actions, who will be affected by the actions, the resources required, and a plan to communicate these actions to managers and employees. We continue to believe that developing and implementing a plan to address employees’ perceptions of NSPS could help DOD make changes to the system that could lead to greater employee acceptance and, ultimately, the system’s successful implementation. Further, we note that having such a plan is an approach that DOD could take to involve employees in the system’s implementation—which is one of the safeguards we previously discussed. As we noted in our first assessment, DOD’s implementation of NSPS placed the department at the forefront of a significant transition facing the federal government. However, toward the end of this review, the future of NSPS became uncertain, given the proposed legislation that, if enacted, would terminate the system and require any future system created by DOD to use safeguards similar to those discussed in our report, including ensuring employee involvement in the system and providing adequate training and retraining. In light of the contingent nature surrounding NSPS and the possibility of implementing a different system, sustained and committed leadership will be imperative to provide focused attention necessary to implement any pay-for-performance system within DOD. Key to implementing a fair, effective, and credible system is including safeguards early on in the design of the system. Since we issued our first assessment of NSPS in 2008, we note that DOD has continued to take steps to meet the intent of each of the safeguards. However, with this latest assessment, we note that the department has not implemented the safeguards systematically; for example, it has not ensured that the training provided to employees on the system’s operations is effective. Further, DOD has not monitored how the safeguards specifically are implemented by lower-level organizations across the department. As a result, decision makers in DOD lack information that could be used to determine whether the department’s actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. Additionally, while DOD has gained experience operating under NSPS, at the time of our review it had not yet developed an action plan for addressing employees’ perceptions of the system, as we recommended in 2008. As DOD moves forward with implementing a pay-for-performance system—whether NSPS or another—we believe that it is important for the department to improve upon its implementation of the safeguards and address employees’ concerns. Left unchecked, these issues could undermine any future human capital reform efforts within DOD. To help implement a fair, effective, and credible performance management system for its civilian employees—whether NSPS or another—we recommend that the Secretary of Defense take the following three actions: Review and evaluate the effectiveness of the department’s training. Ensure that guidance is in place for conducting a postdecisional analysis that specifies what process the components should follow to investigate and eliminate potential barriers to fair and equitable ratings. Include, as part of the department’s monitoring of the implementation of its system, efforts to monitor and evaluate how the safeguards specifically are implemented by lower-level organizations across the department. In September 2009, we provided DOD with a draft of this report that included three recommendations to better address the safeguards and improve implementation of the NSPS performance management system. Specifically, we recommended that DOD (1) evaluate NSPS training, (2) review and revise its guidance for conducting postdecisional analysis of NSPS ratings, and (3) monitor how the safeguards specifically are implemented. In commenting on a draft of our report, DOD partially concurred with our three recommendations. DOD’s comments are reprinted in appendix III. DOD partially concurred with our recommendations, noting the expectation that the Congress would require the department to terminate NSPS by January 1, 2012, and this action, in turn, would require the department to focus on drawing down NSPS in an orderly manner. DOD further stated that it would consider acting on our recommendations to the extent they are relevant as the department moves forward with any future performance management system. We believe that this is a reasonable approach. As discussed above, we recognize the contingent nature surrounding NSPS as a result of provisions in the proposed National Defense Authorization Act for Fiscal Year 2010, which recently passed both Houses of Congress. Accordingly, we revised our recommendations to apply to any future performance management system for the department’s civilian employees—whether NSPS or another system. However, we also note that provisions of the proposed legislation would require DOD to implement certain safeguards and issue regulations for that system to provide a fair, credible and transparent performance appraisal system. We therefore continue to believe that our recommendations have merit. We are sending copies of this report to the appropriate congressional committees. We will make copies available to others upon request. The report also is available at no charge on GAO’s Web site at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to the report are listed in appendix IV. As with our first assessment of the National Security Personnel System (NSPS) in 2008, we limited our scope in conducting this review to the performance management aspect of NSPS. Therefore, we addressed neither performance management of the Senior Executive Service at the Department of Defense (DOD) nor other aspects of NSPS, such as classification and pay. To determine the extent to which DOD has implemented safeguards as part of the NSPS performance management system and monitored the implementation of the safeguards, we used the following safeguards, which we also reported on in our 2008 review: Involve employees, their representatives, and other stakeholders in the design of the system, to include employees directly involved in validating any related implementation of the system. Assure that the agency’s performance management systems link employee objectives to the agency’s strategic plan, related goals, and desired outcomes. Provide adequate training and retraining for supervisors, managers, and employees in the implementation and operation of the performance management system. Provide a process for ensuring ongoing performance feedback and dialogue between supervisors, managers, and employees throughout the appraisal period, and for setting timetables for review. Implement a pay-for-performance evaluation system to better link individual pay to performance, and provide an equitable method for appraising and compensating employees. Assure that certain predecisional internal safeguards exist to help achieve consistency, equity, nondiscrimination, and nonpoliticization of the performance management process (e.g., independent reasonableness reviews by a third party or reviews of performance rating decisions, pay determinations, and promotions before they are finalized to ensure that they are merit-based, as well as pay panels who consider the results of the performance appraisal process and other information in connection with final pay decisions). Assure that there are reasonable transparency and appropriate accountability mechanisms in connection with the results of the performance management process, to include reporting periodically on internal assessments and employee survey results relating to performance management and individual pay decisions while protecting individual confidentiality. Assure that performance management results in meaningful distinctions in individual employee performance. Provide a means for ensuring that adequate agency resources are allocated for the design, implementation, and administration of the performance management system. To assess implementation of the safeguards, we reviewed the legislative requirements and obtained and analyzed regulations and other guidance for implementing the NSPS performance management system. We also obtained and analyzed other documents, such as DOD’s rating results and reconsideration statistics, for the 2007 and 2008 NSPS performance management cycles. We also interviewed knowledgeable officials in DOD’s NSPS Program Executive Office and the NSPS program offices of the four components—the Army, the Air Force, the Navy, and the Fourth Estate—to obtain a comprehensive understanding of their efforts to implement NSPS and each of the safeguards, as well as the processes, procedures, and controls used for monitoring and overseeing implementation of the system. In addition, we conducted site visits to select organizations located outside the continental United States to assess implementation of the safeguards. To allow for appropriate representation by each component, we visited two organizations per component, or eight organizations in total. The organizations we visited were selected based on a number of factors, such as the presence of a large number or concentrated group of civilian employees under NSPS and, when possible, the presence of employees who had converted to NSPS under both spirals 1 and 2. We focused our efforts for this review on visiting organizations located outside the continental United States because our 2008 review focused on assessing implementation of NSPS and the safeguards at locations that were geographically distributed throughout the United States. We elected to focus our site visits in Germany and Hawaii because of the civilian employees located outside the continental United States who had converted to NSPS at the time we initiated our review, more than half were located in either Germany or Hawaii. Also, we wanted to determine whether civilian employees located outside the continental United States were experiencing any unique problems or challenges with the system. In Germany, the organizations we visited were the 5th Signal Command; the 435th Air Base Wing; the Defense Finance and Accounting Service; and the George C. Marshall European Center for Security Studies, part of the Defense Security Cooperation Agency. In Hawaii, the organizations we visited were the Commander, Navy Region Hawaii; Headquarters, Pacific Air Force; the Naval Facilities Engineering Command, Hawaii; and the U.S. Army Corps of Engineers, Honolulu District. For each of the organizations we visited, we met with or interviewed the performance review authority, pay pool managers, pay pool panel members, rating officials, and the NSPS program manager or transition team, among others, to discuss the steps they have taken to implement the safeguards or otherwise ensure the fairness, effectiveness, and credibility of NSPS. To assess the organizations’ implementation of the safeguards, we compared and contrasted the information obtained during our interviews and supplemented this testimonial evidence with the other relevant documentation we obtained, such as the organizations’ pay pool business rules, lessons learned, and training materials. To determine how DOD civilian employees perceive NSPS, we analyzed two sources of employee perceptions or attitudes. First, we analyzed the results of DOD’s survey of civilian employees to identify employee perceptions of NSPS and examine whether and how these perceptions may be changing over time. Second, we conducted small group discussions with civilian employees who had converted to NSPS and administered a short questionnaire to the participants at each of the eight organizations we visited. As with our first assessment of NSPS, our overall objective in using the discussion group approach was to obtain employees’ perceptions about NSPS and its implementation thus far. We analyzed the results of the Defense Manpower Data Center’s (DMDC) Status of Forces Survey of Civilian Employees—including the May 2006, November 2006, May 2007, and February 2008 administrations—to gauge employee attitudes toward NSPS and performance management in general and identify indications of movement or trends in employee perceptions. As we reported in September 2008, we have reviewed the results of prior administrations of DMDC surveys and found the survey results, including the results of the Status of Forces Survey of Civilian Employees, sufficiently reliable to use for several GAO engagements. However, to understand the nature of any changes that were made to its survey methods for administering the survey for 2008 as compared with previous administrations, we also received responses to written questions from and discussed these data with officials at DMDC. Based on these responses and discussions, we determined that DMDC’s survey data remain sufficiently reliable for the purpose of our reports on DOD civilian employees’ perceptions of NSPS. We also conducted small group discussions with DOD civilian employees and administered a short questionnaire during site visits in February and March 2009. Specifically, we conducted two discussion groups—one with nonsupervisory employees and another with supervisory employees—at each of the eight locations we visited, for a total of 16 discussion groups. As with our first assessment of NSPS in 2008, our objective in using this approach was to obtain employees’ perceptions about NSPS and its implementation thus far because discussion groups are intended to provide in-depth information about participants’ reasons for holding certain attitudes about specific topics and to offer insights into the range of concerns about and support for an issue. Further, in conducting our discussion groups, our intent was to achieve saturation—the point at which we were no longer hearing new information. As we previously reported, our discussion groups were not designed to (1) demonstrate the extent of a problem or to generalize the results to a larger population, (2) develop a consensus to arrive at an agreed-upon plan or make decisions about what actions to take, or (3) provide statistically representative samples or reliable quantitative estimates. Instead, our discussion groups provide in-depth information about participants’ reasons for holding certain attitudes about specific topics and offer insights into the range of concerns about and support for an issue. Although the results of our discussion sessions are not generalizable to the entire NSPS civilian population, the composition of our discussion groups was designed to ensure that we spoke with employees from each of the four components at locations outside the continental United States. Because supervisory and nonsupervisory employees have distinct roles with respect to NSPS, we held separate discussion sessions for these groups. To select the discussion group participants, we requested that the organizations we visited provide us with lists of employees who had converted to NSPS. From the lists provided, we selected participants based on their supervisory and nonsupervisory status. To ensure maximum participation of the selected employees, we randomly selected up to 20 participants from each group with the goal of meeting with 8 to 12 individuals in each discussion group and provided the employee names and a standard invitation to GAO’s points of contact to disseminate to the employees. At the majority of locations, we reached our goal of meeting with 8 to 12 individuals in each discussion group; however, since participation was not compulsory, in two instances we did not reach our goal of 8 participants per discussion group. Table 6 provides information on the composition of our discussion groups. To facilitate our discussion groups, we developed a discussion guide to help the moderator in addressing several topics related to employees’ perceptions of the NSPS performance management system. These topics include employees’ overall perception of NSPS and the rating process, the training they received on NSPS, the communication they have had with their supervisors, positive aspects they perceive of NSPS, and any changes they would make to the system, among others. Each discussion group was scheduled for a 2-hour period and began with the GAO moderator greeting the participants, describing the purpose of the study, and explaining the procedures for the discussion group. Participants were assured that all of their comments would be discussed in the aggregate or as part of larger themes that emerged. The GAO moderator asked participants open-ended questions related to NSPS, while at least one other GAO analyst observed the discussion group and took notes. Following the conclusion of all our discussion group sessions, we performed content analysis of the sessions in order to identify the themes that emerged and to summarize the participants’ perceptions of NSPS. We reviewed responses from several of the discussion groups and created a list of themes and subtheme categories. We then reviewed the comments from each of the 16 discussion groups and assigned each comment to the appropriate category, which was agreed upon by two analysts. If agreement was not reached on a comment’s placement in a category, another analyst reconciled the issue by placing the comment in either one or more of the categories. The responses in each category were then used in our evaluation and discussion of how civilian employees perceive NSPS. Following each discussion group we administered a questionnaire to the participants to obtain further information on their background, tenure with the federal government and DOD, and attitudes toward NSPS. We received questionnaires from 164 discussion group participants. In addition to collecting demographic data from participants for the purpose of reporting with whom we spoke (see table 7), the purpose of our questionnaire was to (1) collect information from participants that could not easily be obtained through discussion, for example, information participants may have been uncomfortable sharing in a group setting, and (2) collect some of the same data found in past DOD surveys. Specifically, the questionnaire included questions designed to obtain employees’ perceptions of NSPS as compared to their previous personnel system, the accuracy with which they felt their ratings reflected their performance, and management’s methods for conveying overall rating information. Since the questionnaire was used to collect supplemental information and was administered solely to the participants of our discussion groups, the results represent the opinions of only those employees and cannot be projected across DOD, a component, or any single pay pool we visited. Commander, Navy Region Hawaii, Pearl Harbor, Hawaii Naval Facilities Engineering Command, Hawaii, Pearl Harbor, Hawaii NSPS Program Office, Navy Yard, Washington, D.C. Office of Civilian Human Resources, Navy Yard, Washington, D.C. We conducted this performance audit from November 2008 through September 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient and appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Department of Defense (DOD) civilian employees who receive performance ratings under the National Security Personnel System (NSPS) have the option of challenging their ratings through a formal process known as reconsideration. The reconsideration process is the sole and exclusive agency administrative process for nonbargaining unit employees to challenge their ratings. However, DOD’s NSPS regulations also allow for alternative dispute resolution techniques, such as mediation or interest-based problem solving, to be pursued at any time during the reconsideration process consistent with component policies and procedures. Under the reconsideration process, employees may challenge their ratings of record or individual job objective ratings; employees cannot challenge their performance payout, the number of shares assigned, the share value, or the distribution of their performance payout between salary increase and bonus, nor can they challenge their recommended ratings of record, interim reviews, or applicable closeout assessments. In addition, employees who allege that their performance ratings are based on prohibited discrimination or reprisal may not use the reconsideration process; rather, such allegations are to be processed through the department’s equal employment opportunity discrimination complaint procedure. Employees who wish to challenge their rating have 10 calendar days from the receipt of their ratings of record to submit written requests for reconsideration to their pay pool managers. Within 15 calendar days of the pay pool manager’s receipt of an employee’s request for reconsideration, the pay pool manager must render a written decision that includes a brief explanation of the basis of the decision. The pay pool manager’s decision is final, unless the employee seeks further reconsideration from the performance review authority. Specifically, if the employee is dissatisfied with the pay pool manager’s decision, or if none is provided within the prescribed time frames, the employee may submit a written request for final review by the performance review authority or his or her designee. This request must be submitted within 5 calendar days of receipt of the pay pool manager’s decision or within 5 calendar days of the date the decision should have been rendered. The performance review authority then is allotted 15 calendar days from receipt of the written request from the employee to make a decision, which is final. If the final decision is to change the rating of record or job objective rating, the revised rating takes the place of the original one, and a revised performance appraisal is prepared for the employee. According to DOD’s 2008 evaluation report, for the 2007 NSPS performance management cycle, 2,302 civilian employees out of the 100,465 employees who were rated under NSPS elected to file a request for reconsideration of their ratings, and of these, about 33 percent of the requests were granted. For the 2008 NSPS performance management cycle, according to the NSPS Program Executive Office, as of June 2009, 4,296 civilian employees out of the 170,149 employees who were rated under NSPS elected to file requests for reconsideration of their ratings, and of these, about 52 percent of the requests were granted. In addition to the contact named above, Ron Fecso, Chief Statistician; Marion Gatling, Assistant Director; Margaret G. Braley; Virginia A. Chanley; William Colwell; Emily Gruenwald; K. Nicole Harms; Cynthia Heckmann; Wesley A. Johnson; Lonnie McAllister; Carolyn Taylor; John W. Van Schaik; Jennifer L. Weber; Cheryl A. Weissman; and Gregory H. Wilmoth made key contributions to the report. Human Capital: Continued Monitoring of Internal Safeguards and an Action Plan to Address Employee Concerns Could Improve Implementation of the National Security Personnel System. GAO-09-840. Washington, D.C.: June 25, 2009. Questions for the Record Related to the Implementation of the Department of Defense’s National Security Personnel System. GAO-09-669R. Washington, D.C.: May 18, 2009. Human Capital: Improved Implementation of Safeguards and an Action Plan to Address Employee Concerns Could Increase Employee Acceptance of the National Security Personnel System. GAO-09-464T. Washington, D.C.: April 1, 2009. Human Capital: Opportunities Exist to Build on Recent Progress to Strengthen DOD’s Civilian Human Capital Strategic Plan. GAO-09-235. Washington, D.C.: February 10, 2009. High-Risk Series: An Update. GAO-09-271. Washington, D.C.: January 2009. Human Capital: DOD Needs to Improve Implementation of and Address Employee Concerns about Its National Security Personnel System. GAO-08-773. Washington, D.C.: September 10, 2008. The Department of Defense’s Civilian Human Capital Strategic Plan Does Not Meet Most Statutory Requirements. GAO-08-439R. Washington, D.C.: February 6, 2008. Human Capital: DOD Needs Better Internal Controls and Visibility over Costs for Implementing Its National Security Personnel System. GAO-07-851. Washington, D.C.: July 16, 2007. Human Capital: Federal Workforce Challenges in the 21st Century. GAO-07-556T. Washington, D.C.: March 6, 2007. Post-Hearing Questions for the Record Related to the Department of Defense’s National Security Personnel System (NSPS). GAO-06-582R. Washington, D.C.: March 24, 2006. Human Capital: Observations on Final Regulations for DOD’s National Security Personnel System. GAO-06-227T. Washington, D.C.: November 17, 2005. Human Capital: Designing and Managing Market-Based and More Performance-Oriented Pay Systems. GAO-05-1048T. Washington, D.C.: September 27, 2005. Human Capital: Symposium on Designing and Managing Market-Based and More Performance-Oriented Pay Systems. GAO-05-832SP. Washington, D.C.: July 27, 2005. Human Capital: DOD’s National Security Personnel System Faces Implementation Challenges. GAO-05-730. Washington, D.C.: July 14, 2005. Questions for the Record Related to the Department of Defense’s National Security Personnel System. GAO-05-771R. Washington, D.C.: June 14, 2005. Questions for the Record Regarding the Department of Defense’s National Security Personnel System. GAO-05-770R. Washington, D.C.: May 31, 2005. Post-hearing Questions Related to the Department of Defense’s National Security Personnel System. GAO-05-641R. Washington, D.C.: April 29, 2005. Human Capital: Agencies Need Leadership and the Supporting Infrastructure to Take Advantage of New Flexibilities. GAO-05-616T. Washington, D.C.: April 21, 2005. Human Capital: Selected Agencies’ Statutory Authorities Could Offer Options in Developing a Framework for Governmentwide Reform. GAO-05-398R. Washington, D.C.: April 21, 2005. Human Capital: Preliminary Observations on Proposed Regulations for DOD’s National Security Personnel System. GAO-05-559T. Washington, D.C.: April 14, 2005. Human Capital: Preliminary Observations on Proposed Department of Defense National Security Personnel System Regulations. GAO-05-517T. Washington, D.C.: April 12, 2005. Human Capital: Preliminary Observations on Proposed DOD National Security Personnel System Regulations. GAO-05-432T. Washington, D.C.: March 15, 2005. Human Capital: Principles, Criteria, and Processes for Governmentwide Federal Human Capital Reform. GAO-05-69SP. Washington, D.C.: December 1, 2004. Human Capital: Building on the Current Momentum to Transform the Federal Government. GAO-04-976T. Washington, D.C.: July 20, 2004. DOD Civilian Personnel: Comprehensive Strategic Workforce Plans Needed. GAO-04-753. Washington, D.C.: June 30, 2004. Human Capital: Implementing Pay for Performance at Selected Personnel Demonstration Projects. GAO-04-83. Washington, D.C.: January 23, 2004. Human Capital: Key Principles for Effective Strategic Workforce Planning. GAO-04-39. Washington, D.C.: December 11, 2003. DOD Personnel: Documentation of the Army’s Civilian Workforce- Planning Model Needed to Enhance Credibility. GAO-03-1046. Washington, D.C.: August 22, 2003. Posthearing Questions Related to Proposed Department of Defense (DOD) Human Capital Reform. GAO-03-965R. Washington, D.C.: July 3, 2003. Human Capital: Building on DOD’s Reform Effort to Foster Governmentwide Improvements. GAO-03-851T. Washington, D.C.: June 4, 2003. Posthearing Questions Related to Strategic Human Capital Management. GAO-03-779R. Washington, D.C.: May 22, 2003. Human Capital: DOD’s Civilian Personnel Strategic Management and the Proposed National Security Personnel System. GAO-03-493T. Washington, D.C.: May 12, 2003. Defense Transformation: DOD’s Proposed Civilian Personnel System and Governmentwide Human Capital Reform. GAO-03-741T. Washington, D.C.: May 1, 2003. Defense Transformation: Preliminary Observations on DOD’s Proposed Civilian Personnel Reforms. GAO-03-717T. Washington, D.C.: April 29, 2003. High-Risk Series: Strategic Human Capital Management. GAO-03-120. Washington, D.C.: January 2003. Acquisition Workforce: Status of Agency Efforts to Address Future Needs. GAO-03-55. Washington, D.C.: December 18, 2002. Military Personnel: Oversight Process Needed to Help Maintain Momentum of DOD’s Strategic Human Capital Planning. GAO-03-237. Washington, D.C.: December 5, 2002. Managing for Results: Building on the Momentum for Strategic Human Capital Reform. GAO-02-528T. Washington, D.C.: March 18, 2002. A Model of Strategic Human Capital Management. GAO-02-373SP. Washington, D.C.: March 15, 2002. Human Capital: Taking Steps to Meet Current and Emerging Human Capital Challenges. GAO-01-965T. Washington, D.C.: July 17, 2001. Human Capital: Major Human Capital Challenges at the Departments of Defense and State. GAO-01-565T. Washington, D.C.: March 29, 2001. High-Risk Series: An Update. GAO-01-263. Washington, D.C.: January 2001.

Summary: In 2004, the Department of Defense (DOD) began implementing the National Security Personnel System (NSPS)--a human capital system for DOD civilians. NSPS significantly redesigned the way DOD civilians are hired, compensated, and promoted. Pub. L. No. 110-181 mandated that GAO conduct reviews of the NSPS performance management system in calendar years 2008, 2009, and 2010. In this report, GAO assessed (1) the extent to which DOD has implemented certain internal safeguards to ensure the fairness, effectiveness, and credibility of NSPS, and monitored their implementation, and (2) how DOD civilian personnel perceive NSPS, and the actions DOD has taken to address those perceptions. GAO analyzed relevant documents and employee survey results, interviewed DOD officials, and conducted discussion groups with DOD employees at eight locations outside of the continental United States. Toward the end of GAO's review, both Houses of Congress passed proposed legislation that, if enacted, would terminate NSPS and require any future performance management system for DOD civilians to include certain internal safeguards DOD continues to take steps to implement internal safeguards as part of NSPS, but implementation of some safeguards could still be improved, and continued monitoring of all safeguards' implementation is needed. In general, DOD has taken some steps to meet the intent of each of the safeguards, and it has implemented some of the recommendations from GAO's September 2008 report. However, opportunities exist for DOD to improve implementation of some safeguards. For example, DOD has not yet evaluated the effectiveness of the training employees receive, although doing so could help DOD measure the impact of its training and its progress toward achieving agency goals. In addition, DOD has not specified in its guidance what process the components should follow to investigate and eliminate potential barriers to fair and equitable ratings. Consequently, the components may not follow a consistent approach when investigating potential barriers, which could hinder their efforts to eliminate them. Further, GAO previously noted that continued monitoring of the safeguards was needed to ensure that DOD's actions were effective. While DOD monitors some aspects of the system's implementation, it does not monitor how or the extent to which the safeguards specifically are implemented across the department. As a result, decision makers lack information that could be used to determine whether the department's actions are effective and whether the system is being implemented in a fair, equitable, and credible manner. DOD civilian personnel have mixed perceptions about NSPS, and while the department has taken some steps toward addressing employee concerns, it has not yet developed and implemented an action plan to address areas where employees express negative perceptions of the system, as GAO recommended in 2008. DOD's survey data from 2008 revealed that overall, NSPS employees responded positively about some aspects of performance management, such as connecting pay to performance, and negatively about others, such as the performance appraisal process. According to the most recent survey data, the negative perceptions of employees who worked under NSPS the longest remain largely unchanged from what was reported by GAO in 2008. Further, as GAO reported in 2008, employees and supervisors continue to express negative perceptions in discussion groups about NSPS--for example, voicing concerns about the negative impact of NSPS on employees' motivation and morale, and about the excessive amount of time spent navigating the performance management process. Such negative perceptions are not surprising given that large-scale organizational transformations often require an adjustment period to gain employees' trust and acceptance. DOD has taken some steps to address employees' perceptions of NSPS--for example, by issuing a memorandum with suggested actions the components could take to address employee concerns. However, DOD has not yet developed and implemented an action plan that fully meets the intent of GAO's 2008 recommendation. Specifically, DOD has not yet specified such things as its intended actions, who will be responsible, and the time frames for these actions. GAO continues to believe that implementing such a plan has merit.