GAO 

United States Government Accountability Office

Report to Congressional Committees

HUMAN CAPITAL

Monitoring of 

Safeguards and 

Addressing Employee 

Perceptions Are Key 

to Implementing a 

Civilian Performance 

Management System 

in DOD 

October 2009 

GAO-10-102 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

October 2009

HUMAN CAPITAL

Monitoring of Safeguards and Addressing Employee 

Perceptions Are Key to Implementing a Civilian 

Highlights of GAO-10-102, a report to Performance Management System in DOD 

congressional committees 

In 2004, the Department of Defense 

(DOD) began implementing the 

National Security Personnel 

System (NSPS)—a human capital 

system for DOD civilians. NSPS 

significantly redesigned the way 

DOD civilians are hired, 

compensated, and promoted. Pub. 

L. No. 110-181 mandated that GAO 

conduct reviews of the NSPS 

performance management system 

in calendar years 2008, 2009, and 

2010. In this report, GAO assessed 

(1) the extent to which DOD has 

implemented certain internal 

safeguards to ensure the fairness, 

effectiveness, and credibility of 

NSPS, and monitored their 

implementation, and (2) how DOD 

civilian personnel perceive NSPS, 

and the actions DOD has taken to 

address those perceptions. GAO 

analyzed relevant documents and 

employee survey results, 

interviewed DOD officials, and 

conducted discussion groups with 

DOD employees at eight locations 

outside of the continental United 

States. Toward the end of GAO’s 

review, both Houses of Congress 

passed proposed legislation that, if 

enacted, would terminate NSPS 

and require any future performance 

management system for DOD 

civilians to include certain internal 

safeguards. 

What GAO Recommends

GAO is recommending that DOD 

evaluate its training, revise its 

guidance for analyzing ratings, and 

monitor the implementation of 

safeguards for NSPS or any future 

performance management system. 

DOD generally concurred with 

GAO’s recommendations. 

DOD continues to take steps to implement internal safeguards as part of 

NSPS, but implementation of some safeguards could still be improved, and 

continued monitoring of all safeguards’ implementation is needed. In general, 

DOD has taken some steps to meet the intent of each of the safeguards, and it 

has implemented some of the recommendations from GAO’s September 2008 

report. However, opportunities exist for DOD to improve implementation of 

some safeguards. For example, DOD has not yet evaluated the effectiveness of 

the training employees receive, although doing so could help DOD measure 

the impact of its training and its progress toward achieving agency goals. In 

addition, DOD has not specified in its guidance what process the components 

should follow to investigate and eliminate potential barriers to fair and 

equitable ratings. Consequently, the components may not follow a consistent 

approach when investigating potential barriers, which could hinder their 

efforts to eliminate them. Further, GAO previously noted that continued 

monitoring of the safeguards was needed to ensure that DOD’s actions were 

effective. While DOD monitors some aspects of the system’s implementation, 

it does not monitor how or the extent to which the safeguards specifically are 

implemented across the department. As a result, decision makers lack 

information that could be used to determine whether the department’s actions 

are effective and whether the system is being implemented in a fair, equitable, 

and credible manner. 

DOD civilian personnel have mixed perceptions about NSPS, and while the 

department has taken some steps toward addressing employee concerns, it 

has not yet developed and implemented an action plan to address areas where 

employees express negative perceptions of the system, as GAO recommended 

in 2008. DOD’s survey data from 2008 revealed that overall, NSPS employees 

responded positively about some aspects of performance management, such 

as connecting pay to performance, and negatively about others, such as the 

performance appraisal process. According to the most recent survey data, the 

negative perceptions of employees who worked under NSPS the longest 

remain largely unchanged from what was reported by GAO in 2008. Further, 

as GAO reported in 2008, employees and supervisors continue to express 

negative perceptions in discussion groups about NSPS—for example, voicing 

concerns about the negative impact of NSPS on employees’ motivation and 

morale, and about the excessive amount of time spent navigating the 

performance management process. Such negative perceptions are not 

surprising given that large-scale organizational transformations often require 

an adjustment period to gain employees’ trust and acceptance. DOD has taken 

some steps to address employees’ perceptions of NSPS—for example, by 

issuing a memorandum with suggested actions the components could take to 

address employee concerns. However, DOD has not yet developed and 

implemented an action plan that fully meets the intent of GAO’s 2008 

recommendation. Specifically, DOD has not yet specified such things as its 

intended actions, who will be responsible, and the time frames for these 

actions. GAO continues to believe that implementing such a plan has merit. 

View GAO-10-102 or key components. 

For more information, contact Brenda S. 

Farrell at (202) 512-3604 or farrellb@gao.gov. 
Page i GAO-10-102 

Contents 

Letter 1

Results in Brief 7 

Background 11 

DOD Continues to Make Progress, but Implementation of Some 

Safeguards Could Be Improved, and Continued Monitoring of 

the System’s Implementation, Including the Safeguards, Is 

Needed 17 

Although DOD Civilian Employees under NSPS Have Mixed Views 

about the System, DOD Has Not Yet Developed and 

Implemented a Plan to Address Employees’ Negative 

Perceptions of Some Aspects of the System 35 

Conclusions 49 

Recommendations for Executive Action 50 

Agency Comments and Our Evaluation 50 

Appendix I Scope and Methodology 53 

Appendix II Reconsideration Process 62 

Appendix III Comments from the Department of Defense 64 

Appendix IV GAO Contact and Staff Acknowledgments 66 

Related GAO Products 67 

Tables 

Table 1: Number of DOD Civilian Employees Phased into NSPS, as 

of September 2009 13 

Table 2: Estimated Percentage of Responses from NSPS and NonNSPS Employees to Selected Questions about 

Performance Management in the 2008 Status of Forces 

Survey of Civilian Employees 36 

Human Capital 
Table 3: Estimated Percentage of Responses from NSPS and NonNSPS Employees to Selected Questions about 

Performance Appraisals in the 2008 Status of Forces 

Survey of Civilian Employees 37 

Table 4: Estimated Percentage of Responses from Spiral 1.1 

Employees to Selected Questions about Performance 

Management from the May 2006, November 2006, May 

2007, and February 2008 Administrations of the Status of 

Forces Survey of Civilian Employees 38 

Table 5: Estimated Percentage of Responses from Spiral 1.1 

Employees about the Overall Impact of the National 

Security Personnel System from the May 2006, November 

2006, May 2007, and February 2008 Administrations the 

Status of Forces Survey of Civilian Employees 39 

Table 6: Composition of Discussion Groups 58 

Table 7: Composition of Discussion Groups by Demographic 

Category per Component 60 

Figures 

Figure 1: Example of NSPS Pay Pool Organization 15 

Figure 2: Sample Screen Shots from the NSPS 101 and iSuccess 

Training Modules 21 

Page ii GAO-10-102 Human Capital
Abbreviations 

DMDC Defense Manpower Data Center 

DOD Department of Defense 

GS General Schedule 

NSPS National Security Personnel System 

OPM Office of Personnel Management 

PEO Program Executive Office 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately. 

Page iii GAO-10-102 Human Capital
Page 1 GAO-10-102 

 

United States Government Accountability Office

Washington, DC 20548 

 

October 28, 2009 

Congressional Committees 

In 2004, the Department of Defense (DOD) began implementing the 

National Security Personnel System (NSPS)—a human capital system for 

DOD civilian employees. NSPS significantly redesigned the rules, 

regulations, and processes that govern the way in which civilian 

employees are hired, compensated, and promoted within the department. 

DOD began converting employees to NSPS in 2006 and, according to the 

department, by early 2009 had converted more than 200,000 employees to 

the system. However, a series of events transpired in 2009 leaving the 

system’s future uncertain. Specifically, in February 2009, the Chairman of 

the House Committee on Armed Services and the Chairman of the 

committee’s Subcommittee on Readiness urged DOD to halt conversions 

to NSPS, highlighting concerns over a lack of transparency and 

widespread discontent with the system. Subsequently, in March 2009, DOD 

and the Office of Personnel Management (OPM) announced that they 

would suspend any further conversions of organizations to NSPS pending 

the outcome of a review of the system led by the Defense Business Board.1

In July 2009, the Defense Business Board presented the Secretary of 

Defense with the report of its review of NSPS, which included 

recommendations to initiate a reconstruction of the system and to 

continue the moratorium on conversions to NSPS until reconstruction is 

complete. Most recently, in October 2009, the proposed National Defense 

Authorization Act for Fiscal Year 2010, which passed both Houses of 

Congress,2

 contains provisions that would terminate NSPS and convert 

DOD civilian employees currently under the system to previously existing 

civilian personnel systems no later than January 1, 2012. 

Before the enactment of the legislation that authorized NSPS, we raised a 

number of critical issues about the proposed regulations for NSPS in a 

1

On May 14, 2009, the Deputy Secretary of Defense requested that the Defense Business 

Board form a task group to review NSPS to determine (1) if the underlying design 

principles and methodology for implementation are reflected in the NSPS program 

objectives; (2) whether the program objectives are being met; and (3) whether NSPS is 

operating in a fair, transparent, and effective manner. 

2

H.R. 2647 and H.R. Conf. Rep. No. 111-288, §1113 (2009). 

Human Capital 
 

series of testimonies in 2003.3 Since then, we have provided congressional 

decision makers with insight into DOD’s processes to design and 

implement NSPS and the most significant challenges the department faces 

in implementing the system.4

 While GAO supports human capital reform in 

the federal government, a key challenge is ensuring sustained and 

committed leadership for such a major transformation effort. Further, 

moving too quickly or prematurely to implement such systems, whether at 

DOD or elsewhere, can significantly raise the risk of problems 

implementing them.5

 We have also reported that DOD and other federal 

agencies must ensure that their performance management systems contain 

appropriate accountability mechanisms and internal safeguards to ensure 

that the systems are fair, effective, and credible.6

To help inform congressional deliberations on the implementation of 

NSPS, the National Defense Authorization Act for Fiscal Year 2008 

mandated that GAO conduct reviews of NSPS in calendar years 2008, 2009, 

and 2010 to determine the extent to which DOD has effectively 

incorporated certain accountability mechanisms and internal safeguards 

 

3

See GAO, Defense Transformation: Preliminary Observations on DOD’s Proposed 

Civilian Personnel Reforms, GAO-03-717T (Washington, D.C.: Apr. 29, 2003); Defense 

Transformation: DOD’s Proposed Civilian Personnel System and Governmentwide 

Human Capital Reform, GAO-03-741T (Washington, D.C.: May 1, 2003); and Human 

Capital: Building on DOD’s Reform Effort to Foster Governmentwide Improvements, 

GAO-03-851T (Washington, D.C.: June 4, 2003). 

4

See GAO, Human Capital: DOD Needs to Improve Implementation of and Address 

Employee Concerns about Its National Security Personnel System, GAO-08-773

(Washington, D.C.: Sept. 10, 2008); Human Capital: DOD Needs Better Internal Controls 

and Visibility over Costs for Implementing Its National Security Personnel System, 

GAO-07-851 (Washington, D.C.: July 16, 2007); and Human Capital: Observations on Final 

Regulations for DOD’s National Security Personnel System, GAO-06-227T (Washington, 

D.C.: Nov. 17, 2005). 

5

GAO, Questions for the Record Related to the Implementation of the Department of 

Defense’s National Security Personnel System, GAO-09-669R (Washington, D.C.: May 18, 

2009). 

6

See Related GAO Products at the end of this report for additional reports we have issued 

related to NSPS and performance management in the federal government. 

Page 2 GAO-10-102 Human Capital
 

into the system and to assess employee attitudes toward the system.7 In 

September 2008, we issued our first review of NSPS, and in April 2009 we 

testified on that assessment before the House Committee on Armed 

Services’ Subcommittee on Readiness.8

 We used the following safeguards 

to assess NSPS:9

• Involve employees, their representatives, and other stakeholders in the 

design of the system, to include employees directly involved in 

validating any related implementation of the system. 

• Assure that the agency’s performance management system links 

employee objectives to the agency’s strategic plan, related goals, and 

desired outcomes. 

• Implement a pay-for-performance evaluation system to better link 

individual pay to performance, and provide an equitable method for 

appraising and compensating employees. 

 

7

Pub. L. No. 110-181, § 1106(c) (2008). Specifically, section 1106(c)(1)(B) directs GAO to 

conduct reviews in calendar years 2008 through 2010 to evaluate the extent to which DOD 

has effectively implemented accountability mechanisms, including those established in 5 

U.S.C. § 9902(b)(7) and other internal safeguards, and to evaluate, as well, the extent that 

DOD undertakes internal assessments or employee surveys to assess employee satisfaction 

with NSPS. The accountability mechanisms specified in 5 U.S.C. § 9902(b)(7) include those 

that GAO previously identified as internal safeguards key to successful implementation of 

performance management systems. For example see GAO, Post-Hearing Questions for the 

Record Related to the Department of Defense’s National Security Personnel System 

(NSPS), GAO-06-582R (Washington, D.C.: Mar. 24, 2006). GAO has emphasized the need for 

internal safeguards since DOD first proposed NSPS. For example see GAO, Posthearing 

Questions Related to Strategic Human Capital Management, GAO-03-779R (Washington, 

D.C.: May 22, 2003). Additionally, our review covered merit system principles that pertain 

to performance management systems—specifically those stipulated in 5 U.S.C. § 

2301(b)(2),(3),(7), and (8a). 

8

See GAO-08-773 and GAO, Human Capital: Improved Implementation of Safeguards and 

an Action Plan to Address Employee Concerns Could Increase Employee Acceptance of 

the National Security Personnel System, GAO-09-464T (Washington, D.C.: Apr. 1, 2009). 

9

For the purpose of this report, we define safeguards to include accountability 

mechanisms. We note that we previously identified an independent and credible employee 

appeals mechanism as a safeguard. For example, see GAO-06-582R. However, although the 

National Defense Authorization Act for Fiscal Year 2004 (Pub. L. No. 108-136, § 1101(a), 

(2003)) gave DOD the authority to establish a different process for employees to appeal 

adverse actions than the appeals process available to employees under Title 5, that DOD 

authority was rescinded by the National Defense Authorization Act for Fiscal Year 2008, 

Pub. L. No. 110-181, § 1106(a), (2008). This rescission of authority gave civilian employees 

under NSPS the same right to appeal adverse actions to the Merit Systems Protection 

Board that they had under Title 5. 5 U.S.C. §§ 7513(d), 7701. 

Page 3 GAO-10-102 Human Capital
 

• Provide adequate training and retraining for supervisors, managers, 

and employees in the implementation and operation of the 

performance management system. 

• Institute a process for ensuring ongoing performance feedback and 

dialogue between supervisors, managers, and employees throughout 

the appraisal period, and setting timetables for review. 

• Assure that certain predecisional internal safeguards exist to help 

achieve consistency, equity, nondiscrimination, and nonpoliticization 

of the performance management process (e.g., independent 

reasonableness reviews by a third party or reviews of performance 

rating decisions, pay determinations, and promotions before they are 

finalized to ensure that they are merit-based, as well as pay panels who 

consider the results of the performance appraisal process and other 

information in connection with final pay decisions). 

• Assure that there are reasonable transparency and appropriate 

accountability mechanisms in connection with the results of the 

performance management process, including periodic reports on 

internal assessments and employee survey results relating to 

performance management and individual pay decisions while 

protecting individual confidentiality. 

• Assure that the agency’s performance management system results in 

meaningful distinctions in individual employee performance. 

• Provide a means for ensuring that adequate agency resources are 

allocated for the design, implementation, and administration of the 

performance management system. 

We initially developed a list of safeguards based on our extensive body of 

work looking at the performance management practices used by leading 

public sector organizations both in the United States and in other 

countries. These safeguards were then merged with the list of safeguards 

specified in section 9902(b)(7) of Title 5, U.S. Code, to develop the 

aforementioned safeguards. It is important to keep in mind that the list of 

safeguards we are using to assess DOD’s efforts is not intended to cover 

all attributes of a modern, results-oriented performance management 

system; therefore, other safeguards may exist. Rather, as we previously 

Page 4 GAO-10-102 Human Capital
 

reported, the items on the list cover possible safeguards that help ensure 

those systems are fair, effective, and credible.10

In responding to the congressional mandate for the 2008 review of NSPS, 

we reported, in September 2008, that while DOD has taken some steps to 

ensure that NSPS is fair, effective, and credible, implementation of certain 

safeguards could be improved. We further reported that although DOD 

civilian employees under NSPS responded positively regarding some 

aspects of performance management, the department does not have an 

action plan to address employees’ negative perceptions of NSPS. To 

address our findings, we recommended that DOD improve the 

implementation of certain safeguards and develop and implement an 

action plan to address employees’ concerns about NSPS. DOD generally 

concurred with our recommendations, with the exception of one—

requiring a predecisional analysis of its rating results. We discuss the steps 

DOD has taken to address our previous recommendations later in this 

report. 

To address the congressional mandate for the 2009 review of NSPS, we 

established the following objectives: (1) To what extent has DOD 

implemented safeguards to ensure the fairness, effectiveness, and 

credibility of the NSPS performance management system and monitored 

the implementation of the safeguards? (2) How do DOD civilian personnel 

perceive NSPS, and what actions has DOD taken to address these 

perceptions? 

For our first objective, we reviewed the safeguards specified in section 

9902(b)(7) of Title 5, U.S. Code, for the NSPS performance management 

system, as well as other safeguards we have previously identified as key 

for performance management systems. We also obtained and analyzed 

regulations, guidance, and other documentation provided by officials in 

DOD and the NSPS program offices of the four components—the Army, 

the Air Force, the Navy, and the Fourth Estate11—and interviewed 

knowledgeable officials in all of these offices to obtain a comprehensive 

 

10GAO-03-741T. 

11The Department of the Navy’s NSPS policies encompass Marine Corps civilians. The 

Fourth Estate includes all organizational entities in DOD that are not in the military 

departments or the combatant commands. Examples of Fourth Estate entities are the 

Office of the Secretary of Defense, the Joint Staff, the Office of the DOD Inspector General, 

the defense agencies, and DOD field activities. 

Page 5 GAO-10-102 Human Capital
 

understanding of their efforts to implement the NSPS performance 

management system and the safeguards as well as the processes, 

procedures, and controls used for monitoring and overseeing 

implementation of the system. In addition, we visited eight DOD 

organizations—two for each component—located outside the continental 

United States to assess the extent to which the department has 

implemented safeguards for NSPS. Because our 2008 review focused on 

assessing implementation of NSPS and the safeguards at locations that 

were geographically distributed throughout the United States, we focused 

our efforts for this review on visiting locations outside the continental 

United States.12 During our site visits we reviewed documents, such as pay 

pool business rules,13 and met with the performance review authority, pay 

pool managers, pay pool panel members, rating officials, the NSPS 

program manager or transition team, and DOD civilian employees under 

NSPS to obtain their perspectives on the extent to which the safeguards 

had been implemented. The organizations we visited were selected based 

on a number of factors, such as the presence of a large number or 

concentrated group of civilian employees under NSPS. For our second 

objective, we obtained and analyzed the results of DOD’s May 2006, 

November 2006, May 2007, and February 2008 Status of Forces Survey of 

Civilian Employees, which gauge employee attitudes toward NSPS, among 

other things. We used these data to identify indications of movement or 

trends in employee perceptions. As we reported in our first assessment of 

NSPS, we reviewed the results of prior administrations of DOD’s Status of 

Forces Survey of Civilian Employees and found the results to be 

sufficiently reliable for the purposes of our report. For our current 

assessment, we found no substantial changes in the methodology for the 

2008 Status of Forces Survey of Civilian Employees. In addition, we 

obtained the perspectives of DOD civilian personnel at the eight locations 

we visited by conducting small group discussions with nonsupervisory and 

supervisory employees under NSPS and administering a short 

questionnaire. Because the locations we visited are not representative of 

all DOD locations, the results from our discussions and the comments 

 

12According to the Program Executive Office, as of September 2008, about 85 percent of the 

civilian employees under NSPS were located in the continental United States, whereas 

about 15 percent were located outside the continental United States. Of the civilian 

employees under NSPS located outside the continental United States, about 54 percent 

were in either Germany or Hawaii. 

13Business rules are the policies that govern a pay pool’s operations. They may specify, for 

example, the pay pool panel’s structure, roles and responsibilities, standards of conduct, 

and the processes used for reconciling employee ratings and allocating shares under NSPS. 

Page 6 GAO-10-102 Human Capital
 

provided are not generalizable to the entire NSPS civilian population. 

However, the composition of our discussion groups was designed to 

ensure that we spoke with employees from each of the four DOD 

components at locations outside the continental United States. Further 

details about our scope and methodology can be found in appendix I. 

We conducted this performance audit from November 2008 through 

September 2009 in accordance with generally accepted government 

auditing standards. Those standards require that we plan and perform the 

audit to obtain sufficient, appropriate evidence to provide a reasonable 

basis for our findings and conclusions based on our audit objectives. We 

believe that the evidence obtained provides a reasonable basis for our 

findings and conclusions based on our audit objectives. 

DOD continues to take steps to implement safeguards as part of the NSPS 

performance management system, but implementation of some safeguards 

could be improved, and continued monitoring of the safeguards’ 

implementation is needed. In general, DOD has taken some steps to meet 

the intent of each of the safeguards, and it has implemented some of the 

recommendations from our 2008 report, including requiring commands to 

publish the final overall rating results and providing guidance to pay pools 

encouraging managers to rate employees appropriately. Nonetheless, 

DOD’s implementation of some of the safeguards could be improved. First, 

for example, DOD has not evaluated the effectiveness of the training that it 

provides to employees on the system. Under NSPS, the components are 

responsible for training their employees, while DOD’s NSPS Program 

Executive Office (PEO) supports the components’ efforts by offering a 

variety of departmentwide training courses and other materials. However, 

officials14 at each of the eight organizations we visited expressed concerns 

over the effectiveness of the training provided, noting that additional 

training was needed, that training was not always helpful, and that some 

training was not timely or was outdated. We previously reported that it is 

increasingly important for agencies to measure the real impact of training 

and thus evaluate the training efforts to ascertain progress toward 

Results in Brief 

 

14For the purpose of this report, we use the term officials to refer to the management 

officials we met with during our site visits, including the performance review authority, pay 

pool manager, pay pool panel members, rating officials, and NSPS transition manager or 

members of the NSPS transition team. 

Page 7 GAO-10-102 Human Capital
 

achieving agency goals.15 Second, while, in 2008, DOD did not agree with 

our recommendation to require a predecisional analysis of ratings—stating 

that postdecisional analysis of rating results is useful to identify barriers 

and corrective actions—we found during the course of this review that the 

department’s postdecisional analyses following the 2007 and 2008 NSPS 

performance management cycles lacked consistency and did not generally 

include demographic information. However, in May 2009, the PEO issued 

guidance to promote a degree of standardization in the components’ 

postdecisional analyses. While not predecisional, we recognize that DOD’s 

approach represents a noteworthy step and does provide some benefits, 

some of which are similar to those of a predecisional analysis. For 

example, like predecisional analysis, postdecisional analysis is a 

mechanism to ensure that employees receive fair and equitable treatment 

in all aspects of personnel management.16 However, the guidance does not 

specify what process the components should follow to investigate 

potential barriers to fair and equitable ratings and their causes, nor a 

process for eliminating barriers that are found. Because DOD’s guidance 

does not specify these steps, the components may not follow a consistent 

approach when investigating potential barriers, which could hinder their 

efforts to eliminate them. In our 2008 report, we also noted that continued 

monitoring of the safeguards was needed to ensure that DOD’s actions 

were effective as implementation of NSPS proceeded. We found that DOD 

monitors some aspects of the implementation of NSPS, such as whether 

the system is on track to achieve certain goals it established, but does not 

monitor how or the extent to which the safeguards specifically are 

implemented across the department. Because DOD does not monitor the 

implementation of the safeguards, decision makers in DOD and the 

Congress lack information that could be used to determine whether the 

department’s actions are effective and whether the system is being 

implemented in a fair, equitable, and credible manner. Accordingly, until 

DOD effectively implements the safeguards and monitors their 

implementation across the department, employees will not have assurance 

that NSPS is a fair, equitable, and credible system and decision makers in 

 

15GAO, Human Capital: A Guide for Assessing Strategic Training and Development 

Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). 

16Specifically, 5 U.S.C. § 2301(b)(2) provides that “all employees and applicants for 

employment should receive fair and equitable treatment in all aspects of personnel 

management without regard to political affiliation, race, color, religion, national origin, sex, 

marital status, age, or handicapping condition, and with proper regard for their privacy and 

constitutional rights.” 

Page 8 GAO-10-102 Human Capital
 

DOD and the Congress will not have the information that could be used to 

determine if the department’s actions are effective. 

DOD civilian personnel have mixed perceptions about NSPS, and while the 

department has taken some steps toward addressing their concerns, it has 

not yet developed and implemented an action plan to address areas where 

employees express negative perceptions of the system, as we 

recommended in 2008. DOD’s survey data from 2008 revealed that overall, 

NSPS employees responded positively about some aspects of performance 

management, such as connecting pay to performance, and negatively 

about other aspects, including the performance appraisal process. For 

example, DOD’s survey data for 2008 indicate that an estimated 42 percent 

of NSPS employees agree that pay raises depend on how well employees 

perform their jobs, as compared with an estimated 25 percent of non-NSPS 

employees. However, when asked about their performance appraisal 

system, an estimated 29 percent of NSPS employees, as compared with an 

estimated 34 percent of non-NSPS employees, agreed with the statement 

that their performance appraisal system improves organizational 

performance. Additionally, DOD’s most recent survey data indicated that 

employees who worked under NSPS the longest—spiral 1.1 employees17—

expressed negative perceptions of the system consistent with those we 

reported on in 2008. For example, between the November 2006 and 

February 2008 administrations of DOD’s survey, the percentage of spiral 

1.1 employees that agreed that they understood what they had to do to be 

rated at a different performance level declined from an estimated 59 

percent in November 2006 to an estimated 53 percent in May 2007, then 

remained consistent with an estimated 54 percent responding that they 

agree in February 2008. Further, the results of our discussion groups 

indicated that employees and supervisors continue to have consistent 

concerns and negative perceptions of NSPS. These included the following: 

(1) NSPS’s negative impact on employee motivation and morale; (2) the 

excessive amount of time spent navigating the performance management 

process; (3) challenges with job objectives; (4) factors undermining 

employee confidence in the system, such as the subjectivity of the pay 

pool panel process; and (5) factors unrelated to job performance affecting 

employees’ final performance ratings, such as supervisors’ and employees’ 

writing skills. As we have previously reported, these negative perceptions 

 

17DOD used a phased approach to converting civilian employees to NSPS. Each phase is 

called a spiral and each spiral has an identifying number associated with it—for example, 

spiral 1.1. 

Page 9 GAO-10-102 Human Capital
 

are not surprising given that large-scale organizational transformations—

for example, the adoption of a new performance management system—

often entail fundamental and radical changes that require an adjustment 

period to gain employees’ trust and acceptance. With regard to DOD’s 

efforts to address employee perceptions of the system, DOD issued a 

memorandum, in June 2009, highlighting actions the department had taken 

in this area, as well as suggested future actions that the components could 

take to address identified negative perceptions. While we believe that 

issuing this memorandum is an important first step in addressing 

employee concerns, it does not fully meet the intent of our 2008 

recommendation that DOD develop and implement a specific action plan 

to address employee perceptions of NSPS because it does not specify such 

things as the actions DOD intends to take, who will be responsible for 

taking the action, and timelines for doing so—items stipulated in OPM 

guidance on action planning.18 We continue to believe that implementing 

such an action plan is important, and note that doing so would be a step 

that DOD could take to involve employees in the system’s 

implementation—which is one of the safeguards we discuss.

In September 2009, we provided DOD with a draft of this report that 

included three recommendations to better address the safeguards and 

improve implementation of the NSPS performance management system. 

Subsequently, as we were preparing to issue this report, the proposed 

National Defense Authorization Act for Fiscal Year 2010 passed both 

Houses of Congress.19 That legislation contained provisions that, if 

enacted, would terminate NSPS while also requiring that DOD include 

certain safeguards as part of any future performance management system. 

Citing the uncertainty regarding the system‘s future, DOD partially 

concurred with our three recommendations specific to the NSPS 

performance management system. DOD also stated that it would consider 

acting on our recommendations to the extent they are relevant as the 

department moves forward with any future performance management 

system. We believe that this is a reasonable approach. Accordingly, we 

have revised our recommendations to apply to any future performance 

management system for DOD’s civilian employees—whether NSPS or 

another system. Specifically, we are recommending that DOD (1) review 

 

18Office of Personnel Management Memorandum, “Guidance on Employee Satisfaction 

Action Planning” (Aug. 19, 2009). 

19H.R. Conf. Rep. No. 111-288, §1113 (Oct. 7, 2009) to accompany H.R. 2647, the proposed 

National Defense Authorization Act for 2010. 

Page 10 GAO-10-102 Human Capital
 

and evaluate the effectiveness of the department’s performance 

management training; (2) ensure that guidance is in place for conducting a 

postdecisional analysis that specifies what process the components should 

follow to investigate and eliminate potential barriers to fair and equitable 

ratings; and (3) include, as part of the department’s monitoring of the 

implementation of its system, efforts to monitor and evaluate how the 

safeguards specifically are being implemented by lower-level organizations 

across the department. DOD’s comments are reprinted in appendix III.

The General Schedule (GS) is the federal government’s primary pay and 

classification system for white-collar employees. Under this system, 

federal employees are paid according to governmentwide rules and 

procedures, and federal positions are classified according to the difficulty 

and responsibility of the work performed. The GS system was created in 

1949, when most federal positions involved clerical work or revolved 

around the execution of established, stable processes—for example, 

posting census figures in ledgers or retrieving taxpayer records from vast 

file rooms. Over time, however, federal positions have become 

increasingly specialized and more highly skilled. In light of this change, a 

number of federal agencies have attempted to provide managers with 

greater flexibility in hiring and awarding pay raises to employees by 

implementing human capital initiatives, such as performance management 

systems, that reward employees’ performance and contribution to the 

agency’s mission. The need for human capital reform regarding these 

systems has been the subject of a number of previous GAO reviews.20

These reviews have noted, for example, that federal agencies must have 

modern, effective, credible, and validated performance management 

systems that are capable of supporting pay and other personnel decisions, 

and that pay for performance works only with adequate safeguards and 

appropriate accountability mechanisms in place to ensure that the 

safeguards are implemented in a fair, effective, and credible manner. 

Background 

 

20See GAO, Results-Oriented Management: Opportunities Exist for Refining the 

Oversight and Implementation of the Senior Executive Performance-Based Pay System, 

GAO-09-82 (Washington, D.C.: Nov. 21, 2008); Office of Personnel Management: Key 

Lessons Learned to Date for Strengthening Capacity to Lead and Implement Human 

Capital Reforms, GAO-07-90 (Washington, D.C.: Jan. 19, 2007); Results-Oriented Cultures: 

Modern Performance Management Systems Are Needed to Effectively Support Pay for 

Performance, GAO-03-612T (Washington, D.C.: Apr. 1, 2003); and GAO-03-741T. 

Page 11 GAO-10-102 Human Capital
 

In November 2003, the Congress included a provision in the National 

Defense Authorization Act for Fiscal Year 2004 providing DOD with the 

authority to establish a pay-for-performance management system as part 

of NSPS.21 In April 2004, the Secretary of Defense appointed an NSPS 

Senior Executive to, among other things, design, develop, and implement 

NSPS. Under the Senior Executive’s authority, the PEO was established as 

the central policy and program office for NSPS. The PEO’s responsibilities 

include designing the human resource/pay-for-performance systems, 

developing communication and training strategies, modifying personnel 

information technology, and preparing joint enabling regulations and 

internal DOD implementing regulations, called implementing issuances. As 

the central DOD-wide program office, the PEO also directs and oversees 

the four components’ NSPS program managers, who report to their parent 

components and the PEO. These program managers serve as their 

components’ NSPS action officers and also participate in the development, 

planning, implementation, and deployment of NSPS. 

Beginning in April 2006, DOD began phasing (or spiraling) civilian 

employees into NSPS;22 however, in January 2008 the National Defense 

Authorization Act for Fiscal Year 2008 prohibited the Secretary of Defense 

from converting more than 100,000 employees to NSPS in any calendar 

year and excluded Federal Wage System employees from coverage under 

NSPS.23 Further, in March 2009, DOD announced that it would delay the 

conversion of additional organizations to NSPS pending the outcome of a 

joint review of the system by DOD and OPM. According to DOD, this 

decision affected roughly 2,000 employees in organizations scheduled to 

convert to NSPS during the spring of 2009. As a result of these and other 

legislative changes that resulted in revisions to the NSPS regulations, the 

PEO has not developed new timelines for phasing any additional civilian 

employees into NSPS. As table 1 shows, according to DOD, almost 220,000 

civilian employees have been phased into NSPS as of September 2009. 

 

21Pub. L. No. 108-136, § 1101 (2003) (codified at 5 U.S.C. §§ 9901-9904). The National 

Defense Authorization Act for Fiscal Year 2008 amended 5 U.S.C. § 9902. Pub. L. No. 110-

181, § 1106 (2008). 

22According to PEO officials, DOD originally planned to convert approximately 700,000 

civilian employees to NSPS; however, legislative changes have decreased the number of 

civilian employees eligible to convert to the system. 

23Pub. L. No. 110-181, § 1106(a) (2008). The Federal Wage System is a uniform pay-setting 

system that covers federal appropriated fund and nonappropriated fund blue-collar 

employees who are paid by the hour. 

Page 12 GAO-10-102 Human Capital
 

Table 1: Number of DOD Civilian Employees Phased into NSPS, as of September 2009 

Spiral Army Air Force Navy Fourth Estate

Total number

of employees

1.1 2,666 3,187 4,828 1,611 12,292

1.2 18,084 38,693 10,087 9,239 76,103

1.3 28,582 1,102 8,559 391 38,634

2.1 11,103 - 8,194 85 19,382

2.2 20,175 - 22,487 15,145 57,807

2.3 483 - 14,128 38 14,649

2.4 - - 294 15 309

Total number of employees 81,093 42,982 68,577 26,524 219,176

Source: DOD. 

Note: In June 2009, we reported that according to DOD, about 212,000 civilian employees were 

under NSPS. See GAO, Human Capital: Continued Monitoring of Internal Safeguards and an Action 

Plan to Address Employee Concerns Could Improve Implementation of the National Security 

Personnel System, GAO-09-840 (Washington, D.C.: June 25, 2009). According to a PEO official, no 

additional organizations have converted to NSPS since March 2009 and the growth of the NSPS 

workforce is due to routine hiring, transfer, reassignment, and promotion actions into organizations 

already under NSPS. 

The NSPS performance management process is ongoing and consists of 

several phases that are repeated during each annual performance cycle. 

The process begins with a planning phase that involves supervisors (or 

rating officials) and employees working together to establish performance 

plans. This includes developing job objectives—the critical work 

employees perform that is aligned with their organizational goals and 

focused on results—and identifying contributing factors—the attributes 

and behaviors that identify how the critical work established in the job 

objectives is going to be accomplished (e.g., cooperation and teamwork). 

After the planning phase comes the monitoring and developing phase, 

during which ongoing communication between supervisors and employees 

occurs to ensure that work is accomplished; attention is given to areas 

that need to be addressed; and managers, supervisors, and employees have 

a continued and shared understanding of expectations and results. In the 

rating phase, the supervisor prepares a written assessment that captures 

the employee’s accomplishments during the appraisal period. In the 

final—or reward—phase, employees should be appropriately rewarded or 

compensated for their performance with performance payouts. During this 

phase, employee assessments are reviewed by multiple parties to 

determine employees’ ratings and, ultimately, performance payouts. 

Page 13 GAO-10-102 Human Capital
 

The performance management process under NSPS is organized by pay 

pools. A pay pool is a group of employees who share in the distribution of 

a common pay-for-performance fund.24 The key participants that make up 

pay pools are the employee, supervisor, higher-level reviewer, pay pool 

panel, pay pool manager, performance review authority, and, in some 

instances, the sub-pay pool,25 as shown in figure 1. 

 

24Criteria to distinguish pay pools may include, but are not limited to, organization 

structure, employee job function, location, and organization mission. 

25Where determined appropriate because of the size of the pay pool population, the 

complexity of the mission, the need to prevent conflicts of interest, or other similar criteria, 

sub-pay pool panels may be organized in a structure subordinate to the pay pool panel. 

Sub-pay pool panels normally operate under the same requirements and guidelines 

provided to the pay pool panels to which they belong. 

Page 14 GAO-10-102 Human Capital
 

Figure 1: Example of NSPS Pay Pool Organization 

Performance 

review authority

(PRA)

Pay pool

manager

Pay pool panel member 

(sub-pay pool manager)

Employee

Employee

Employee

Employee

Employee

Rating official 

(sub-pay pool 

panel member)

Rating official 

(sub-pay pool 

panel member)

Rating official 

(sub-pay pool 

panel member)

Rating official Rating official

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Employee

Sub-pay

pool

Source: DOD.

Pay pool

panel member

Within a pay pool, each participant has defined responsibilities under the 

performance management process: 

• Employees are encouraged to be involved throughout the performance 

management cycle. This includes initially working with their 

supervisors to develop job objectives and identify associated 

contributing factors; identifying and recording accomplishments and 

results throughout the appraisal period; and participating in interim 

Page 15 GAO-10-102 Human Capital
 

reviews and end-of-year assessments, for example, by preparing a selfassessment of their performance during the annual appraisal period. 

• Supervisors (or rating officials) are responsible for effectively 

managing the performance of their employees. This includes clearly 

communicating performance expectations; aligning performance 

expectations and employee development with organization mission 

and goals; working with employees to develop written job objectives 

that reflect expected accomplishments and contributions for the 

appraisal period and identifying applicable contributing factors; 

providing employees meaningful, constructive, and candid feedback 

relative to performance expectations, including at least one 

documented interim review; making meaningful distinctions among 

employees based on performance and contribution; and providing 

recommended ratings of record, share assignments, and payout 

distributions to the pay pool. 

• The higher-level reviewer, typically the rating official’s supervisor, is 

responsible for reviewing and approving job objectives and 

recommended employee assessments. The higher-level reviewer is the 

first step in ensuring consistency of ratings because this individual 

looks across multiple ratings. 

• The pay pool panel (or, in some cases, the sub-pay pool panel) is a 

board of management officials who are usually in positions of line 

authority or in senior staff positions with resource oversight for the 

organizations, groups, or categories of employees making up the pay 

pool membership.26 The primary function of the pay pool panel is the 

reconciliation of ratings of record, share distribution, and payout 

allocation decisions. For example, the pay pool panel may adjust a 

supervisor’s recommended rating of record in order to help ensure the 

equity and consistency of ratings across the pay pool. 

• Each pay pool has a manager who is responsible for providing 

oversight of the pay pool panel. The pay pool manager is the final 

approving official of the rating of record. Performance payout 

 

26Pay pool panel members may not participate in payout deliberations or decisions that 

directly affect their own performance assessments or pay. 

Page 16 GAO-10-102 Human Capital
 

determinations may be subject to higher management review by the 

performance review authority27 or equivalent review process. 

• Finally, the performance review authority provides oversight of several 

pay pools and addresses the consistency of performance management 

policies within a component, major command, field activity, or other 

organization as determined by the component. 

DOD continues to take some steps to implement each of the safeguards we 

reported on in September 2008. However, opportunities exist to 

continually involve employees in the system’s implementation, and 

implementation of some safeguards—for example, providing effective 

training—could be improved. Also, we previously reported that continued 

monitoring of the safeguards was needed to help ensure that DOD’s efforts 

were effective as implementation of NSPS proceeded. However, we found 

that while DOD monitors some aspects of the implementation of NSPS, it 

does not monitor how the safeguards specifically are implemented across 

the department. Because DOD does not monitor the safeguards’ 

implementation, decision makers in DOD lack information that could be 

used to determine whether the department’s actions are effective and 

whether the system is being implemented in a fair, equitable, and credible 

manner. 

DOD Continues to 

Make Progress, but 

Implementation of 

Some Safeguards 

Could Be Improved, 

and Continued 

Monitoring of the 

System’s 

Implementation, 

Including the 

Safeguards, Is Needed 

 

27The senior organization official, usually a member of the Senior Executive Service or a 

General/Flag officer, serves as the performance review authority. DOD components may 

provide additional guidance for the establishment of performance review authorities. The 

responsibilities of the performance review authority may be assigned to an individual 

management official or organizational unit or group. 

Page 17 GAO-10-102 Human Capital
 

DOD has taken a number of steps to involve employees and stakeholders 

in the design and implementation of NSPS. In our September 2008 report, 

we noted that DOD solicited comments from employees and unions 

representing DOD employees during the design of NSPS. We also noted 

that DOD involved employees during the implementation of the system in 

workshops and other efforts aimed at refining the system’s performance 

factors. DOD continues to take such steps. For example: 

Implementation of Some 

Safeguards Could Be 

Improved 

Involve Employees in the 

Design and Implementation of 

the System 

• DOD solicited comments from employees and unions on the system’s 

final rule,28 which was published in the Federal Register in September 

2008. According to the Federal Register, the final regulations, which 

became effective in November 2008, include revisions based on 526 

comments submitted during the public comment period and on 

comments from 9 of the department’s 10 unions with national 

consultation rights. 

• DOD involved employees in efforts to improve the usability of the 

automated tools that support the NSPS performance and pay pool 

management processes. Specifically, the PEO and the department’s 

Civilian Personnel Management Service held a series of meetings with 

employees, rating officials, pay pool managers, and human resource 

practitioners in early 2008 to address concerns regarding the usability 

of the automated tools. These meetings allowed the department to 

gather requirements for the next version of the NSPS automated tools 

based on lessons learned and user input. Subsequently, DOD 

established six separate working groups to develop and evaluate the 

requirements for each of its automated tools. In addition, DOD 

initiated separate efforts to enhance the usability of the Performance 

Appraisal Application—the DOD-wide tool for employee selfassessments and appraisals. Specifically, the contractor that developed 

the Performance Appraisal Application enlisted the assistance of 

software usability experts to evaluate the tool and recommend 

changes that would enhance users’ experience with it. As a part of this 

effort, the contractor observed and worked with employees and rating 

officials to identify changes that could be made to the Performance 

Appraisal Application to make it more user-friendly. DOD also tested 

 

285. C.F.R. § 9901 (2008). 

Page 18 GAO-10-102 Human Capital
 

the functionality and usability of the enhancements that were made to 

the Performance Appraisal Application with over 300 users. 

• DOD has taken steps to involve the components in the implementation 

of NSPS through biweekly conference calls held at key phases of the 

performance management process. According to the PEO, during these 

calls, PEO and Civilian Personnel Management Service representatives 

discuss topics submitted by the components, respond to questions 

regarding such things as NSPS policy and the system’s automated 

tools, and share lessons learned with participants. Further, according 

to the PEO, these conference calls allow participants to address 

systemic problems through feedback shared between different levels 

of the organization. 

At the locations we visited outside the continental United States, we found 

that some steps were generally being taken to involve employees in the 

implementation of NSPS as well. For example, officials at each of the eight 

locations reported that organizations identified lessons learned that were 

generally based upon employee feedback gathered following the mock pay 

pool.29 According to these officials, lessons learned were used to make 

changes to, among other things, training materials, business rules, and the 

use of job objectives and contributing factors. For example, two locations 

limited the number of contributing factors employees should use in their 

performance plans based upon lessons learned, while one location 

responded to employee feedback regarding a lack of time to devote to 

NSPS by mandating that employees be allowed to take time to complete 

NSPS training. 

While DOD has taken a number of steps to involve employees in the design 

and implementation of NSPS thus far, as stated above, we note that one 

way the department could continue to involve employees as 

implementation of the system proceeds is to develop and implement an 

action plan to address employees’ perceptions of NSPS, as we 

recommended in our first assessment of NSPS. However, DOD has not yet 

done so, which we discuss further in the second objective of this report. 

 

29The mock pay pool is a way for organizations to understand the pay pool process. During 

the exercise, organizations identify ways to improve their pay pool process to achieve 

greater consistency and ensure fairness in ratings and payouts. Rating officials review their 

employees and assign mock ratings, numbers of shares, and payout distributions. The panel 

then practices advocating for employees and reconciling results. 

Page 19 GAO-10-102 Human Capital
 

DOD continues to take steps to link employee job objectives to the 

agency’s strategic goals, mission, and desired outcomes. As we noted in 

our 2008 report, DOD’s automated tool for employee self-assessments and 

appraisals—the Performance Appraisal Application—provides a 

designated area for the mission of the employee’s command to be inserted 

as a guide while employees compose their job objectives and selfassessments. In May 2009, DOD published its evaluation of NSPS for 2008, 

entitled National Security Personnel System (NSPS) – 2008 Evaluation 

Report, which included an evaluation of employee performance plans to 

determine the extent to which employee job objectives are aligned with 

higher-level organizational goals and thus ensure that employee 

performance contributes to the achievement of organizational success. 

The evaluation included 240 employee performance plans encompassing a 

range of job series, pay schedules, pay bands, and organizations within 

each of the four components. The evaluation found that 95 percent of the 

objectives were strongly aligned to higher-level goals and demonstrated a 

clear, direct, and strong linkage to the organizational mission or relevant 

strategic goals. 

Link Employee Objectives to 

the Agency’s Strategic Goals 

and Mission 

During our site visits, we found that the organizations were taking steps to 

ensure that employees understood how their job objectives link to the 

organization’s strategic goals. This was generally accomplished through 

documentation requirements in the Performance Appraisal Application 

and reinforced during employees’ discussions with their supervisors. Some 

organizations have taken additional steps to ensure that employee job 

objectives link to the organization’s strategic goals. For example, at one 

location we visited the commanding general issued a memorandum to 

managers emphasizing the importance of ensuring that employee 

objectives are linked to the command’s mission and objectives and that 

employees understand their roles in achieving those objectives. However, 

officials at five locations also reported that employees do not always 

understand this linkage. 

DOD continues to take steps to provide employees with required and other 

training on the implementation and operation of the NSPS performance 

management system, but has not yet evaluated the effectiveness of the 

training that it provides. In our September 2008 report, we noted that DOD 

encouraged employees who were transitioning to NSPS to receive training 

that covered skills and behaviors necessary to implement and sustain 

NSPS; foster support and confidence in the system; and facilitate the 

transition to a performance-based, results-oriented culture. DOD and each 

of the components continue to take such steps to provide employees with 

required and other training on the system, including introductory training 

Training and Retraining in the 

System’s Implementation and 

Operation 

Page 20 GAO-10-102 Human Capital
 

for employees converting to NSPS and sustainment training for employees 

already under the system. While the components are responsible for 

providing employees with training on the NSPS performance management 

system, the PEO supports their efforts by offering a variety of 

departmentwide training courses and other materials. For example, Webbased training modules that the PEO has developed, such as its NSPS 101 

and iSuccess courses (see fig. 2 for sample screen shots), provide 

employees with basic knowledge about NSPS and performance 

management principles in general, and are used by employees to develop 

their job objectives. As another example, the PEO developed training 

guides to educate employees on changes to the NSPS classroom materials 

resulting from the revised NSPS regulations and implementing issuances. 

The PEO also developed a Web site for accessing NSPS learning materials, 

resources, and other tools. In addition, we found that the Air Force has 

begun incorporating training on NSPS as a normal part of its operations 

and is working to embed NSPS topics in the regular training provided to 

Air Force civilians and servicemembers. 

Figure 2: Sample Screen Shots from the NSPS 101 and iSuccess Training Modules 

Source: DOD.

Although DOD and the components continue to take steps to provide 

employees with training on NSPS, the department has not yet evaluated 

the effectiveness of the training provided. We previously reported that it is 

increasingly important for agencies to measure the real impact of training 

Page 21 GAO-10-102 Human Capital
 

and thus evaluate their training—for example, by establishing clear goals 

about what the training is expected to achieve along with agreed-upon 

measures or performance indicators to ascertain progress toward 

achieving those goals.30 DOD has outlined the fundamental requirements, 

or goals, of the training that it provides to employees on NSPS. 

Specifically, DOD has noted that for NSPS, a training program must be 

implemented that enables employees to understand better how to succeed, 

and enables supervisors to communicate performance expectations to 

their employees, provide feedback to them on their performance against 

these expectations, and tell them what steps they can take to improve 

their performance and competencies and manage their careers. However, 

while DOD has undertaken efforts to understand employees’ perceptions 

of its training, the department has not yet evaluated the effectiveness of 

the training that it provides. For example, in early 2009 the PEO conducted 

what PEO officials describe as an ad hoc study of training needs. The 

PEO’s study consisted of conducting sensing sessions with 120 human 

resource practitioners at 11 locations across the department.31 According 

to the PEO, the purpose of these sessions included obtaining feedback on 

existing NSPS learning products and support and exploring options for 

next generation products. While the PEO’s study identified some needed 

improvements, it does not constitute an evaluation of the department’s 

training—for example, because it did not assess the department’s progress 

toward achieving the goals specified for the training. 

As another example, DOD’s 2008 evaluation report also highlighted 

deficiencies with the department’s training on NSPS.32 Specifically, the 

report notes that without effective communication and training, NSPS 

cannot achieve its goal of being a credible and trusted system. Further, 

three of the report’s six key recommendations focus on the need to 

enhance the effectiveness of the training provided to employees on NSPS: 

(1) provide more training on the performance management system; (2) 

provide high-level training for employees and supervisors that explains the 

pay pool process; and (3) hold mock pay pool panels, which serve as 

 

30GAO-04-546G. 

31Of these 120 human resource practitioners, only 1 was located outside the continental 

United States. 

32DOD’s 2008 evaluation focused on determining whether NSPS, as implemented in spiral 1 

organizations, was on track to achieve certain goals specified by DOD and if any 

improvements were needed. These goals include whether a supporting infrastructure is in 

place for the system. 

Page 22 GAO-10-102 Human Capital
 

refreshers for continuing panel members and as training for new members. 

However, like the PEO’s study, DOD’s 2008 evaluation report does not 

constitute an evaluation of the department’s training—for example, 

because it did not include an in-depth assessment of DOD’s training and 

also did not assess the department’s progress toward achieving the goals 

for the training, per agreed-upon measures or performance indicators. Of 

the components, we found that only the Army has taken some steps to 

evaluate the training it provides to employees on the system. Specifically, 

the Army assesses the adequacy of NSPS training during on-site reviews 

that it conducts as part of its implementation of the system. The Army 

conducted three such assessments during 2008 and an additional four such 

assessments during 2009. 

With respect to our site visits, although we found that each of the eight 

locations provided training on NSPS to employees, officials at each 

location also expressed concerns over the effectiveness of the training 

provided. For example, officials at seven locations told us that additional 

training was needed on writing job objectives or employee selfassessments under the system, while other officials noted that additional 

training was needed on the pay pool process, use of the system’s 

automated tools, and how the reconsideration process works. Similarly, 

officials at two locations raised concerns that the training they received 

did not provide them with the skills they needed to use the Performance 

Appraisal Application. For example, officials told us that they received 

training too early and had either forgotten it or the training had become 

outdated by the time they actually used the Performance Appraisal 

Application. Further, some program officials raised concerns about their 

organizations’ ability to provide employees with adequate training on the 

system when the employees are located outside the continental United 

States. For example, program officials at one location in Germany 

reported challenges providing employees located in the field with training 

on NSPS because they lack the resources to send NSPS trainers to those 

locations. However, until DOD evaluates its training, it will not be able to 

determine whether the training meets the needs of its employees or the 

department is making progress toward achieving the goals it specified for 

the training. 

DOD continues to take steps to ensure that employees receive timely 

performance feedback that is meaningful, constructive, and in accordance 

with the department’s requirements. In our first assessment of NSPS we 

noted that DOD’s implementing issuances required at least one 

documented interim performance review and an annual performance 

appraisal and that the Performance Appraisal Application allowed 

Ongoing Performance 

Feedback and Dialogue 

between Supervisors and 

Employees 

Page 23 GAO-10-102 Human Capital
 

supervisors and employees to document these feedback sessions. Since 

then, DOD has taken additional steps to enhance the Performance 

Appraisal Application by modifying the tool to allow supervisors and 

employees to identify where they are in the performance appraisal process 

and help them accomplish required actions in a timely manner. 

During our site visits, officials at all eight locations told us that NSPS helps 

ensure the occurrence of performance feedback between employees and 

supervisors. For example, officials noted that use of the Performance 

Appraisal Application encourages employee feedback by allowing 

employees to document and track feedback sessions, and that NSPS 

encourages direct discussions about performance-related issues, such as 

developing effective job objectives and establishing performance 

expectations. 

DOD continues to take steps to better link individual pay to performance 

as well. As we noted in our 2008 report, the NSPS performance 

management system uses a multirating system of five rating categories—of 

which the lowest rating is “1” (unacceptable performance) and the highest 

rating is a “5” (role model performance)—that allows distinctions to be 

made in employee performance and therefore compensation. Since then, 

DOD added details to the NSPS regulations to facilitate uniform, equitable 

practices across the department that accord with merit system principles. 

These include specifying specific share assignment ranges, rounding rules 

for converting raw performance scores to ratings, and formulas for 

determining share value and the calculation of performance payouts under 

NSPS. DOD also clarified that a common share value should apply 

throughout an entire pay pool. According to the PEO, these changes, in 

addition to the higher-level review of performance expectations, 

recommendations for ratings of record, share assignment, and payout 

distribution have helped to promote a more equitable method for 

appraising and compensating employees. 

System to Better Link 

Individual Pay to Performance 

in an Equitable Manner 

However, during our site visits, officials at seven of the eight locations told 

us that they saw the potential for factors other than performance to 

influence employees’ ratings, such as the quality of employees’ and 

supervisors’ writing skills. For example, rating officials at one location 

commented that NSPS does not reward employees based on their 

performance but rather on how well employees and supervisors can 

communicate in writing what the employee achieved during the 

performance management cycle. Similarly, at another location, a pay pool 

panel member told us that the paperwork submitted to the panel by both 

the employee and the supervisor must be of very high quality because it 

Page 24 GAO-10-102 Human Capital
 

can be difficult to defend a high rating recommended for an employee if 

the assessments are poorly written. DOD’s 2008 evaluation report also 

highlighted concerns from employees and managers over the department’s 

success in linking pay to performance under NSPS. For example, DOD’s 

report noted that while some employees believed that they saw some level 

of pay for performance under NSPS, others were uncertain. Further, 

DOD’s report also noted that some managers observed that the quality of 

written assessments contributed significantly to the way in which ratings 

were substantiated. 

We found that DOD also continues to take steps to ensure that adequate 

agency resources are allocated to NSPS. According to 5 U.S.C. § 

9902(e)(4), to the maximum extent practicable, for fiscal years 2004 

through 2012, the overall amount of money allocated for compensation of 

civilian employees in organizations under NSPS shall not be less than the 

amount that would have been allocated under the GS system. To meet the 

requirements of 5 U.S.C. § 9902(e)(4), DOD’s implementing issuances 

require that the components certify in writing to the Deputy Secretary of 

Defense through the Under Secretary of Defense for Personnel and 

Readiness that the amount expended for NSPS performance-based pay 

increases is no less than what would have been expended had these 

positions not been converted into NSPS. Following the 2008 NSPS 

performance management cycle, each of the components certified that it 

met this requirement. 

Means to Ensure That 

Adequate Agency Resources 

Are Allocated for System 

Design, Implementation, and 

Administration 

DOD also continues to capture NSPS implementation costs, and for fiscal 

year 2008, the PEO reported that NSPS implementation costs were about 

$61.8 million. According to the PEO, continuing implementation of NSPS 

will result in some additional program implementation costs, such as for 

training for NSPS, conducting outreach to employees and others, and 

improving the system’s automated tools. However, the PEO estimates that 

once the conversion of all non-bargaining unit employees is complete, the 

system’s implementation costs will decrease significantly unless there is a 

decision to convert bargaining unit employees. Thereafter, according to 

the PEO, the cost of administering NSPS will be no different than that of 

any other personnel system. 

In our 2008 report, we recommended that DOD take steps to better ensure 

the consistency and equity of the performance management process by 

requiring a third party to perform predecisional demographic and other 

analysis of the pay pool results. DOD did not concur with our 

recommendation, stating that its postdecisional analysis of the rating 

results was useful for identifying barriers and any needed corrective 

Predecisional Internal 

Safeguards to Determine if 

Rating Results Are Consistent, 

Equitable, and 

Nondiscriminatory 

Page 25 GAO-10-102 Human Capital
 

action. DOD also stated that if the information obtained from a 

postdecisional demographic analysis demonstrated that the results were 

not fair or equitable, for any reason, the process used to achieve those 

results could be examined with the intent to identify and eliminate barriers 

to a fair and equitable outcome. 

However, our review of the postdecisional analyses that the PEO and each 

of the components completed following the 2007 NSPS performance 

management cycle and the analyses that each of the eight organizations 

we visited for our review completed following the most recent 

performance management cycle in 2008 found these analyses were 

inconsistent, did not always include an analysis of the rating results by 

demographics, and were generally conducted at too high a level to provide 

decision makers with sufficient information to identify potential barriers 

or corrective actions. For example, we found that following the 2007 

performance management cycle, the PEO analyzed the rating results for 

more than 100,000 employees by select demographic groups, such as race, 

gender, ethnicity, age, veteran status, and targeted disability, but limited 

its analysis to the aggregate data from its pay pools. That is, the PEO did 

not analyze the rating results at the level where decisions are made—in the 

case of NSPS ratings and payouts, the pay pool level. Similarly, in 

analyzing the postdecisional analyses that the components conducted 

following the 2007 performance management cycle, we found 

inconsistencies in their approaches, primarily because the components 

were allowed to develop their own approaches for conducting this 

analysis. For example, only the Army and Fourth Estate included an 

analysis of the rating results by demographics as part of their respective 

postdecisional analyses. However, we also found that neither the Army’s 

nor the Fourth Estate’s demographic analysis of the ratings provided 

decision makers with sufficient information to identify possible barriers or 

corrective actions that could be taken to address such barriers. As with 

the PEO, this problem results because neither the Army nor the Fourth 

Estate conducted its analysis at the pay pool level. Of the eight locations 

we visited for our review, we found that only one organization’s 

postdecisional analysis following the 2008 performance management cycle 

included an analysis of its ratings results by demographics. 

Since we issued our 2008 report, DOD has taken steps to promote a degree 

of consistency in its postdecisional analysis of NSPS ratings and payout 

data. Specifically, in December 2008, DOD revised its implementing 

issuance to require the heads of DOD components to conduct an annual 

analysis of NSPS performance ratings and payouts for subordinate 

elements, and issue guidance to lower echelons and otherwise act to 

Page 26 GAO-10-102 Human Capital
 

identify, examine, and remove barriers to similar rating and payout 

potential for demographic and other groups in the workforce. Further, in 

May 2009, the PEO issued guidance, entitled Guidance for Conducting 

Annual Analysis of NSPS Performance Ratings and Payouts, in order to 

provide the components with a framework and suggested approaches for 

conducting their annual analysis and to serve as a starting point for 

identifying and examining barriers. For example, the guidance notes that 

the NSPS performance management system’s rating and payout process 

has four main outcomes—the rating of record, number of shares awarded, 

payout, and the distribution of the payout—and that each outcome should 

be reviewed. The guidance also notes that analysis is best done at the level 

where decisions are made—in the case of NSPS ratings and payouts, the 

pay pool level. Further, the guidance 

• expresses the expectation that as the components conduct their 

analyses, changes and improvements to the guidance will be made; 

• is careful to ensure that the components understand base parameters 

for conducting the analysis so it is conducted in a manner that is 

methodologically sound; 

• encourages consultation with experts, such as statisticians and human 

resources researchers, to assist with determining the most suitable 

analytical models to employ, the statistical tools to utilize, and the 

standards to adopt in relation to understanding, measuring, and 

reporting significant findings; and 

• makes responsibility for conducting the analysis a shared 

responsibility between various offices, including the components’ 

legal, equal employment opportunity, and human resources offices, but 

notes that the components should consider tasking their Office of 

General Counsel or Office of the Judge Advocate General, whose staff 

are well positioned to ensure that the components are in compliance 

with applicable statutes, regulations, and policies, with primary 

responsibility for conducting the analysis and ensuring that adequate 

resources are provided in support of the function. 

While issuance of the May 2009 guidance represents a noteworthy step, the 

guidance does not address all steps suggested by the Equal Employment 

Opportunity Commission for identifying and addressing potential barriers 

to fair, consistent, and equitable ratings. The Equal Employment 

Opportunity Commission’s Instructions to Federal Agencies for EEO 

Page 27 GAO-10-102 Human Capital
 

Management Directive 71533 provides guidance that agencies can use in 

identifying and addressing potential barriers.34 The instructions state that 

“barrier identification and elimination is the process by which agencies 

uncover, examine, and remove barriers to equal participation at all levels 

of the workforce.” Management Directive 715 further states that “where it 

is determined that an identified barrier serves no legitimate purpose with 

respect to the operation of an agency, this Directive requires that agencies 

take immediate steps to eliminate the barrier.” In conducting their 

analysis, the components’ data may uncover barriers or other potential 

problems. However, understanding why the barrier or problem exists, or 

what to do to address it, may require that the components take additional 

steps. To identify and eliminate potential barriers, the directive outlines a 

four-step process: (1) analyzing workforce data to identify potential 

barriers; (2) investigating actual barriers and their causes; (3) eliminating 

barriers, which includes devising a plan for improvement and developing 

overall objectives for barrier elimination, with corresponding action items, 

responsible personnel, and target dates; and (4) assessing the success of 

the plan. The PEO’s guidance aims to promote a degree of uniformity and 

standardization in conducting postdecisional analyses. However, the 

PEO’s guidance does not specify what process the components should 

follow to investigate potential barriers and their causes, nor does it specify 

a process for eliminating barriers that are found. By not specifying such 

steps in its guidance, the components may not follow a consistent 

approach when investigating barriers, which could hinder their efforts to 

eliminate them. 

While not predecisional, we recognize that DOD’s approach does provide 

some benefits, some of which are similar to those of a predecisional 

analysis. For example, DOD’s approach lays out a method of analyzing 

ratings that would address some of the merit principles in 5 U.S.C. § 

2301—for example, that employees should receive fair and equitable 

 

33Management Directive 715 is the policy guidance which the Equal Employment 

Opportunity Commission provides to federal agencies for their use in establishing and 

maintaining effective affirmative programs of equal employment opportunity under Section 

717 of Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. § 2000(e) et seq., and 

effective affirmative action programs under Section 501 of the Rehabilitation Act of 1973, 

as amended by Pub. L. No. 99-506, § 103 (1986). 

34Management Directive 715 defines a barrier as “an agency policy, principle, practice or 

condition that limits or tends to limit employment opportunities for members of a 

particular gender, race or ethnic background or for an individual (or individuals) based on 

disability status.” 

Page 28 GAO-10-102 Human Capital
 

treatment in all aspects of personnel management and that employees 

should be protected against arbitrary action, personal favoritism, or 

coercion for partisan political purposes.35 However, as stated previously, 

because DOD does not specify what process the components should 

follow to investigate and eliminate potential barriers, the components may 

not follow a consistent approach, which could hinder their efforts to 

ensure fair, consistent, and equitable ratings. 

While DOD continues to take steps to ensure a reasonable amount of 

transparency in its implementation of NSPS, concerns about the overall 

transparency of the system continue to exist. To improve the transparency 

of the NSPS performance management system, our September 2008 report 

recommended that DOD require commands to publish the final rating 

results to employees. DOD concurred with our recommendation and, in 

November 2008, amended its NSPS regulations and implementing 

issuances to require commands to publish the final rating results to 

employees. Under DOD’s revised guidance, performance review 

authorities are required to communicate the general pay pool results to the 

NSPS workforce in writing. At a minimum, this includes the number of pay 

pools (if aggregate pay pool results are necessary), the number of 

employees rated, the rating and share distributions, the average rating, the 

average share assignment, the share value (or average share value), and 

the average payout expressed as a percentage of base salary. At the eight 

locations we visited, we found that each of pay pools shared this 

information with employees following the 2008 NSPS performance 

management cycle. 

Reasonable Transparency of 

the System and Its Operation 

DOD continues to take other steps to ensure a reasonable amount of 

transparency of the NSPS performance management system. In May 2009, 

the PEO launched “NSPS Connect,” a centralized Web portal for 

employees, managers, and others to access NSPS products, such as online 

training courses, fact sheets, tips sheets, and information on the system’s 

automated tools. The PEO also continues to take steps to collect and share 

lessons learned on the department’s experiences implementing NSPS and 

facilitate lessons learned briefings with the components following each 

performance management cycle. Further, the PEO continues to report 

periodically on internal assessments and employee survey results relating 

to the NSPS performance management system. For example, in May 2009, 

the PEO published the results of its 2008 evaluation of NSPS. Similarly, as 

 

355 U.S.C. § 2301(b)(2) and (b)(8)(A). 

Page 29 GAO-10-102 Human Capital
 

we previously reported, DOD posts the results of its survey of civilian 

employees on a Web site that is accessible to DOD employees, 

supervisors, and managers. 

Officials at each of the eight locations we visited told us that efforts were 

being made to help ensure transparency of the NSPS performance 

management system and the rating process. According to officials, among 

the steps being taken to help ensure transparency, for example, were 

establishing multiple communities of practice, disseminating business 

rules and other guidance on NSPS to employees and managers under the 

system, and publishing monthly newsletters on NSPS. At seven of the 

locations, officials told us that town hall meetings were held to keep 

employees informed of NSPS-related developments, and officials at six 

locations told us that mock pay pool panels were held to show employees 

how the pay pool process works. 

However, our site visits revealed some concerns about the overall 

transparency of the system. For example, at three locations officials 

expressed concerns over a lack of transparency with regard to their pay 

pools’ business rules, indicating that their business rules either had not 

been published or were published well after the performance management 

cycle had begun. At one location, pay pool panel members told us that 

though it was 6 months into the current performance management cycle, 

they did not yet have copies of their business rules. Similarly, rating 

officials at four locations told us that they did not understand what 

constituted a rating of “4” because neither their pay pools nor business 

rules provided clear criteria. 

Although our site visits revealed concerns over the extent to which 

meaningful distinctions in individual employee performance are being 

made under NSPS, DOD has taken some steps that include addressing a 

recommendation we made in our 2008 report aimed at encouraging the use 

of all available rating categories. Specifically, at all eight locations we 

visited, officials told us that they did not believe NSPS was being 

implemented in a manner that encouraged meaningful distinctions in 

individual employee performance. For example, officials at four locations 

told us that they were hesitant to give ratings lower than a “3,” and 

officials at six locations told us that they believed that there was a forced 

distribution of the ratings or manipulation of the ratings in order to 

achieve a predetermined quota. Further, at one location we found that 

management stressed the importance of maintaining employee share value 

at above 2.0, which would result in a higher payout, regardless of the 

employee’s rating. According to the PEO, any effort to limit share value is 

Meaningful Distinctions in 

Individual Employee 

Performance 

Page 30 GAO-10-102 Human Capital
 

a roundabout way to establish preset limits on ratings and would 

constitute forced distribution, which the NSPS regulations prohibit.36

However, in response to concerns about the potential for the forced 

distribution of performance ratings under NSPS, in April 2009 the 

department posted to its NSPS Web site a fact sheet emphasizing that the 

forced distribution of ratings is prohibited under NSPS and describing how 

meaningful distinctions in performance are made under the system. DOD’s 

fact sheet provides guidance specifying what constitutes the forced 

distribution of ratings, why the forced distribution of ratings is prohibited, 

how use of standard performance indicators minimizes the potential for 

individual bias or favoritism, and how organizations can best apply this 

information when rating and rewarding employee performance under 

NSPS. DOD has also noted that if employees believe their rating did not 

result from meaningful distinctions or are unfair, the system affords them 

the opportunity to challenge their ratings through a formal process known 

as reconsideration. See appendix II for further information on the NSPS 

reconsideration process. 

 

365 C.F.R. § 9901.412(a). 

Page 31 GAO-10-102 Human Capital
 

Within DOD, both the PEO and the components are responsible for 

monitoring the implementation of NSPS. As part of its efforts to monitor 

the implementation of NSPS, the PEO conducts broad, annual evaluations 

of NSPS to determine whether the system is on track to achieve certain 

goals, or key performance parameters,37 by, among other things, 

monitoring employee perceptions of the system using DOD’s survey of 

civilian employees. In May 2009, the PEO published its first evaluation of 

NSPS, which focused on determining whether NSPS, as implemented in 

spiral 1 organizations, was on track to achieve some of the goals specified 

by DOD and if any improvements were needed.38 While some of DOD’s 

goals for the system lend themselves to comparisons to the safeguards—

for example, one of its goals is ensuring a credible and trusted system 

(which could align with the transparency safeguard)—PEO officials stated 

that to date, their focus has been limited to understanding how the 

components have generally implemented NSPS and not on monitoring or 

assessing the components’ implementation of the safeguards. 

Although DOD Monitors 

Some Aspects of the 

Implementation of NSPS, 

It Does Not Monitor How 

the Safeguards Specifically 

Are Implemented across 

the Department 

With respect to the components, DOD’s implementing issuances state that 

the heads of DOD components are accountable for the manner in which 

management in their organizations carries out NSPS policy, procedures, 

and guidance. However, we found that only the Army has taken steps 

similar to the PEO to assess whether it is on track to achieve DOD’s goals 

 

37DOD has established certain goals for the NSPS performance management system, known 

as key performance parameters. DOD defines a key performance parameter as “a capability 

or characteristic that is so significant that failure to meet a minimum ‘threshold’ can be 

cause for that element, concept or system to be reevaluated, or the program to be 

reassessed or terminated.” DOD originally identified six key performance parameters for 

NSPS: (1) High Performing Workforce and Management—whether employees and 

supervisors are compensated and retained based on their performance and contribution to 

mission; (2) Agile and Responsive Workforce and Management—whether the workforce 

can be easily sized, shaped, and deployed to meet changing mission requirements; 

(3) Credible and Trusted System—whether the system assures openness, clarity, 

accountability and adherence to the public employment principles of merit and fitness; 

(4) Fiscally Sound System—whether aggregate increases in civilian payroll, at the 

appropriations level, will conform to Office of Management and Budget fiscal guidance, 

and whether managers will have flexibility to manage to budget at the unit level; 

(5) Supporting Infrastructure—whether information technology support, training, and 

change management plans are available and funded; and (6) Schedule—whether NSPS will 

be operational and stable before November 2009. 

38DOD’s 2008 evaluation focused only on spiral 1 organizations under NSPS—which 

included a total of 108,758 employees. Also, according to the PEO, externally driven delays 

to converting employees to NSPS subsequently resulted in the department’s elimination of 

the sixth key performance parameter, Schedule. Therefore, DOD’s 2008 evaluation did not 

assess the extent to which the department was on track to achieve this particular goal. 

Page 32 GAO-10-102 Human Capital
 

for the system—the other components have not done so. Furthermore, 

none of the components monitors how the safeguards specifically are 

implemented within their organizations because there is no requirement to 

do so. 

We have previously reported that transitioning to a more performanceoriented pay system is a huge undertaking that requires constant 

monitoring.39 Further, in our 2008 assessment of NSPS, we noted that 

continued monitoring of the safeguards was needed to ensure that DOD’s 

actions were effective as implementation of NSPS proceeded. While DOD’s 

efforts to date have helped provide decision makers with some 

information on how the department is implementing NSPS, including 

potential areas for changes or improvements, they do not provide decision 

makers in DOD and the Congress with information to determine whether 

the safeguards specifically have been implemented effectively. Without 

monitoring the safeguards’ implementation, decision makers in DOD and 

the Congress lack information that could be used to determine whether 

the department’s actions are effective and whether the system is being 

implemented in a fair, equitable, and credible manner. For example, in 

conducting our review, we identified some issues related to the 

implementation of the safeguards that illustrate the need to monitor their 

implementation. These include the following: 

• Ensuring that adequate agency resources are allocated for the 

system’s design, implementation, and administration. We found 

that each of the components generally lacks visibility over the reasons 

why organizations have supplemented their pay pool funds. For 

example, Air Force NSPS program officials told us that for the 2009 

payout, while they knew that 8 of the Air Force’s 18 major commands 

supplemented their pay pool funds, they did not know the specific 

reasons why. According to PEO officials, organizations might elect to 

supplement their pay pool funds for a variety of reasons—for example, 

to recruit or retain employees or to compete with other organizations 

for talent. However, because they do not understand the reasons why 

pay pools supplement their pay pool funding—which could help DOD 

and the components understand the extent to which adequate 

resources have been allocated to the system—decision makers cannot 

identify or assess any trends in these practices. Indeed, DOD’s 2008 

 

39GAO, Human Capital: Symposium on Designing and Managing Market-Based and 

More Performance-Oriented Pay Systems, GAO-05-832SP (Washington, D.C.: July 27, 

2005). 

Page 33 GAO-10-102 Human Capital
 

evaluation report notes that some employees in organizations that 

supplemented their pay pools’ funding questioned whether the higher 

funding levels could be sustained over the long term. 

• Ensuring reasonable transparency of the system and its 

operation. We found evidence that three of the pay pools we visited 

deviated from their business rules during the last performance 

management cycle, indicating a lack of transparency of the 

performance management process in some instances. DOD’s guidance 

states that a pay pool’s business rules are the guiding principles or 

ground rules that are used throughout the pay pool process, that pay 

pool panels should establish these principles and hold one another 

accountable for following them, and that a pay pool’s policies—which 

would include its business rules—will be made available to employees 

before the end of the performance cycle. However, at three of the 

locations we visited, we found evidence that the pay pool had deviated 

from its business rules during the course of the last performance 

management cycle. For example, at one location the pay pool’s 

business rules required all recommended ratings be reviewed, noting 

that the pay pool panel will ensure that all employees receive 

appropriate consideration and that ratings are fair and consistent. 

However, officials we spoke with at that location told us that they did 

not review all recommended ratings in accordance with their business 

rules. Rather, only the recommended ratings of “4” or “5” were 

reviewed. As another example, we found evidence that a pay pool at 

another location used different criteria than what was specified in its 

business rules for allocating the number of shares to employees. 

According to component-level NSPS program officials, in order to 

ensure transparency of the system, pay pools should not deviate from 

their business rules once those rules are published. However, none of 

the components requires its pay pools to notify it when such an event 

occurs, or of the reasons why, though doing so could help provide 

decision makers with information on the extent to which pay pools are 

implementing the system in a manner that is transparent to employees. 

Page 34 GAO-10-102 Human Capital
 

Although DOD 

Civilian Employees 

under NSPS Have 

Mixed Views about 

the System, DOD Has 

Not Yet Developed 

and Implemented a 

Plan to Address 

Employees’ Negative 

Perceptions of Some 

Aspects of the System 

DOD civilian personnel have mixed perceptions about NSPS, and although 

DOD has taken some steps toward addressing employees’ concerns, it has 

not yet developed and implemented an action plan to address areas where 

employees express negative perceptions of the system. DOD’s most recent 

survey of civilian employees reveals that NSPS employees have mixed 

perceptions about NSPS. The responses to questions specifically asking 

about NSPS show positive perceptions about some aspects of 

performance management under NSPS, including connecting pay to 

performance, but show negative perceptions about other aspects of 

performance management, such as the appraisal process. Further, the 

most recent data indicate that the perceptions of those employees who 

have worked under NSPS the longest appear to have remained largely 

unchanged from the negative perceptions we reported in 2008. Moreover, 

during discussion groups we held at eight locations outside the continental 

United States, civilian employees and supervisors expressed consistent 

concerns and negative perceptions of NSPS that are similar to those 

identified in our 2008 report, although they also identified positive aspects 

of the system. DOD has taken some steps to address employees’ negative 

perceptions of the system; however, the department has yet to develop 

and implement an action plan that meets the intent of our prior 

recommendation because it does not specify such things as the actions 

DOD intends to take, who will be responsible for taking the action, and the 

timelines for doing so. We continue to believe that implementing such an 

action plan is important and note that doing so would be a step that DOD 

could take to involve employees in the system’s implementation—which is 

one of the safeguards we previously discussed. 

NSPS Employees Express 

Mixed Perceptions about 

the System 

According to DOD’s most recent survey data, some NSPS employees 

recognize that positive aspects of performance management, such as 

connecting pay to performance, exist under the system. For example, as 

shown in table 2, DOD’s survey data for 2008 indicate that an estimated 38 

percent of NSPS employees agree that differences in their performance are 

recognized in meaningful ways, as compared with an estimated 33 percent 

of non-NSPS employees. Further, an estimated 42 percent of NSPS 

employees agree that pay raises depend on how well employees perform 

their jobs, as compared with an estimated 25 percent of non-NSPS 

employees. When asked about how poor performers are dealt with, an 

estimated 34 percent of NSPS employees, versus an estimated 27 percent 

NSPS Employees in All Spirals 

Have Positive Views of Some 

Aspects of Performance 

Management but Negative 

Views of Other Aspects 

Page 35 GAO-10-102 Human Capital
 

of non-NSPS employees, agreed that steps are taken to deal with poor 

performers. 

Table 2: Estimated Percentage of Responses from NSPS and Non-NSPS Employees to Selected Questions about 

Performance Management in the 2008 Status of Forces Survey of Civilian Employees 

Percentage responding 

Performance management questions 

Employee 

description “Agree” “Neither/nor” “Disagree”

Differences in performance are recognized in meaningful NSPS 38 31 28

ways. Non-NSPS 33 32 31

Pay raises depend on how well employees perform their NSPS 42 26 30

jobs. Non-NSPS 25 29 42

Performance standards/expectations are directly related to NSPS 70 21 9

the organization’s mission. Non-NSPS 65 25 10

In my work unit, steps are taken to deal with a poor NSPS 34 31 30

performer who cannot or will not improve. Non-NSPS 27 30 38

My bonus and cash awards depend on how well I perform NSPS 61 17 23

my job. Non-NSPS 55 20 25

Source: Defense Manpower Data Center. 

Note: The estimated percentages are based on a 95 percent confidence interval and the margin of 

error is within +/- 1 percent. The response categories are collapsed for positive (“agree”) and negative 

(“disagree”) responses. That is, “agree” is the estimated percentage of employees who responded 

either “agree” or “strongly agree,” while “disagree” is the estimated percentage of employees who 

responded either “disagree” or “strongly disagree.” Totals may sum to less than or more than 100 

percent as a result of collapsing the positive and negative response categories. 

In comparison, an estimated 47 percent of non-NSPS employees, as 

compared with an estimated 44 percent of NSPS employees, agreed that 

their current performance appraisal system motivates them to perform 

well. Further, an estimated 34 percent of non-NSPS employees, as 

compared with an estimated 29 percent of NSPS employees, agreed that 

their performance appraisal system improves organizational performance. 

Table 3 shows additional comparisons between NSPS and non-NSPS 

employee responses to questions about performance appraisals. 

Page 36 GAO-10-102 Human Capital
 

Table 3: Estimated Percentage of Responses from NSPS and Non-NSPS Employees to Selected Questions about 

Performance Appraisals in the 2008 Status of Forces Survey of Civilian Employees 

Percentage responding 

Performance appraisal questions 

Employee 

description “Agree” “Neither/nor” “Disagree”

Performance appraisal is a fair reflection of performance. NSPS 62 18 19

Non-NSPS 65 19 14

In my most recent appraisal, I understood what I had to do NSPS 59 16 22

to be rated at different performance levels. Non-NSPS 65 17 14

My current performance appraisal system motivates me to NSPS 44 28 28

perform well. Non-NSPS 47 31 22

The performance appraisal system I am under improves NSPS 29 36 36

organizational performance. Non-NSPS 34 40 26

Procedures for reconsidering performance appraisal NSPS 37 45 19

ratings are fair. Non-NSPS 38 45 17

Source: Defense Manpower Data Center. 

Note: The estimated percentages are based on a 95 percent confidence interval and the margin of 

error is within +/- 1 percent. The response categories are collapsed for positive (“agree”) and negative 

(“disagree”) responses. That is, “agree” is the estimated percentage of employees who responded 

either “agree” or “strongly agree,” while “disagree” is the estimated percentage of employees who 

responded either “disagree” or “strongly disagree.” Totals may sum to less than or more than 100 

percent as a result of collapsing the positive and negative response categories. 

In our first assessment of NSPS, we reported that the results of DOD’s 

Status of Forces Survey of Civilian Employees indicated that the 

perceptions of employees who had been under the system the longest had 

become more negative on questions related to performance management. 

However, the results of DOD’s most recent administration of the survey in 

2008 indicate that spiral 1.1 employee perceptions are about the same as 

the May 2007 survey, as shown in table 4. For example, from the 

November 2006 through February 2008 administrations of DOD’s survey, 

the percentage of spiral 1.1 employees that agreed that they understood 

what they had to do to be rated at a different performance level declined 

from an estimated 59 percent in November 2006 to an estimated 53 percent 

in May 2007, then remained largely unchanged in February 2008 at an 

estimated 54 percent.40

Perceptions of Employees 

under NSPS the Longest 

Remain Largely Unchanged 

since GAO’s 2008 Report 

 

40The first Web-based Status of Forces Survey was conducted in October 2003. Regular 

administrations of the Status of Forces Survey of Civilian Employees occurred every 6 

months from October 2004 through November 2006, and annual administrations 

commenced in 2007. 

Page 37 GAO-10-102 Human Capital
 

Table 4: Estimated Percentage of Responses from Spiral 1.1 Employees to Selected Questions about Performance 

Management from the May 2006, November 2006, May 2007, and February 2008 Administrations of the Status of Forces 

Survey of Civilian Employees 

Percentage of spiral 1.1 employees responding Performance 

management questions 

Survey 

administration “Agree” “Neither/ nor” “Disagree”

May 2006 67 20 12

November 2006 59 22 16

May 2007 52 21 25

Performance appraisal is a fair reflection 

of performance. 

February 2008 54 20 25

May 2006 37 33 28

November 2006 35 31 31

May 2007 32 31 34

Differences in performance are 

recognized in meaningful ways. 

February 2008 35 30 32

May 2006an/a n/a n/a

November 2006 59 17 22

May 2007 53 17 29

In most recent appraisal, I understood 

what I had to do to be rated at different 

performance levels. 

February 2008 54 17 27

Source: Defense Manpower Data Center. 

Note: The estimated percentages are based on a 95 percent confidence interval and the margin of 

error is within +/- 2 percent. The response categories are collapsed for positive (“agree”) and negative 

(“disagree”) responses. That is, “agree” is the estimated percentage of employees who responded 

either “agree” or “strongly agree,” while “disagree” is the estimated percentage of employees who 

responded either “disagree” or “strongly disagree.” Totals may sum to less than or more than 100 

percent as a result of collapsing the positive and negative response categories. 

a

This question was not asked on the May 2006 administration of the Status of Forces Survey of 

Civilian Employees. 

In addition, when asked about the overall impact that NSPS will have on 

personnel practices in DOD, spiral 1.1 employees’ perceptions have 

become significantly more negative since first converting to NSPS in 2006, 

but showed little change between the May 2007 and February 2008 

surveys. Specifically, the results of the 2008 survey indicate that that an 

estimated 22 percent of spiral 1.1 employees responded that the overall 

impact of NSPS on the department’s personnel practices would be 

positive, as compared to an estimated 23 percent in May 2007 and an 

estimated 25 percent in November 2006. Table 5 shows a comparison of 

spiral 1.1 employee responses over time about the overall impact of NSPS 

on personnel practices in DOD. 

Page 38 GAO-10-102 Human Capital
 

Table 5: Estimated Percentage of Responses from Spiral 1.1 Employees about the Overall Impact of the National Security 

Personnel System from the May 2006, November 2006, May 2007, and February 2008 Administrations the Status of Forces 

Survey of Civilian Employees 

Percentage of spiral 1.1 employees responding 

Overall Impact question 

Survey 

administration “Positive” “Neither/ nor” “Negative”

May 2006 40 35 25

November 2006 25 32 42

May 2007 23 30 48

Overall, what type of impact will NSPS 

have on personnel practices in the 

DOD? 

February 2008 22 28 50

Source: Defense Manpower Data Center. 

Note: The estimated percentages are based on a 95 percent confidence interval and the margin of 

error is within +/- 3 percent. The response categories are collapsed for positive and negative 

responses. That is, “positive” is the estimated percentage of employees who responded either 

“positive” or “very positive,” while “negative” is the estimated percentage of employees who 

responded either “negative” or “very negative.” Totals may sum to less than or more than 100 as a 

result of collapsing the positive and negative response categories. 

NSPS Employees and 

Supervisors in Our 

Discussion Groups 

Expressed Consistent 

Concerns 

As with our first review of NSPS, DOD civilians in our discussion groups at 

locations outside the continental United States continue to express wideranging but consistent concerns about the NSPS performance 

management system. Although the results of our discussion groups are not 

generalizable to the entire population of DOD civilian employees, the 

themes that emerged provide valuable insight into employees’ perceptions 

about the implementation of NSPS thus far. 

Specifically, during these discussion groups, participants at six locations 

told us that they were initially optimistic about the intent of NSPS and the 

concept of pay for performance. Further, some participants indicated that 

they remain positive about the amount of performance pay and flexibilities 

afforded to supervisors to rate their employees under the system. 

However, as with our first review, discussion group participants at all 

eight locations we visited primarily expressed frustration with and 

concern about certain aspects of NSPS implementation and the system. 

The prevalent themes that emerged during our discussion groups include 

concerns over (1) the negative impact of NSPS on employee motivation 

and morale, (2) the excessive amount of time spent navigating the 

performance management process, (3) challenges with job objectives, 

(4) factors undermining employee confidence in the system, and 

(5) factors unrelated to job performance affecting employees’ final 

performance ratings. As we noted in 2008, the themes that emerged during 

Page 39 GAO-10-102 Human Capital
 

our discussion group sessions are not surprising. Our prior work, as well 

as that of OPM,41 has recognized that organizational transformations, such 

as the adoption of a new performance management system, often entail 

fundamental and radical changes that require an adjustment period to gain 

employees’ trust and acceptance. As a result, we expect major change 

management initiatives in large-scale organizations to take several years to 

be fully successful. 

A prevalent theme from our discussions with both employees and 

supervisors was that several aspects of NSPS have had a negative impact 

on employee motivation and morale—consistent with our first assessment 

of NSPS. Specifically, discussion group participants at all eight locations 

discussed how various aspects of NSPS—for example, their perception 

that a rating of “3” is only average—have negatively affected their 

motivation and morale. Discussion group participants at six of the eight 

locations also told us they have negative perceptions of what a rating of 

“3” means. At five of those locations, discussion group participants told us 

that they continue to believe that a rating of “3” means only “average,” in 

contrast to “valued performer,” as it was initially defined to the workforce 

by DOD. 

Concerns about a Negative 

Impact on Motivation and 

Morale 

Discussion group participants at five locations also discussed how 

achieving a rating higher than a “3” seemed to be an unattainable goal. For 

example, employees at four locations told us that they felt NSPS either 

does not provide incentives for high performance or encourages only 

mediocre performance from employees under the system because of the 

high number of employees receiving “3”-level ratings each year. As another 

example, supervisors at one location noted that across the installation 

there is a general feeling that everyone receives a rating of “3,” and 

therefore such a rating is considered average, no matter how DOD defines 

it. Similarly, discussion group participants at seven locations told us that 

they felt it was difficult for employees in certain positions to receive a 

rating of “5” because of the nature of their work or the perceived value 

their management placed on those positions. At one of those locations, 

supervisors told us that they felt such things as how the pay pool’s 

business rules were structured affected whether an employee could 

receive a high rating. At that location, the pay pool’s business rules 

specified that an employee must receive a higher-level award, such as a 

 

41Office of Personnel Management, Working for America: Alternative Personnel Systems 

in Practice and a Guide to the Future (Washington, D.C., October 2005). 

Page 40 GAO-10-102 Human Capital
 

command or agencywide award, to receive a rating of “5.” However, 

discussion group participants told us that they felt some employees were 

not in a position to receive such awards because of their positions or the 

type of work they did. 

In addition, discussion group participants at all eight locations we visited 

expressed frustration over the amount of the annual performance payout 

provided under NSPS. For example, discussion group participants noted 

that they felt the payout was not significant enough to encourage anything 

other than average performance. Discussion group participants at six of 

the eight locations also told us that they felt NSPS discourages employees 

from seeking additional responsibilities and opportunities that fall outside 

the scope of their objectives because their payout may not reflect their 

additional work. In addition, discussion group participants at six locations 

told us that because supervisory positions under NSPS require such a 

significant increase in responsibility and effort, and because the maximum 

allowable pay increase for reassignments is capped at 5 percent,42 some 

employees may not seek promotion opportunities. Similarly, a discussion 

group participant at another location expressed frustration that some 

employees only received their payout in the form of a bonus and not an 

increase in salary. Discussion group participants at three of the eight 

locations also expressed concerns that they felt performance payouts 

under NSPS tended to benefit higher-paid employees at the expense of 

lower-paid employees. For example, employees at one location expressed 

concerns that in their pay pool, the higher payouts under NSPS seemed to 

go to employees at the top of the pay bands. 

Another prevalent theme at seven of the eight locations and also 

highlighted in our first assessment was that employees spend an excessive 

amount of time navigating the performance management process. While 

the discussion group participants’ complaints about the time- and laborintensive nature of the system were not limited to any one part of the 

process, discussion group participants at seven locations pointed out that 

the time and effort required to complete the steps of the NSPS 

performance management process were significantly greater than what 

was required of them under previous systems. For example, one 

Concerns about an Excessive 

Amount of Time Spent 

Navigating the NSPS 

Performance Management 

Process 

 

42The NSPS regulations (5 C.F.R § 9901.353) state that an employee may only receive up to 

a total of a 5 percent cumulative increase to his or her base salary in any 12-month period 

as the result of an employee-initiated action, unless an exception is approved by an 

authorized management official. There are no limits to the number of times an employee 

may be reassigned by management, however. 

Page 41 GAO-10-102 Human Capital
 

supervisor we spoke with speculated that his supervisory duties under 

NSPS took him six times as long to perform as they had under the GS 

system, while another supervisor told us that he may have spent from 45 to 

50 hours assessing the performance of three employees, a task he could 

have completed in 10 hours under the GS system. 

At five of the locations we visited, employees expressed concerns about 

NSPS potentially affecting their ability to complete their jobs or affecting 

the mission because of what they perceived as an excessive amount of 

time required of employees and supervisors in navigating the NSPS 

performance management process. In some instances, employees spoke of 

impacts NSPS was having on their supervisors, while others spoke of their 

own experiences navigating the NSPS performance management process. 

At three locations, discussion group participants described how what they 

perceived as an excessive amount of time navigating the NSPS 

performance management process affected their ability to complete their 

job-related duties, requiring completion of some NSPS tasks, such as selfassessments and employee ratings, after work hours or on weekends. One 

employee described feeling inundated with information on NSPS and that 

it was difficult to stay on top of things while simultaneously performing his 

job, while another employee estimated that she spent about 2 hours per 

week on NSPS-related tasks. In some instances, discussion group 

participants told us that they saw the potential for the excessive time 

commitment required by NSPS to affect the missions of their 

organizations. According to one supervisor, any task that takes employees 

away from their daily work affects the mission, and any task that takes the 

time and patience of the command’s leadership detracts from the mission. 

Further, in discussing during a site visit the potential for NSPS to impact 

the organization’s mission, one general officer we spoke with described 

NSPS as “mission ineffective.” 

Another prevalent theme that emerged from our discussions with both 

supervisors and employees at all eight locations was that there are 

challenges with employee job objectives under NSPS. According to DOD, 

the NSPS performance management system is designed to provide a fair 

and equitable method for appraising and evaluating performance. As part 

of the system, DOD established the concept of “job objectives,” which are 

the required tasks of a given job as determined by managers and 

supervisors, and directed that job objectives be developed and used as the 

standards for evaluating employee performance. However, supervisors 

and employees at each of the eight locations discussed challenges they 

experienced developing their job objectives under NSPS. Specifically: 

Concerns about Challenges 

with Job Objectives under 

NSPS 

Page 42 GAO-10-102 Human Capital
 

• Although DOD guidance encourages employees to develop job 

objectives that are specific, measurable, aligned, realistic, and timed—

an approach summarized by the acronym S.M.A.R.T.—employees and 

supervisors we met at six of eight locations discussed how they found 

it challenging to develop job objectives that are measurable or that 

follow the S.M.A.R.T. approach. Supervisors at one location objected 

to the S.M.A.R.T. approach, particularly the “specific” portion, because 

they felt that job objectives needed to be broad enough to allow 

employees to discuss any accomplishments they make if they complete 

additional job activities or other tasks that might arise during the year. 

Supervisors at two locations discussed how the work they did was 

nebulous and unpredictable, which made it challenging to develop job 

objectives that not only reflected the nature of their work but that they 

could exceed. Similarly, supervisors at another location expressed 

concerns that employees’ job objectives may not reflect the work they 

do by the end of the performance management cycle because of 

constant changes within their organization. 

• According to discussion group participants at four locations, guidance 

for developing job objectives is either limited or nonexistent, which 

may result in different approaches to developing job objectives across 

an organization. At one of these locations, employees told us that their 

management had not established consistent ground rules for 

developing job objectives and that as a result some employees’ job 

objectives were based on out-of-date position descriptions. One 

organization we visited used a mixture of mandatory and employeespecific job objectives; but, according to one employee, little guidance 

exists to help employees and supervisors when they need to develop 

personalized job objectives. Employees at another location told us that 

there were significant differences in the amount of involvement they 

had in developing their job objectives. For example, one individual told 

us that employees in her office develop their own objectives, while 

another said employees in her office are assigned mandatory 

objectives and were thus unable to provide input into their objectives. 

• Discussion group participants at six locations expressed concerns that 

it can be difficult to achieve a high rating for some job objectives. 

Some locations we visited used mandatory job objectives, which left 

employees concerned that their job objectives did not accurately 

capture the full responsibilities of the work they performed. For 

example, at one location, a uniform, mandatory supervisory objective 

accounted for half of supervisors’ ratings, which, according to one 

supervisor, diminished the value of the other responsibilities they had. 

The supervisor expressed further concern that some mandatory job 

Page 43 GAO-10-102 Human Capital
 

objectives, such as those assigned to government purchase card 

holders, require a pass-fail evaluation, making it difficult, if not 

impossible, for the employee to receive a high rating. In one instance, a 

location we visited required all employees to be rated against a 

mandatory safety objective. However, according to some supervisors, 

it did not make sense for everyone to have the mandatory safety 

objective because for many employees, safety issues were out of their 

control. 

During our discussion groups, participants at all eight locations also 

discussed how various factors undermine employees’ confidence in the 

system and its implementation thus far. For example, discussion group 

participants at six locations commented that they do not believe that the 

NSPS performance management system has the ability to rate employees 

fairly. At the locations we visited, discussion of these concerns centered 

on such things as the perception of subjectivity and the potential for 

favoritism under NSPS; a lack of transparency surrounding the pay pool 

panel process, including a lack of understanding of what employees 

needed to do to receive higher ratings; and the perception of 

inconsistencies in interpretations of the standards used to determine 

employee ratings. 

Concerns about Factors That 

Undermine Employee 

Confidence in the System 

One prevalent theme at all eight locations involved perceptions of 

subjectivity, such as the potential for favoritism under NSPS during the 

rating and pay pool panel processes. At five locations, participants 

discussed their frustration with how NSPS takes the responsibility for 

rating employees out of the hands of supervisors and places it in the hands 

of the pay pool panel members, who may or may not have any direct 

knowledge of individual employees’ performance. One supervisor told us 

that NSPS may inadvertently favor employees who work closely or are in 

direct contact with members of the pay pool panel because those 

individuals have direct knowledge of the employees and, sometimes, their 

performance. Similarly, supervisors at another location told us that they 

did not feel that their pay pool panel understood their jobs and what they 

do and expressed frustration that the pay pool panel did not seem to be 

reaching out to their supervisors and higher-level reviewers for additional 

input on their performance. At five of the eight locations, discussion group 

participants also told us that they saw the potential for the employeesupervisor relationship to affect an employee’s rating—either to the 

benefit or detriment of the employee. 

Another prevalent theme at six of the eight locations—a theme also 

highlighted in our first assessment of NSPS—was a lack of transparency 

Page 44 GAO-10-102 Human Capital
 

and understanding of the pay pool panel process. Specifically, supervisors 

at two locations commented that their organizations’ pay pool panel 

processes were not transparent. A supervisor at one location commented 

that everything “goes into a black vacuum,” while another likened the 

process to a “black box.” Employees at that same location referred to the 

organization’s pay pool panel process as a “star chamber,” where decisions 

are made but are not explained to employees. Employees and supervisors 

at five locations expressed concerns about the amount of information they 

received from their pay pools and about the process itself; some desired 

further information to help them better understand the pay pool panel 

process. In addition, at six of the eight locations, discussion group 

participants told us that they did not understand what they needed to do to 

receive a higher rating. For example, an employee at one location told us 

that she was told by her supervisor that all employees had to receive a 

rating of “3” because they would have had to “walk on water” to receive a 

higher rating. Discussion group participants at two other locations also 

discussed how “walking on water” was a perceived standard for receiving 

a high rating under NSPS. At three locations, supervisors commented that 

they were unclear about what they could do to help their employees 

receive better ratings, while employees at four locations were unclear 

about what they could do to achieve higher ratings. 

Discussion group participants at six locations also raised concerns about 

inconsistent interpretation of the standards used when evaluating civilian 

employees under NSPS. Discussion group participants reported concerns 

that military supervisors may rate employees using more stringent 

standards than their civilian counterparts. Discussion group participants 

also reported concerns that some military supervisors may not value the 

NSPS performance management process and sometimes devote less time 

and effort to the process, which could affect employees’ ratings. One 

civilian supervisor told us that some military supervisors with whom he 

attended NSPS training had a much harsher perspective of employee 

performance than their civilian counterparts. For example, he noted that 

the military supervisors indicated that giving a rating of “1” or “2” was 

acceptable, whereas he believed civilian supervisors would be more 

inclined to give an employee a rating of “3.” Employees also told us that 

they do not believe some military supervisors value the work of employees 

who perform certain job functions, such as providing child care on an 

installation. 

A prevalent theme expressed by discussion group participants at all eight 

locations we visited is that factors unrelated to performance may affect 

employees’ final performance ratings. Such factors include the existence 

Concerns about Factors 

Unrelated to Performance 

Affecting Final Ratings 

Page 45 GAO-10-102 Human Capital
 

of a forced distribution or quota of ratings, the writing ability of employees 

and supervisors, and pay pool panel members’ knowledge of employees. 

For example: 

• Discussion group participants at all eight locations expressed concerns 

that their pay pool panels used a forced distribution or quota for 

ratings, which dictated the number of ratings in each category that 

could be awarded.43 Employees at one location told us that they were 

aware of their management’s attempts to artificially preserve a higher 

share value for employees by primarily awarding ratings of “3,” 

regardless of the employees’ performance. Further, at three locations 

discussion group participants told us that their management told them 

that all employees should expect to receive a rating of “3.” Moreover, 

some discussion group participants told us that they doubted that their 

actual performance had the bearing it was supposed to have on their 

final ratings, while others felt the use of a forced distribution or quotas 

was in direct conflict with the principles of pay for performance under 

NSPS. While no discussion group participants we met with were aware 

of any explicit guidance provided to pay pool panels or supervisors 

that limited the number of certain ratings they assigned employees, 

employees and supervisors from at least three locations believed that 

informal guidelines existed or that pay pool panels or supervisors were 

encouraged to limit the number of certain ratings they could assign. 

• Discussion group participants at all eight locations also expressed 

concerns that the writing ability of employees and supervisors may 

affect ratings—a theme also highlighted in our first assessment of 

NSPS. Supervisors at one location likened the process of developing 

employees’ assessments under NSPS to a writing contest. Moreover, 

supervisors told us that they felt their writing ability could 

unintentionally affect their employees’ ratings, noting, for example, 

that a supervisor’s ability to articulate an employee’s achievements in 

writing plays a significant role in supporting a higher rating for that 

employee. Employees shared the supervisors’ concerns, noting that 

they believed that succeeding under NSPS depended on the quality of 

their written assessments, rather than their job performance, and that 

their ratings could suffer if their supervisors did not provide the pay 

pool panel with well-written assessments. In discussing the potential 

influence that employees’ and supervisors’ writing skills may have on a 

 

43The NSPS regulations (5 C.F.R. § 9901.412(a)) state that the forced distribution of ratings 

(setting preestablished limits for the percentage or number of ratings that may be assigned 

at any level) is prohibited. 

Page 46 GAO-10-102 Human Capital
 

pay pool panel’s assessment of an employee, officials at seven of the 

eight locations acknowledged that in some instances writing skills had 

affected employees’ ratings and could overshadow employees’ 

performance. 

• Discussion group participants at seven locations also expressed 

frustration that employee ratings were potentially affected by the 

extent to which pay pool panel members have personal knowledge of 

employees or understand the nature of their work in general. Some 

discussion group participants felt that pay pool panel members’ 

personal knowledge of employees helped some employees receive 

higher ratings, while others told us that they felt that members of the 

pay pool panel were too far removed from them and lacked direct 

knowledge of the work they performed. One employee believed that 

individuals who were involved in implementing NSPS worked closely 

with pay pool panel members, or were senior managers who were 

more likely to receive higher ratings under NSPS than others. Other 

employees told us that they were concerned about the potential for 

pay pool panel members to advocate in some way for employees they 

personally know—for example, by encouraging the pay pool panel to 

contact a specific employee’s supervisor to seek additional 

information or justification for a rating. As a result, they felt that pay 

pool panel members’ personal knowledge of employees could benefit 

some employees, but not others. 

In our first assessment of NSPS, we recommended that DOD develop and 

implement a specific action plan to address employees’ perceptions of 

NSPS, based on guidance published by OPM for conducting annual 

employee surveys and providing feedback to employees on the results.44

The guidance suggests that after an agency’s survey results have been 

reviewed, the agency has a responsibility to provide feedback to 

employees on the results of the survey, as well as to let employees know 

the intended actions to address the results and the progress made on these 

actions. Further, the guidance suggests that agencies consider the 

following when developing action plans: 

DOD Has Not Yet Developed a 

Comprehensive Action Plan to 

Address Employees’ 

Perceptions of the System 

• the resources required; 

• who will be responsible for taking action; 

• who will be responsible for providing oversight; 

 

44Office of Personnel Management, Annual Employee Survey Guidance (Washington, D.C., 

November 2006). 

Page 47 GAO-10-102 Human Capital
 

• if the individuals taking the action have the necessary authority to 

make things happen; 

• what coordination, if any, is required, and how it will be accomplished; 

and 

• how agencies will adjust given any changes or delays in their actions. 

Since then, in June 2009, the PEO issued a departmentwide memorandum 

entitled “Addressing Key NSPS Workforce Concerns”; however, issuance 

of this memorandum does not fully meet the intent of our 2008 

recommendation. Specifically, the PEO’s June 2009 memorandum 

summarizes key concerns from the department’s 2008 evaluation of NSPS, 

summarizes departmentwide actions that had been taken to date to 

address employees’ concerns about the system, and suggests approaches 

to enhance local efforts to address workforce concerns. The PEO 

identified five key areas of concern, which are similar to those identified in 

our own discussion group sessions with DOD employees and supervisors: 

(1) performance communication and feedback, (2) understanding of 

performance management and the pay pool process, (3) trust in the system 

and its processes, (4) training and information, and (5) the amount of time 

needed to fulfill performance management responsibilities. The PEO’s 

memorandum urged the components to leverage information from the 

department’s 2008 evaluation of NSPS and focus on the five areas 

discussed above as they plan their own actions. Further, the PEO’s 

memorandum noted that DOD has taken some steps to address employees’ 

concerns about NSPS—for example, developing and fielding a pay pool 

training course for employees and rating officials, modifying its 

implementing issuances to require all performance review authorities to 

review pay pool panel results on an annual basis, and providing guidance 

to employees on the prohibition against the forced distribution of ratings. 

Issuance of the PEO’s memorandum represents an important first step. 

However, because the memorandum does not specify actions the 

department intends to take, who will be responsible for taking the action, 

and timelines for addressing areas where employees express negative 

perceptions of the system, it does not fully meet the intent of our 2008 

recommendation. In developing an action plan, we note that OPM recently 

issued guidance that agencies can use in developing action plans for 

improving employee satisfaction.45 According to OPM, action plans should 

clearly (1) state the objectives, (2) identify actions to be taken, (3) provide 

 

45Office of Personnel Management Memorandum, “Guidance on Employee Satisfaction 

Action Planning” (Aug. 19, 2009). 

Page 48 GAO-10-102 Human Capital
 

outcome measures and improvement targets, and (4) describe how 

progress will be tracked. In addition to identifying the specific actions that 

will be taken to achieve improvements, OPM’s guidance also suggests that 

agencies specify 

• time frames for accomplishing the actions, 

• who will be responsible for implementing the actions, 

• who will be affected by the actions, 

• the resources required, and 

• a plan to communicate these actions to managers and employees. 

We continue to believe that developing and implementing a plan to 

address employees’ perceptions of NSPS could help DOD make changes to 

the system that could lead to greater employee acceptance and, ultimately, 

the system’s successful implementation. Further, we note that having such 

a plan is an approach that DOD could take to involve employees in the 

system’s implementation—which is one of the safeguards we previously 

discussed. 

As we noted in our first assessment, DOD’s implementation of NSPS 

placed the department at the forefront of a significant transition facing the 

federal government. However, toward the end of this review, the future of 

NSPS became uncertain, given the proposed legislation that, if enacted, 

would terminate the system and require any future system created by DOD 

to use safeguards similar to those discussed in our report, including 

ensuring employee involvement in the system and providing adequate 

training and retraining. In light of the contingent nature surrounding NSPS 

and the possibility of implementing a different system, sustained and 

committed leadership will be imperative to provide focused attention 

necessary to implement any pay-for-performance system within DOD. Key 

to implementing a fair, effective, and credible system is including 

safeguards early on in the design of the system. Since we issued our first 

assessment of NSPS in 2008, we note that DOD has continued to take steps 

to meet the intent of each of the safeguards. However, with this latest 

assessment, we note that the department has not implemented the 

safeguards systematically; for example, it has not ensured that the training 

provided to employees on the system’s operations is effective. Further, 

DOD has not monitored how the safeguards specifically are implemented 

by lower-level organizations across the department. As a result, decision 

makers in DOD lack information that could be used to determine whether 

the department’s actions are effective and whether the system is being 

implemented in a fair, equitable, and credible manner. Additionally, while 

Conclusions 

Page 49 GAO-10-102 Human Capital
 

DOD has gained experience operating under NSPS, at the time of our 

review it had not yet developed an action plan for addressing employees’ 

perceptions of the system, as we recommended in 2008. As DOD moves 

forward with implementing a pay-for-performance system—whether NSPS 

or another—we believe that it is important for the department to improve 

upon its implementation of the safeguards and address employees’ 

concerns. Left unchecked, these issues could undermine any future human 

capital reform efforts within DOD. 

To help implement a fair, effective, and credible performance management 

system for its civilian employees—whether NSPS or another—we 

recommend that the Secretary of Defense take the following three actions: 

• Review and evaluate the effectiveness of the department’s training. 

• Ensure that guidance is in place for conducting a postdecisional 

analysis that specifies what process the components should follow to 

investigate and eliminate potential barriers to fair and equitable 

ratings. 

• Include, as part of the department’s monitoring of the implementation 

of its system, efforts to monitor and evaluate how the safeguards 

specifically are implemented by lower-level organizations across the 

department. 

In September 2009, we provided DOD with a draft of this report that 

included three recommendations to better address the safeguards and 

improve implementation of the NSPS performance management system. 

Specifically, we recommended that DOD (1) evaluate NSPS training, 

(2) review and revise its guidance for conducting postdecisional analysis 

of NSPS ratings, and (3) monitor how the safeguards specifically are 

implemented. In commenting on a draft of our report, DOD partially 

concurred with our three recommendations. DOD’s comments are 

reprinted in appendix III. 

Recommendations for 

Executive Action 

Agency Comments 

and Our Evaluation 

DOD partially concurred with our recommendations, noting the 

expectation that the Congress would require the department to terminate 

NSPS by January 1, 2012, and this action, in turn, would require the 

department to focus on drawing down NSPS in an orderly manner. DOD 

further stated that it would consider acting on our recommendations to 

the extent they are relevant as the department moves forward with any 

future performance management system. We believe that this is a 

Page 50 GAO-10-102 Human Capital
 

reasonable approach. As discussed above, we recognize the contingent 

nature surrounding NSPS as a result of provisions in the proposed 

National Defense Authorization Act for Fiscal Year 2010, which recently 

passed both Houses of Congress. Accordingly, we revised our 

recommendations to apply to any future performance management system 

for the department’s civilian employees—whether NSPS or another 

system. However, we also note that provisions of the proposed legislation 

would require DOD to implement certain safeguards and issue regulations 

for that system to provide a fair, credible and transparent performance 

appraisal system. We therefore continue to believe that our 

recommendations have merit. 

We are sending copies of this report to the appropriate congressional 

committees. We will make copies available to others upon request. The 

report also is available at no charge on GAO’s Web site at 

http://www.gao.gov. 

If you or your staff have any questions about this report, please contact me 

at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made major contributions to the report are 

Brenda S. Farrell 

listed in appendix IV. 

Director, Defense Capabilities and Management 

Page 51 GAO-10-102 Human Capital
 

List of Committees 

The Honorable Carl Levin 

Chairman 

The Honorable John McCain 

Ranking Member 

Committee on Armed Services 

United States Senate 

The Honorable Joseph I. Lieberman 

Chairman 

The Honorable Susan M. Collins 

Ranking Member 

Committee on Homeland Security and Governmental Affairs 

United States Senate 

The Honorable Ike Skelton 

Chairman 

The Honorable Howard P. McKeon 

Ranking Member 

Committee on Armed Services 

House of Representatives 

The Honorable Edolphus Towns 

Chairman 

The Honorable Darrell Issa 

Ranking Member 

Committee on Oversight and Government Reform 

House of Representatives 

Page 52 GAO-10-102 Human Capital
Appendix I: Scope and Methodology 

Page 53 GAO-10-102 

Appendix I: Scope and Methodology 

As with our first assessment of the National Security Personnel System 

(NSPS) in 2008,1

 we limited our scope in conducting this review to the 

performance management aspect of NSPS. Therefore, we addressed 

neither performance management of the Senior Executive Service at the 

Department of Defense (DOD) nor other aspects of NSPS, such as 

classification and pay. 

Determining 

Implementation of 

Safeguards and Monitoring 

Their Implementation 

To determine the extent to which DOD has implemented safeguards as 

part of the NSPS performance management system and monitored the 

implementation of the safeguards, we used the following safeguards, 

which we also reported on in our 2008 review: 

• Involve employees, their representatives, and other stakeholders in the 

design of the system, to include employees directly involved in 

validating any related implementation of the system. 

• Assure that the agency’s performance management systems link 

employee objectives to the agency’s strategic plan, related goals, and 

desired outcomes. 

• Provide adequate training and retraining for supervisors, managers, 

and employees in the implementation and operation of the 

performance management system. 

• Provide a process for ensuring ongoing performance feedback and 

dialogue between supervisors, managers, and employees throughout 

the appraisal period, and for setting timetables for review. 

• Implement a pay-for-performance evaluation system to better link 

individual pay to performance, and provide an equitable method for 

appraising and compensating employees. 

• Assure that certain predecisional internal safeguards exist to help 

achieve consistency, equity, nondiscrimination, and nonpoliticization 

of the performance management process (e.g., independent 

reasonableness reviews by a third party or reviews of performance 

rating decisions, pay determinations, and promotions before they are 

 

1

GAO, Human Capital: DOD Needs to Improve Implementation of and Address Employee 

Concerns about Its National Security Personnel System, GAO-08-773 (Washington, D.C.: 

Sept. 10, 2008). 

Human Capital 
Appendix I: Scope and Methodology 

finalized to ensure that they are merit-based, as well as pay panels who 

consider the results of the performance appraisal process and other 

information in connection with final pay decisions). 

• Assure that there are reasonable transparency and appropriate 

accountability mechanisms in connection with the results of the 

performance management process, to include reporting periodically on 

internal assessments and employee survey results relating to 

performance management and individual pay decisions while 

protecting individual confidentiality. 

• Assure that performance management results in meaningful 

distinctions in individual employee performance. 

• Provide a means for ensuring that adequate agency resources are 

allocated for the design, implementation, and administration of the 

performance management system. 

To assess implementation of the safeguards, we reviewed the legislative 

requirements and obtained and analyzed regulations and other guidance 

for implementing the NSPS performance management system. We also 

obtained and analyzed other documents, such as DOD’s rating results and 

reconsideration statistics, for the 2007 and 2008 NSPS performance 

management cycles.2

 We also interviewed knowledgeable officials in 

DOD’s NSPS Program Executive Office and the NSPS program offices of 

the four components—the Army, the Air Force, the Navy, and the Fourth 

Estate3

—to obtain a comprehensive understanding of their efforts to 

implement NSPS and each of the safeguards, as well as the processes, 

procedures, and controls used for monitoring and overseeing 

implementation of the system. In addition, we conducted site visits to 

select organizations located outside the continental United States to 

assess implementation of the safeguards. To allow for appropriate 

representation by each component, we visited two organizations per 

component, or eight organizations in total. The organizations we visited 

 

2

GAO has not independently verified the reliability of DOD’s reported reconsideration 

statistics. 

3

The Department of the Navy’s NSPS policies encompass Marine Corps civilians. The 

Fourth Estate includes all organizational entities in DOD that are not in the military 

departments or the combatant commands. Examples of Fourth Estate entities are the 

Office of the Secretary of Defense, the Joint Staff, the Office of the DOD Inspector General, 

the defense agencies, and DOD field activities. 

Page 54 GAO-10-102 Human Capital
Appendix I: Scope and Methodology 

were selected based on a number of factors, such as the presence of a 

large number or concentrated group of civilian employees under NSPS 

and, when possible, the presence of employees who had converted to 

NSPS under both spirals 1 and 2. We focused our efforts for this review on 

visiting organizations located outside the continental United States 

because our 2008 review focused on assessing implementation of NSPS 

and the safeguards at locations that were geographically distributed 

throughout the United States. We elected to focus our site visits in 

Germany and Hawaii because of the civilian employees located outside the 

continental United States who had converted to NSPS at the time we 

initiated our review, more than half were located in either Germany or 

Hawaii.4

 Also, we wanted to determine whether civilian employees located 

outside the continental United States were experiencing any unique 

problems or challenges with the system. In Germany, the organizations we 

visited were the 5th Signal Command; the 435th Air Base Wing; the 

Defense Finance and Accounting Service; and the George C. Marshall 

European Center for Security Studies, part of the Defense Security 

Cooperation Agency. In Hawaii, the organizations we visited were the 

Commander, Navy Region Hawaii; Headquarters, Pacific Air Force; the 

Naval Facilities Engineering Command, Hawaii; and the U.S. Army Corps 

of Engineers, Honolulu District. For each of the organizations we visited, 

we met with or interviewed the performance review authority, pay pool 

managers, pay pool panel members, rating officials, and the NSPS program 

manager or transition team, among others, to discuss the steps they have 

taken to implement the safeguards or otherwise ensure the fairness, 

effectiveness, and credibility of NSPS. To assess the organizations’ 

implementation of the safeguards, we compared and contrasted the 

information obtained during our interviews and supplemented this 

testimonial evidence with the other relevant documentation we obtained, 

such as the organizations’ pay pool business rules,5

 lessons learned, and 

training materials. 

 

4

According to the Program Executive Office, as of September 2008, about 85 percent of the 

civilian employees under NSPS were located in the continental United States, whereas 

about 15 percent were located outside the continental United States. Of the civilian 

employees under NSPS located outside the continental United States, about 54 percent 

were in either Germany or Hawaii. 

5

Business rules are the policies that govern a pay pool’s operations. They may specify, for 

example, the pay pool panel’s structure, roles and responsibilities, standards of conduct, 

and the processes used for reconciling employee ratings and allocating shares under NSPS. 

Page 55 GAO-10-102 Human Capital
Appendix I: Scope and Methodology 

To determine how DOD civilian employees perceive NSPS, we analyzed 

two sources of employee perceptions or attitudes. First, we analyzed the 

results of DOD’s survey of civilian employees to identify employee 

perceptions of NSPS and examine whether and how these perceptions 

may be changing over time. Second, we conducted small group 

discussions with civilian employees who had converted to NSPS and 

administered a short questionnaire to the participants at each of the eight 

organizations we visited. As with our first assessment of NSPS, our overall 

objective in using the discussion group approach was to obtain employees’ 

perceptions about NSPS and its implementation thus far. 

We analyzed the results of the Defense Manpower Data Center’s (DMDC) 

Status of Forces Survey6

 of Civilian Employees—including the May 2006, 

November 2006, May 2007, and February 2008 administrations—to gauge 

employee attitudes toward NSPS and performance management in general 

and identify indications of movement or trends in employee perceptions.7

As we reported in September 2008, we have reviewed the results of prior 

administrations of DMDC surveys and found the survey results, including 

the results of the Status of Forces Survey of Civilian Employees, 

sufficiently reliable to use for several GAO engagements. However, to 

understand the nature of any changes that were made to its survey 

methods for administering the survey for 2008 as compared with previous 

administrations, we also received responses to written questions from and 

discussed these data with officials at DMDC. Based on these responses 

and discussions, we determined that DMDC’s survey data remain 

Determining DOD Civilian 

Employees’ Perceptions of 

NSPS 

Analysis of DOD Survey 

Results 

 

6

The Status of Forces Survey is a series of Web-based surveys of the total force that allows 

DOD to (1) evaluate existing programs/policies, (2) establish baselines before 

implementing new programs/policies, and (3) monitor progress of programs/policies and 

their effects on the total force. 

7

DMDC has conducted large-scale, departmentwide surveys of active military personnel 

since 2002, called the Status of Forces Active Duty Survey. DMDC has also conducted 

surveys of reserve military personnel for DOD called the Status of Forces Reserve Survey. 

Since 2003, DMDC has administered its Status of Forces Survey of Civilian Employees, 

which includes questions about compensation, performance, and personnel processes. All 

surveys include outcome or “leading indicator” measures such as overall satisfaction, 

retention intention, and perceived readiness, as well as demographic items needed to 

classify individuals into various subpopulations. Regular administrations of the Status of 

Forces Survey of Civilian Employees occurred every 6 months from October 2004 through 

November 2006, while annual administrations commenced in 2007. 

Page 56 GAO-10-102 Human Capital
Appendix I: Scope and Methodology 

sufficiently reliable for the purpose of our reports on DOD civilian 

employees’ perceptions of NSPS.8

We also conducted small group discussions with DOD civilian employees 

and administered a short questionnaire during site visits in February and 

March 2009. Specifically, we conducted two discussion groups—one with 

nonsupervisory employees and another with supervisory employees—at 

each of the eight locations we visited, for a total of 16 discussion groups. 

As with our first assessment of NSPS in 2008, our objective in using this 

approach was to obtain employees’ perceptions about NSPS and its 

implementation thus far because discussion groups are intended to 

provide in-depth information about participants’ reasons for holding 

certain attitudes about specific topics and to offer insights into the range 

of concerns about and support for an issue. Further, in conducting our 

discussion groups, our intent was to achieve saturation—the point at 

which we were no longer hearing new information. 

GAO’s Discussion Groups with 

DOD Civilian Employees under 

NSPS 

As we previously reported, our discussion groups were not designed to 

(1) demonstrate the extent of a problem or to generalize the results to a 

larger population, (2) develop a consensus to arrive at an agreed-upon 

plan or make decisions about what actions to take, or (3) provide 

statistically representative samples or reliable quantitative estimates. 

Instead, our discussion groups provide in-depth information about 

participants’ reasons for holding certain attitudes about specific topics and 

offer insights into the range of concerns about and support for an issue. 

Although the results of our discussion sessions are not generalizable to the 

entire NSPS civilian population, the composition of our discussion groups 

was designed to ensure that we spoke with employees from each of the 

four components at locations outside the continental United States. 

Because supervisory and nonsupervisory employees have distinct roles 

with respect to NSPS, we held separate discussion sessions for these 

groups. 

To select the discussion group participants, we requested that the 

organizations we visited provide us with lists of employees who had 

converted to NSPS. From the lists provided, we selected participants 

based on their supervisory and nonsupervisory status. To ensure 

 

8

In our September 2008 report, we identified areas for improvement with regard to DOD’s 

survey results—for example, use of nonresponse analysis, which is a good survey research 

practice, to clarify whether those employees who did not respond to DOD’s surveys may 

provide substantively different answers than those who did respond. 

Page 57 GAO-10-102 Human Capital
Appendix I: Scope and Methodology 

maximum participation of the selected employees, we randomly selected 

up to 20 participants from each group with the goal of meeting with 8 to 12 

individuals in each discussion group and provided the employee names 

and a standard invitation to GAO’s points of contact to disseminate to the 

employees. At the majority of locations, we reached our goal of meeting 

with 8 to 12 individuals in each discussion group; however, since 

participation was not compulsory, in two instances we did not reach our 

goal of 8 participants per discussion group. Table 6 provides information 

on the composition of our discussion groups. 

Table 6: Composition of Discussion Groups 

Discussion group participants 

Organization 

Nonsupervisory

employees

Supervisory

 employees

Total discussion 

group participants 

Total NSPS 

employees 

assigned to the 

pay pool visiteda

Army 

5th Signal Command, Funari Barracks, Germany 11 10 21 557

U.S. Army Corps of Engineers, Honolulu District, 

Fort Shafter, Hawaii 

7 9 16 246

Air Force 

435th Air Base Wing, Ramstein Air Base, 

Germany 

13 9 22 314

Headquarters, Pacific Air Force, Hickam Air Force 

Base, Hawaii 

9 9 18 100

Navy 

Commander, Navy Region Hawaii, Pearl Harbor, 

Hawaii 

12 15 27 320

Naval Facilities Engineering Command, Hawaii, 

Pearl Harbor, Hawaii 

11 10 21 97

Fourth Estate 

Defense Finance and Accounting Service, 

Kaiserslautern, Germany 

12 4 16 1,497

George C. Marshall European Center for Security 

Studies, Garmisch-Partenkirchen, Germany 

10 13 23 434

Total 85 79 164 3,565

Source: GAO (analysis) and DOD (number of employees assigned to the respective pay pool). 

a

The totals listed include the number of civilian employees each organization rated during the 2008 

NSPS performance management cycle. 

To facilitate our discussion groups, we developed a discussion guide to 

help the moderator in addressing several topics related to employees’ 

perceptions of the NSPS performance management system. These topics 

Page 58 GAO-10-102 Human Capital
Appendix I: Scope and Methodology 

include employees’ overall perception of NSPS and the rating process, the 

training they received on NSPS, the communication they have had with 

their supervisors, positive aspects they perceive of NSPS, and any changes 

they would make to the system, among others. Each discussion group was 

scheduled for a 2-hour period and began with the GAO moderator greeting 

the participants, describing the purpose of the study, and explaining the 

procedures for the discussion group. Participants were assured that all of 

their comments would be discussed in the aggregate or as part of larger 

themes that emerged. The GAO moderator asked participants open-ended 

questions related to NSPS, while at least one other GAO analyst observed 

the discussion group and took notes. Following the conclusion of all our 

discussion group sessions, we performed content analysis of the sessions 

in order to identify the themes that emerged and to summarize the 

participants’ perceptions of NSPS. We reviewed responses from several of 

the discussion groups and created a list of themes and subtheme 

categories. We then reviewed the comments from each of the 16 

discussion groups and assigned each comment to the appropriate 

category, which was agreed upon by two analysts. If agreement was not 

reached on a comment’s placement in a category, another analyst 

reconciled the issue by placing the comment in either one or more of the 

categories. The responses in each category were then used in our 

evaluation and discussion of how civilian employees perceive NSPS. 

Following each discussion group we administered a questionnaire to the 

participants to obtain further information on their background, tenure 

with the federal government and DOD, and attitudes toward NSPS. We 

received questionnaires from 164 discussion group participants. In 

addition to collecting demographic data from participants for the purpose 

of reporting with whom we spoke (see table 7), the purpose of our 

questionnaire was to (1) collect information from participants that could 

not easily be obtained through discussion, for example, information 

participants may have been uncomfortable sharing in a group setting, and 

(2) collect some of the same data found in past DOD surveys. Specifically, 

the questionnaire included questions designed to obtain employees’ 

perceptions of NSPS as compared to their previous personnel system, the 

accuracy with which they felt their ratings reflected their performance, 

and management’s methods for conveying overall rating information. 

Since the questionnaire was used to collect supplemental information and 

was administered solely to the participants of our discussion groups, the 

results represent the opinions of only those employees and cannot be 

projected across DOD, a component, or any single pay pool we visited. 

Page 59 GAO-10-102 Human Capital
Appendix I: Scope and Methodology 

Table 7: Composition of Discussion Groups by Demographic Category per Component 

Component 

Category Army Air Force Navy Fourth Estate Total

Male 22 25 21 17 85

Female 15 15 27 22 79

Total 37 40 48 39 164

American Indian or Alaskan Native 0 1 0 0 1

Asian 14 4 21 1 40

Black/African American 4 2 1 4 11

Hispanic 0 1 2 3 6

Native Hawaiian or other Pacific Islander 0 0 7 0 7

White 16 28 14 31 89

Other 3 4 3 0 10

Total 37 40 48 39 164

Source: GAO. 

Note: Participants voluntarily self-reported demographic information in our questionnaire. 

We visited or contacted the following organizations during our review: 

Department of Defense • Defense Finance and Accounting Service, Indianapolis, Indiana; Rome, 

New York; Columbus, Ohio; and Kaiserslautern, Germany 

• Defense Manpower Data Center, Arlington, Virginia 

• Defense Security Cooperation Agency, Arlington, Virginia 

• George C. Marshall European Center for Security Studies, 

Garmisch-Partenkirchen, Germany 

• Equal Employment Opportunity Office, Arlington, Virginia 

• Fourth Estate NSPS Program Management Office, Arlington, Virginia 

• NSPS Program Executive Office, Arlington, Virginia 

Department of the Army • 5th Signal Command, Funari Barracks, Germany 

• Civilian Personnel Evaluation Agency, Alexandria, Virginia 

• NSPS Program Management Office, Alexandria, Virginia 

• U.S. Army Corps of Engineers 

• Headquarters, Washington, D.C. 

• Honolulu District, Fort Shafter, Hawaii 

• Pacific Ocean Division, Fort Shafter, Hawaii 

Page 60 GAO-10-102 Human Capital
Appendix I: Scope and Methodology 

Department of the Air 

Force 

• 435th Air Base Wing, Ramstein Air Base, Germany 

• Headquarters, Pacific Air Force, Hickam Air Force Base, Hawaii 

• NSPS Program Office, Arlington, Virginia 

Department of the Navy • Commander, Navy Region Hawaii, Pearl Harbor, Hawaii 

• Naval Facilities Engineering Command, Hawaii, Pearl Harbor, Hawaii 

• NSPS Program Office, Navy Yard, Washington, D.C. 

• Office of Civilian Human Resources, Navy Yard, Washington, D.C. 

We conducted this performance audit from November 2008 through 

September 2009 in accordance with generally accepted government 

auditing standards. Those standards require that we plan and perform the 

audit to obtain sufficient and appropriate evidence to provide a reasonable 

basis for our findings and conclusions based on our audit objectives. We 

believe that the evidence obtained provides a reasonable basis for our 

findings and conclusions based on our audit objectives. 

Page 61 GAO-10-102 Human Capital
Appendix II: Reconsideration Process Appendix II: Reconsideration Process 

Department of Defense (DOD) civilian employees who receive 

performance ratings under the National Security Personnel System (NSPS) 

have the option of challenging their ratings through a formal process 

known as reconsideration. The reconsideration process is the sole and 

exclusive agency administrative process for nonbargaining unit 

employees to challenge their ratings.1

 However, DOD’s NSPS regulations 

also allow for alternative dispute resolution techniques, such as mediation 

or interest-based problem solving, to be pursued at any time during the 

reconsideration process consistent with component policies and 

procedures. Under the reconsideration process, employees may challenge 

their ratings of record or individual job objective ratings; employees 

cannot challenge their performance payout, the number of shares 

assigned, the share value, or the distribution of their performance payout 

between salary increase and bonus, nor can they challenge their 

recommended ratings of record, interim reviews, or applicable closeout 

assessments. In addition, employees who allege that their performance 

ratings are based on prohibited discrimination or reprisal may not use the 

reconsideration process; rather, such allegations are to be processed 

through the department’s equal employment opportunity discrimination 

complaint procedure. 

Employees who wish to challenge their rating have 10 calendar days from 

the receipt of their ratings of record to submit written requests for 

reconsideration to their pay pool managers. Within 15 calendar days of the 

pay pool manager’s receipt of an employee’s request for reconsideration, 

the pay pool manager must render a written decision that includes a brief 

explanation of the basis of the decision. The pay pool manager’s decision 

is final, unless the employee seeks further reconsideration from the 

performance review authority. Specifically, if the employee is dissatisfied 

with the pay pool manager’s decision, or if none is provided within the 

prescribed time frames, the employee may submit a written request for 

final review by the performance review authority or his or her designee. 

This request must be submitted within 5 calendar days of receipt of the 

pay pool manager’s decision or within 5 calendar days of the date the 

 

1

In contrast, negotiated grievance procedures are the exclusive administrative procedures 

for bargaining unit employees. However, if a negotiated grievance procedure is not 

available to a bargaining unit employee or challenging a rating of record or job objective 

rating is outside the scope of the employee’s negotiated grievance procedure, a bargaining 

unit employee may challenge his or her rating of record or job objective rating in 

accordance with the DOD’s NSPS regulations and implementing issuances. As of May 2009, 

DOD reported that there were approximately 685 bargaining unit employees and 18 

certified bargaining units representing NSPS employees.

Page 62 GAO-10-102 Human Capital 
Appendix II: Reconsideration Process 

decision should have been rendered. The performance review authority 

then is allotted 15 calendar days from receipt of the written request from 

the employee to make a decision, which is final. If the final decision is to 

change the rating of record or job objective rating, the revised rating takes 

the place of the original one, and a revised performance appraisal is 

prepared for the employee. 

According to DOD’s 2008 evaluation report, for the 2007 NSPS 

performance management cycle,2

 2,302 civilian employees out of the 

100,465 employees who were rated under NSPS elected to file a request for 

reconsideration of their ratings, and of these, about 33 percent of the 

requests were granted. For the 2008 NSPS performance management 

cycle,3

 according to the NSPS Program Executive Office, as of June 2009, 

4,296 civilian employees out of the 170,149 employees who were rated 

under NSPS elected to file requests for reconsideration of their ratings, 

and of these, about 52 percent of the requests were granted.4

 

2

DOD civilian employees under NSPS received their payout for the 2007 NSPS performance 

management cycle in January 2008. 

3

DOD civilian employees under NSPS received their payout for the 2008 NSPS performance 

management cycle in January 2009. 

4

GAO has not independently verified the reliability of DOD’s reported reconsideration 

statistics. 

Page 63 GAO-10-102 Human Capital
Appendix III: Comments from the Department Appendix III: Comments from the of Defense 

Department of Defense 

Page 64 GAO-10-102 Human Capital 
Appendix III: Comments from the Department 

of Defense 

Page 65 GAO-10-102 Human Capital
Appendix IV: 

A

GAO Contact and Staff 

cknowledgments 

Page 66 GAO-10-102 

Appendix IV: GAO Contact and Staff 

Acknowledgments 

Brenda S. Farrell, (202) 512-3604 or farrellb@gao.gov GAO Contact 

In addition to the contact named above, Ron Fecso, Chief Statistician; 

Marion Gatling, Assistant Director; Margaret G. Braley; Virginia A. 

Chanley; William Colwell; Emily Gruenwald; K. Nicole Harms; Cynthia 

Heckmann; Wesley A. Johnson; Lonnie McAllister; Carolyn Taylor; John W. 

Van Schaik; Jennifer L. Weber; Cheryl A. Weissman; and Gregory H. 

Wilmoth made key contributions to the report. 

Acknowledgments 

Human Capital 
Related GAO Products Related GAO Products 

Human Capital: Continued Monitoring of Internal Safeguards and an 

Action Plan to Address Employee Concerns Could Improve 

Implementation of the National Security Personnel System. GAO-09-840. 

Washington, D.C.: June 25, 2009. 

Questions for the Record Related to the Implementation of the 

Department of Defense’s National Security Personnel System. 

GAO-09-669R. Washington, D.C.: May 18, 2009. 

Human Capital: Improved Implementation of Safeguards and an Action 

Plan to Address Employee Concerns Could Increase Employee 

Acceptance of the National Security Personnel System. GAO-09-464T. 

Washington, D.C.: April 1, 2009. 

Human Capital: Opportunities Exist to Build on Recent Progress to 

Strengthen DOD’s Civilian Human Capital Strategic Plan. GAO-09-235. 

Washington, D.C.: February 10, 2009. 

High-Risk Series: An Update. GAO-09-271. Washington, D.C.: 

January 2009. 

Human Capital: DOD Needs to Improve Implementation of and Address 

Employee Concerns about Its National Security Personnel System. 

GAO-08-773. Washington, D.C.: September 10, 2008. 

The Department of Defense’s Civilian Human Capital Strategic Plan 

Does Not Meet Most Statutory Requirements. GAO-08-439R. Washington, 

D.C.: February 6, 2008. 

Human Capital: DOD Needs Better Internal Controls and Visibility over 

Costs for Implementing Its National Security Personnel System. 

GAO-07-851. Washington, D.C.: July 16, 2007. 

Human Capital: Federal Workforce Challenges in the 21st Century. 

GAO-07-556T. Washington, D.C.: March 6, 2007. 

Post-Hearing Questions for the Record Related to the Department of 

Defense’s National Security Personnel System (NSPS). GAO-06-582R. 

Washington, D.C.: March 24, 2006. 

Human Capital: Observations on Final Regulations for DOD’s National 

Security Personnel System. GAO-06-227T. Washington, D.C.: 

November 17, 2005. 

Page 67 GAO-10-102 Human Capital
Related GAO Products 

Human Capital: Designing and Managing Market-Based and More 

Performance-Oriented Pay Systems. GAO-05-1048T. Washington, D.C.: 

September 27, 2005. 

Human Capital: Symposium on Designing and Managing Market-Based 

and More Performance-Oriented Pay Systems. GAO-05-832SP. 

Washington, D.C.: July 27, 2005. 

Human Capital: DOD’s National Security Personnel System Faces 

Implementation Challenges. GAO-05-730. Washington, D.C.: July 14, 2005. 

Questions for the Record Related to the Department of Defense’s National 

Security Personnel System. GAO-05-771R. Washington, D.C.: June 14, 

2005. 

Questions for the Record Regarding the Department of Defense’s 

National Security Personnel System. GAO-05-770R. Washington, D.C.: 

May 31, 2005. 

Post-hearing Questions Related to the Department of Defense’s National 

Security Personnel System. GAO-05-641R. Washington, D.C.: April 29, 

2005. 

Human Capital: Agencies Need Leadership and the Supporting 

Infrastructure to Take Advantage of New Flexibilities. GAO-05-616T. 

Washington, D.C.: April 21, 2005. 

Human Capital: Selected Agencies’ Statutory Authorities Could Offer 

Options in Developing a Framework for Governmentwide Reform. 

GAO-05-398R. Washington, D.C.: April 21, 2005. 

Human Capital: Preliminary Observations on Proposed Regulations for 

DOD’s National Security Personnel System. GAO-05-559T. Washington, 

D.C.: April 14, 2005. 

Human Capital: Preliminary Observations on Proposed Department of 

Defense National Security Personnel System Regulations. GAO-05-517T. 

Washington, D.C.: April 12, 2005. 

Human Capital: Preliminary Observations on Proposed DOD National 

Security Personnel System Regulations. GAO-05-432T. Washington, D.C.: 

March 15, 2005. 

Page 68 GAO-10-102 Human Capital
Related GAO Products 

Human Capital: Principles, Criteria, and Processes for Governmentwide 

Federal Human Capital Reform. GAO-05-69SP. Washington, D.C.: 

December 1, 2004. 

Human Capital: Building on the Current Momentum to Transform the 

Federal Government. GAO-04-976T. Washington, D.C.: July 20, 2004. 

DOD Civilian Personnel: Comprehensive Strategic Workforce Plans 

Needed. GAO-04-753. Washington, D.C.: June 30, 2004. 

Human Capital: Implementing Pay for Performance at Selected 

Personnel Demonstration Projects. GAO-04-83. Washington, D.C.: January 

23, 2004. 

Human Capital: Key Principles for Effective Strategic Workforce 

Planning. GAO-04-39. Washington, D.C.: December 11, 2003. 

DOD Personnel: Documentation of the Army’s Civilian WorkforcePlanning Model Needed to Enhance Credibility. GAO-03-1046. 

Washington, D.C.: August 22, 2003. 

Posthearing Questions Related to Proposed Department of Defense 

(DOD) Human Capital Reform. GAO-03-965R. Washington, D.C.: July 3, 

2003. 

Human Capital: Building on DOD’s Reform Effort to Foster 

Governmentwide Improvements. GAO-03-851T. Washington, D.C.: June 4, 

2003. 

Posthearing Questions Related to Strategic Human Capital 

Management. GAO-03-779R. Washington, D.C.: May 22, 2003. 

Human Capital: DOD’s Civilian Personnel Strategic Management and 

the Proposed National Security Personnel System. GAO-03-493T. 

Washington, D.C.: May 12, 2003. 

Defense Transformation: DOD’s Proposed Civilian Personnel System 

and Governmentwide Human Capital Reform. GAO-03-741T. 

Washington, D.C.: May 1, 2003. 

Defense Transformation: Preliminary Observations on DOD’s Proposed 

Civilian Personnel Reforms. GAO-03-717T. Washington, D.C.: April 29, 

2003. 

Page 69 GAO-10-102 Human Capital
Related GAO Products 

High-Risk Series: Strategic Human Capital Management. GAO-03-120. 

Washington, D.C.: January 2003. 

Acquisition Workforce: Status of Agency Efforts to Address Future 

Needs. GAO-03-55. Washington, D.C.: December 18, 2002. 

Military Personnel: Oversight Process Needed to Help Maintain 

Momentum of DOD’s Strategic Human Capital Planning. GAO-03-237. 

Washington, D.C.: December 5, 2002. 

Managing for Results: Building on the Momentum for Strategic Human 

Capital Reform. GAO-02-528T. Washington, D.C.: March 18, 2002. 

A Model of Strategic Human Capital Management. GAO-02-373SP. 

Washington, D.C.: March 15, 2002. 

Human Capital: Taking Steps to Meet Current and Emerging Human 

Capital Challenges. GAO-01-965T. Washington, D.C.: July 17, 2001. 

Human Capital: Major Human Capital Challenges at the Departments of 

Defense and State. GAO-01-565T. Washington, D.C.: March 29, 2001. 

High-Risk Series: An Update. GAO-01-263. Washington, D.C.: January 

2001. 

(351277) Page 70 GAO-10-102 Human Capital
GAO’s Mission The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday afternoon, GAO 

posts on its Web site newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.” 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s Web site, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information. 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, DC 20548 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

Public Affairs 

Please Print on Recycled Paper