Office of Personnel Management: Retirement Modernization Planning and Management Shortcomings Need to Be Addressed
Published: 2009-04-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: For the past two decades, the Office of Personnel Management (OPM) has been working to modernize the paper-intensive processes and antiquated systems used to support the retirement of federal employees. By moving to an automated system, OPM intends to improve the program's efficiency and effectiveness. In January 2008, GAO recommended that the agency address risks to successful system deployment. Nevertheless, OPM deployed a limited initial version of the modernized system in February 2008. After unsuccessful efforts to address system quality issues, OPM suspended system operation, terminated a major contract, and began restructuring the modernization effort, also referred to as RetireEZ. For this study, GAO was asked to (1) assess the status of OPM's efforts to plan and implement the RetireEZ program and (2) evaluate the effectiveness of the agency's management of the modernization initiative. To do this, GAO reviewed OPM program documentation and interviewed agency and contractor officials.
section_title: What GAO Found
paragraphs: OPM remains far from achieving the modernized capabilities it had planned. Specifically, the agency has partially implemented two of eight planned capabilities: (1) an integrated database of retirement information accessible to OPM and agency retirement processing personnel and (2) enhanced customer service capabilities that support customer needs and provide self-service tools. However, the remaining six capabilities have yet to be implemented because they depended on deliverables that were to be provided by a contract that is now terminated. Examples of these missing capabilities include: (1) automated submission of retirement information through interfaces with federal agencies and (2) Web-accessible self-service retirement information for active and retired federal employees. Further, OPM has not yet developed a complete plan that describes how the program is to proceed without the system that was to be provided under the terminated contract. Although agency documents describe program implementation activities, they do not include a definition of the program, its scope, lines of responsibility and authority, management processes, and a schedule. Also, modernization program documentation does not describe results-oriented performance goals and measures. Until the agency completes and uses a plan that includes all of the above elements to guide its efforts, it will not be properly positioned to move forward with its restructured retirement modernization initiative. Further, OPM has significant weaknesses in five key management areas that are vital for effective development and implementation of its modernization program: cost estimating, earned value management (a recognized means for measuring program progress), requirements management, testing, and oversight. For example, the agency has not developed a cost estimating plan or established a performance measurement baseline--prerequisites for effective cost estimating and earned value management. Further, although OPM is revising its previously developed system requirements, it has not established processes and plans to guide this work or addressed test activities even though developing processes and plans, as well as planning test activities early in the life cycle, are recognized best practices for effective requirements development and testing. Finally, although OPM's Executive Steering Committee and Investment Review Board have recently become more active regarding RetireEZ, these bodies did not exercise effective oversight in the past, which has allowed the aforementioned management weaknesses to persist and OPM has not established guidance regarding how these entities are to intervene when corrective actions are needed. Until OPM addresses these weaknesses, many of which GAO and others made recommendations to correct, the agency's retirement modernization initiative remains at risk of failure. Institutionalizing effective management is critical not only for the success of this initiative, but also for that of other modernization efforts within the agency.

section_title: Letter
paragraphs: For the past two decades, the Office of Personnel Management (OPM) has  been attempting to modernize the paper-intensive processes and  antiquated systems it uses to support the retirement of federal employees,  in part because the current process does not provide prompt and complete  benefit payments upon retirement. The agency’s retirement modernization  initiative, also referred to as RetireEZ, was intended to address this  situation and improve the efficiency and effectiveness of the retirement  program, which serves federal employees who are eligible to receive  benefits in the future, employees who are already retired, and their  survivors and beneficiaries.
In January 2008, we reported on challenges that OPM faced in deploying  RetireEZ, noted a number of concerns regarding the agency’s management  of the initiative, and recommended corrective actions. In late February  2008, OPM deployed a version of the modernized retirement system;  however, the agency suspended the system’s operation in May 2008. After  5 months of attempting to address system quality issues, the agency  terminated its contract with the primary modernization vendor in October  2008 and began restructuring the retirement modernization initiative in  November 2008.
At your request, we conducted a follow-up study of OPM’s management of  RetireEZ. Specifically, our objectives were to    assess the status of OPM’s efforts toward planning and implementing the    evaluate the effectiveness of the agency’s management of the  modernization initiative.
To accomplish our objectives, we reviewed relevant program  documentation and interviewed appropriate OPM and contractor officials.  Specifically, to assess the status of the program, we reviewed  documentation such as the retirement modernization program  management plan and supporting documents. We also visited OPM’s  retirement operations facilities in Washington, D.C., and Boyers,  Pennsylvania. To evaluate the effectiveness of the agency’s management of  the modernization initiative, we reviewed OPM’s cost estimation, earned  value management (EVM), requirements management, test management  activities, and oversight. We compared OPM’s plans and activities to best  practices and assessed the agency’s progress toward implementing our  prior recommendations.
We conducted this performance audit from May 2008 through March 2009  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. See appendix II for a more  complete description of our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: OPM’s mission is to ensure that the federal government has an effective  civilian workforce. In this regard, one of the agency’s major human  resources tasks is to manage and administer the retirement program for  federal employees. According to the agency, the program serves federal  employees by providing (1) retirement compensation and (2) tools and  options for retirement planning. OPM’s Center for Retirement and  Insurance Services administers the two defined benefit retirement plans  that provide retirement, disability, and survivor benefits to federal  employees. The first plan, the Civil Service Retirement System (CSRS),  provides retirement benefits for most federal employees hired before 1984.  The second plan, the Federal Employees Retirement System (FERS),  covers most employees hired in or after 1984 and provides benefits that  include Social Security and a defined contribution system.
According to OPM, there are approximately 2.9 million active federal  employees and nearly 2.5 million retired federal employees. The agency’s  March 2008 analysis of federal employment retirement data estimates that  nearly 1 million active federal employees will be eligible to retire and  almost 600,000 will most likely retire by 2016. Figure 1 summarizes the  estimated number of employees eligible and likely to retire.
subsections: section_title: Federal Employee Retirement Application Processing
paragraphs: OPM and employing agencies’ human resources and payroll offices are  responsible for processing federal employees’ retirement applications. The  process begins when an employee submits a paper retirement application  to his or her employer’s human resources office and is completed when  the individual begins receiving regular monthly benefit payments (as  illustrated in fig. 2).
Once an employee submits an application, the employing agency’s human  resources office provides retirement counseling services to the employee  and augments the retirement application with additional paperwork, such  as a separation form that finalizes the date the employee will retire. Then  the agency provides the retirement package to the employee’s payroll  office. After the employee separates for retirement, the payroll office is  responsible for reviewing the documents for correct signatures and  information, making sure that all required forms have been submitted, and  adding any additional paperwork that will be necessary for processing the  retirement package. Once the payroll office has finalized the paperwork,  the retirement package is mailed to OPM to continue the retirement  process. Payroll offices are expected to submit the package to OPM within  30 days of the retiree’s separation date.
Upon receipt of the retirement package, OPM calculates an interim  payment based on information provided by the employing agency. The  interim payments are partial payments that typically provide retirees with  80 percent of the total monthly benefit they will eventually receive. OPM  then starts the process of analyzing the retirement application and  associated paperwork to determine the total monthly benefit amount to  which the retiree is entitled. This process includes collecting additional  information from the employing agency’s human resources and payroll  offices or from the retiree to ensure that all necessary data are available  before calculating benefits. After OPM completes its review and authorizes  payment, the retiree begins receiving 100 percent of the monthly  retirement benefit payments. OPM then stores the paper retirement folder  at the Retirement Operations Center in Boyers, Pennsylvania.
According to the agency’s 2008 performance report, the average  processing time from the date OPM receives the initial application to the  time the retiree receives a full payment is 42 days. According to the Deputy  Associate Director for the Center of Retirement and Insurance Services,  about 200 employees are directly involved in processing the approximately  100,000 retirement applications OPM receives annually. This processing  includes functions such as determining retirement eligibility, inputting  data into benefit calculators, and providing customer service. The agency  uses over 500 different procedures, laws, and regulations, which are  documented on the agency’s internal Web site, to process retirement  applications. For example, the site contains memorandums that outline  new procedures for handling special retirement applications, such as  those for disability or court orders. Further, OPM’s retirement processing  involves the use of over 80 information systems that have approximately  400 interfaces with other internal and external systems. For instance, 26  internal systems interface with the Department of the Treasury to provide,  among other things, information regarding the total amount of benefit  payments to which an employee is entitled.
OPM has stated that the federal employee retirement process currently  does not provide prompt and complete benefit payments upon retirement,  and that customer service expectations for more timely payments are  increasing. The agency also reports that a greater workload is expected  due to an anticipated increase in the number of retirement applications  over the next decade, yet current retirement processing operations are at  full capacity. Further, the agency has identified several factors that limit  its ability to process retirement benefits in an efficient and timely manner.  Specifically, it noted that    current processes are paper-based and manually intensive, resulting in a  higher number of errors and delays in providing benefit payments;  the high costs, limited capabilities, and other problems with the existing  information systems and processes pose increasing risks to the accuracy  of benefit payments;    current manual capabilities restrict customer service;  federal employees have limited access to their retirement records, making  planning for retirement difficult; and    attracting qualified personnel to operate and maintain the antiquated  retirement systems, which have about 3 million lines of custom  programming, is challenging.
section_title: OPM Has a Long History of Retirement Modernization Initiatives
paragraphs: In the late 1980s, OPM recognized the need to automate and modernize its  retirement processing and began retirement modernization initiatives that  have continuously called for automating its antiquated paper-based  processes. The agency’s previously established program management  plans included the objectives of having timely and accurate retirement  benefit payments and more efficient and flexible processes. For example,  the agency’s plans call for processing retirement applications and  providing retirees 100 percent of their monthly benefit payments the day it  is due versus providing interim monthly payments. Its initial  modernization vision called for providing prompt and complete benefit  payments by developing an integrated system and automated processes.  However, the agency has faced significant and long-standing challenges in  doing so.
In early 1987, OPM began a program called the FERS Automated  Processing System (FAPS). However, after 8 years of planning, the agency  decided it needed to reevaluate the program, and the Office of  Management and Budget (OMB) requested that an independent board  conduct a review to identify critical issues impeding progress and  recommend ways to address the issues. The review identified various  management weaknesses, including the lack of an established strategic  plan, cost estimation methodologies, and baseline; improperly defined and  ineffectively managed requirements; and no clear accountability for  decision making and oversight. Accordingly, the board suggested areas  for improvement and recommended terminating the program if immediate  action was not taken. In mid-1996, OPM terminated the program.
In 1997, OPM began planning a second modernization initiative, called the  Retirement Systems Modernization (RSM) program. The agency originally  intended to structure the program as an acquisition of commercially  available hardware and software that would be modified in-house to meet  its needs. From 1997 to 2001, OPM developed plans and analyses and  began developing business and security requirements for the program.  However, in June 2001, it decided to change the direction of the retirement  modernization initiative.
In late 2001, retaining the name RSM, the agency embarked upon its third  initiative to modernize the retirement process and examined the  possibility of privately sourced technologies and tools. To this end, OPM  issued a request for information to obtain private sourcing options and  determined that contracting was a viable alternative that would be cost  efficient, less risky, and more likely to be completed on time and on  budget. In 2006, the agency awarded three contracts for: (1) a  commercially available, defined benefits technology solution (DBTS) to  automate retirement processing; (2) services to convert paper records to  electronic files; and (3) consulting services to support the redesign of its  retirement operations. The contract for DBTS was awarded to Hewitt  Associates, and the additional contracts to support the technology were  awarded to Accenture Ltd. and Northrop Grumman Corporation, as  reflected in table 1.
OPM produced a December 2007 program management plan that, among  other things, described capabilities the agency expected to implement as  outcomes of retirement modernization. Among these capabilities, the  agency expected to implement retirement benefit modeling and planning  tools for active federal employees, a standardized retirement benefit  calculation system, and a consolidated system to support all aspects of  retirement processing.
In February 2008, OPM renamed the program RetireEZ and deployed a  limited initial version of DBTS. As the foundation of the modernization  initiative, DBTS was to be a comprehensive technology solution that  would provide capabilities to substantially automate retirement  processing. This technology was to be provided by the contractor for a  period of 10 years and was intended to provide, among other things, an  integrated database with calculation functionality for retirement  processing. In addition to calculating retirement benefit amounts, DBTS  was intended to provide active and retired federal employees with self- service, Internet-based tools for accessing accounts, updating retirement  records, submitting transactions, monitoring the status of claims, and  forecasting retirement income. The technology was also expected to  enhance customer service by providing OPM and agency personnel with  the capability to access retirement information online. Further, the  technology was expected to be integrated with OPM and federal agency  electronic retirement records and processes. When fully implemented, the  modernized program was expected to serve OPM retirement processing  personnel, federal agency human resources and payroll offices, active  federal employees, retirees, and the beneficiaries of retirees.
According to the agency, in late February 2008, the DBTS was deployed  with limited functionality to 26,000 federal employees serviced by the  General Services Administration’s (GSA) payroll offices. In April 2008,  OPM reported that 13 of the 37 retirement applications received from  GSA’s payroll office had been processed through DBTS with manual  intervention and provided the retirees 100 percent of their monthly  benefits within 30 days from their retirement date. However, a month later,  the agency determined that DBTS had not worked as expected and  suspended system operation. In October 2008, after 5 months of  attempting to address system quality issues, the agency terminated the  contract.
In November 2008, OPM began restructuring the program and reported  that its efforts to modernize retirement processing would continue. Figure  3 illustrates the timeline of retirement modernization initiatives from 1987  to the present.
section_title: Current Retirement Modernization Management
paragraphs: Various entities within OPM are responsible for managing RetireEZ.  Specifically, the management is composed of committees, a program  office, and operational support, as reflected in table 2.
section_title: Prior GAO Reviews of OPM’s Retirement Modernization Initiatives
paragraphs: Since 2005, we have conducted several studies of OPM’s retirement  modernization noting weaknesses in its management of the initiative. In  February of that year, we reported that the agency lacked processes for  retirement modernization acquisition activities, such as determining  requirements, developing acquisition strategies, and implementing a risk  program. Further, the agency had not established effective security  management, change management, and program executive oversight. We  recommended that the Director of OPM ensure that the retirement  modernization program office expeditiously establish processes for  effective oversight of the retirement modernization in the areas of system  acquisition management, information security, organizational change  management, and information technology (IT) investment management. In  response, between 2005 and 2007, the agency initiated steps toward  establishing management processes for retirement modernization and  demonstrated the completion of activities with respect to each of our nine  recommendations.
However, in January 2008, we reported that the agency still needed to  improve its management of the program to ensure a successful outcome  for its modernization efforts. Specifically, we reported that initial test  results had not provided assurance that DBTS would perform as intended,  the testing schedule increased the risk that the agency would not have  sufficient resources or time to ensure that all system components were  tested before deployment, and trends in identifying and resolving system  defects had indicated a growing backlog of problems to be resolved prior  to deployment. Further, we reported that although the agency had  established a risk management process, it had not reliably estimated the  program costs, and its progress reporting was questionable because it did  not reflect the actual state of the program. We recommended that the  Director of OPM address these deficiencies by conducting effective system  tests and resolving urgent and high priority system defects prior to system  deployment, in addition to improving program cost estimation and  progress reporting.
In response to our report, OPM stated that it concurred with our  recommendations and was taking steps to address them. However, in  March 2008, we determined that the agency was moving forward with  system deployment and had not yet implemented its planned actions.  OPM subsequently affirmed its agreement with our recommendations in  April 2008 and reported that it had implemented or was in the process of  implementing each recommendation. As of March 2009, however, these  recommendations still had not been fully addressed.
section_title: Few Planned Capabilities Have Been Implemented, and Future Efforts Are Not Guided by a Complete Plan
paragraphs: OPM remains far from fully implementing the retirement modernization  capabilities described when it documented its plans for RetireEZ in 2007.  The agency only partially implemented two of eight capabilities that it  identified to modernize retirement processing. The remaining six  capabilities, which were to be delivered through the DBTS contract, have  not been implemented, and OPM’s plans to continue implementing them  are uncertain. While the agency has taken steps to restructure the  RetireEZ program without the DBTS contract, it has not developed a plan  to guide its future modernization efforts.
subsections: section_title: Few Retirement Modernization Capabilities Have Been Implemented
paragraphs: OPM’s retirement modernization plans from 2007 described eight  capabilities that were to be implemented to achieve modernized processes  and systems. As of late March 2009, the agency had partially implemented  two of these capabilities while the remaining six had not been  implemented (see table 3).
Specifically, it had achieved partial implementation of an integrated  database of retirement information that was intended to be accessible to  OPM and agency retirement processing personnel. In this regard, the  agency implemented a new database, populated with images of retirement  information, which is accessible to OPM retirement processing personnel  online. This database contains over 8 million files which, according to  agency officials, represent approximately 80 to 90 percent of the available  retirement information for all active federal employees. However, the  capability for the information in the database to be integrated with OPM’s  legacy retirement processing systems and to be accessible to other agency  retirement processing personnel has not yet been implemented.
OPM has also partially implemented enhanced customer service  capabilities. Specifically, the agency acquired a new telephone  infrastructure (i.e., additional lines) and hired additional customer service  representatives to reduce wait times and abandonment rates. However,  the agency has not yet developed the capabilities for OPM retirement  processing personnel to provide enhanced customer support to active and  retired federal employees through online account access and management.
Moreover, six other capabilities have not been implemented—and plans to  implement them are uncertain—because they were to be delivered  through the now-terminated DBTS contract, which had been expected to  provide a single system that would automate the processing of retirement  applications, calculations, and benefit payments.
Among the capabilities not implemented was one for other agencies’  automated submissions of retirement information to OPM that could be  used to process retirement applications. While OPM began developing this  capability by establishing interfaces with other agencies as part of its  effort to implement DBTS, it discontinued the use of the interfaces for  processing retirement applications when the DBTS contract was  terminated. Thus, federal agencies that submit retirement information to  OPM continue to provide paper packages and information when  employees are ready to retire.
Further, OPM has not implemented a planned capability for active and  retired federal employees to access online retirement information through  self-service tools. While the agency provided demonstrations of DBTS in  April 2008 that showed the ability for employees to access information  online, including applying for retirement and modeling future retirement  benefits, this capability was to be provided by DBTS, and thus, no longer  exists. The contractor had also been expected to deliver a consolidated  system to support all aspects of retirement processing and an electronic  case management system to support retirement processing. In the absence  of these capabilities, the agency continues to manage cases through paper  tracking and stand-alone systems. Additionally, OPM and federal agencies  continue to rely on nonstandardized systems to determine and calculate  retirement benefits, and federal retirees currently have only limited online,  self-service tools.
section_title: OPM Does Not Have a Complete Plan for the Future of the RetireEZ Program
paragraphs: Program management principles and best practices emphasize the  importance of using a program management plan that, among other things,  establishes a complete description that ties together all program  activities. An effective plan includes a description of the program’s scope,  implementation strategy, lines of responsibility and authority,  management processes, and a schedule. Such a plan incorporates all the  critical areas of system development and is to be used as a means of  determining what needs to be done, by whom, and when. Furthermore,  establishing results-oriented (i.e., objective, quantifiable, and measurable)  goals and measures, that can be included in a plan, provides stakeholders  with the information they need to effectively oversee and manage  programs.
A plan for the future of the RetireEZ program has not been completed. In  November 2008, OPM began restructuring the program and reported it was  continuing toward retirement modernization without the DBTS contract.  The restructuring efforts have resulted in a wide variety of documentation,  including multiple descriptions of the program in formal agency reports,  budget documentation, agency briefing slides, and related documents.
For example, OPM’s November Fiscal Year 2008 Agency Financial Report  described what the RetireEZ program is expected to achieve (e.g., provide  retirement modeling tools for federal employees) once implemented. The  agency’s Annual Performance Report, dated January 2009, outlined that  the new vision for the restructured program is “to support benefit planning  and management throughout a participant’s lifecycle through an enhanced  federal retirement program.” The agency also presented information to  OMB that identified eight fiscal year 2009 program initiatives, as listed in  table 4.
The agency has developed a variety of informal program documents and  briefing slides that describe retirement modernization activities. For  instance, one document prepared by the program office describes a five- phased approach that is intended to replace its previous DBTS-reliant  strategy. The approach includes the following activities: (1) collecting  electronic retirement information, (2) automating the retirement  application process, (3) integrating retirement information, (4) developing  retirement calculation technologies and tools, and (5) improving post- retirement processes through a technology solution. In addition, briefing  slides also prepared by the program office outline a schedule for efforts to  identify new technologies to support retirement modernization by drafting  a request for information, which OPM expects to issue in late April 2009.
Regardless, OPM’s various reports and documents describing its planned  retirement modernization activities do not provide a complete plan for its  restructured program. Specifically, although agency documents describe  program implementation activities, they do not include a definition of the  program, its scope, lines of responsibility and authority, management  processes, and schedule. Also, the modernization program documentation  does not describe results-oriented (i.e., objective, quantifiable, and  measurable) performance goals and measures. According to the RetireEZ  program manager, the agency is developing plans, but they will not be  ready for release until the new OPM director has approved them, which is  expected to occur in April 2009. Until the agency completes and uses a  plan that includes all of the above elements to guide its efforts, it will not  be properly positioned to obtain agreement with relevant stakeholders  (e.g., Congress, OMB, federal agencies, and OPM senior executives) for its  restructured retirement modernization initiative. Further, the agency will  also not have a key mechanism that it needs to help ensure successful  implementation of future modernization efforts.
section_title: OPM Is Not Positioned to Effectively Manage Its Retirement Modernization Initiative
paragraphs: OPM has significant management weaknesses in five areas that are  important to the success of its retirement modernization program: cost  estimating, EVM, requirements management, testing, and program  oversight. For example, the agency has not performed key steps, including  the development of a cost estimating plan or completion of a work  breakdown structure, both of which are necessary to develop a reliable  program cost estimate. Also, OPM has not established and validated a  performance measurement baseline, which is essential for reliable EVM.  Further, although OPM is revising its previously developed system  requirements, it has not established processes and plans to guide this  work. Nor has the agency addressed test activities, even though  developing processes and planning test activities early in the life cycle are  recognized best practices for effective testing. Furthermore, although  OPM’s Executive Steering Committee and Investment Review Board have  recently become more active regarding RetireEZ, these bodies did not  exercise effective oversight in the past, which has allowed the  aforementioned management weaknesses to persist. Notably, OPM has not  established guidance regarding how these entities are to engage with the  program when corrective actions are needed. Until OPM addresses these  weaknesses, many of which we and others made recommendations to  correct, the agency’s retirement modernization initiative remains at risk of  failure.
subsections: section_title: OPM Has Yet to Complete Key Steps in Developing a Reliable Cost Estimate for Retirement Modernization
paragraphs: The establishment of a reliable cost estimate is a necessary element for  informed investment decision making, realistic budget formulation, and  meaningful progress measurement. A cost estimate is the summation of  individual program cost elements that have been developed by using  established methods and validated data to estimate future costs.  According to federal policy, programs must maintain current and well- documented estimates of program costs, and these estimates must span  the full expected life of the program. Our Cost Estimating and  Assessment Guide includes best practices that agencies can use for  developing and managing program cost estimates that are comprehensive,  well-documented, accurate, and credible, and provide management with a  sound basis for establishing a baseline to measure program performance  and formulate budgets. This guide identifies a cost estimating process  that includes initial steps such as defining the estimate’s purpose (i.e., its  intended use, scope, and level of detail); developing the estimating plan  (i.e., the estimating approach, team, and timeline); defining the program  (e.g., technical baseline description); and determining the estimating  structure (e.g., work breakdown structure). According to best practices,  these initial steps in the cost estimating process are of the utmost  importance, and should be fully completed in order for the estimate to be  considered valid and reliable.
OPM officials stated that they intend to complete a modernization program  cost estimate by July 2009. However, the agency has not yet fully  completed initial steps for developing the new estimate. Specifically, the  agency has not yet fully defined the estimate’s purpose, developed the  estimating plan, defined program characteristics in a technical baseline  description, or determined the estimating structure.
With respect to the estimate’s purpose, agency officials stated that the  estimate will inform the budget justification of RetireEZ for fiscal year  2011 and beyond. However, the agency has not clearly defined the scope  or level of detail of the estimate.
Regarding the estimating plan, agency officials stated that they have  created a timeline to complete the estimate by July 2009. However, the  agency has not documented an estimating plan that includes the approach  and resources required to complete the estimate in the time period  identified.
With respect to the technical baseline description, agency officials stated  that they are in the advanced stages of developing a request for  information and a concept of operations that will serve as the basis for a  technical baseline description. These documents are expected to be  reviewed for approval in April 2009.
Regarding the estimating structure, the agency has developed a work  breakdown structure that identifies elements of the program to be  estimated. However, the agency has not yet developed a work breakdown  structure dictionary that clearly defines each element.
Weaknesses in the reliability of OPM’s retirement modernization cost  estimate have been long-standing. We first reported on the agency’s lack of  a reliable cost estimate in January 2008 when we noted that critical  activities, including documentation of a technical baseline description, had  not been performed, and we recommended that the agency revise the  estimate. Although OPM agreed to produce a reliable program cost  estimate, the agency has not yet done so. Until OPM fully completes each  of the steps, the agency increases the risk that it will produce an unreliable  estimate and will not have a sound basis for measuring program  performance and formulating retirement modernization program budgets.
section_title: OPM Is Not Prepared to Perform Reliable Earned Value Management for Retirement Modernization
paragraphs: OMB and OPM policies require major IT programs to use EVM to measure  and report program progress. EVM is a tool for measuring program  progress by comparing the value of work accomplished with the amount  of work expected to be accomplished. Such a comparison permits actual  performance to be evaluated, based on variances from the planned cost  and schedule, and future performance to be forecasted. Identification of  significant variances and analysis of their causes helps program managers  determine the need for corrective actions. Before EVM analysis can be  reliably performed, developing a credible cost estimate is necessary. In  addition to developing a cost estimate, an integrated baseline review must  be conducted to validate a performance measurement baseline and attain  agreement of program stakeholders (e.g., agency and contractor officials)  before reliable EVM reporting can begin. The establishment of a baseline  depends on the completion of a work breakdown structure, an integrated  master schedule, and budgets for planned work.
Although the agency plans to begin reporting on the restructured  program’s progress using EVM in April 2009, the agency is not yet  prepared to do so because initial steps have not been completed and are  dependent on decisions about the program that have not been made.  Specifically,  the agency has not yet developed a reliable cost estimate for the program;  such an estimate, which is critical for establishing reliable EVM, is not  expected to be complete until July 2009;  the agency does not plan to conduct an integrated baseline review to  establish a reliable performance measurement baseline before beginning  EVM reporting; and  the work breakdown structure and integrated master schedule that agency  officials report they have developed may not accurately reflect the full  scope and schedule because key program documentation, such as the  concept of operations, has not been completed.
This situation resembles the state of affairs that existed in January 2008,  when we reported that OPM’s EVM was unreliable because an integrated  baseline review had not been conducted to validate the program baseline.
At that time we recommended, among other things, that the agency  establish a basis for effective use of EVM by validating a program  performance measurement baseline through a program-level integrated  baseline review. Although the agency stated that it agreed, it did not  address this recommendation.
Until the agency has developed a reliable cost estimate, performed an  integrated baseline review, and validated a performance measurement  baseline that reflect its program restructuring, the agency is not prepared  to perform reliable EVM. Engaging in EVM reporting without first  performing these fundamental steps could again render the agency’s  assessment unreliable.
section_title: OPM Has Not Established Processes Needed to Effectively Develop and Manage Retirement Modernization Requirements
paragraphs: Well-defined and managed requirements are a cornerstone of effective  system development and acquisition. According to recognized guidance,  disciplined processes for developing and managing requirements can help  reduce the risks of developing a system that does not meet user and  operational needs. Such processes include (1) developing detailed  requirements that have been derived from the organization’s concept of  operations and are complete and sufficiently detailed to guide system  development and (2) establishing policies and plans, including defining  roles and responsibilities, for managing changes to requirements and  maintaining bidirectional requirements traceability.
OPM’s retirement modernization requirements processes include some,  but not all, of the elements needed to effectively develop and manage  requirements. The agency began an effort to better develop its retirement  modernization requirements in November 2008. This effort was in  response to the agency’s recognition that its over 1,400 requirements  lacked sufficient detail, were incomplete, and required further  development. The agency intends to complete this requirements  development effort in April 2009. However, the requirements will not be  derived from OPM’s concept of operations because the agency is revising  the concept of operations expected to be completed by April 2009, to  reflect the program restructuring. Further, OPM documentation indicates  that the agency has not yet determined the level of detail to which  requirements should be developed. Additionally, agency officials stated  that OPM is developing a requirements development process for  retirement modernization. With respect to requirements management,  OPM developed an organizational charter that outlined roles and  responsibilities for supporting efforts to manage requirements. However,  the agency does not yet have a requirements management plan.
OPM’s prior experience with DBTS illustrates the importance of effective  requirements development and management. According to RetireEZ  program officials, insufficiently detailed requirements, poorly controlled  requirements changes, and inadequate requirements traceability were  factors that contributed to DBTS not performing as expected. Moreover,  these requirements development and management weaknesses were  identified, and recommendations for improvement were made by OPM’s  independent verification and validation contractor before DBTS  deployment. However, the agency has not yet corrected these weaknesses.
Until OPM fully establishes requirements development and management  processes, the agency increases the risk that it will   (1) identify requirements that are neither complete nor sufficiently  detailed and (2) not effectively manage requirements changes or maintain  bidirectional traceability, thus further increasing agency risk that it will  produce a system that does not meet user and operational needs.
section_title: OPM Is Not Positioned to Effectively Manage Retirement Modernization Testing
paragraphs: Effective testing is an essential component of any program that includes  developing systems. Generally, the purpose of testing is to identify defects  or problems in meeting defined system requirements and satisfying user  needs. To be effectively managed, testing should be planned and  conducted in a structured and disciplined fashion that adheres to  recognized guidance and is coordinated with the requirements  development process. Beginning the test planning process in the early  stages of a program life cycle can reduce rework later in the program.  Early test planning in coordination with requirements development can  provide major benefits. For example, planning for test activities during the  development of requirements may reduce the number of defects identified  later and the costs related to requirements rework or change requests.  Further, planning test activities early in a program’s life cycle can inform  requests for proposals and help communicate testing expectations to  potential vendors.
OPM has not begun to plan test activities in coordination with developing  its requirements for the RetireEZ program. According to OPM officials, the  agency intends to begin its test planning by revising the previously  developed DBTS test plans after requirements have been developed.  However, the agency has not yet added test planning to its project  schedule.
Early test planning is especially important to avoid repeating the agency’s  experience during DBTS testing when it identified more defects than it  could resolve before system deployment. In January 2008, we reported  that an unexpectedly high number of defects were identified during  testing; yet, the deployment schedule had increased the risk of not  resolving all defects that needed to be corrected before deploying DBTS.  According to the RetireEZ program officials, the failure to fully address  these defects contributed to the limited number of federal employees who  were successfully processed by the system when it was deployed in  February 2008.
If it does not plan test activities early in the life cycle of RetireEZ, OPM  increases the risk that it will again deploy a system that does not satisfy  user expectations and meet requirements (i.e., accurately calculate  retirement benefits) because of its potential inability to address a higher  number of defects than expected. Moreover, criteria used to develop  requests for proposals and communicate testing expectations to potential  vendors could be better informed if the agency plans RetireEZ test  activities early in the life cycle.
section_title: OPM Is Not Providing Effective Oversight for Retirement Modernization
paragraphs: GAO and OMB guidance calls for agencies to ensure effective oversight of  IT projects throughout all life-cycle phases. Critical to effective oversight  are investment management boards made up of key executives who  regularly track the progress of IT projects such as system acquisitions or  modernizations. These boards should maintain adequate oversight and  track project performance and progress toward predefined cost and  schedule goals, as well as monitor project benefits and exposure to risk.  Another element of effective IT oversight is employing early warning  systems that enable management boards to take corrective actions at the  first sign of cost, schedule, and performance slippages.
OPM’s Investment Review Board was established to ensure that major  investments are on track by reviewing their progress and determining  appropriate actions when investments encounter challenges. Despite  meeting regularly and being provided with information that indicated  problems with the retirement modernization, the board did not ensure that  the investment was on track, nor did it determine appropriate actions for  course correction when needed. For example, from January 2007 to  August 2008 the board met and was presented with reports that described  problems the retirement modernization program was facing, such as the  lack of an integrated master schedule and earned value data that did not  reflect the “reality or current status” of the program. However, meeting  minutes indicate that no discussion or action was taken to address these  problems. According to a member of the board, OPM guidance regarding  how the board is to communicate recommendations and corrective  actions when needed for the investments it is responsible for overseeing  has not been established.
In addition, OPM established an Executive Steering Committee to oversee  retirement modernization. According to its charter, the committee is to  provide strategic direction, oversight, and issue resolution to ensure that  the program maintains alignment with the mission, goals, and objectives of  the agency and is supported with required resources and expertise.  However, the committee was inactive for most of 2008 and, consequently,  did not exercise oversight of the program during a crucial period in its  development. For example, from January 2008 until October 2008, the  committee discontinued its formal meetings, and as a result, it was not  involved in key program decisions, including the deployment of DBTS.  Further, a member of the committee noted that OPM guidance for making  recommendations and taking corrective actions also has not been  provided.
The ineffectiveness of the board and the inactivity of the committee  allowed program management weaknesses in the areas of cost estimation,  EVM, requirements management, and testing to persist and raise concerns  about OPM’s ability to provide meaningful oversight as the agency  proceeds with its retirement modernization. Without fully functioning  oversight bodies, OPM cannot monitor modernization activities and make  the course corrections that effective boards and committees are intended  to provide.
section_title: Conclusions
paragraphs: OPM’s retirement modernization initiative is in transition from a program  that was highly dependent on the success of a major contract that no  longer exists, to a restructured program that has yet to be fully defined.  Although the agency has been able to partially implement a database of  retirement information and improvements to customer service, it remains  far from implementing six other key capabilities. Recognizing that much  work remains, OPM has undertaken steps to restructure the retirement  modernization program, but it has not yet produced a complete  description of its planned program, including fundamental information  about the program’s scope, implementation strategy, lines of responsibility  and authority, management processes, and schedule. Further, OPM’s  retirement modernization program restructuring does not yet include  definitions of results-oriented goals and measures against which program  performance can be objectively and quantitatively assessed.
In addition, OPM has not overcome managerial shortcomings in key areas  of program management, including areas that we have previously  reported. Specifically, the agency is not yet positioned to develop a  reliable program cost estimate or perform reliable EVM, both of which are  critical to effective program planning and oversight. Nor has OPM  overcome weaknesses in its management of system testing and defects,  two activities that proved problematic as the agency was preparing to  deploy the RetireEZ system that subsequently was terminated. Adding to  these long-standing concerns are weaknesses in OPM’s process to  effectively develop and manage requirements for whatever system or  service it intends to acquire or develop. Finally, these weaknesses have  been allowed to persist by entities within the agency that were ineffective  in overseeing the retirement modernization program.
As a consequence, the agency is faced with significant challenges on two  fronts: defining and transitioning to its restructured program, and  addressing new and previously identified managerial weaknesses. Until  OPM addresses these weaknesses, many of which were previously  identified by GAO and others, the agency’s retirement modernization  initiative remains at risk of failure. Institutionalizing effective planning and  management is critical not only for the success of this initiative, but also  for that of other modernization efforts within the agency.
section_title: Recommendations for Executive Action
paragraphs: To improve OPM’s effort toward planning and implementing its retirement  modernization program by addressing management weaknesses, we  recommend that the Director of the Office of Personnel Management  provide immediate attention to ensure the following six actions are taken:    Develop a complete plan for the restructured program that defines the  scope, implementation strategy, lines of responsibility and authority,  management processes, and schedule. Further, the plan should establish  results-oriented (i.e., objective, quantifiable, and measurable) goals and  associated performance measures for the program.
Develop a reliable cost estimate by following the best practice steps  outlined in our Cost Estimating and Assessment Guide, including  definition of the estimate’s purpose, development of an estimating plan,  definition of the program’s characteristics, and determination of the  estimating structure.
Establish a basis for reliable EVM, when appropriate, by developing a  reliable program cost estimate, performing an integrated baseline review,  and validating a performance measurement baseline that reflects the  program restructuring.
Develop a requirements management plan and execute processes  described in the plan to develop retirement modernization requirements in  accordance with recognized guidance.
Begin RetireEZ test planning activities early in the life cycle.
Develop policies and procedures that would establish meaningful program  oversight and require appropriate action to address management  deficiencies.
section_title: Agency Comments and Our Evaluation
paragraphs: The Director of the Office of Personnel Management provided written  comments on a draft of this report. (The comments are reproduced in app.  II.) In the comments, OPM agreed with our recommendations and stated  that it had begun to address them. To this end, the Director stated that the  agency had, among other actions, begun revising its retirement  modernization plans, developing a new program cost estimate, planning  for accurate EVM reporting, incorporating recognized guidance in  requirements management planning, and planning test activities during  requirements development. If the recommendations are properly  implemented, they should better position OPM to effectively manage its  retirement modernization initiative. The agency also provided comments  on the draft report regarding our description of the federal retirement  application process, as well as our characterizations of OPM’s EVM and  requirements management capabilities vis-à-vis the retirement  modernization program. In each of these instances, we made revisions as  appropriate.
We are sending copies of this report to the Director of the Office of  Personnel Management, appropriate congressional committees, and other  interested parties. In addition, the report is available at no charge on the  GAO Web site at http://www.gao.gov.
If you or your staffs have questions about this report, please contact me at  (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: As requested, the objectives of our study were to (1) assess the status of  the Office of Personnel Management’s (OPM) efforts toward planning and  implementing the RetireEZ program and (2) evaluate the effectiveness of  the agency’s management of the modernization initiative.
To assess the status of OPM’s efforts toward planning and implementing  the RetireEZ program, we  reviewed and analyzed program documentation, including program  management plans, briefing slides, and project status documentation, to  identify planned retirement modernization capabilities and determine to  what extent these capabilities have been implemented;    evaluated the agency’s documentation about restructuring the program  and analyzed the extent to which the documentation describes current and  planned RetireEZ program activities;  identified and evaluated the agency’s program goals and measures and  compared them to relevant guidance to determine the extent to which the  goals and measures are described in results-oriented terms;    supplemented agency program documentation and our analyses by  interviewing agency and contractor officials, including the OPM Director,  Chief Information Officer, Chief Financial Officer, Director of  Modernization, Associate Director for Human Resources Products and  Services Division, and executives from Hewitt Associates and Northrop  Grumman Corporation; and    observed retirement operations and ongoing modernization activities at  OPM and contractor facilities in Washington, D.C.; Boyers, Pennsylvania;  and Herndon, Virginia.
To determine the effectiveness of OPM’s management of the retirement  modernization initiative, we evaluated the agency’s management of  program cost estimating, earned value management (EVM), requirements,  test planning, and oversight and compared the agency’s work in each area  with recognized best practices and guidance. Specifically,  to evaluate whether OPM effectively developed a reliable program cost  estimate, we analyzed the agency’s program documentation and  determined to what extent the agency had completed key activities  described in our Cost Estimating and Assessment Guide;  to assess OPM’s implementation of EVM, we reviewed program progress  reporting documentation and compared the agency’s plans for restarting  its EVM-based progress reporting against relevant guidance, including our  Cost Estimating and Assessment Guide;  regarding requirements management, we evaluated OPM’s processes for  developing and managing retirement systems modernization requirements  and compared the effectiveness of those processes against recognized  guidance;  to determine the effectiveness of the agency’s test planning for the  retirement modernization, we reviewed program activities and test plans  against best practices and evaluated the extent to which the agency has  begun planning for these activities; and    we reviewed and analyzed documentation from program oversight entities  and evaluated the extent to which these entities took actions toward  ensuring the RetireEZ program was being effectively overseen.
We also evaluated OPM’s progress toward implementing our open  recommendations and interviewed OPM and contractor officials as noted.
We conducted this performance audit at OPM headquarters in Washington,  D.C., the Retirement Operations Center for OPM in Boyers, Pennsylvania,  and contractor facilities in Herndon, Virginia, from May 2008 through April  2009, in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Office of Personnel Management
section_title: Appendix III: GAO Staff Contact and Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, key contributions to this report  were made by Mark T. Bird, Assistant Director; Barbara S. Collier; Neil J.  Doherty; David A. Hong; Thomas J. Johnson; Rebecca E. LaPaze; Lee A.  McCracken; Teresa M. Neven; Melissa K. Schermerhorn; Donald A. Sebers;  and John P. Smith.