Information Technology: U.S. Postal Service Needs to Strengthen System Acquisition and Management Capabilities to Improve Its Intelligent Mail Full Service Program
Released: 2009-11-30T00:00:00

Letter

Conclusions

The current schedule for the full implementation of the Full Service program has been delayed by almost 10 months, and key functionality that was originally intended to be delivered in the program has been deferred indefinitely. In addition, the life-cycle cost estimate that program officials prepared does not capture all the costs associated with the acquisition and implementation of the program. As a result, program officials lack an accurate total cost estimate. Moreover, the first deployed release is experiencing performance issues.

While the Full Service program has implemented initial acquisition management activities, it does not have the full set of capabilities needed to fully manage the acquisition. A key cause of the program’s acquisition management weaknesses in the areas of project planning, risk management, and product integration is that USPS organizational policies do not set forth sufficient requirements for establishing effective practices in these areas. Weaknesses exist in the program monitoring and control area because the program management contract creates a conflict of interest by requiring that the contractor assess the quality of its own deliverables and oversee the program’s schedule, issues, and risks.

While organizational policies exist for requirements development and management, weaknesses exist in this area, in part, because USPS decided not to follow USPS’s organizational policies for system acquisition and instead followed a truncated program management approach in an effort to deliver the system in a compressed time frame. Without these processes in place, USPS increases the risk that this project will continue to encounter problems in meeting its performance, schedule, and cost objectives.

Given that release 2 is expected to be implemented by the end of November 2009 and decisions about future releases need to be made, having the key elements of a sound acquisition management capability in place will be crucial to the program’s success in meeting its goal.

Recommendations for Executive Action

To ensure that USPS adequately manages the acquisition of the Intelligent Mail® Full Service program, we recommend that the Postmaster General take seven actions. Specifically, we recommend that the Postmaster General direct the Chief Information Officer and Senior Vice President of Intelligent Mail and Address Quality to: Develop a comprehensive cost estimate to include both government and contractor costs over the program’s full life cycle, from the inception of the program through design, development, deployment, and operation and maintenance to retirement.

Complete an overall program plan for the entire Full Service program, including an overview of the program’s scope of all releases, deliverables and functionality within these releases, plans to phase out the approximately 30 barcodes currently being utilized, assumptions and constraints, roles and responsibilities, staffing and training plans, and the strategy for maintaining the plan.

Reconsider the current contract arrangement to avoid having the contractor evaluate its own performance.

Define the core set of requirements for the entire program and use them as a basis for developing a reliable cost estimate.

Develop a risk management process that enables the program officials to develop an adequate risk management plan that fully addresses the scope of their risk management efforts; ensures that a comprehensive list of risks and complete mitigation plans are identified and tracked; and includes milestones, mitigating actions, thresholds, and resources for significant risks.

Develop and maintain a systems integration plan for release 2 and beyond.

We are also recommending that the Postmaster General direct USPS’s Chief Information Officer to include in USPS’s Technical Solution Life Cycle policy guidance for programs to develop (1) complete program plans that define overall budget and schedule, key deliverables and milestones, assumptions and constraints, description and assignment of roles and responsibilities, staffing and training plans, and an approach for maintaining these plans; (2) specific requirements for programs to establish a robust risk management process that identifies potential problems before they occur, such as requiring programs to develop a risk management plan; and (3) system integration plan that include all systems to be integrated with the system, roles and responsibilities for all relevant participants, the sequence and schedule for every integration step, and how integration problems are to be documented and resolved.

Agency Comments and Our Evaluation

We obtained written comments on a draft of this report from the USPS Senior Vice President of Intelligent Mail and Address Quality, which are reprinted in appendix II. USPS agreed with three of our recommendations, disagreed with three, and did not comment on one.

Specifically, USPS agreed that (1) the current contract arrangement should be reconsidered to avoid having the contractor evaluate its own performance, (2) a comprehensive risk management process should be developed, and (3) a system integration plan for release 2 and beyond should be developed and maintained. The agency further stated that it has and will continue to enable these capabilities. In previously commenting on our briefing slides, USPS disagreed with aspects of our findings on these issues or provided additional information that we incorporated as appropriate. USPS’s subsequent written comments on this draft report, which recognize the need to implement these recommendations, provide greater assurance of program success.

The Senior Vice President stated that the disagreement with three of our recommendations may be the result of our use of the 2003 Intelligent Mail® strategy document to measure the program’s performance. However, we reviewed and analyzed many documents to form the basis of our findings and conclusions on the program’s performance. The 2003 strategy was just one of the many documents we used, as it represented the original baseline and justification for the program. To report on the progress of the program since its inception, we measured the program against original plans, while acknowledging that USPS has made multiple modifications to the implementation dates.

USPS also stated that we relied on the 2003 strategy to determine delays to the program. This comment is inaccurate. As we stated in this report, we relied on the January 2008 Intelligent Mail® Advance Notice of Proposed Rulemaking in the Federal Register to identify the originally proposed implementation time frame. We also reported the subsequent revisions that USPS made to the program’s implementation schedule.

The Senior Vice President disagreed with our recommendation to develop a comprehensive cost estimate. He stated such an activity would consume a significant amount of funding, time, and resources, while providing little or no value. However, as stated in our GAO Cost Estimating and Assessment Guide, developing a realistic cost estimate is essential because it enables program officials to evaluate resource requirements at key decision points, develop performance measurement baselines, and establish effective resource allocations. Additionally, cost estimates should be comprehensive and should include both government and contractor costs throughout the program’s full life cycle, from the inception of the program through design, development, deployment, and operation and maintenance to retirement. While we acknowledge that preparing a realistic cost estimate may require some amount of effort, we believe the benefits of having an accurate total cost estimate for the entire program to make better informed resource allocation decisions, clearly merit its completion.

With regard to our recommendation to complete an overall program plan for the entire Full Service program, the Senior Vice President stated that, while USPS plans to start updating the Intelligent Mail® strategy on an annual basis, it plans to remain focused on its clearly defined actions for the current releases, rather than planning for future releases. However, industry best practices specifically state that a project plan is the essential document used to manage and control the execution of a project. In order for the project plan to be an effective and useful document, it should consider all phases of the project’s life cycle. Program officials should also ensure that all plans affecting the project are consistent with the overall project plan to ensure that all releases and associated functionality seamlessly fit together. As we state in our report, without such a plan that describes the full scope of the program, including how many releases are envisioned, USPS lacks an overarching approach for incorporating future releases into the program. Additionally, without this information, USPS may not be able to ensure the program is accomplishing its complete set of goals within the specified cost and schedule objectives.

Regarding our recommendation to define a core set of requirements for the entire program and use them to develop reliable cost estimates, the Senior Vice President stated that the program must remain dynamic and that any attempt to define the entire program and its associated cost is a waste of funding and resources. We are not recommending that USPS define all detailed system-level requirements at the outset of the program; rather, we are recommending that USPS develop a roadmap of the program’s high-level requirements. As we state in this report, without a core set of high-level requirements, it will be difficult for USPS to focus appropriately on the next release and to hold itself accountable for delivering a system that meets USPS’s and mailers’ needs. Defining these requirements is especially important given the functionality that is being deferred in the first two releases.

USPS program officials did not state whether they agreed or disagreed with our recommendation that USPS include in its Technical Solution Life Cycle policy guidance for programs to develop (1) complete program plans, (2) specific requirements for programs to establish a robust risk management process, and (3) a system integration plan.

As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies of this report to interested congressional committees and other interested parties. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

Should you or your offices have questions on matters discussed in this report, please contact me at (202) 512-9286 or at pownerd@gao.gov. Contact points for our Offices on Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Briefing to the House Oversight and Government Reform Subcommittee Staff

Information Technology: U.S. Postal Service Needs to Strengthen System Acquisition and Management Capabilities to Improve Its Intelligent Mail® Full Service Program

September 9, 2009

The U.S. Postal Service (USPS) relies heavily on information technology (IT) to support its mission of providing prompt, reliable, and efficient mail service to all areas of the country. Starting in May 2009, as part of a program referred to as the Intelligent Mail® program, USPS began to encourage commercial mailers to use new standardized barcodes which are intended to make it easier to track and provide information about the mail’s progress as it flows through the mail stream. According to USPS officials, this information is important to their efforts to improve efficiency and reduce costs.

The Intelligent Mail® program encompasses numerous programs, including a major initiative known as the Full Service program. This initiative is intended to build a system that improves the visibility into end-to-end mail processing operations through the use of new barcodes, gather more comprehensive and detailed service performance information and measure it against established performance standards, and create efficiencies by streamlining and automating certain aspects of the process USPS uses to verify mail from commercial mailers.

Commercil miler inclde business, orgniztion, nd other prtie thend nd rely on mil to mintin contct with their ustomer. The commercil miler o encompassil preprer, inclding printer nd business thend or receive mil on ehlf of third prty. A of 2008, thee miler cconted for 86 percent of ll mil processed y USPS.

USPS is planning to implement the program in multiple software releases—thus far it has committed to implementing two releases: the first one was deployed in May 2009 and the other is planned to be implemented by November 2009. Program officials have recently stated that they also plan to have future releases; however, they have not made any commitments to do so or obtained funding approval.

USPS says the Full Service program is one of the most complex programs it has undertaken—it will involve the integration of approximately 30 different systems and is intended to benefit both commercial mailers and USPS.

As agreed, our objectives were to determine the current status and plans for the Intelligent Mail® Full Service program and if the Postal Service has capabilities in place to successfully acquire and manage the Intelligent Mail® Full Service program.

For our first objective, we analyzed system documentation, including plans, status reports, meeting minutes, cost estimates, schedule estimates, reports on program management reviews, test plans, and other acquisition-related documents. We also compared the cost and schedule estimates to actual cost and schedule information. In addition, we compared contract deliverables to the actual milestones and deliverables achieved. Finally, we interviewed Postal Service officials and reviewed our previous reports and Inspector General reports to determine the program’s status and plans.

5 Objectives, Scope, and Methodology For our second objective, we identified widely recognized industry standards for good acquisition and development practices, including processes defined in the Software Engineering Institute’s Capability Maturity Model® Integration for Acquisition (CMMI- ACQ) and for Development (CMMI-DEV). From this guidance we identified the following process areas as being the most relevant to our review: (1) project planning, (2) project monitoring and control, (3) requirements development and management, (4) risk management, and (5) product integration. We compared USPS documentation, such as organizational policies, contract information, status reports, meeting minutes, requirements for the program, process documentation, and risk information to SEI’s guidance on sound IT systems acquisition and management practices in the five process areas. We also interviewed Postal Service officials about these key process areas to help us understand whether the agency has the capabilities in place to successfully acquire and manage the program. rnegie Mellon Softwre Engineering Intitte, Cability Mrity Model® Integrtion for Acqition (CMMI-ACQ), Verion 1.2 (Novemer 2007) nd Crnegie Mellon Softwre Engineering Intitte, Cability Mrity Model® Integrtion for Development (CMMI-DEV) Verion 1.2 (Aust 2006).

6 Objectives, Scope, and Methodology We conducted this performance audit from February 2009 to August 2009 at United States Postal Service headquarters in Washington, D.C., in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. 7 Although USPS officials originally intended to deliver the entire Full Service program by January 2009, they currently plan to deliver the program in multiple releases—the first of two planned releases of the Full Service program was deployed on May 18, 2009. The second release is expected to be implemented by the end of November 2009. Additionally, key functionality that was originally intended to be delivered in these two planned releases has been deferred, including automating aspects of the mail acceptance process. Program officials have recently stated that they plan to have future releases to incorporate the deferred functionality; however, they have not made any commitments to do so or obtained funding approval. Program officials estimate that the life cycle cost of the program is $116.4 million, of which $65.9 million has been spent as of June 3, 2009. However, the life cycle cost estimate that program officials prepared does not capture all the costs associated with the acquisition and implementation of the program, such as costs to integrate approximately 30 systems with the Full Service program. Moreover, the first deployed release is currently experiencing operational problems (e.g., applying inconsistent charges for certain mail pieces). Therefore, program officials are developing patches to resolve these issues and to implement system enhancements. 8 While the Full Service program has implemented initial acquisition management activities, it does not have the full set of capabilities it needs to fully manage the acquisition.

A key cause of the program’s immature management approach in the areas of project planning, risk management, and product integration is that USPS organizational policies do not set forth sufficient requirements for establishing effective practices in these areas. Weaknesses exist in the program monitoring and control area because the program management contract itself creates a conflict of interest by requiring the contractor to assess the quality of its own deliverables, and oversee the program’s schedule, issues, and risks. Although USPS officials have told us they use strategies to avoid potential conflicts, such as developing a separate program management team from the system development team, they have not provided us with evidence that they have a formal mitigation plan in place to address the conflict that exists. While organizational policies exist for requirements development and management, weaknesses exist in this area, in part, because USPS decided not to follow its organizational policies for system acquisition and instead followed a truncated program management approach in an effort to deliver the system in a compressed timeframe.

Until USPS fully implements these key acquisition management processes, the Intelligent Mail® Full Service program is at risk of continuing to encounter problems in meeting its performance, schedule, and cost objectives. 10 We are recommending that the Postmaster General direct USPS’s Chief Information Officer and Senior Vice President of Intelligent Mail and Address Quality to take the following actions to improve the management of its acquisition capabilities: (1) develop a comprehensive cost estimate; (2) complete a program plan for the entire Full Service program; (3) reconsider the current contract arrangement to avoid having the contractor review its own performance; (4) define requirements for the entire program and use them as a basis for developing a reliable cost estimate; (5) develop a robust risk management process; and (6) develop and maintain a systems integration plan for release 2 and beyond.

We are also recommending that the Postmaster General direct USPS’s Chief Information Officer to include in USPS’s Technical Solution Life Cycle policy guidance for programs to develop (1) complete program plans; (2) specific requirements for programs to establish a robust risk management process; and (3) system integration plan. 11 In e-mail comments on a draft of these briefing slides, the Senior Vice President of Intelligent Mail and Address Quality did not state whether he agreed or disagreed with our recommendation to develop a comprehensive cost estimate for the program. He disagreed with our findings and conclusions regarding the program’s acquisition management capabilities. Specifically, With regard to project planning, the Senior Vice President stated that federal best practices do not reflect the dynamic environment that drives the scope, requirements, and schedule of future releases of the Full Service program. We disagree, as industry best practices call for an overarching plan that describes a program’s full scope in order to ensure that it is accomplishing its goals.

He further disagreed with our findings on requirements development and management, stating that there is not enough funding to define requirements for the full program. Without a core set of high-level requirements, however, USPS will face challenges in focusing on the next release and holding itself accountable to users’ needs.

Regarding risk management, the Senior Vice President noted that the program has a risk manager and a risk management process in place. However, we found that several key risks were not included in risk reports, complete mitigation plans were not developed, and they did not have a comprehensive risk management plan.

Regarding product integration, the Senior Vice President stated that the program provided us with a systems integration plan. However, these documents defined testing strategies, not a comprehensive system integration plan.

Finally, USPS’s program officials did not state whether they agreed or disagreed with our recommendations that USPS modify its current policy to provide guidance for USPS programs to, among other things, develop complete program plans.

13 Since the 1970s, the use of barcodes and automation has improved efficiency in USPS mail processing operations. Commercial mailers have been encouraged to use barcodes through pricing incentives, allowing USPS to cut costs and increase efficiency in its mail processing operations. In particular, automated mail processing machines can sort mail with barcodes containing delivery information faster than manual sorting. Over the past three decades, the number and type of barcodes increased along with technology changes, and in 2003 USPS estimated that there were more than 30 different barcodes in use. Two of the most commonly used barcodes are the following: POSTNET, which contains delivery information that enables automated sorting of the mail to the carrier’s route level. Mailers receive a postage discount when they print POSTNET barcodes on their mail.

PLANET, which is a barcode that contains identification numbers to enable tracking mail in USPS’s mail processing system but contains less information than the new Intelligent Mail® barcode.

14 According to USPS, the use and maintenance of numerous barcodes have become increasingly burdensome. For example, whenever USPS adds or upgrades its mail processing equipment, it has to ensure that the equipment remains compatible with each of the relevant barcodes. Additionally, printing numerous barcodes on mail pieces clutters the pieces, thus reducing the “real estate” that mailers have to advertise or print other information on their envelopes (see fig. 1).

In 2003, USPS initiated the Intelligent Mail® program, which is intended to use information-rich standardized barcodes to track mail and thus provide USPS and mailers with better and timelier information about the mail. Figure 2 illustrates the components of the Intelligent Mail® barcode.

USPS has identified several ways it expects the implementation of Intelligent Mail® to benefit USPS and mailers: Improve efficiency, reduce costs, and improve timeliness of delivery. USPS says it will be able to use information from Intelligent Mail® to improve its processing system. Also, USPS plans to use Intelligent Mail® to create efficiencies by streamlining and automating the process it uses to accept mail from commercial mailers, which is currently time- and labor-intensive.

Reduce the amount of mail that must be forwarded, which can involve extra handling by USPS and delays in delivery. USPS will provide free notification when intended recipients have moved and filed a change of address with USPS. Mailers previously had to pay for this service. This feature, known as the Address Correction Service, could help USPS meet its goal of reducing the amount of mail that cannot be delivered.

Provide better service to mailers. Through Intelligent Mail®, USPS plans to provide better service to mailers through real-time feedback. Also, since mail will be uniquely identified, USPS anticipates having the ability to isolate and give special handling to a specific mail piece, which creates an opportunity for USPS to offer mailers new products and services.

Financial incentives. USPS is also offering a financial incentive to mailers.

Specifically, those who adopt Full Service Intelligent Mail® will receive a postage discount, in addition to other worksharing discounts.

Service performance measurement capability. Intelligent Mail® is expected to allow USPS to gather more comprehensive and detailed service performance information and measure it against established performance standards, which is intended to help keep USPS accountable to its stakeholders. This feature was also intended to enable USPS to meet requirements in the Postal Accountability and Enhancement Act of 2006. concept, known as workring, generlly involveilerualifying for redced poge rte y performing certin ctivitie such as prepring nd barcoding mil o it corted y USPS automted eqipment.

The 29 other systems are a combination of existing and new systems that need to integrate with the IMB/SASP system in order to share data. An example of one of the existing USPS systems that must be integrated with the IMB/SASP is Postal One—the main communications interface between USPS and mailers. Upgrades to PostalOne include designing a better external user interface, adding electronic documentation acceptance capabilities, and providing more options for mailers to access mail tracking information.

Table 2 provides a summary of the key functionality of the program.

Several USPS groups support the execution of the Full Service program:

The Full Service Program Directors—the Senior Vice President of Intelligent Mail and Address Quality and the Chief Information Officer—head the program. Their responsibilities include reviewing deliverables and conducting governance meetings that focus on the status of the program, issues, and risks.

The Program Management Office activities are performed by a contractor, Accenture, who reports directly to the Program Directors. The responsibilities for the contractor include program status reporting, communications management, scope and release management, issue and risk management, and selective quality deliverable audits.

The Marketing Technology and Channel Management group is the business organization responsible for business mail acceptance process re-engineering and field deployment activities, including field preparedness, developing test plans, newsletters, and training and awareness.

The Sales and Marketing Portfolio is the IT organization responsible for development, integration, and systems deployment activities.

The program’s estimated life cycle cost is $116.4 million. In November 2007, USPS awarded its first contract in a series of contracts for the Full Service program to the prime contractor, Accenture. As of June 3, 2009, USPS has awarded $65.9 million in contracts for the Full Service program. The contracts for the requirements development, designing, building, and testing are firm fixed price.

ct, the gency nd the contrctor gree on price nd the contrctor assumell reponility for ll co nd the resulting profit or loss.

In May 2009, we issued a report that described the Intelligent Mail® program and stated that key management actions were not taken, such as developing a comprehensive strategic plan; preparing information about the program’s costs, including its anticipated savings or cost reductions; and establishing a risk mitigation plan. In addition, we highlighted commercial mailers’ concerns about the implementation of the program. Specifically, the mailers stated that USPS communication efforts were insufficient; USPS and mailers may not be ready for implementation given USPS’s short time frame to simultaneously design, develop, test, and implement the Intelligent Mail® program; and the program’s pricing and benefits may not be sufficient to encourage mailers to participate.

As such, we recommended that USPS develop a comprehensive Intelligent Mail® strategic plan, as well as develop a plan that addresses how USPS will mitigate program- level risks. In its response to our recommendations, USPS agreed to develop a comprehensive Intelligent Mail® strategy, including all planned phases and the associated functions and systems, program goals, and measures of success; and a plan that addresses how it will mitigate risks. y 2009).

26 Objective 1: Full Service Program Status and Plans Program officials have completed key activities for implementing the Full Service program, including deploying the first release on May 18, 2009, and beginning activities for release 2. However, while USPS officials originally intended to deliver the entire Full Service program by January 2009, they currently plan to deliver the program in multiple releases—the first of two planned releases of the program was deployed on May 18, 2009. The second release is expected to be implemented by the end of November 2009. Therefore, full implementation of the program has been delayed by almost 10 months. Additionally, key functionality that was originally intended to be delivered in these two planned releases has been deferred, including automating aspects of the mail acceptance process. Program officials have recently stated that they plan to have future releases to incorporate the deferred functionality; however, they have not made any commitments to do so. In addition, the life cycle cost that program officials prepared does not capture all the costs associated with the acquisition and implementation of the program, such as costs to integrate several USPS systems. Moreover, the first deployed release is currently experiencing operational problems, thus requiring program officials to develop patches to resolve the issues. 27 Objective 1: Full Service Program Status and Plans Current Implementation Status Although USPS officials originally intended to deliver the entire Full Service program by January 2009, they currently plan to deliver the program in multiple releases. The current implementation schedule for the Full Service program is as follows: May 11, 2009. USPS internally implemented the first release which enables certain functions, such as the Address Correction Service and electronic documentation.

May 18, 2009 and beyond. Mailers began testing their systems’ ability to access and electronically transmit documentation to USPS’s system.

November 29, 2009. USPS plans to deploy the second release of the Full Service program and expects to begin offering price incentives to mailers that utilized the Full Service program.

No commitments have been made by program officials for future releases. m offici nnonced on Aust 12, 2009, tht they re iming to develop third releasy Mrch 12, 2010, progrm offici indicted tht they hve not oined fnding to implement thi release or fre releas.

28 Objective 1: Full Service Program Status and Plans Current Implementation Status By May 2011, the use of POSTNET and PLANET barcodes will be phased out and mailers seeking reduced automation-postage rates will be required to use Intelligent Mail® barcodes. Based on the current revised schedule, table 3 summarizes the key functionality by releases.

As of June 3, 2009, $65.9 million had been spent on the acquisition and implementation of the first release and the development of requirements for the second release.

30 Objective 1: Full Service Program Status and Plans Implementation Has Been Delayed The current implementation schedule represents a significant delay from its original and revised implementation dates. Specifically, In January 2008, USPS published the Intelligent Mail® Advance Notice of Proposed Rulemaking in the Federal Register, which originally proposed implementing all functionality of the program by January 2009.

In April 2008, USPS issued a revised Intelligent Mail® Federal Register notice which pushed back the implementation date to May 2009. This was due to several concerns by the mailers, such as the compressed time period in which USPS planned to simultaneously design, test, and implement the program. Mailers were also concerned that they had not been provided with finalized IT requirements.

31 Objective 1: Full Service Program Status and Plans Implementation Has Been Delayed Subsequently, in November 2008, program officials planned to incrementally deliver functionality in multiple releases and delay full implementation further. Specially, they committed to delivering three releases—the first in May 2009, a second in September 2009, and a third in November 2009. According to USPS officials, the schedule was revised to accommodate the implementation of the Intelligent Mail® barcode and to allow mailers more time to make appropriate modifications to their systems and processes.

However, in January 2009, program officials again revised its schedule: they planned to deliver select functionality in a release in May 2009 and additional functionality in another release in November 2009. According to program officials, they decided that delivering three releases in such a short time frame was too ambitious.

See figure 4 for a summary of the Full Service program’s original and revised implementation schedule as of June 2009.

32 Objective 1: Full Service Program Status and Plans Implementation Has Been Delayed Objective 1: Full Service Program Status and Plans Functionality Has Been Deferred In addition to implementation delays, key functionality that was originally intended to be delivered in the two planned releases of the Full Service program has been deferred. Specifically, despite the fact that automating several aspects of the business mail verification process was one of the key justifications for the Full Service program, this function is not going to be delivered in the two releases. Additionally, USPS recently announced, in August 2009, that enabling the ability to better measure and report USPS’s service performance is no longer going to be delivered in the second release, as planned. According to USPS officials, service performance measurement functionality has been deferred because it was taking longer than planned to implement, and they wanted to be able to deliver other promised functionality in release 2 by November 29, 2009. This is especially problematic since USPS is legislatively required to develop a system to better measure and report its service performance to the Postal Regulatory Commission, and the Full Service program was the vehicle USPS planned to use to meet that mandate. l Accontability nd Enhncement Act reqired USPS to develop tem to measure nd report ervice performnce to the Pol Regtory Commission.

35 Objective 1: Full Service Program Status and Plans Functionality Has Been Deferred While program officials announced on August 12, 2009, that they are aiming to develop a third release by March 12, 2010, program officials indicated that they have not obtained funding to implement this release or future releases. According to program officials, one of the primary reasons for not moving forward with such decisions is that funding for future releases may not be available as a result of USPS’s current financial situation. We recently reported that amid challenging economic conditions and a changing business environment, USPS is facing a deteriorating financial situation in which it does not expect to cover its expenses and financial obligations in fiscal years 2009 and 2010. As a result, we added the financial condition of USPS to our high-risk list of federal areas in need of transformation. tmaster Generl tetified in Juary 2009 tht USPSascing potentil net loss of $6 illion or more for fil yer 2009. He noted tht USPS nticipted contined deteriortion de to the economic lowdown, as the finncil, credit, nd housing ector re mong it key business driver. ly 2009).

36 Objective 1: Full Service Program Status and Plans Program Life Cycle Cost Was Not Completely Defined According to industry best practices, programs must maintain current and well- documented cost estimates, and these estimates must encompass the full life cycle of the program. Specifically, as stated in the GAO Cost Estimating and Assessment Guide, cost estimates should be comprehensive in that they should include both government and contractor costs throughout the program’s full life cycle, from the inception of the program through design, development, deployment, and operation and maintenance to retirement.

According to the business case, the life cycle cost estimate of the Full Service program is $116.4 million. This includes the costs to develop the custom software capabilities, necessary hardware to support the software capabilities, and operating and maintenance cost.

However, the life cycle cost estimate excludes key costs associated with the acquisition and implementation of the Full Service program. For example, the estimate does not include costs related to the integration of the systems or the cost of future releases beyond release 2. rch 2009).

37 Objective 1: Full Service Program Status and Plans Program Life Cycle Cost Was Not Completely Defined Additionally, the life cycle cost has not been updated to reflect the significant changes that have been made to the program. While a revised business case to reflect the modified schedule and scope for the program was approved in June 2009, program officials did not update the life cycle cost of the program.

According to program officials, they did not include all of the costs associated with integrating the system because they did not regard the costs to be significant enough to include. Additionally, officials stated that they did not include the costs of the future releases because they are uncertain if they are going to be able to deliver those releases.

Examples of issues reportedly resolved in these two patches include:

certain mailers and mail pieces were being incorrectly charged, the system was not recognizing certain zone values, the system was preventing mailers from putting Intelligent Mail® barcodes on certain reporting functions were not working as intended, system was not allowing mailers to enter certain information on individual mail after logging into account and clicking through the available links, mailers were receiving an error message when they tried to return to the homepage.

A third patch is scheduled to be deployed on August 16, 2009. Examples of operational problems and system enhancements intended to be addressed include:

system is incorrectly creating a finalized postage statement for mailers who canceled or updated a job, system does not accept certain updates after postage statements are final and ready inconsistent charges are being applied to a secured group of mail pieces, and certain electronic documentation is not being transmitted through the system.

40 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed USPS Lacks Key Management Capabilities Essential to Effectively Acquire and Manage the Full Service Program USPS is in the process of implementing key acquisition management controls, but it has yet to implement the full set of controls essential for acquiring and managing the Full Service program in a disciplined and rigorous manner. Specifically, it has not implemented certain process controls in the areas of project planning, project monitoring and control, requirements development and management, risk management, and product integration.

The primary cause of the program’s immature management approach in the areas of project planning, risk management, and product integration is that USPS organizational policies do not set forth sufficient requirements for establishing effective practices in these areas. While organizational policies exist for requirements development and management, weaknesses exist in this area in part because USPS decided not to follow its organizational policies for system acquisition and instead took a 41 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed truncated program management approach in an effort to deliver the system in a compressed timeframe.

Until USPS implements the full set of controls essential to effectively managing the program, it increases the risk that the Full Service program will continue to encounter problems in meeting its performance, schedule, and cost objectives.

42 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed As we have previously reported, to effectively manage major IT programs, organizations must use sound acquisition and management processes to minimize risks and thereby maximize chances for success. Such processes have been identified by leading organizations such as the Software Engineering Institute, the Chief Information Officer’s Council, and in our prior work analyzing best practices in industry and government. In particular, the CMMI-ACQ and CMMI-DEV have defined a suite of key acquisition process control areas that are necessary to manage system acquisitions in a rigorous and disciplined fashion. These process areas include project planning, project monitoring and control, requirements development and management, risk management, and product integration. mple, GAO, Information Technology: Management Improvements Needed on Immigration and Customs Enforcement’s Infrastructure Modernization Program, GAO-05-805 (Washington, D.C.: Septemer 7, 2005) nd Census Bureau: Important Activities for Improving Management of Key 2010 Decennial Acquisitions Remain to be Done, GAO-06-444T (Washington, D.C.: Mrch 1, 2006). rnegie Mellon Softwre Engineering Intitte, Cability Mrity Model® Integrtion for Acqition (CMMI-ACQ), Verion 1.2 (Novemer 2007). rnegie Mellon Softwre Engineering Intitte, Cability Mrity Model® Integrtion for Development (CMMI-DEV), Verion 1.2 (Aust 2006).

Project Planning Effective project planning involves establishing and maintaining plans that define project scope and activities, including overall budget and schedule, key deliverables and milestones for key deliverables, assumptions and constraints, description and assignment of roles and responsibilities, staffing and training plans, and an approach for maintaining these plans. It also involves obtaining stakeholder commitment to the project plan.

The Full Service program officials have established a program office for the Full Service program and assigned a USPS hired a contractor to carry out program management activities including tracking schedule, issues, and risks for the program; identified the tasks and organizational roles and responsibilities for release 1; developed a program plan for release 2 that identifies key deliverables and milestones for these deliverables; and developed a business case for the Full Service program. ion 1.2 (Novemer 2007) nd The Intitte of Electricnd Electronic Engineer, IEEE Sndrd for Softwre Life Cycle Process—Project Mgement Pl, IEEE Sndrd 1058-1998 (Decemer 8, 1998).

44 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed Project Planning (continued)

While officials have developed a business case for the Full Service program and a program plan for release 2, there still is no comprehensive program plan that includes the full scope of the program, including how many releases are planned and the specific functions and systems to be implemented in each release; its plans to standardize and consolidate the over 30 barcodes currently being used; assumptions and constraints about the program; a description and assignment of roles and responsibilities; staffing and training plans; and the strategy for maintaining the program plan. In addition, program officials have not yet obtained commitment from internal and external stakeholders on the program plan for release 2. Such a plan is often used to form a baseline for the program and to obtain buy-in from stakeholders.

A key reason that these activities have not been completed is that USPS’s policy that outlines the steps that programs should follow when developing, acquiring, enhancing, and/or maintaining IT systems—referred to as the Technical Solution Life Cycle policy— does not require that officials develop a comprehensive plan for their programs. 45 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed Project Planning (continued)

Until program officials develop a complete program plan that supports the Intelligent Mail® Strategic Plan, which we previously recommended, and includes the details on the full scope of the Full Service program, USPS may not be able to ensure that the program is moving in the right direction. Without this assurance, it is more likely to encounter unanticipated changes in direction—which could affect cost, schedule, and deliverables.

Project Monitoring and Control Project monitoring and control involves providing oversight of the contractor’s and the project office’s performance, in order to allow appropriate corrective actions if actual performance deviates significantly from the plan. Key activities in tracking both the contractor’s and the project office’s performance include communicating status, taking corrective actions, and determining progress. In addition, organizations should have IT investment management boards comprised of key executives to regularly track the progress of major systems acquisitions. These boards should be able to adequately oversee the project’s progress toward cost and schedule milestones and its risks. The board should also employ early warning systems that enable it to take corrective actions at the first sign of cost, schedule, and performance slippages.

With regard to project monitoring and control activities, program officials track the milestones and dependencies of the program; and review the activities, status, and results of the process with higher level program management, USPS senior executives representing both IT and business units, and the contractor. ion 1.2 (Novemer 2007). rch 2004).

47 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed Project Monitoring and Control (continued)

However, the main contractor performing the development and implementation functions of the Full Service program is also the contractor carrying out USPS’s program management activities. Specifically, according to the Program Management Office contract, the contractor is responsible for assessing the quality of program deliverables; overseeing the program’s schedule, issues, and risks; assessing the project plan’s critical path which is necessary for examining the effects of any activity slipping along this path; developing project status materials for USPS program officials, including bi-weekly detailed status reports to the program manager and weekly status reports to IT management and project teams; and participating in weekly deliverable reviews from other USPS internal and external suppliers, including documenting all meeting minutes, and action items. l pth i the longet dtion pth throgh the eqenced lit of key progrctivitie.

48 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed The roles that the contractor plays as both a manager of the Full Service program and as a supplier of products for the program create a conflict of interest because of the risk that the contractor will not evaluate its own products in a completely objective manner. USPS program officials stated that they do not think that this is an issue because the company’s program management staff work on a separate team from the system development staff and the two teams do not interact; however, this arrangement still requires the contractor to assess the quality of its own deliverables, and oversee the program’s schedule, issues, and risks. USPS officials have not provided us with evidence that they have a formal mitigation plan in place to address the conflict that exists under the contract.

While we recognize that USPS is not required to comply with the Federal Acquisition Regulation (FAR), these regulations can be instructive since they are used by federal agencies for acquiring goods and services. According to the FAR, an underlying principle is that, in order to avoid a conflict of interest, a contractor should not have conflicting roles that might bias a contractor’s judgment. ) (“Acqition” defined). The FAR generlly pplie to cqitionde with pproprited fnd used to oin supplie or ervice for the federl government. ).

49 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed Until program officials reconsider having the same contractor that is developing and implementing the system be responsible for helping USPS oversee the program, USPS will increase its risk of unexpected cost increases, schedule delays, and performance shortfalls.

Requirements Development and Management Requirements development involves eliciting, analyzing, and validating customer and stakeholder needs and expectations. Requirements management involves establishing an agreed-upon set of requirements, ensuring traceability between operational and product requirements, and managing any changes to the requirements in collaboration with stakeholders.

With regard to requirements development and management, program officials have defined the initial business requirements dated August 16, 2007, for release 1; defined requirements for release 2; and developed a change control process for managing changes to the requirements. ion 1.2 (Novemer 2007).

51 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed Requirements Development and Management (continued)

While USPS has defined the requirements for release 1 and release 2, it has not finalized or validated the core set of requirements for the Full Service program, which would include high-level requirements that USPS plans to deliver in future releases. These core requirements would need to be further defined as the program begins to focus on the next release. Program officials stated that they did not fully define the program’s requirements because the requirements are still evolving.

Risk Management An effective risk management process identifies potential problems before they occur, so that risk- handling activities may be planned and invoked as needed across the life of the product and project in order to mitigate adverse impacts on achieving objectives. Key activities include assigning resources, identifying and analyzing risks, and developing risk mitigation plans and milestones for key mitigation deliverables. Additionally, a risk management strategy addresses the specific actions and management approach used to perform and control the risk management program. It also includes identifying and involving relevant stakeholders in the risk management process.

With regard to risk management, program officials have assigned responsibility for managing the risks and identified and analyzed selected risks associated with schedule, performance, and testing. Examples of the selected program-level risks include o limited mailer adaptation and adoption can affect future Full Service releases, o program success measurements are not defined, o parallel program activities have caused resource constraints, o components of scope have not been planned for release 2, and o mailers require significantly more support than estimated to assist them with the implementation of the Intelligent Mail® barcode. ion 1.2 (Novemer 2007).

Risk Management (continued)

However, they did not adequately identify all risks. For example, While the USPS contracting officer indicated in the program management contract’s price negotiation memorandum that having the same company perform program management activities as well as development and implementation activities for the Full Service program is a major concern, program officials have not identified this as a risk or established a complete mitigation strategy.

Program officials stated that they include the list of risks the system development contractor identifies as part of the program management office’s risk reports. However, as of July 16, 2009, there was no evidence in the reports that contractor risks were being identified or mitigated.

While program officials are concurrently conducting activities for release 2 and unplanned post-deployment efforts for release 1, they have not identified potential schedule delays in release 2 as a risk or established a mitigation plan.

54 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed Risk Management (continued)

Moreover, as we have previously reported, USPS lacks a risk mitigation plan, and therefore we recommended that USPS develop a plan that addresses how it will mitigate program-level risks. Although USPS agreed with this recommendation, it has not yet developed complete risk mitigation plans. During this review we found that while USPS recently finalized a risk management plan for release 2, it is not comprehensive and does not fully address the scope of the risk management effort, including discussing techniques for risk mitigation, defining adequate risk sources and categories, and identifying and involving relevant stakeholders to promote commitment and understanding of the process.

The program’s weaknesses in the risk management area are partly due to the fact that USPS’s Technical Solution Life Cycle policy does not set forth sufficient requirements regarding risk management.

Product Integration The scope of this process area is to achieve complete product integration through progressive assembly of product components. A critical aspect of this area is the management of internal and external interfaces of the products and product components to ensure compatibility among the interfaces. Attention should be paid to interface management throughout the project. In addition, a systems integration plan should be developed to identify all systems to be integrated, define roles and responsibilities of all relevant participants, establish the sequence and schedule for every integration step, and describe how integration problems are to be documented and resolved.

With regard to product integration, program officials have identified approximately 30 systems that will need to be integrated in the Full Service program. However, USPS officials have stated that the number of systems that need to be integrated could change, and they are not yet aware of which specific systems will need to be integrated in release 2 or in possible future releases. In addition, while program officials stated they have several program documents, such as testing strategies, they do not have a system integration plan, which is intended to support the deployment strategy and describe to key stakeholders in each integration step what needs to be done to effectively integrate the various systems. The program office also lacks documentation of the process associated with updating and maintaining the integration of the systems. ion 1.2 (Aust 2006). rtment of Trporttion, Federl Highwy Adminitrtion, Stem Engineering Gideook for ITS Verion 2, Juary 2, 2007.

56 Objective 2: Adequacy of Acquisition Management Capabilities Acquisition Management Capabilities Are Needed Part of the reason that these activities have not been completed is that USPS’s Technical Solution Life Cycle policy does not set forth sufficient requirements regarding product integration and does not require that programs develop system integration plan and associated documentation regarding updating and maintaining the integration of the systems.

Until program officials develop these key product integration documents, USPS will be limited in its ability to ensure that the product is integrated, functioning properly, and delivered on time and within budget to the users.

57 The current schedule for the full implementation of the Full Service program has been delayed by almost 10 months, and key functionality that was originally intended to be delivered in the program has been deferred indefinitely. In addition, the life cycle cost estimate that program officials prepared does not capture all the costs associated with the acquisition and implementation of the program. As a result, program officials lack an accurate total cost estimate. Moreover, the first deployed release is experiencing performance issues.

While organizational policies exist for requirements development and management, weaknesses exist in this area, in part, because USPS decided not to follow USPS’s organizational policies for system acquisition and instead followed a truncated program management approach in an effort to deliver the system in a compressed timeframe. Without these processes in place, USPS increases the risk that this project will continue to encounter problems in meeting its performance, schedule, and cost objectives.

Given that release 2 is expected to be implemented by the end of November 2009 and decisions about future releases need to be made, having the key elements of a sound acquisition management capability in place will be crucial to the program’s success in meeting its goal.

To ensure that USPS adequately manages the acquisition of the Intelligent Mail® Full Service program, we recommend that the Postmaster General direct the Chief Information Officer and Senior Vice President of Intelligent Mail and Address Quality to take the following six actions: Develop a comprehensive cost estimate to include both government and contractor costs over the program’s full life cycle, from the inception of the program through design, development, deployment, and operation and maintenance to retirement.

Complete an overall program plan for the entire Full Service program, including an overview of the program’s scope of all releases, deliverables and functionality within these releases, plans to phase out the approximately 30 barcodes currently being utilized, assumptions and constraints, roles and responsibilities, staffing and training plans, and the strategy for maintaining the plan.

Reconsider the current contract arrangement to avoid having the contractor evaluate its own performance.

Define the core set of requirements for the entire program and use them as a basis for developing a reliable cost estimate.

Develop a risk management process that enables the program officials to develop an adequate risk management plan that fully address the scope of their risk management efforts; ensures that a comprehensive list of risks and complete mitigation plans are identified and tracked; and includes milestones, mitigating actions, thresholds, and resources for significant risks.

Develop and maintain a systems integration plan for release 2 and beyond.

We are also recommending that the Postmaster General direct USPS’s Chief Information Officer to include in USPS’s Technical Solution Life Cycle policy, guidance for programs to develop (1) complete program plans that define overall budget and schedule, key deliverables and milestones, assumptions and constraints, description and assignment of roles and responsibilities, staffing and training plans, and an approach for maintaining these plans; (2) specific requirements for programs to establish a robust risk management process that identifies potential problems before they occur, such as requiring programs to develop a risk management plan; and (3) system integration plan that include all systems to be integrated with the system, roles and responsibilities for all relevant participants, the sequence and schedule for every integration step, and how integration problems are to be documented and resolved. 61 Agency Comments and Our Evaluation We received comments via e-mail from the Senior Vice President of Intelligent Mail and Address Quality on a draft of these briefing slides. He did not state whether he agreed or disagreed with our recommendation to develop a comprehensive cost estimate. The Senior Vice President disagreed with our findings and conclusions regarding the program’s acquisition management capabilities.

With regard to project planning, he stated he disagreed with the static approach suggested in our briefing. He stated that the scope, requirements, and schedule of future releases are driven by a dynamic environment both internal and external to USPS. However, as we state in the briefing, industry best practices show that it is important to have an overarching program plan that describes the full scope of the program, including how many releases are planned, in order to ensure that the program is accomplishing its goals within the specified cost and schedule objectives. ident revied the gency’ poition on GAO’ even recommendtion in written commentted Octoer 15, 2009. See ppendix II.

62 Agency Comments and Our Evaluation Regarding project monitoring and control activities, the Senior Vice President provided additional clarifying information on executive-level oversight, which we have incorporated into the briefing as appropriate. He did not acknowledge that a conflict of interest exists. While the USPS contracting officer identified that having the same company perform program management activities as well as development and implementation activities for the Full Service program is a concern, the program office has not identified that this arrangement is a conflict of interest. In fact, the program office has not even identified this arrangement as a risk that it tracks in its risk tracking process. Despite this, program officials have explained actions they are taking to mitigate the potential risk of a conflict of interest, such as establishing separate teams for the program management staff and system development staff. However, they have not presented us with any evidence of a formal mitigation plan that is in place to address the actual conflict of interest that is introduced by the responsibilities that are specified in the program management office contract, which states that the contractor must assess the quality of deliverables and oversee the program’s schedule, issues, and risks. Unless USPS reconsiders the current contract arrangement to avoid having the contractor evaluate its own performance, there is an increased risk that the conflict of interest will negatively impact the program. 63 Agency Comments and Our Evaluation Regarding our findings with regard to requirements development and management activities, the Senior Vice President stated that he disagreed because there is not enough approved funding to define the requirements for the full program—they only received funding for a portion of the program. However, as we state in the briefing, without a core set of high-level requirements, it will be difficult for USPS to focus appropriately on the next release and to hold itself accountable to delivering a system that meets USPS's and mailers' needs. Defining these requirements is especially important given the functionality that is being deferred in the first two releases.

Regarding risk management activities, he stated that the program has a defined, active, cross-program process and a risk manager who is responsible for managing this process. While we acknowledge that the program has developed a tracking process, which includes assigning responsibility for managing and identifying risks, several key risks, such as the risk for potential schedule delays in release 2 as a result of conducting concurrent activities for release 2 and release 1, were not included by the program office as part of their risk reports and complete mitigation plans were not developed. Additionally, the recently finalized risk management plan for release 2 is not comprehensive and does not fully address the scope of the risk management effort. Until USPS develops a strategy for ensuring a comprehensive 64 Agency Comments and Our Evaluation list of risks and that includes mitigation efforts, it increases the probability that unanticipated risks may occur that could have a critical impact on the program’s cost, schedule, and performance.

The Senior Vice President disagreed with our finding on product integration activities.

He stated that a systems integration plan for conducting product integration was provided to us. However, the documents provided to us included testing strategies and not a system integration plan, which is intended to support the deployment strategy and to describe to key stakeholders in each integration step what needs to be done to effectively integrate the various systems. Until these key product integration documents are developed, USPS will be limited in its ability to ensure that the product is integrated and functioning properly.

Additionally, USPS program officials did not state whether they agreed or disagreed with our recommendation that USPS include in its Technical Solution Life Cycle policy, guidance for programs to develop (1) complete program plans; (2) specific requirements for programs to establish a robust risk management process; and (3) system integration plan.

Appendix II: Comments from the United States Postal Service

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Shannin G. O’Neill, Assistant Director; Neil Doherty; Rebecca E. Eyler; Mary D. Fike; Franklin Jackson; Lee McCracken; Niti Tandon; Christy A. Tyson; and Adam Vodraska made key contributions to this report.