United States Postal Service: Opportunities to Strengthen IT Investment Management Capabilities
Published: 2002-10-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The U.S. Postal Service invests hundreds of millions of dollars in information technology (IT) each year to support its mission of providing prompt, reliable, and efficient mail service to all areas of the country. It must support these operations through the revenues it earns for its services. Growing operating expenses and capital needs in the face of reduced revenues highlight the need for the Postal Service to invest its IT dollars wisely. Accordingly, the Senate Committee on Governmental Affairs and its Subcommittee on International Security, Proliferation, and Federal Services asked GAO to evaluate how well the Postal Service manages its IT investments.
section_title: What GAO Found
paragraphs: The Postal Service has in place many of the foundational capabilities required for managing IT investments described in GAO's IT Investment Management framework, illustrated below. Proposed major projects go through established review processes and must be approved at a high level before being implemented. Control processes also are in place. Although the Postal Service evaluates proposed IT projects before investing in them, it does not fully manage these investments from a portfolio perspective by assessing projects on the basis of indicators that clearly link performance to initial selection criteria. Such a portfolio approach would enable the Postal Service to consider proposed projects along with those that have already been funded and to select the mix of investments that best meets its mission needs. The Postal Service has not yet attained the key attributes associated with most capable organizations, such as evaluating the performance of investments as a whole, capturing "lessons learned," and institutionalizing these lessons to benefit the organization. Until it addresses areas such as these, the Postal Service will not be in a position to continually improve its investment process and leverage its IT capabilities for strategic outcomes.

section_title: Letter
paragraphs: In response to your request, this report addresses the Postal Service’s capabilities in information  technology (IT) investment management. The Postal Service invests hundreds of millions of dollars  each year in information technology to provide prompt, reliable, and efficient mail service to all areas  of the country. Our evaluation determined that the Postal Service is executing many of the  foundational practices necessary for managing these investments, but the Postal Service has  additional opportunities to implement more mature and effective processes. We are making  recommendations to strengthen the Postal Service’s investment management capabilities.
We are sending copies of this report to the Chairman and Ranking Minority Member of the House  Committee on Government Reform, the Chairmen and Ranking Minority Members of the House and  Senate Committees on Appropriations, the Postal Service’s Postmaster General/Chief Executive  Officer, and the Postal Service’s Chief Financial Officer. We will also make copies available to others  on request. In addition, the report will be available at no charge on the GAO Web site at  http://www.gao.gov.
Staff acknowledgments are included in appendix IV. If you have any questions about this report,  please contact me by telephone at (202) 512-6408 or by E-mail at WillemssenJ@gao.gov.
subsections: section_title: Introduction
subsections: section_title: Postal Service’s Mission and Organization
paragraphs: The Postal Reorganization Act of 1970 (P.L. 91-375) created the United  States Postal Service, an independent, self-supporting organization,  replacing the former United States Post Office Department. The act  charges the Postal Service with binding the nation together through the  personal, educational, literary, and business correspondence of the people  and providing reliable and efficient mail services to all areas of the country.  The Postal Service is intended to be self-supporting from postal operations  and is mandated to break even over time. With nearly 800,000 career  employees, the Postal Service is the second largest employer compared  with U.S. private sector organizations. It has an extensive infrastructure,  consisting of more than 38,000 post offices, branches, and stations; 240,000  delivery routes to over 137 million delivery addresses; a fleet of 215,000  vehicles; 350 major processing and distribution facilities; and nearly  800,000 career employees.
The Postal Service is now facing a financial crisis brought about by  declining revenues and growing operating expenses and capital needs,  including the cost of existing and new investments in information  technology (IT). In February 2002, we reported significant declines in the  Postal Service’s net income from fiscal year 1995 to fiscal year 2001 and a  net loss of $1.68 billion in fiscal year 2001 alone, resulting in part from  declining mail volumes and from terrorist incidents. In April 2001, we  placed the Postal Service’s transformational efforts and long-term outlook  on our High-Risk list, noting that the Postal Service is at growing risk of not  being able to continue its mission of providing the current level of universal  service throughout the nation while maintaining reasonable rates and  remaining largely self-supporting through postal revenues.
The Postal Service has acknowledged the need for a new business model in  light of these events and various trends now shaping the delivery services  marketplace, such as consumer interest in new service types and  increasing security concerns. Other increases in the cost of doing business,  such as the rising costs of retirement and health benefits, heighten the need  for action. To conserve cash and limit debt, the Postal Service has  continued its freeze on capital spending for most facility projects, and its  total budgeted capital outlays have declined in fiscal year 2002 for the third  consecutive year to $2.2 billion.
The Postal Service has reported that it plans to respond to these trends by  providing customers with added value, improving the efficiency of  operations, containing costs, fostering a performance-based culture, and  improving its management of enabling functions such as financial  management, purchasing, and IT. The Postal Service has established the  specific goal of connecting all of its components through IT, to enable it to  enhance security, add valuable product features, and manage its operations  in real time.
The Postal Service accounts for its expenditures in separate expense and  capital accounts, according to Generally Accepted Accounting Principles  to which public financial reporting by U.S. corporations must conform.  Expenditures categorized as “expense” generally comprise operating costs  and are primarily funded through a general operating budget. Expenditures  categorized as “capital” are for one-time costs, are project-specific, and are  depreciated.
The Postal Reorganization Act vested direction of the Postal Service in an  eleven-member Board of Governors, including nine appointed by the  President. The nine governors appoint the Postmaster General, who is the  Chief Executive Officer, and who, with the nine governors, appoints the  Deputy Postmaster General. The Postal Service’s executive vice presidents  are the Chief Operating Officer and the Chief Financial Officer. The Postal  Service has senior vice presidents for Government Relations and Public  Policy, Human Resources, Operations, Office of the Chief Marketing  Officer, and Office of the Chief Technology Officer. Figure 2 shows an  overview of the Postal Service’s current organizational structure.
section_title: Postal Service’s Information Technology Environment
paragraphs: The Postal Service has come to rely increasingly on IT. In the early 1980s, it  used data centers and mainframe computers to support administrative  functions such as personnel, accounting, and payroll processing. In the  mid-1980s, the Postal Service began to incorporate IT into its core business  activities by interconnecting various components of its mail processing  system through telecommunications and automation. Today, the  organization relies on IT throughout the full range of its operations and  management processes to run the machines that process and sort mail,  assign mail efficiently to alternative surface and air carriers, support point- of-service terminals, collect and analyze inventory and sales information,  process payroll and other accounts payable, and perform other activities.  Communication networks also play a vital role in linking together various  elements of the Postal Service’s infrastructure and transmitting information  to various locations for storage, processing, and analysis. The Postal  Service expended approximately $700 million for IT in fiscal year 2002 and  plans to spend about $1 billion for IT in fiscal year 2003.
The Postal Service currently manages almost 650 IT systems and  applications that operate in support of postal functions. It has 24 IT-related  projects in development or recently completed, each estimated to cost at  least $10 million. The total investment cost estimated for these projects  since 1997 is more than $2 billion, ranging from about $10 million to about  $404 million per project. (See app. I for a list of the Postal Service’s IT- related projects currently in progress.) Projects with major IT components  in development or implementation phases include the following:  Point of Service ONE—A retail point-of-sale information system  that is intended to replace outdated retail terminals at postal retail  windows and provide more timely and accurate information.
Associate Office Infrastructure—Expected to support a common  information system for retail, delivery, and administrative operations  in post offices.
Delivery Operations Information System—Scheduled to replace  three current information systems and assist delivery unit  supervisors in managing office activities, planning street activities,  and managing route inspection and adjustment activities.
Time and Attendance Collection System—Expected to replace  five existing time and attendance systems and enable labor resources  to be more efficiently allocated by providing supervisors with  accurate, real-time labor data by type of work being performed.
Advanced Computing Environment—A major infrastructure  modernization initiative that is expected to replace existing  workstations and transitions applications to a Web-based environment.
section_title: Weaknesses in the Postal Service’s Investment Management Process
paragraphs: Given the challenges the Postal Service currently faces, effective  management of its existing and new IT investments is crucial if it is to  provide the service expected while remaining self-supporting. However,  recent reviews, performed by the Postal Service’s Office of Inspector  General (OIG) and by us, have raised some concerns regarding the  Service’s investment management. The OIG has identified weaknesses in  the management of some investments in recent years. For example, in  September 2001, the OIG reported that projects have been proposed to the  Board of Governors for approval without adequate documentation and  analyses and that other projects may not achieve anticipated performance  and financial results.In March 2001, the OIG’s review of the Delivery  Operations Information System found weaknesses in the methods and  assumptions that were used to derive figures on estimated savings and  return on investment. In September 1999, the OIG found that Point of  Service ONE was not achieving the results outlined in its business case.  The Postal Service has made enhancements to its investment policies and  procedures to address the issues the OIG raised. In September 2000, we  identified a number of issues with the management of the Postal Service’s  e-commerce program, including inconsistencies in reviewing and  approving e-commerce initiatives and deficiencies in the financial data  reported. We made several recommendations to the Postal Service that  addressed these issues. This program was subsequently scaled back by the  Postal Service, as both revenues and customer response fell below  expectations.
section_title: Postal Service’s Approach to Investment Management
paragraphs: Several individuals and oversight boards are involved in managing IT  investments, from reviewing and approving a proposed IT project, through  the process of budgeting for it and monitoring it once it is implemented,  and evaluating it at its conclusion. These individuals and oversight boards  and their roles are described below.
Board of Governors—Eleven-member board that governs the Postal  Service; comprises the Postmaster General, the Deputy Postmaster  General, and nine Presidential appointees; expected to approve any  project with capital and “expense investment” costs of $10 million or  more.
Capital Projects Committee (CPC)—Three members of the Board of  Governors who are to review proposals for any new project with capital  and expense investment costs of $10 million or more and make  recommendations to the full Board on whether to approve it.
Postmaster General—Chief Executive Officer of the Postal Service  and a member of the Board of Governors; expected to approve or review  any project with capital and expense investment costs of $7.5 million or  more.
Establish Team—Comprises the Deputy Postmaster General, the Chief  Financial Officer, the Chief Operating Officer, the Chief Marketing  Officer, the Senior Vice Presidents of Operations and Human Resources,  the Controller, and a field vice president; is to set financial and  nonfinancial goals for the Postal Service at the start of its annual  planning and budgeting process and determine funding for existing and  proposed IT projects as part of the budget formulation process.
Capital Investment Committee (CIC)—Comprises the Chief  Technology Officer (CTO) and other senior executives. Is to review  proposals for any project with capital and expense investment costs of  $7.5 million or more.
Deploy Team—Comprises several vice presidents; with the Establish  Team, is to determine funding for IT projects as part of the Postal  Service’s annual planning and budgeting process.
Vice President of Finance (Controller)—Is to review and validate  proposals for any project with capital and expense investment costs of  $5 million or more.
Capital and Program Evaluation (CAPE)—Group within the  Finance Department under the Controller. During the review process for  new projects, is expected to validate the assumptions and cost, benefit,  and schedule estimates; prepare the Postal Service’s 5-year Capital  Investment Plan (CIP); monitor projects with capital and expense  investment costs of $5 million or more; and perform cost studies of  selected completed projects.
Chief Technology Officer (CTO) Organization—Comprises the  Office of the CTO and the Information Technology Department headed  by the Chief Information Officer (CIO). The CTO organization assists  other functional units in developing business cases for projects that  have an IT component. It is also involved in the project concurrence  process, where feedback on a project is given to the sponsoring  organization by functional areas and relevant field units. The CTO  organization is also responsible for developing systems standards and  requirements for organizationwide compliance. At the strategic level,  the CTO and CIO recommend and present corporatewide IT projects  before the Establish Team during the annual capital planning cycle.
CTO Investment Review Board—Three-member board comprises the  CTO, CIO, and Manager of IT Value; is to manage the process of  selecting projects within the CTO organization, review the performance  of all IT projects in development, and conduct detailed reviews of  selected IT projects on a monthly basis.
The Postal Service has established a number of capital planning,  investment control, and budgeting processes to manage its IT investments.  These include processes for (1) developing the investment portfolio, (2)  approving major new projects, and (3) controlling and evaluating projects.
subsections: section_title: Process for Developing the Investment Portfolio
paragraphs: The Postal Service’s annual capital planning and budgeting cycle begins in  January with a process called the CustomerPerfect! management cycle.  The Establish Team and the Deploy Team, composed of Postal Service  executives, manage this annual organizationwide direction-setting process,  led by Operations and aided by the Budget and Financial Analysis (B&FA)  and the Capital and Program Evaluation (CAPE) groups within the Finance  Department. The Establish Team is expected to align the organization’s  targets and goals with its commitment to listen to the three “voices” that  represent aspects of its mission: the Voice of the Business (financial  benefits), the Voice of the Customer (customer satisfaction), and the Voice  of the Employee (employee satisfaction). The Establish Team is to review  project and program funding requests and make preliminary selection and  funding decisions on the basis of how the requests fit the organization’s  mission and budget. This process sets the Postal Service’s financial and  nonfinancial goals for the year. Figure 3 provides detail on the Postal  Service’s capital planning and budgeting cycle.
section_title: Process for Approving Major New Projects
paragraphs: The process for approving major new IT projects is the same as for any  other new projects with capital costs of $5 million or more. These major  projects are to proceed through the formal approval process and are  monitored by the Finance Department in conjunction with the program  sponsors when they are in development and implementation phases.
The process for approving proposed capital investments is defined in the  Postal Service’s F-66 manual. The process begins with the sponsoring unit  preparing a Decision Analysis Report (DAR), which presents the business  case for the proposed project. Figure 4 provides detail on the process for  approving major new projects.
section_title: Control and Evaluation Process
paragraphs: During a capital project’s life cycle, control and evaluation are  accomplished through two processes. Project sponsors are to produce  quarterly compliance reports that summarize the project’s status. These  reports are to be used by CAPE, along with other financial information, to  produce the quarterly Investment Highlights that are distributed to the  Board of Governors and others to present the status of Board-approved  projects. This project oversight process continues for 18 months beyond a  project’s initial implementation. The Program Performance Group, part of  CAPE, studies selected projects that are still in development to determine  whether they remain on track to achieve cost goals. The Program  Performance Group may also conduct cost studies, after implementation,  to determine whether cost goals have been met. Changes in scope,  schedule, or total capital funding needed for a project trigger the  requirement for a modified DAR, which must be reviewed and approved  through the same process as the original DAR.
At the operational level, the CTO organization’s project managers and  portfolio managers conduct the day-to-day oversight of IT projects,  including those sponsored outside of the CTO organization, by tracking  performance of IT projects in the Program Tracking and Reporting System  (PTRS) and reporting project status every month to the CTO Investment  Review Board. When problems are identified, they are addressed through  interaction with the sponsoring organization, which may choose to bring  the issue to senior executives if the problem is likely to affect their ability  to meet their objectives.
IT investments that are not funded by capital funds are controlled and  evaluated through the annual budget process. Executive-level oversight is  performed through annual reviews of program descriptions called  “program narratives,” which provide input to the budget decision. At the  operational level, ongoing oversight is performed through routine tracking  of system operation. Figure 5 shows the Postal Service’s project control  and evaluation process.
section_title: IT Investment Management Framework
paragraphs: Based on research into the IT investment management practices of leading  private- and public-sector organizations, we have developed an information  technology investment management maturity (ITIM) framework. This  framework identifies critical processes for successful IT investment  organized into a framework of five increasingly mature stages. The ITIM  is intended to be used both as a management tool for implementing these  processes incrementally and as an evaluation tool for determining an  organization’s current level of maturity. The overriding purpose of the  framework is to encourage investment processes that increase business  value and mission performance, reduce risk, and increase accountability  and transparency in the decision process. This framework has been used in  several GAO evaluationsand has been adopted by a number of agencies.  These agencies have used ITIM for purposes ranging from self-assessment  to redesign of their IT investment management processes.
ITIM is a hierarchical model comprising five “maturity stages.” These  maturity stages represent steps toward achieving stable and mature  processes for managing IT investments. Each stage builds upon the lower  stages; the successful achievement of each stage leads to improvement in  the organization’s ability to manage its investments. With the exception of  the first stage, each maturity stage is composed of “critical processes” that  must be implemented and institutionalized for the organization to achieve  that stage. These critical processes are further broken down into key  practices that describe the types of activities an organization should be  performing to successfully implement each critical process. An  organization may be performing key practices from more than one maturity  stage at one time. This is not unusual, but efforts to improve investment  management capabilities should focus on becoming compliant with lower  stage practices before addressing higher stage practices.
Stage two in the ITIM framework encompasses building a sound  investment management process—by developing the capability to control  projects so they finish predictably within established cost and schedule  expectations—and establishing basic capabilities for selecting new IT  projects. Stage three requires that an organization continually assess  proposed and ongoing projects as parts of a complete investment portfolio:  an integrated and competing set of investment options. This approach  enables the organization to consider the relative cost, benefit, and risk of  newly proposed investments along with those previously funded and to  identify the optimal mix of IT investments to meet its mission, strategies,  and goals. Stages four and five require the use of evaluation techniques to  continuously improve both the investment portfolio and investment  processes to better achieve strategic outcomes. Figure 6 shows the five  maturity stages and the associated critical processes.
As defined by the model, each critical process consists of “core elements”  that indicate whether the implementation and institutionalization of a  process can be effective and repeated. Key practices must be executed to  fulfill the core elements and implement the critical process. The core  elements are as follows:  Organizational commitments—Actions taken by management to  ensure that the critical process is established and will endure. Key  practices typically involve establishing organizational policies and  engaging the sponsorship of senior management.
Prerequisites—Conditions that must exist within an organization to  enable it to successfully implement a critical process. Key practices  typically involve allocating resources, establishing organizational  structures, and providing training.
Activities—Actions that must be taken to implement a critical process.  An activity occurs over time and has recognizable results. Key practices  typically involve establishing procedures, performing and tracking  work, and taking corrective actions as necessary.
section_title: Objective, Scope, and Methodology
paragraphs: The objective of our review was to assess the Postal Service’s capabilities  for effectively managing its IT investments. To determine these capabilities  and the organization’s level of maturity in managing its IT investments, we  applied our ITIM framework and the associated assessment method. As a  part of the ITIM assessment method, we obtained documentary and  testimonial evidence and observed demonstrations of several internal  systems showing the organization’s execution of various key practices. We  evaluated the Postal Service against 14 critical processes in maturity stages  two, three, four, and five. We did not evaluate the Postal Service on key  practices for one critical process in stage three—Authority Alignment of IT  Investment Boards—because major IT capital investments are managed by  the same oversight entities, and we determined that this critical process  was not applicable.
To determine whether the Postal Service had implemented the 14 critical  processes we assessed, we first reviewed documentation relating to the  organization’s IT investment management practices, including written  policies, procedures, and guidance that it had developed, and other forms  of documentation that provided evidence that these practices had been  executed. Documents included the Postal Service’s F-66 manual,  Investment Highlights reports, executive memoranda, program narratives  required for the annual budget formulation, DARs, performance indicators,  and the minutes from meetings of the CIC, the CPC, and the Board of  Governors. We also reviewed a variety of administrative and system  documents from the CTO organization, including evidence of its  formulation process for IT investment proposals and its oversight process  for IT investments.
We interviewed a number of senior officials, including the Chief Financial  Officer (CFO), the CTO, and the CIO. Within the Office of the CFO, we also  spoke with the Manager of Capital and Program Evaluation and the  Manager of Corporate Budget. Within the Office of the CTO, we  interviewed the Manager of IT Value and a representative from the  Enterprise Architecture Office. We also spoke with senior officials from the  functional units, such as the Manager of Logistics Systems, the Manager for  Human Resources Technology Management, and the Manager of Customer  Service Operations.
As part of the analysis, we selected four projects, representing a range of  functional units, stages of development, and sizes, and examined them to  determine the extent to which the Postal Service’s policies and procedures  for IT investment management were being implemented. The projects we  selected for review were (1) Enhanced Security Capability,  (2) Organization Structure, Staffing and Management, (3) Point of Service  ONE, and (4) Surface-Air Management System. Appendix II contains  additional information on each of the projects we reviewed. To perform the  project reviews, we reviewed project management documentation such as  DARs, project management plans, and PTRS reports. To clarify information  in these documents and gain further insight we also interviewed managers  in the sponsoring functional units, project managers, and the members of  the project management teams. The teams included staff who had been  assigned responsibility for project oversight within the Office of the Chief  Technology Officer.
We compared the evidence we collected through document reviews and  interviews to the detailed requirements for each key practice and critical  process that is specified in the ITIM. In accordance with the ITIM  assessment method, we considered a key practice to have been “executed”  when we determined, by team consensus, that sufficient evidence existed  to confirm that the Postal Service was executing the practice in accordance  with stated ITIM criteria. When we determined that there were significant  weaknesses in the Postal Service’s execution of a practice or found  insufficient evidence of its execution, we concluded that the practice was  not executed. Once the key practices were assessed, we determined which  of the 14 critical processes had been implemented. A critical process was  determined to be “implemented” when all related key practices were  designated as executed. Otherwise, according to the ITIM assessment  method, the critical process would not be considered to have been  implemented.
We conducted our work at the Postal Service’s headquarters offices in  Washington, D.C., from October 2001 through July 2002, in accordance with  generally accepted government auditing standards.
section_title: Postal Service Executes Most Key Foundational Practices
paragraphs: At the stage two level of maturity in the IT investment management  framework, an organization has attained repeatable, basic selection and  control processes and successful IT investment control processes at the  project level. In other words, the organization can select projects that meet  established selection criteria and can identify expectation gaps early and  take appropriate steps to address them. According to ITIM, critical  processes at this stage include (1) defining investment review board  operations, (2) developing processes to determine the progress of  individual IT projects, (3) creating an inventory of IT investments,  (4) identifying IT project and systems business needs, and (5) developing a  basic process for selecting new IT proposals. Table 1 shows the purpose of  each critical process in stage two.
The Postal Service is executing nearly 90 percent of the key practices  associated with stage two critical processes. Specifically, the Postal Service  is carrying out all of the key practices associated with selecting proposals  that meet established criteria, aligning IT projects with the organization’s  business needs, and maintaining information on IT projects and systems in  an inventory.
The Postal Service has yet to execute a few key practices associated with  establishing an IT investment management foundation. For example, the  Postal Service does not have guidance defining the overall framework for  its IT investment management process, and policies and procedures for  project oversight are not documented. When the Postal Service implements  the remaining critical processes associated with stage two, it will acquire  the additional key controls needed to fully implement basic control  processes. For example, with an investment management process guide,  the Postal Service will gain assurance that IT investment activities will be  performed in a consistent and cost-effective manner.
Table 2 summarizes the status of the Postal Service’s critical processes for  stage two, showing how many associated key practices it has executed.
The following discussion provides information on steps the Postal Service  has taken to implement each of these critical processes.
subsections: section_title: Boards Are Established but Operating without a Complete Process Guide
paragraphs: The creation of decision-making bodies or boards is central to the IT  investment management process. At the stage two level of maturity,  organizations define one or more boards, provide resources to support  their operations, and appoint members who have expertise in both  operational and technical aspects of proposed investments. Resources  provided to support the operations of IT investment boards typically  include top management’s participation in creating the board(s) and  defining their scope and formal evidence acknowledging management’s  support for board decisions. The boards operate according to a written IT  investment process guide tailored to the organization’s unique  characteristics, thus ensuring that consistent and effective management  practices are implemented across the organization.Once board members  are selected, the organization ensures that they are knowledgeable about  policies and procedures for managing investments. Organizations at the  stage two level of maturity also take steps to ensure that executives and  line managers support and carry out the decisions of the IT investment  board. According to ITIM, an IT investment management process guide  should be a key authoritative document that the organization uses to  initiate and manage IT investment processes and should provide a  comprehensive foundation for policies and procedures developed for all  other related processes.
The Postal Service has executed four of the six key practices for this  critical process by establishing investment boards; providing adequate  resources for related activities; appointing experienced senior-level  executives to the boards; and implementing policies, procedures, and  processes to ensure that executives and line managers support and carry  out decisions made by the boards.
However, the Postal Service has yet to develop a written, organization- specific process guide to direct the operations of the investment boards.  While the F-66 manual provides general guidance on the organization’s  investment management process, it does not constitute an IT investment  process guide because it does not sufficiently define the investment  process. Specifically, the manual does not include information on the roles  of the Establish Team and the CTO Investment Review Board. In addition,  it does not provide detail on the processes followed by other boards  involved in the investment management process (e.g., the CIC and CPC).  Finally, the manual does not identify the manner in which investment  boards’ processes are to be coordinated with other key organizational  plans and processes (such as the budget formulation process). Without an  investment management process guide, the Postal Service lacks the  assurance that IT investment activities will be coordinated and performed  in a consistent and cost-effective manner.
Table 3 shows the rating for each key practice required to implement the  critical process for establishing IT investment board operation at the stage  two level of maturity. Each of the “executed” ratings shown below  represents an instance where, based on the evidence provided by Postal  Service officials, we concluded that a specific key practice was currently  being executed by the organization.
section_title: CTO Organization Oversees IT Investments
paragraphs: Investment boards should effectively oversee IT projects throughout all  life-cycle phases (concept, design, testing, implementation, and  operations/maintenance). At the stage two level of maturity, investment  boards should review each project’s progress toward predefined cost and  schedule expectations, using established criteria and performance  measures, and should take corrective actions to address cost and milestone  variances.
According to ITIM, effective project oversight requires, among other things,  (1) having written polices and procedures for project management;  (2) developing and maintaining an approved management plan for each IT  project; (3) making up-to-date cost and schedule data for each project  available to the oversight boards; (4) reviewing each project’s performance  by regularly comparing actual cost and schedule data with expectations;  (5) ensuring that corrective actions for each under-performing project are  documented, agreed to, implemented, and tracked until the desired  outcome is achieved; and (6) having written policies and procedures for  oversight of IT projects.
The Postal Service has executed most of the key practices in the area of  project oversight. For example, the Postal Service has developed several  policies and procedures for project management, including the Program  Management Process Guidelines, which are high-level project  management guidelines used for all projects; the more-detailed Software  Process Standards and Procedures used by the Postal Service’s business  solution centers to develop and maintain systems; and the recently-issued  Integrated Solutions Methodology, which provides a process for managing  a system’s development throughout the life-cycle phases. In addition, IT  projects have an approved, up-to-date project management plan, in  accordance with project management guidelines. Data on a project’s actual  cost and schedule are provided to the CTO Investment Review Board,which is responsible for overseeing the performance of IT projects, and to  other oversight groups as appropriate. Actual cost and schedule data for  the four projects we reviewed were provided to (1) the CTO Investment  Review Board in the form of PTRS reports, (2) the Board of Governors  through quarterly Investment Highlights reports featuring capital  expenditures and schedule data, and (3) field and headquarters offices  through accounting and management reports featuring data on projects’  actual capital and expense costs.
Finally, the CTO Investment Review Board regularly oversees the  performance of projects by comparing actual cost and schedule data to  expectations and performs special reviews of projects that do not meet  expectations. When these reviews are performed, corrective actions are  defined, documented, agreed to by the program manager and the CTO  Investment Review Board, and tracked until the desired outcome is  achieved. According to the IT program manager for Organization Structure,  Staffing and Management (OSS&M), special meetings were held for this  project to address schedule performance issues. Also, officials from the  CTO organization stated that the office generates reports listing projects  that are not meeting cost, schedule, or customer satisfaction expectations  and brings them to management’s attention so that “special reviews” can be  performed. These reports identify the manager and group responsible for  the project, provide a summary of the problem, the status of the resolution,  and a target date for resolving the problem. The CTO Investment Review  Board tracks action items to resolve the problem until they are completed.
Notwithstanding these strengths, the Postal Service has a few weaknesses  in its oversight of IT projects. First, while the Postal Service has written  policies and procedures addressing how the CTO Investment Review Board  is to oversee IT investments, it does not have any that sufficiently define  the Establish Team’s role in the oversight process. The F-66 manual, for  example, notes that senior management is to continually review the  performance of capital projects and discusses some mechanisms that could  be used for this purpose (e.g., compliance reports). However, it does not  provide specifics on the role of the Establish Team or define processes for  oversight of projects beyond the initial deployment phase. Without  adequate policies and procedures, project oversight may not be performed  consistently. In addition, without these policies and procedures, the Postal  Service lacks the transparency that is helpful in both communicating and  demonstrating how project oversight is performed.
Second, the Postal Service’s investment boards do not adequately oversee  project performance by comparing actual cost data to expectations.  Specifically, while the Establish Team and CTO Investment Review Board  each compare actual cost data to annual budget expectations, the Postal  Service could not demonstrate that these boards compared the data to  original expectations established in the DAR. In addition, while the  Investment Highlights used by executives to monitor project performance  contains schedule information, it does not contain complete information  on actual project costs because it does not report operating expenses.  Without comparisons of complete actual cost data to original expectations,  Postal Service executives may not be able to easily determine whether the  projects they have selected are progressing as planned or whether  corrective actions are needed.
Table 4 shows the rating for each key practice required to implement the  critical process for project oversight at the stage two level of maturity and  summarizes the evidence that supports these ratings.
section_title: IT Project and System Information Is Maintained to Support Project Management
paragraphs: To make good management decisions, an organization must know how  funds are being expended toward acquiring, maintaining, and deploying its  IT investments. Implementing this critical process requires an organization  to identify all projects and systems within the organization and create one  or more repositories or inventories of information about them. This  information is required to track the organization’s IT resources to provide a  basis for analyses showing major cost and management factors and trends.  An IT project and systems inventory can take many forms and does not  have to be centrally located or consolidated. The guiding principles for  developing the inventory are that the information maintained should be  accessible where it is of the most value to investment decision makers and  relevant to the management processes and decisions that are being made.
According to ITIM, organizations at the stage two level of maturity provide  adequate resources for tracking IT projects and systems, designate  responsibility for managing the project and system identification process,  and develop related written policies and procedures. Resources required  for this purpose typically include managerial attention to the process; staff;  supporting tools; an inventory database; inventory reporting, updating and  query tools; and a method for communicating inventory changes to  affected parties. Stage two organizations develop and maintain information  on their IT projects and systems in one or more inventories according to  written procedures, recording changes in data as required, and maintaining  historical records. Access to this information is provided on demand to  decision makers and other affected parties.
The Postal Service has executed all of the key practices for this critical  process. The Service has established a number of repositories of  information on its IT projects and systems in the form of the Enterprise  Information Repository (EIR), and automated systems such as PTRS that  track actual cost, schedule, benefit, and risk associated with the Postal  Service’s IT programs and projects. Members of the Postal Service’s  investment boards have access to the systems used to maintain information  on the organization’s IT programs and projects. Information is maintained  in these databases because they are also used for other purposes. For  example, project managers input up-to-date systems and project status  information to PTRS; the Corporate Planning System (CPS) and PTRS are  updated automatically as financial transactions are processed. Finally, the  Postal Service retains records showing changes in the information  maintained on each IT investment over time and provides these records to  its investment boards.
Table 5 shows the rating for each key practice required to implement the  critical process for IT project and system identification at the stage two  level of maturity and summarizes the evidence that supports these ratings.
section_title: Processes Ensure That IT Investments Support Business Needs and Meet User Needs
paragraphs: Defining business needs for IT projects helps ensure that projects support  the organization’s mission goals and meet users’ needs. This critical  process creates the link between the organization’s business objectives and  its IT management strategy. According to ITIM, effectively identifying  business needs requires, among other things, (1) developing policies and  procedures for identifying business needs and associated users for IT  projects, (2) defining the organization’s business needs or stated mission  goals, (3) defining business needs for projects, and (4) identifying users for  projects who will participate in the project’s development and  implementation.
The Postal Service has executed all of the key practices for this critical  process. The Service’s business needs are defined in a number of  documents, including the organization’s strategic plan and recent  Transformation Plan. Business needs and project users are being  identified and defined in accordance with policies and procedures, and  users are involved in project management throughout a project’s life cycle.  For example, the project management team for Point of Service ONE  conducts interviews to ensure that the system is providing the information  needed for decision making, and staff working in field locations tested  Point of Service ONE software to provide input on modifications required  to support their needs. The business needs and associated users of the four  projects we reviewed were clearly identified and defined in the DARs used  to obtain project approval and in other project justification documentation.  In addition, users of these projects were involved in project development  activities through direct collaboration with CTO staff, user groups, and/or  change control groups. Because the Postal Service is executing all the key  practices associated with identifying business needs, it has increased  confidence that its IT projects will meet both business needs and users’  needs.
Table 6 shows the rating for each key practice required to implement the  critical process for business needs identification at the stage two level of  maturity and summarizes the evidence that supports these ratings.
section_title: Structures Are in Place for Selecting IT Investment Proposals
paragraphs: As a basic step in the direction of implementing mature stage two  processes, an organization must develop a sound process for selecting IT  proposals and projects. Once adequate resources are provided and an  official is designated with responsibility for selecting proposals, stage two  organizations establish a structured selection process. Resources required  for selecting proposals typically include managerial time and attention,  staff, and supporting tools and methodologies. Executives analyze and  prioritize the proposals and make related funding decisions according to an  established, structured process.
The Postal Service has executed all of the key practices pertaining to  selecting IT proposals: executives and managers follow established  selection processes, the CFO has been designated with responsibility for  the organization’s budget formulation process, adequate resources are  being provided to support related activities, a structured process is in place  for developing new IT proposals, and executives analyze and prioritize the  proposals according to established selection criteria.
Postal Service executives and managers follow established processes for  selecting IT investments. Specifically, functional units, the Finance  Department’s CAPE group, the Establish Team, and the organization’s  enterprise-level investment boards all follow established processes for  proposing, prioritizing, and selecting IT investments. Officials reported that  the Establish Team operates in accordance with established management  cycle processes supported by the organization’s CPS and that these  processes, although not documented, are generally understood by  members of the team. Finally, the CTO organization has developed  selection criteria for that unit’s proposed IT investments that are  incorporated in its new Business Case System (BCS).
Table 7 shows the rating for each key practice required to implement the  critical process for proposal selection at the stage two level of maturity and  summarizes the evidence that supports these ratings.
section_title: Postal Service Shows Mixed Progress in Managing Its IT Investments as a Portfolio
paragraphs: An IT investment portfolio is an integrated, enterprisewide collection of  investments that are assessed and managed collectively based on common  criteria. Managing investments within the context of such a portfolio is a  conscious, continuous, and proactive approach to expending limited  resources on an organization’s competing initiatives in light of the relative  benefits expected from these investments. Taking an enterprisewide  perspective enables an organization to consider its investments  comprehensively so that the collective investments optimally address its  mission, strategic goals, and objectives. This portfolio approach also allows  an organization to determine priorities and make decisions about which  projects to fund based on analyses of the relative organizational value and  risks of all projects, including projects that are proposed, under  development, and in operation.
According to ITIM, critical processes performed by organizations at the  stage three level of maturity include (1) defining portfolio selection criteria,  (2) engaging in project-level investment analysis, (3) developing a complete  portfolio based on the investment analysis, and (4) maintaining oversight  over the investment performance of the portfolio. In addition,  organizations with more than one board that selects IT projects for funding  must align the authority of their IT investment boards. Although authority  alignment is a critical process for the stage three level of maturity, we did  not assess it in this study, because the Postal Service has a single set of  organizationwide investment processes that apply to IT investments. Table  8 shows the purpose of each critical process in stage three.
The Postal Service has executed many of the key practices associated with  stage three critical processes. For example, the organization’s portfolio  selection criteria are distributed throughout the organization, and they are  reviewed and modified as appropriate. In addition, executives examine the  mix of proposals and investments across portfolio categories in making  funding selections. However, many key practices still need to be executed  before the Postal Service can effectively manage its IT investments from a  portfolio perspective. For example, the Postal Service has not defined the  policies and procedures for any of the stage three critical processes. In  addition, the Service has not developed portfolio selection criteria that  adequately address cost, benefit, schedule, and risk. Until the Service fully  implements critical processes associated with managing investments as a  complete portfolio, it will not have ready access to the data needed to make  informed decisions about competing investments.
Table 9 summarizes the status of the Postal Service’s stage three critical  processes, showing how many associated key practices it has executed.
The following discussion provides information on the steps the Postal  Service has taken toward implementing each of the critical processes.
subsections: section_title: Portfolio Selection Criteria Are Defined, but Do Not Adequately Address All Factors
paragraphs: To manage IT investments effectively, an organization needs to establish  rules or “selection criteria” for determining how to allocate scarce funding  to existing and proposed investments. Thus, the process of developing an  IT investment portfolio necessarily involves defining appropriate cost,  benefit, schedule, and risk criteria for evaluating individual proposals for  investments. To ensure that the organization’s strategic goals, objectives,  and mission will be satisfied by the investments, the criteria should have an  enterprisewide focus that reflects these strategic goals. Further, if an  organization’s mission or business needs and strategies change, criteria for  selecting investments should be reexamined at the portfolio level. Portfolio  selection criteria should be disseminated throughout the organization to  ensure that decisions concerning investments are made in a consistent  manner and that this critical process is institutionalized. To achieve this  result, project managers, organizational planners, and other decision  makers should receive information on the organization’s selection criteria  and address the criteria in IT proposals and business cases, project  oversight activities, and strategic and business planning processes.  Resources required for this critical process typically include the time and  attention of executives involved in the process, adequate staff, and  supporting tools.
The Postal Service has executed four of the six key practices for this  critical process. First, adequate resources are available to conduct  portfolio selection criteria definition activities. Second, several working  groups, including the Establish Team, are tasked with creating and  modifying portfolio selection criteria. Third, portfolio selection criteria in  the form of performance indicators and targets and program narratives that  are required for budget formulation are distributed throughout the  organization. Fourth, the Establish Team performs periodic reviews of the  portfolio selection criteria and, in doing so, considers the organization’s  current strategic goals and objectives, changing the criteria from year to  year as required by current circumstances and priorities.
Nonetheless, the Postal Service has yet to develop written guidance  establishing procedures to be followed in creating, modifying, and using  criteria for selecting a portfolio. Postal Service officials use annual  performance plans, performance indicators and targets, and program  narrative requirements as portfolio selection criteria. While these criteria  are based on the Postal Service’s mission, goals, strategies, and priorities,  they are not adequate because they do not address cost, benefit, schedule,  and risk considerations in a manner that (1) provides sufficient and  meaningful cost, benefit, schedule, and risk information to effectively  assess investments and (2) would allow the Service to compare them  against one another, prioritize them, and select those that best meet its  needs and priorities. For example, program narratives do not include  complete cost information. While expected capital costs are reported for  the next 6 years, expense is only reported through the end of the current  fiscal year. Further, the criteria do not include a weighting schema or other  method that would allow the Establish Team to compare the risk-adjusted  returns of competing investments. The CTO organization uses such criteria  to prioritize investments and assist in making selection decisions. Without  portfolio selection criteria that adequately address cost, benefit, schedule,  and risk considerations, Postal Service officials have less assurance that  they are selecting the mix of investments that best meets the organization’s  needs and priorities.
Table 10 shows the rating for each key practice required to implement the  critical process for defining proposal selection criteria at the stage three  level of maturity and summarizes the evidence that supports these ratings.
section_title: IT Investments Are Not Consistently Analyzed and Prioritized within the Context of a Portfolio
paragraphs: This critical process ensures that all IT investments are consistently  analyzed and prioritized according to the organization’s portfolio selection  criteria, which should include cost, benefit, schedule, and risk  considerations. According to ITIM, effective investment analysis requires,  among other things, that (1) portfolio selection criteria have been  developed; (2) cost, benefit, schedule, and risk data are assessed and  validated for each investment; (3) the investment review board compares  each investment against the organization’s portfolio selection criteria; and  (4) the investment review board creates a ranked list of investments using  the portfolio selection criteria.
The Postal Service has executed two of the key practices in this area. First,  the Postal Service has adequate resources for analyzing investments,  including CAPE and other dedicated staff. Second, the Postal Service  ensures that cost, benefit, schedule, and risk data concerning IT  investments are validated. The Service does this in two particular  instances: (1) during the development of the DAR, the document for  approving capital projects, there is a validation step in which Finance  Department staff independently verify the accuracy and integrity of the  data presented and a validation memo is signed by the Controller to  confirm that the data are correct; (2) as part of the annual budget  formulation process, the data submitted on the various budget proposals  are reviewed, and thus validated, by various levels of management up to the  senior vice president of the functional unit sponsoring a proposal.
Nevertheless, the Postal Service has a number of weaknesses in the way it  analyzes investments for portfolio management. First, it does not have  policies and procedures that sufficiently address this critical process. Its  F-66 manual includes some procedures for analyzing proposed  investments; however, it does not specify an approach for analyzing  existing investments to make portfolio selection decisions. Nor does it  describe a process to establish portfolio selection criteria that adequately  incorporate cost, benefit, schedule, and risk considerations. In addition, it  does not address capital projects that have been deployed for more than 18  months or ongoing infrastructure-type projects.
Second, while investments are analyzed by executives during the approval  process, through the review of quarterly status reports, and during the  annual budget formulation activities, these investments are not assessed  against portfolio selection criteria that adequately consider cost, benefit,  schedule, and risk factors. The F-66 manual does not explicitly require the  preparation of a risk assessment when the DAR is developed for a new  investment. Further, when the Establish Team reviews budget documents  as part of the annual budget formulation process, these documents do not  provide sufficient information on cost, benefit, and risk to determine  whether investments are progressing according to the approved DAR  parameters.
Table 11 shows the rating for each key practice required to implement the  critical process for analyzing investments at the stage three level of  maturity and summarizes the evidence that supports these ratings.
section_title: An IT Investment Portfolio Is Developed, but Project Expectations Are Not Routinely Revised
paragraphs: At the stage three level of maturity, organizations design processes for  developing an IT portfolio and develop written policies and procedures to  ensure that projects are selected that best fit their strategic business  direction, needs, and priorities. Each organization has practical limits on  funding, the risks it is willing to take, and the length of time for which it will  incur costs on a given investment before benefits are realized. To address  these limits, stage three organizations group existing and proposed IT  investments into predefined logical categories, for example, by cost or by  type of investment (i.e., facilities or equipment). Once this is accomplished,  organizations can compare investments and proposals within and across  the portfolio categories and select the best overall portfolio for funding.
According to ITIM, the portfolio development process cannot be performed  well unless certain conditions are first satisfied, including (1) providing  adequate resources for a portfolio development process; (2) appointing to  IT investment boards people who exhibit core competencies in developing  portfolios; (3) analyzing individual IT investments, including validating  associated cost, benefit, schedule, and risk data; and (4) defining  investment categories. Organizations should also create written policies  and procedures for establishing and maintaining the portfolio development  process. Assuming that this foundation is in place, the IT investment  boards of stage three organizations assign each investment to a portfolio  category, examine the mix of existing and proposed investments across  these categories, and make selections for funding. Each IT investment  board also establishes annual cost, benefit, schedule, and risk expectations  for individual IT projects and gathers and validates data on actual  performance. A repository of information on developing portfolios is  established, updated, and maintained. Resources required for this critical  process typically include staff, supporting tools for developing portfolios,  and managerial time and attention to portfolio development.
The Postal Service has executed six of the nine key practices for this  critical process by providing adequate resources to implement this critical  process; assigning competent managers to the board responsible for the  portfolio development process; developing common portfolio categories;  assigning IT programs and projects to portfolio categories on the basis of  established criteria; examining the mix of proposals and investments  across the common portfolio categories and making selection decisions for  funding; and establishing, updating, and maintaining repositories of  portfolio information.
Postal Service officials reported that adequate management time and staff  resources are available for this critical process. In addition, several systems  are in use that support portfolio development activities, including PTRS,  BCS, and CPS. Postal Service officials stated that the organization provides  training in the use of these systems. Moreover, members of the Postal  Service’s enterprise-level investment boards are senior-level executives  who have had many years of experience in the organization and in working  with the IT investment management process. The Postal Service also has  defined common IT investment portfolio categories for the organization.  The Postal Service’s IT investments are considered to relate either to  corporatewide or functional unit activities and are further classified by  funding type (capital or expense) and investment type (facilities,  equipment, field, or other), as provided for in the organization’s F-66  manual and budget instructions.
Postal Service programs and projects are now being assigned to portfolio  categories based on the criteria described above. Further, the Establish  Team examines the organization’s entire portfolio of IT investments  annually and then selects programs and projects for funding. The Postal  Service collects and stores information relating to the portfolio  development process in a variety of forms ranging from IT project and  systems inventories and finance/budget and corporate planning systems to  manual backup books maintained by the Finance Department.
Even though these important steps in stage three portfolio development  have been taken, some weaknesses remain. The Postal Service has yet to  develop written policies and procedures for establishing and maintaining  portfolio information on its IT investments. The Postal Service has defined  investment categories in its F-66 manual but has not developed written  policies and procedures for establishing and maintaining portfolio  information on IT investments. Moreover, even though the CTO  organization monitors data on the performance of IT projects, the Establish  Team does not perform complete analyses of the performance of individual  investments or establish cost, benefit, schedule, and risk expectations for  each investment annually.
While the Establish Team reviews investments each year from a strategic  planning and funding perspective, neither the analyses it performs nor the  Investment Highlights reports on the projects provided to the Board of  Governors adequately consider actual benefit and risk or contain complete  information on cost. For example, the business case for the Surface-Air  Management System includes information on over a dozen different types  of qualitative benefits expected to be obtained by investing in that project.  However, Investment Highlights reports provided to the Board of  Governors only include information on the number of installations  completed to date. In addition, although information on projects’ capital  costs is included in Investment Highlights, information on operating  expenses is not. As a result, information on these aspects of project  performance is not routinely provided to the Board of Governors. Without  complete cost, benefit, schedule, and risk data, Postal Service executives  do not have the information needed to analyze and compare all investments  and select those that best fit with the strategic business direction, needs,  and priorities, of the organization.
Table 12 shows the rating for each key practice required to implement the  critical process for portfolio development at the stage three level of  maturity and summarizes the evidence that supports these ratings.
section_title: Portfolio Performance Oversight Is Performed, but without Comprehensive Guidance
paragraphs: The purpose of this critical process is to ensure that each IT investment  achieves its cost, benefit, schedule, and risk expectations. It builds on the  critical process for IT project oversight at stage two by adding elements of  benefit measurement and risk management to an organization’s investment  control capability. Executive-level oversight of project-level risk and  benefit management activities provides the organization with increased  assurance that each investment will achieve the desired cost, benefit,  schedule, and risk expectations.
According to ITIM, effective oversight of portfolio performance requires,  among other things, that the investment board (1) has access to up-to-date  cost, benefit, schedule, and risk data; (2) monitors the performance of each  investment in its portfolio by comparing actual project-level cost, benefit,  schedule, and risk data to the predefined expectations for the project; and  (3) corrects poorly performing projects.
The Postal Service is executing six of the nine key practices for this critical  process by providing adequate resources for monitoring and controlling IT  project performance and giving investment boards access to data on actual  and expected cost, benefit, schedule, and risk that are maintained in the  organization’s IT project and system inventory. In addition, the CTO  Investment Review Board provides oversight for all IT projects by  monitoring these data and providing feedback on performance to  sponsoring organizations. These oversight activities include working with  IT project management teams to identify and address any development and  deployment issues that may arise.
Despite these strengths, however, the Postal Service has yet to develop  policies and procedures that address performance oversight from a  portfolio perspective. Moreover, while expectations are established in  DARs or business cases that include cost, benefit, schedule, and risk, and  the CTO organization monitors actual performance results, the Postal  Service has not established a mechanism for revising expected benefit and  risk expectations after its boards approve the investments or for notifying  the Establish Team when an investment has not met cost, benefit, schedule,  and risk expectations. Until the Postal Service executes all key practices  associated with this critical process, senior executives will be less likely to  determine whether the investments they have selected are delivering  mission value at the expected cost and risk.
Table 13 shows the rating for each key practice required to implement the  critical process for portfolio performance oversight at the stage three level  of maturity and summarizes the evidence that supports these ratings.
section_title: Postal Service Has Yet to Implement Processes to Better Meet Strategic Goals
paragraphs: Organizations that achieve the stage four level of maturity evaluate their IT  investment processes and portfolios to identify opportunities for  improvement. At the same time, these organizations are able to maintain  the mature control and selection processes that are characteristic of stage  three in the ITIM model. A key tool for accomplishing this critical process  is the post-implementation review, in which outcomes of individual IT  investments are compared to the organization’s plans and expectations.  This review typically results in identifying lessons learned from the  investment experience that are used by the organization to improve its  understanding of the key variables in the investment’s business case.  Analyzing a number of post-implementation reviews can also provide  insights into the organization’s overall IT investment management process.  This analysis is facilitated by classifying individual investments into logical  categories and using the lessons learned to fine-tune associated processes,  as well as aspects of the portfolio. In addition, at stage four maturity,  organizations are capable of systematically planning for and implementing  decisions to discontinue or deselect obsolete, high-cost, and low-value IT  investments and planning for successor investments that better support  strategic goals and business needs.
Organizations acquire stage five capabilities when they create  opportunities to shape strategic outcomes by learning from other  organizations and continuously improving the manner in which they use IT  to support and improve business outcomes. Thus, organizations at the  stage five level of maturity benchmark their IT investment processes  relative to other best-in-class organizations and conduct proactive  monitoring for breakthrough information technologies that will allow them  to significantly improve business performance. Table 14 shows the purpose  of each critical process in stages four and five.
The Postal Service is executing five of the thirty-four key practices  associated with the five critical processes in stages four and five. For  example, it has policies and guidance for conducting post-implementation  reviews and provides training to individuals involved in these activities.  The Postal Service also provides resources for identifying opportunities for  IT-driven strategic business change. However, it does not regularly capture  lessons learned from post-implementation reviews, the performance of its  portfolio, or benchmarking in order to improve its investment processes. In  addition, it does not actively manage the succession of its IT systems or  investments. Until it implements stage four and five critical processes, the  Postal Service will not be positioned to effectively improve its IT  investment management processes and successfully leverage IT to improve  business outcomes.
Table 15 summarizes the status of the Postal Service’s critical processes for  stages four and five and shows how many associated key practices it has  executed.
The following discussion provides information on steps the Postal Service  has taken to implement each of the critical processes.
subsections: section_title: Policies and Procedures Are Defined, but Post- Implementation Review Process Is Not Institutionalized
paragraphs: Post-implementation reviews are performed (1) to examine differences  between estimated and actual investment costs and benefits and possible  ramifications for unplanned funding needs in the future and (2) to extract  lessons learned about the investment selection and control processes that  can be used as the basis for management improvements. Investments that  have completed development and those that were terminated before  completion should be reviewed promptly to identify potential management  and process improvements. According to ITIM, this critical process  involves identifying the projects to be reviewed; initiating reviews and  developing policies and procedures for conducting the reviews; and  ensuring that quantitative and qualitative data are collected, evaluated for  reliability, and analyzed during the course of the reviews.
The Postal Service has executed three of the six key practices required to  implement the critical process for post-implementation reviews. First, the  CTO organization and Finance Department have each developed policies  and procedures for performing post-implementation reviews. These  include the CTO organization’s Program Management Process Guidelines  and the Finance Department’s National Cost Study Process. Second,  according to Postal Service officials, the Service has adequate resources to  perform review activities. Third, Postal Service staff are trained in  conducting post-implementation reviews.
The Postal Service, however, has several weaknesses in this critical  process. First, no investment board has been assigned responsibility for  (1) identifying projects for which post-implementation reviews are to be  conducted and (2) ensuring that post-implementation reviews are initiated.  Second, the Postal Service has no institutionalized process for routinely  (1) identifying projects for which post-implementation reviews are to be  conducted, (2) collecting quantitative and qualitative data while performing  post-implementation reviews, and (3) developing lessons learned and  improvement recommendations about the investment process and  capturing them in a written product or knowledge base. This is evidenced  by the fact that, while the Finance Department’s Program Performance  Group is responsible for conducting post-implementation cost studies, only  three of them have been performed since 1990. Until the Postal Service  implements an institutionalized process for routinely performing post- implementation reviews, senior executives will lack key information  needed to improve the performance of the IT investment portfolio as well  as the investment management process.
Table 16 shows the rating for each key practice required to implement the  critical process for post-implementation reviews at the stage four level of  maturity and summarizes the evidence that supports these ratings.
section_title: IT Investments Are Not Evaluated from the Perspective of Portfolio Performance
paragraphs: Stage four evaluations of portfolio performance enable organizations to  determine what contribution their collected pools of IT investments are  making to mission goals and needs. Evaluations of this sort are similar to  post-implementation reviews involving individual projects, but different in  that they apply to entire IT investment portfolios. This critical process  seeks to determine how well IT investments are helping to achieve the  strategic needs of the enterprise, satisfying the needs of individual units  and users, and improving business performance through IT. Performance  information for an organization’s entire portfolio of investments has to be  compiled and analyzed and trends examined. Developing baseline  performance data is critical to making this a meaningful exercise.  According to ITIM, the process of addressing problems and opportunities  for improving the investment process and the investment portfolio usually  involves developing written policies and procedures for the investment  management process, creating recommendations for the IT investment  board, documenting the decision criteria used to measure portfolio  performance, deciding whether or not to implement each recommendation,  and tracking the progress made. Resources required for this critical  process typically include staff support, methods and tools to aid the teams  conducting post-implementation reviews, and current and historical  portfolio data.
To advance to the stage four level of maturity, an organization must first  ensure that all of the prerequisites, commitments, and activities that are  characteristic of levels two and three have been put into place. The next  step is to develop written policies and procedures for evaluating and  improving its IT investment portfolio that include defining requirements for  measuring performance data. Cost, benefit, schedule, and risk must all be  fully considered to enable an organization to construct a picture of the  overall performance of its IT investment portfolio.
The Postal Service is not executing any of the six key practices for this  critical process. First, while the Establish Team reviews existing and  proposed IT investments each year as a part of the organization’s budget  formulation process, no evaluations are being done that are designed to  identify opportunities for improving portfolio performance. Also lacking  are written policies and procedures that define the organization’s key  measures and the methods used to assess portfolio performance,  evaluation methods, reporting requirements, and other applicable policies  and procedures. Because the Postal Service has not collected data for this  critical process, including baseline performance information on its IT  portfolio, it is more difficult to perform evaluations that could result in  recommendations for improving its process for selecting a portfolio.
Table 17 shows the rating for each key practice required to implement the  critical process for evaluating and improving the performance of the  portfolio at the stage four level of maturity and summarizes the evidence  that supports these ratings.
section_title: Process for Managing Succession of Systems and Technology Is Not Established
paragraphs: Managing the succession of systems and technology entails periodically  evaluating IT investments to determine whether they should be retained,  modified, replaced, or otherwise disposed of. According to ITIM, system  and technology succession management includes (1) defining policies and  procedures for managing the IT succession process, (2) assigning  responsibility for the succession management process, (3) developing  criteria for identifying IT investments that may meet succession status, and  (4) periodically analyzing IT investments to determine whether they are  ready for succession. This critical process enables the organization to  recognize low-value or high-cost IT investments and augments the routine  replacement of systems at the end of their useful lives. This critical process  supports the development of a forward-looking, solution-oriented view of  IT investments that anticipates future resource requirements and allows  the organization to plan appropriately.
The Postal Service has not performed any of the nine key practices  required to implement this critical process. For example, while the Postal  Service’s project management guidelines define procedures for retiring  investments, they do not describe how to review systems regularly to  identify candidates for retirement. According to officials from the CTO  organization, decisions on succession management are usually made  between business unit managers and CTO office staff (e.g., portfolio  managers), but no individual or group has been assigned responsibility for  managing the succession process from an enterprise perspective, which  would allow the Postal Service to better anticipate future resource  requirements. Finally, the Postal Service has neither defined the criteria for  identifying investments that may meet succession status nor taken steps to  regularly analyze IT investments for possible succession.
According to CTO organization officials, the Postal Service has retired or  replaced roughly 250 systems since 1998. However, this was not done  within the structure of an institutionalized succession management  process. Postal Service officials have stated that IT investments are  reviewed, for example, during the annual budget formulation process to  analyze them for possible succession. However, without an  institutionalized process for succession management, the Postal Service  may not be able to identify those IT investments that are eligible for  succession in enough time to minimize the effect of the transition on their  successors.
Table 18 shows the rating for each key practice required to implement the  critical process for managing the succession of systems and technology at  the stage four level of maturity and summarizes the evidence that supports  these ratings.
section_title: Activities for Benchmarking the Investment Process Are Not Institutionalized
paragraphs: In stages two through four, organizations ensure that sound investments  are selected, controlled, and evaluated within the context of the IT  investment management process and the enterprisewide portfolio. In the  stage five level of maturity, a shift in orientation occurs as organizations  evolve toward using information on leading technologies to identify  opportunities for business change and to implement changes in their  overall business process. Benchmarking the investment process allows  organizations to identify opportunities for improvement and to implement  measurable improvements in their IT investment management processes so  that these processes meet or exceed those used by best-in-class  organizations. Improvements can include using innovative investment  oversight tools and techniques or improving the feedback mechanisms for  lessons learned. According to ITIM, investment process benchmarking  includes (1) defining policies and procedures for using benchmarking to  improve the IT investment management process, (2) collecting baseline  data on the organization’s current IT investment management process,  (3) identifying and benchmarking external comparable best-in-class  processes for IT investment management, and (4)  improving the  organization’s investment management processes.
The Postal Service has not fully executed any of the seven key practices  required to implement this critical process. While there have been some  efforts to identify best practices from best-in-class organizations and  incorporate these practices into the Postal Service’s IT investment  management processes (such as the CTO organization’s use of lessons  learned in benchmarking to develop the BCS), the Postal Service has  not defined policies and procedures for improving the IT investment  management process using benchmarking. It also does not have any  institutionalized processes to routinely (1) collect baseline data on the  organization’s current IT investment management process, (2) identify and  benchmark external best-in-class processes for IT investment management  in comparable organizations, or (3) actually improve the organization’s  investment management processes. Without these processes, the Postal  Service is less likely to learn from best-in-class organizations, which will  hinder any concerted effort to improve its IT investment management  processes.
Table 19 shows the rating for each key practice required to implement the  critical process for investment process benchmarking at the stage five level  of maturity and summarizes the evidence that supports these ratings.
section_title: Potential Impacts of Leading Technologies Are Not Routinely Considered in Strategic Planning Efforts
paragraphs: Information technologies can provide opportunities for an organization to  move dramatically in new directions to meet its goals. Thus, once an  organization finds it can competently manage its enterprisewide portfolio  of investments, it should actively seek out opportunities to use alternative  technologies.
According to ITIM, stage five organizations provide adequate resources for  conducting IT-driven activities that can result in strategic business change.  These may include developing an advanced IT laboratory, test center, or  library; conducting technical research; employing internal staff and  external experts or reviewers; and obtaining supporting tools. Stage five  organizations also develop applicable written policies and procedures and  designate an official to oversee their implementation. The central focus of  these activities is to follow technological events and to identify and  evaluate technologies that appear to offer strategic business-changing  capabilities. Once a conclusion has been reached that specific technology  offers the organization significant opportunities, senior managers plan for  and implement changes to the organization’s business processes.  Organizations at a stage five level of maturity may create an advanced  technology group, a cross-departmental group of experts, or technology  centers of excellence. Finally, to strengthen management on these types of  activities, mature organizations designate responsibility for this key  practice to a single senior-level manager.
The Postal Service has executed two of the six key practices required to  implement this critical process by designating responsibility to specific  organizational units to support activities aimed at IT-driven strategic  business change and by providing a range of related resources. However,  steps have yet to be taken to execute the remaining key practices, including  creating and maintaining a knowledge base of state-of-the-technology IT  products and processes; actively identifying technologies with business- changing capabilities; and planning and implementing strategic changes to  business processes on the basis of the capabilities of these technologies.
The Postal Service has assigned responsibilities to several units that could  leverage IT to implement strategic business change, including its  Transformation Plan Office, Office for Strategic Planning, and the CTO  organization. Also, within the CTO organization, the Information  Technology unit has established the positions of Enterprise Architect and  Manager of Technology Standards. To ensure standardization, the Postal  Service has also developed the IT Infrastructure Toolkit process and  established the Enterprise Architecture Councils and the Management  Steering Committee.
The Postal Service is also providing a range of resources that could be used  to support the critical process of IT-driven strategic business change. The  Service is funding a testing laboratory and has established Integrated  Business Solutions Systems Centers and developed an IT Toolkit system  and associated processes. The IT Toolkit system serves as a repository of  information on technologies and application systems that have been  approved for use within the Postal Service.
In addition, the Postal Service’s CTO organization is taking several steps to  initiate changes to the business process based on currently available state- of-the-practice IT approaches. First, the CTO has developed a plan for a  corporate database called the Corporate Data Mart, which could serve as a  repository of data from 35 separate Postal Service systems. According to  Postal Service officials, the CTO organization is working with each  functional unit to determine which legacy systems will transition to the  data mart and plans to incorporate future systems in the data mart. This  transition may eliminate costly legacy systems or avoid the investment cost  to replace them. The CTO organization is sponsoring the Advanced  Computing Environment initiative to transition to a less costly distributed  computing environment. According to officials, under this approach,  activities will be standardized, centralized, and reengineered such that the  costs per Postal Service user will be reduced.
These accomplishments can be helpful to the Postal Service, particularly in  light of its financial difficulties and the need to identify new, more cost- effective ways of accomplishing its mission. By continuing to foster a more  coordinated approach to using IT investments to achieve its business goals,  using resources from across the organization, and disseminating  information that is gathered more broadly, the Postal Service can more  effectively capitalize on opportunities uncovered by efforts already  underway.
Table 20 shows the rating for each key practice required to implement the  critical process for IT-driven strategic business change at the stage five  level of maturity and summarizes the evidence that supports these ratings.
section_title: Conclusions, Recommendations, and Agency Comments
subsections: section_title: Conclusions
paragraphs: Information technology provides key core operational capabilities that the  Postal Service must rely on to achieve its mission. Only by effectively and  efficiently managing its IT resources can the Postal Service gain  opportunities to further leverage its IT investments and make better  allocation decisions among many investment alternatives.
The Postal Service has in place most of the foundational practices required  to ensure that IT investments are being selected and monitored to support  its overall objectives. A comprehensive process guide for investment  management and written policies and procedures for management  oversight of investments will allow the Postal Service to better coordinate  its IT investment activities and ensure that they are performed consistently.  Once the Service has fully implemented all the critical processes for stage  two, it will have the controls necessary to allow it to effectively manage its  IT investments.
The Postal Service shows mixed progress in managing its IT investments as  a portfolio. The Service performs many portfolio development and  oversight activities. However it lacks policies and procedures for managing  its portfolio. It has not defined criteria that allow it to effectively analyze,  prioritize, and select its investments from a portfolio perspective. In  addition, the Postal Service’s reporting of performance data is largely  limited to capital projects, which are a smaller portion of its portfolio than  are operating expenses. Until the Service fully implements critical  processes associated with managing investments as a complete portfolio, it  will not have ready access to the data it needs to make informed decisions  about competing investments.
The ability of the Postal Service to continue to improve its investment  management process is contingent on its ability to learn from its current  practices and investments and from other organizations. The Service  currently has no institutionalized processes to learn from its own  experience and from other organizations. Such processes can contribute to  the long-term success of the Postal Service’s IT portfolio and support its  mission.
section_title: Recommendations for Executive Action
paragraphs: To strengthen the Postal Service’s capabilities for investment management  and address the weaknesses discussed in this report, we recommend that  the Postmaster General develop a plan that initially focuses on correcting  the weaknesses in critical processes associated with maturity stages two  and three before addressing the weaknesses at maturity stages four and  five, because critical processes at the lower stages provide the foundation  for building those at higher maturity stages. The plan should be developed  within 6 months. At a minimum, the plan should specify an approach to  develop comprehensive guidance that defines and describes the  complete investment management process, unifies existing processes  enterprisewide, and reflects changes in processes as they occur;  develop additional process guidance, as needed, to completely define  the operations and decision-making processes of investment boards and  other management entities involved in managing IT investments;  ensure that cost, benefit, schedule, and risk expectations are set and  approved in the original business case for each investment; that  accurate and complete actual cost, benefit, schedule, and risk data are  tracked against these expectations; and that status information on these  four criteria is periodically reported to executive-level investment  boards; and  establish a structured, transparent, and documented portfolio selection  process that assesses, prioritizes, selects, and funds investments  according to established portfolio selection criteria, including explicit  cost, benefit, schedule, and risk criteria.
The Postmaster General should ensure that the plan specifies measurable  goals and time frames, prioritizes initiatives, designates a senior manager  responsible and accountable for directing and controlling the  improvements, and establishes review milestones. After addressing the  stage two and three processes, the Postal Service should create processes  required for stages four and five that, at a minimum  ensure that guidance for conducting post-implementation reviews is  complete, including criteria for selecting systems for review, and that  post-implementation reviews are conducted on all appropriate systems;  establish a process for evaluating and improving portfolio performance;  establish a process for managing the succession of systems and  technology;  establish a process to benchmark the investment processes of leading  organizations to identify opportunities for improvement; and  establish a process to employ IT investments strategically to improve  business outcomes.
section_title: Agency Comments and Our Evaluation
paragraphs: The Postal Service’s Chief Financial Officer provided written comments on  a draft of this report (reprinted in app. III). In these comments, the Postal  Service stated that the report offered an opportunity to consider changes  and improvements in its IT investment management processes. The Service  added that it would carefully evaluate each of the report’s  recommendations to determine the necessary actions for adopting and  integrating key practices outlined in the GAO ITIM model that are  appropriate for the Postal Service.
The Postal Service also identified key points where it stated that it differs  from GAO’s IT investment management framework. The Postal Service also  explained that it uses a hierarchy of delegations to select and oversee its  investments, from the Board of Governors through the lowest level of  management, to ensure that senior management can concentrate on  strategic issues and the most significant projects. We did observe this  structured approach to the selection and oversight process and have  recognized it in our report.
In succession planning, the Postal Service stated that it uses an  institutionalized portfolio approach to address the succession of its IT  hardware, software, and systems. According to the Postal Service, this  approach enables senior management to determine strategically driven  solutions based on priorities, lessons learned, available technology, best  practices, affordability, risk assessments, and business needs. Our  guidance suggests that, while each of these aspects may be appropriate as  part of a succession management process, effective succession  management entails regularly reviewing the performance of existing  systems against established criteria. Such a process allows an organization  to identify systems that should be retained, modified, replaced, or  otherwise disposed of in a timely manner. However, as we stated in our  report, the Postal Service does not have such a process.
The Postal Service provided comments pertaining to post-implementation  reviews that describe cost studies, the budget process, and the activities of  the Office of Inspector General (OIG) as satisfying this critical process. We  disagree with the Postal Service in this matter. While guidance for cost  studies does exist, the Service provided evidence of only three post- implementation cost studies having been conducted since 1990. The Postal  Service’s budget process does not satisfactorily address this critical  process. Specifically, the budget process does not capture lessons learned  and disseminate them to other projects and work processes in order to  improve them, and this is a major objective of post-implementation  reviews. Finally, while OIG does conduct evaluations from which lessons  learned may be drawn and used to improve other projects and work  processes, OIG evaluations are not part of the regular systems life cycle.