DOD Business Systems Modernization: Long-Standing Weaknesses in Enterprise Architecture Development Need to Be Addressed
Released: 2005-07-22T00:00:00

Letter

Background

DOD is a massive and complex organization. In fiscal year 2004, the department reported that its operations involved $1.2 trillion in assets, $1.7 trillion in liabilities, over 3.3 million military and civilian personnel, and over $605 billion in net cost of operations. For fiscal year 2005, the department received appropriations of about $417 billion. Moreover, execution of its operations spans a wide range of defense organizations, including the military services and their respective major commands and functional activities, numerous large defense agencies and field activities, and various combatant and joint operational commands, which are responsible for military operations for specific geographic regions or theaters of operations.

In executing these military operations, the department performs an assortment of interrelated and interdependent business functions, including logistics management, procurement, health care management, and financial management. DOD reports that, in order to support these business functions, it currently relies on about 4,200 systems—including accounting, acquisition, finance, logistics, and personnel. As we have previously reported, this systems environment is overly complex and error prone and is characterized by (1) little standardization across the department, (2) multiple systems performing the same tasks, (3) the same data stored in multiple systems, and (4) manual data entry into multiple systems. These problems continue despite the department’s spending billions of dollars annually to operate, maintain, and modernize its business systems. DOD received approximately $13.3 billion for fiscal year 2005 to operate, maintain, and modernize this environment. In addition, our reports continue to show that the department’s stovepiped and duplicative systems contribute to fraud, waste, and abuse. Of the 25 areas on GAO’s governmentwide “high-risk” list, 8 are DOD program areas, and the department shares responsibility for 6 other high-risk areas that are governmentwide in scope.

Because of the department’s size and complexity, modernizing its business systems is a huge management challenge that we first designated as one of the department’s high-risk areas in 1995, and we continue to do so today. To help meet this challenge, DOD established its business systems modernization program in 2001. As we testified in 2003, one of the seven key elements that are necessary to successfully execute this modernization program is to establish and implement an enterprise architecture. Subsequently, in its Fiscal Year 2004 Performance and Accountability Report, DOD acknowledged that deficiencies in its systems and business processes hindered the department’s ability to collect and report financial and performance information that was accurate, reliable, and timely. The DOD report noted that to address its systemic problems and assist in the modernization of its business operations, the department had undertaken the development and implementation of a BEA.

Enterprise Architecture Is Critical to Achieving Successful Modernization

Effective use of an enterprise architecture, or a modernization blueprint, is a trademark of successful public and private organizations. For more than a decade, we have promoted the use of architectures to guide and constrain systems modernization, recognizing them as a crucial means to a challenging goal: agency operational structures that are optimally defined in both business and technological environments. Congress, the Office of Management and Budget (OMB), and the federal Chief Information Officer (CIO) Council have also recognized the importance of an architecture- centric approach to modernization. The Clinger-Cohen Act of 1996 mandates that an agency’s CIO develop, maintain, and facilitate the implementation of an information technology (IT) architecture. Further, the E-Government Act of 2002 requires OMB to oversee the development of enterprise architectures within and across agencies. In addition, OMB has issued guidance that, among other things, requires system investments to be consistent with these architectures.

What Is an Enterprise Architecture?

An enterprise architecture provides a clear and comprehensive picture of an entity, whether it is an organization (e.g., a federal department) or a functional or mission area that cuts across more than one organization (e.g., financial maagement). This picture consists of snapshots of both the enterprise’s current or “As Is” environment and its target or “To Be” environment, as well as a capital investment road map for transitioning from the current to the target environment. These snapshots consist of “views,” which are one or more architecture products that provide logical or technical representations of the enterprise.

The suite of products and their content that form a given entity’s enterprise architecture are largely governed by the framework used to develop the architecture. Since the 1980s, various architecture frameworks have emerged and been applied. See appendix III for a discussion of these various frameworks.

The importance of developing, implementing, and maintaining an enterprise architecture is a basic tenet of both organizational transformation and systems modernization. Managed properly, an enterprise architecture can clarify and help to optimize the interdependencies and relationships among an organization’s business operations and the underlying IT infrastructure and applications that support these operations. Employed in concert with other important management controls, such as portfolio-based capital planning and investment control practices, architectures can greatly increase the chances that an organization’s operational and IT environments will be configured to optimize its mission performance. Our experience with federal agencies has shown that investing in IT without defining these investments in the context of an architecture often results in systems that are duplicative, not well integrated, and unnecessarily costly to maintain and interface.

DOD’s Business Management Modernization Program: A Brief Description and Chronology

In July 2001, the Secretary of Defense established the Business Management Modernization Program (BMMP) to improve the efficiency and effectiveness of DOD’s business operations and provide the department’s leaders with accurate and timely information through the development and implementation of a BEA. At that time, the Secretary tasked the Under Secretary of Defense (Comptroller), in coordination with the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD(AT&L)) and the Assistant Secretary of Defense (Networks and Information Integration)/Chief Information Officer (ASD(NII)/CIO), with responsibility for overseeing the program. To accomplish this, in October 2001, the Comptroller established governance bodies and assigned them responsibilities associated with developing, maintaining, and implementing the BEA. These entities and their respective roles and responsibilities are shown in table 1. DOD is currently revising its BEA governance structure, including recently eliminating its long-standing governance entities. These revisions are discussed later in this report.

Also in 2001, the BMMP program office was established to execute the program’s day-to-day activities, including implementing internal management controls and other mechanisms to provide reasonable assurance that the office would develop and implement an effective BEA. The office is led by a program director and comprised of seven program divisions, each of which is headed by an assistant deputy director. Figure 1 is a simplified diagram of the organizational structure of the program office, and table 2 shows the roles and responsibilities of the program divisions.

Initially, DOD planned to develop the architecture in 1 year. Subsequently, the department stated that it would develop its architecture in three increments, each increment addressing a subset of objectives and consisting of specific architecture releases. Table 3 shows these increments, the corresponding architecture releases, and the planned completion dates for the increments.

To develop the architecture, DOD entered into a 5-year, $95 million contract with International Business Machines (IBM) in April 2002, under which the department has issued a series of task orders aimed at developing the architecture. In 2004, DOD increased the contract amount to $250 million; however, the contract did not provide a reason for this increase, and program officials have yet to provide an explanation. As of September 2004, DOD reported that it had obligated approximately $318 million for the program, which is primarily for contractor support.

Prior Reviews of DOD’s Architecture Efforts Have Identified Many Weaknesses and Challenges

Over the last 4 years, we have identified the need for, and reviewed DOD’s efforts to develop an enterprise architecture for modernizing its business operations and systems, and we have made a series of recommendations to assist the department in successfully developing the architecture and using it to guide and constrain its ongoing and planned business systems investments.

In particular, we reported in May 2001 that the department did not have an architecture for its financial and financial-related business operations, nor did it have the management structures, processes, and controls in place to effectively develop and implement one. We concluded that continuing to spend billions of dollars on new and modified systems would result in more processes and systems that were duplicative, not interoperable, unnecessarily costly to maintain and interface, and not optimizing mission performance and accountability. We made eight recommendations to the Secretary of Defense that were aimed at providing the means for effectively developing and implementing an architecture and limiting DOD components’ systems investments until it had a well-defined architecture and the means to enforce it. In July 2001, DOD initiated the BMMP.

In February 2003, we reported that the department was following some architecture best practices, such as establishing a program office to be responsible for managing the enterprise architecture development effort. We also reported challenges and weaknesses in DOD’s architecture efforts. For example, we reported that DOD had not yet (1) established a governance structure and the process controls needed to ensure ownership of and accountability for the architecture across the department; (2) clearly communicated to intended stakeholders its purpose, scope, and approach for developing the architecture; and (3) defined and implemented an independent quality assurance process. We reiterated our earlier recommendations and made six new recommendations aimed at enhancing DOD’s ability to develop its architecture and to guide and constrain its business systems modernization investments.

In March 2003, we reported that DOD’s draft release of its architecture did not include a number of items that were recommended by relevant architectural guidance, and that DOD’s plans would not fully satisfy the requirements of the National Defense Authorization Act for Fiscal Year 2003. For example, the draft architecture did not include a “To Be” security view, which would define the security requirements—including relevant standards to be applied in implementing security policies, procedures, and controls. DOD officials generally agreed, and they stated that subsequent releases of the architecture would provide these missing details.

In July and September 2003, we reported that the initial release of the department’s architecture, including the transition plan, did not adequately address statutory requirements and other relevant architectural requirements. For example, the description of the “As Is” environment did not include (1) the current business operations in terms of entities and people who perform the functions, processes, and activities and (2) the locations where the functions, processes, and activities are performed. The description of the “To Be” environment did not include actual systems to be developed or acquired to support future business operations and the physical infrastructure that would be needed to support the business systems. The transition plan did not include time frames for phasing out existing systems within DOD’s then reported inventory of about 2,300 business systems. We concluded that the department’s initial release of the architecture did not contain sufficient scope and detail either to satisfy the act’s requirements or to effectively guide and constrain departmentwide business systems modernization. In our September 2003 report, we reiterated the open recommendations that we had made in our May 2001 and February 2003 reports, and we made 10 new recommendations to, among other things, improve DOD’s efforts for developing the next release of the architecture.

In March and July 2004, we testified that DOD’s substantial long-standing financial and business management problems adversely affected the economy, effectiveness, and efficiency of its operations and had resulted in a lack of adequate transparency and appropriate accountability across all major business areas. Further, we said that substantial work remained before the BEA would begin to have a tangible impact on improving the department’s overall business operations. We concluded that until DOD completed a number of actions, including developing a well-defined BEA, its business systems modernization efforts would be at a high risk of failure.

In May 2004, we reported that after 3 years of effort and over $203 million in obligations, DOD had not made significant change in the content of the BEA or in its approach to investing billions of dollars annually in existing and new systems. We reported that few actions had been taken to address the recommendations we had made in our September 2003 report, which were aimed at improving the department’s plans for developing the next release of the architecture and implementing the institutional means for selecting and controlling both planned and ongoing business systems investments. We also reported that DOD had still not adopted key architecture management best practices that we had recommended, such as assigning accountability and responsibility for directing, overseeing, and approving the architecture and explicitly defining performance metrics to evaluate the architecture’s quality, content, and utility. Further, DOD had not added the scope and detail to its architecture that we had previously identified as missing. For example, in the latest release of the BEA— Release 2.0—DOD did not provide sufficient descriptive content related to future business operations and supporting technology to permit effective acquisition and implementation of system solutions and associated operational change. Moreover, the department had not yet explicitly defined program plans, including milestones, detailing how it intended to extend and evolve the architecture to incorporate this missing content. We concluded that the future of DOD’s architecture development and implementation activities was at risk, which in turn placed the department’s business transformation effort in jeopardy of failing. Therefore, we added that it was imperative that the department move swiftly to implement our open recommendations. Because many of our prior recommendations remained open, we did not make any new recommendations in our May 2004 report, but we reiterated the open recommendations that we had made in our May 2001, February 2003, and September 2003 reports.

In November 2004 and April 2005, we testified that for DOD to successfully transform its business operations, it would need a comprehensive and integrated business transformation plan; people with the skills, responsibility, and authority to implement the plan; an effective process and related tools, such as a BEA; and results-oriented performance measures that would link institutional, unit, and individual personnel goals and expectations to promote accountability for results. We testified that over the last 3 years, we had made a series of recommendations to DOD and suggested legislative changes that, if implemented, could help the department move forward in establishing the means to successfully address the challenges it faces in transforming its business operations. We also testified that, after about 3 years of effort and over $203 million in reported obligations, the architecture’s content and the department’s approach to investing billions of dollars annually in existing and new systems had not changed significantly, and that the program had yielded very little, if any, tangible improvements in DOD’s business operations.

DOD Has Yet to Implement Effective Governance and Communications, but Improvements Are Under Way

Long-standing weaknesses in DOD’s BEA governance structure and communications strategy still remain. While DOD has established a new senior committee to oversee its business transformation efforts, including BEA development, much remains to be accomplished before proposed governance and communications concepts are fully defined and implemented. Until the department has made its intended governance and communications concept operational, the success of DOD’s architecture efforts will remain in doubt.

Long-standing Program Governance Weaknesses Remain, Although Recent Proposals Are Intended to Address Weaknesses

An enterprise architecture is a corporate asset that, among other things, is intended to represent the strategic direction of the enterprise. Accordingly, best practices recommend that to demonstrate commitment, organizations should vest accountability and assign responsibility for directing, overseeing, and approving the architecture to a committee or group with representation from across the enterprise. Sustained support and commitment by this committee to the architecture, as well as the committee’s ownership of it, are critical to a successful enterprise architecture development effort. We have previously recommended that DOD establish this kind of architecture responsibility and accountability structure. (See app. II for our prior recommendations and their current status.)

During the last 4 years, DOD has relied on three primary management entities to govern BEA development, maintenance, and implementation— the Executive Committee, the Steering Committee, and the Domain Owners Integration Team (DO/IT). (See table 1 for their roles and responsibilities.) This governance approach, however, does not assign accountability and responsibility for directing, overseeing, and approving the BEA to these entities either singularly or collectively. As we reported in February 2003, the Executive and Steering Committees were advisory in nature, and their responsibilities were limited to providing guidance to the program office and advising the Comptroller and the Executive Committee. Moreover, since they were established, neither the two committees nor the DO/IT have adequately fulfilled their assigned responsibilities, as we discuss below.

The Executive Committee was chartered to, among other things, provide strategic direction to the Steering Committee, champion program execution, and hold DOD components—including the military services—responsible for program results. To accomplish these things, the charter states that the committee should establish a meeting schedule. However, no schedule was established, and the committee met only four times in over 3½ years. Moreover, no minutes of the four meetings were prepared, according to the program’s Acting Assistant Deputy Director for Communications, and no other documentation exists to demonstrate the committee’s performance of its chartered functions. In fact, during numerous DO/IT meetings that we attended, participants stated that the Executive Committee was not providing strategic direction, nor was it championing program execution.

The Steering Committee was chartered to advise the Executive Committee on program performance, serve as the forum for discussion of component issues, and provide guidance to the program office. According to the program’s Acting Assistant Deputy Director for Communications, neither Executive Committee meeting minutes nor any other documentation exists to demonstrate that the Steering Committee advised the Executive Committee. Moreover, during the Steering Committee meetings that we attended over the last 4 years, we saw no evidence that this committee either planned to or actually did advise the Executive Committee on program performance. While we did observe in these meetings that issues were raised and discussed, which is a chartered responsibility of the committee, we also observed that the committee did not provide guidance and direction to the program office during these meetings. The Steering Committee last met about 1 year ago (June 2004).

The DO/IT was chartered to provide recommendations to the Steering Committee; provide guidance regarding architecture updates and their effects; and identify, address, and resolve domain and program issues. The charter also states that the DO/IT was to ensure that its representation included the military services. However, during the Steering Committee meetings that we attended, DO/IT representatives did not provide any recommendations to the Steering Committee, nor did they provide guidance on architecture updates and their effects. Moreover, the DO/IT did not include military service representatives, and it did not establish any policies and procedures for how to address and resolve issues. As a result, issues that were identified during meetings were not resolved. For example, in July 2004, there were discussions regarding the lack of involvement from the services, the lack of detail in the architecture content, and the lack of clear understanding of the roles and responsibilities among the domains and the services. During this meeting, however, no decisions were made about how these issues were to be resolved, and no actions were taken to provide recommendations to the Steering Committee for resolving the issues. As a result, we observed the same issues being discussed, without resolution, 5 months later.

DOD has recently taken steps to improve the program’s governance structure and, according to program officials, further steps are planned. For example, the department has implemented the provisions in the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, which are aimed at increasing senior DOD leadership involvement in the program. In particular, it includes DOD’s establishment of the Defense Business Systems Management Committee (DBSMC) to replace both the Executive and Steering Committees and to serve as the highest ranking governance body overseeing business transformation. According to the DBSMC charter, the committee is accountable and responsible for directing, overseeing, and approving all program activities. Specific responsibilities of the committee include establishing strategic direction and plans for the business mission area in coordination with the warfighting and enterprise information environment mission areas; approving business mission area transformation plans and initiatives and coordinating transition planning in a documented program baseline with critical success factors, milestones, metrics, deliverables, and periodic program reviews; establishing key metrics and targets by which to track business establishing policies and approving the business mission area strategic plan, the transition plan for implementation for business systems modernization, the transformation program baseline, and the BEA; and executing a comprehensive communications strategy.

Consistent with recent legislation, the DBSMC is chaired by the Deputy Secretary of Defense, and the USD(AT&L) serves as the vice chair. Its membership consists of senior leadership from across the department, including the military service secretaries, the defense agency heads, and the principal staff assistants. The department has also moved the program office from the Comptroller to the USD(AT&L), reporting to the Special Assistant for Business Transformation. According to DOD, this transfer of functions and responsibilities will allow USD(AT&L) to establish the level of activity necessary to support and coordinate the activities of the newly established DBSMC.

Other entities may be established to support the DBSMC. According to program officials, including the Program Director, the DO/IT has been eliminated and may be replaced by a DOD Enterprise Transformation Integration Group. Further, the DO/IT had identified the need for six additional boards to assist the program office. These boards have not yet been chartered, but potential board members have met to discuss the boards’ roles, responsibilities, and operating procedures, as well as program issues. However, the program director stated that not all of these boards may be established under the new structure.

According to the Special Assistant for Business Transformation, a new governance structure will be in place and operational by September 30, 2005. To this end, DOD officials told us that the DBSMC held its first meeting in February 2005 to finalize its charter and member composition, and that to move governance reforms forward, the committee will initially hold monthly meetings.

The weaknesses in the BEA governance structure over the last 4 years, according to program officials, are attributable to a lack of authority and accountability in program leadership by the various management entities (e.g., Executive and Steering Committees) and to the limited direction provided by these entities. While DOD’s recent actions are intended to address these root causes, almost 4 years and approximately $318 million in obligations have been invested, and the department is still attempting to establish an effective governance structure. Moreover, much remains to be accomplished until DOD’s new governance structure becomes an operational reality. Until it does, it is unlikely that the department will be able to develop an effective BEA.

An Effective Communications Strategy Has Yet to Be Implemented, but Some Activities Are Under Way

Effective communications among architecture stakeholders are closely aligned with effective governance. According to relevant guidance, once initial stakeholder participation in an architecture program is achieved, a communications strategy should be defined and implemented to facilitate the exchange of information among architecture stakeholders about all aspects of the program, such as program purpose, scope, methodologies, progress, results, and key decisions. Such communication is essential to executing an architecture program effectively, including obtaining institutional buy-in for developing and using the architecture for corporate decision making. Accordingly, in 2003 we recommended that DOD provide for ensuring stakeholder commitment and buy-in through proactive architecture marketing and communication activities. (See app. II for our prior recommendation and its current status.)

In response, the program office defined and approved a strategic communications plan in March 2004. According to the plan, its purpose is to direct the flow of information to inform, collaborate with, and educate DOD internal and external stakeholders about the program. To accomplish this, the plan identifies categories of stakeholders and includes a five-phase implementation approach. The five phases are as follows: 1. Plan Start-up: Conduct a formal tactical review, including an assessment of current communication tools, communication procedures, available resources, and agency or industry best practices. 2. Discovery and Assessment: Identify and verify all internal and external stakeholders and begin defining the benchmarking and reporting requirements. 3. Branding: Determine key messages to be communicated and select tools for disseminating these messages. 4. Communications Planning and Execution: Execute and assess the application of the strategy and continue to develop the communication tools. 5. Evaluation: Evaluate the progress and overall success of the plan’s implementation, using metrics.

The strategic communications plan states that the five phases were to have been fully implemented by December 2004. Although 5 months have passed since the plan was to be implemented, none of the five phases have been completed, and communications activities that have been performed by those responsible for implementing the plan have been limited in scope. Specifically, communications activities have focused on raising program awareness through the distribution of posters and press releases and the establishment of a Web site that provides information about the program. However, an assessment to evaluate the effectiveness of communication tools and procedures, available resources, and agency or industry best practices in communicating the program’s purpose to the various stakeholders (e.g., the domains and the military services) has not been performed. In addition, the program’s Acting Assistant Deputy Director for Communications stated that a systematic approach, including metrics, to measure the effectiveness of the communications strategy has not been defined. According to this official, communications activities have been ad hoc.

Further, DO/IT representatives told us that the program’s Web site does not contain consistent and current information about the program; as a result, stakeholders’ understanding of the BEA is limited. Moreover, the plan focuses first on internal stakeholders, and it defers efforts to proactively promote understanding and buy-in with external stakeholders, such as Congress. The plan defines the internal stakeholders to exclude key departmental components, such as the military services and the defense agencies. Instead, it defines internal stakeholders to include only the program office and the domains.

As a result, both the internal and external DOD stakeholders with whom we spoke said that they did not have a clear understanding of the program, including its purpose, scope, approach, activities, and status, as well as stakeholders’ roles and responsibilities. For example, the Installation and Environment domain representative stated that it was unclear how the program would achieve one of the program’s original goals (i.e., achieving an unqualified audit opinion by fiscal year 2007), and what the role of this representative’s domain would be in achieving this goal. Similarly, the Acquisition domain representative stated that the roles and responsibilities of the domains for the program are confusing, because the domains should be the entities that are developing the BEA, and not the program office. According to this representative, the current approach will result in redundancy and increased program costs.

In addition, the program office’s Acting Assistant Deputy Director for Communications acknowledged that stakeholders are confused, stating that they do not have a good understanding of either the BEA’s goals, objectives, and intended outcomes or stakeholders’ roles and responsibilities. This official also stated that cross-domain integration issues remain that require strong DOD leadership to move the communications efforts beyond conducting awareness activities to achieving departmentwide buy-in.

The Special Assistant for Business Transformation has recently begun to better communicate with key external stakeholders, such as Congress. For example, this official stated that department officials have met with and intend to hold future meetings with congressional staff to brief them on the department’s plans and efforts to date. Without effective communication with BEA stakeholders, the likelihood that DOD will be able to effectively develop and implement its BEA is greatly reduced.

DOD Has Yet to Develop Program Plans and Supporting Workforce Plans, but Intends to Make Improvements

DOD does not have the program plans that it needs to effectively manage the development, maintenance, and implementation of its BEA. In particular, the department has yet to specify measurable goals and outcomes to be achieved, nor has it defined the tasks to be performed to achieve these goals and outcomes, the resources needed to perform the tasks, or the time frames within which the tasks will be performed. DOD has not assessed, as part of its program planning, the workforce capabilities that it has in place and that it needs to manage its architecture development, maintenance, and implementation efforts, nor does it have a strategy for meeting its human capital needs. The absence of effective program planning, including program workforce planning, has contributed to DOD’s limited progress in developing a well-defined architecture and clearly reporting on its progress to date. Unless its program planning improves, it is unlikely that the department will be successful in its attempt to develop, maintain, and implement its BEA. Recognizing this long- standing void in planning, DOD stated that it intends to complete, by September 30, 2005, a transition plan that will include a program baseline that will be used to oversee and manage program activities.

DOD Has Yet to Develop Effective Program Plans

Architecture management guidance states that organizations should develop and execute program plans, and that these plans should provide an explicit road map for accomplishing architecture development, maintenance, and implementation goals. Among other things, effective plans should specify tasks to be performed, resources needed to perform these tasks (e.g., funding, staffing, tools, and training), program management and contractor roles and responsibilities, time frames for completing tasks, expected outcomes, performance measures, program management controls, and reporting requirements. We have previously reported on the program’s lack of effective planning and have recommended that DOD develop BEA program plans. (See app. II for our prior recommendation and its current status.)

Since the program was launched in 2001, DOD has operated without a program plan. Instead, the department has set target dates for producing a series of architecture releases as part of three generally defined architecture increments (see table 3). However, DOD has not clearly defined what the purpose of the respective increments are, either individually or collectively, and it has not developed near-, mid-, or long- term plans for producing these increments. At a minimum, such plans would identify specific tasks (i.e., provide a detailed work breakdown structure) for producing the architecture releases that possess predefined content and utility. These plans would also contain specific and reliable estimates of the time and resources it will take to perform these tasks.

Also in lieu of program plans, the program office provided us with documents in November 2004 that the deputy program director stated were to address our open recommendations for (1) adding needed content and consistency to the BEA’s “As Is” and “To Be” architectural products and (2) developing a well-defined BEA transition plan. However, the documents that we were provided were plans for developing plans to address our recommendations and, thus, were not documents explaining how and when our recommendations would be addressed.

DOD has recognized the need for program planning. According to the Acting Assistant Deputy Director for Strategic Planning, the program office had committed to developing a program baseline by December 2004. According to the program director, this baseline was to include program goals, objectives, and activities as well as performance, cost, and schedule commitments. Further, the baseline was to establish program roles, responsibilities, and accountabilities and was to be used as a managerial and oversight tool to allocate resources, manage risk, and measure and report progress. However, the department has yet to develop this program baseline. Moreover, while this program baseline would be a useful tool for strengthening program management, it nevertheless was not to have included all of the elements of an effective program plan, such as a detailed work breakdown structure and associated resource estimates. According to senior officials, including the Special Assistant for Business Transformation and the Deputy Under Secretary of Defense (Financial Management), the department is drafting a transition plan that is to be approved by the DBSMC and issued by September 30, 2005. According to these officials, this plan will include a program baseline and a BEA development approach.

The lack of well-defined program plans has contributed significantly to the limited BEA progress that we have reported over the last several years. Moreover, this absence of program plans has created a lack of transparency and understanding about what is occurring on the program and what will occur—both inside and outside the department. It has also inhibited BEA governance entities’ ability to ensure that resources (e.g., program office, domains, and contractors) are being effectively used to achieve worthwhile outcomes and results. Although the contractor’s work statements have provided some additional detail, these task descriptions lack the specificity necessary to use them effectively to monitor the contractor’s progress and performance. For example, the latest work statement includes the task “develop business rules based on the available sources of information,” which has been included in prior work statements. However, the latest work statement does not define the scope of this effort, nor does it define how this latest task will support prior efforts to develop business rules. As a result, the extent to which business rules have been developed and the work remaining to complete the development of these rules is unclear. Further, the relationship of this task to DOD’s ability to satisfy the objectives for increment 1 also has not been defined. These architecture relationships need to be defined before the department can develop explicit plans for effective BEA development.

Moreover, because there is no plan linking them together, it is not clear how the contractor’s work statements and other BEA working groups’ efforts relate to or contribute to larger BEA goals and objectives. For example, the program office has continued to task the contractor to develop architecture releases, although the intended use, and thus the explicit content, of the various releases has not been clearly linked to the goals and objectives of increment 1. Representatives of the DOD business domains raised similar concerns with the contractor’s work statements, telling us that the work statements have been vague and have not been linked to the specific architecture products. According to these representatives, the department does not know if it is investing resources on tasks that are needed and add value to the program.

According to the deputy program director, continuous changes in the direction and scope of the program have hindered DOD’s ability to develop effective program plans. Without such plans, DOD has been, and will continue to be, limited in its ability to develop a well-defined architecture on time and within budget.

DOD Has Not Performed Effective Workforce Planning

As we have previously reported, workforce planning is an essential component of program management. Workforce planning enables an entity to be aware of and prepared for its current and future human capital needs, such as workforce size, knowledge and skills, and training. Such planning involves assessing the knowledge and skills needed to execute the program, inventorying existing staff knowledge and skills, identifying any shortfalls, and providing for addressing these shortfalls. Through effective workforce planning, programs and organizations can have the right people with the right skills doing the right jobs in the right place at the right time. Relevant architecture management guidance recognizes the importance of planning for and having adequate human resources in developing, maintaining, and implementing enterprise architectures.

DOD has yet to perform workforce planning for its BEA program. Nevertheless, it has established a program office consisting of seven program divisions (see fig. 1) and staffed the office with 60 government employees and approximately 300 contractor staff. In addition, the department has assigned other staff to support the various program government entities, such as the domains and the DO/IT, and it has established various formal and informal working groups. However, DOD has not taken steps to ensure that the people assigned to the program have the right skill sets or qualifications. In particular, DOD has not defined the requisite workforce skills and abilities that the department needs in order to develop, maintain, and implement the architecture. To illustrate, the program’s Assistant Deputy Director for Administrative Support told us that the position descriptions used to staff the program office were generic and were not tailored to the needs of an enterprise architecture program. This official added that, as a result, a person might satisfy the qualifications contained in the position description, but still not meet the needs of the BEA program. In addition to not defining its workforce needs, the program also does not have an inventory of the capabilities of its currently assigned program workforce. For example, the Assistant Deputy Director for Administrative Support told us that the department did not know the number of individuals assigned to support the various governance entities.

In the absence of an inventory of existing workforce knowledge and skills, we requested any available information on the qualifications and training of program staff in key leadership positions (e.g., assistant deputy directors). In response, the Assistant Deputy Director for Administrative Support said that such information was not readily available for all staff, and this official provided us with résumés for 15 program officials, 4 of whose résumés we were told were created in response to our request for key staff qualifications and training.

Exacerbating the program’s lack of workforce planning is the fact that several key program office positions are vacant. For example, four of the seven program division leadership positions (i.e., assistant deputy directors’ positions) are temporarily filled by persons “acting” in this position (see fig. 2). In addition, key supporting positions within the program divisions, such as the positions for performance management and independent verification and validation/quality assurance, are vacant. As a result, 1 program official is currently acting in three positions—strategic planning/organization development (includes risk management), performance management (earned value management system), and independent verification and validation/quality assurance. Further, two of the positions this person occupies are incompatible and do not allow for appropriate separation of duties. Specifically, this individual is responsible for both program performance management and independent oversight of performance management.

In addition, significant staff turnover has occurred in key program positions. For example, the program office has had three directors in 4 years, three transition planning directors since March 2004, and four contracting officer technical representatives responsible for the prime contract since January 2003.

The Assistant Deputy Director for Administrative Support stated that the program lacks valid and reliable data about its human capital needs and current capabilities. This official told us that plans are being developed to begin addressing this situation. For example, to begin monitoring staff turnover, the program office recently began maintaining a list of program staff with start dates. However, this official also told us that the plans for improving the program’s management of its human capital were not complete, and dates for when the plans would be complete have not been set.

The absence of effective workforce planning has contributed significantly to DOD’s limited progress to date in developing its architecture. Unless the program’s approach to human capital management improves, it is unlikely that the department’s future efforts to develop, maintain, and implement the BEA will be successful.

DOD Is Not Performing Effective Configuration Management

Relevant architecture guidance, including DOD guidance, recognizes the importance of configuration management when developing and maintaining an architecture. The purpose of configuration management is to maintain integrity and traceability, and control modifications or changes to the architecture products throughout their life cycles. Effective configuration management, among other things, enables integration and alignment among related architecture products. As we have previously reported, an effective configuration management process comprises four primary elements, each of which should be described in a configuration management plan and implemented according to the plan. In addition, responsibility, accountability, and authority for configuration management should be assigned to a configuration manager. The four elements are: Configuration identification: Procedures for identifying, documenting, and assigning unique identifiers (e.g., serial number and name) to product types generated for the architecture program, generally referred to as configuration items.

Configuration control: Procedures for evaluating and deciding whether to approve changes to a product’s baseline configuration, generally accomplished through configuration control boards, which evaluate proposed changes on the basis of costs, benefits, and risks and decide whether to permit a change.

Configuration status accounting: Procedures for documenting and reporting on the status of configuration items as a product evolves. Documentation, such as historical change lists and original architecture products, are generated and kept in a library, thereby allowing organizations to continuously know the state of a product’s configuration and to be in a position to make informed decisions about changing the configuration.

Configuration auditing: Procedures for determining alignment between the actual product and the documentation describing it, thereby ensuring that the documentation used to support the configuration control board’s decision making is complete and correct. Configuration audits, both functional and physical, are performed when a significant product change is introduced, and they help to ensure that only authorized changes are being made.

DOD has a draft configuration management plan and related procedures that address all four of these areas. However, the plan is not being followed. For example, according to the plan and procedures, certain product types should be placed under configuration management and be assigned a unique identifier. However, in one case, the verification and validation contractor reported that DOD had updated one of the BEA products (i.e., All View-1) that was initially published in Release 2.3, but that this updated product was not given a unique identifier and a new release date, and no entry was made in the version history to enable stakeholders to differentiate between the two versions.

Configuration naming conventions also have not been consistently followed, resulting in the updates to a single document being given different unique identifiers than the original document. For example, the November 2003 configuration management plan had the unique identifier “C0008_05_03_BMMP_Configuration_Management_Plan.doc,” which was comprised of the call number, the task number, the subtask number, and the name of the document. However, the department later assigned this document the following unique identifier “Configuration_Management_Plan.doc,” which did not include the call and task numbers. Such inconsistencies could permit changes to be made to the wrong version of a product, thereby compromising the integrity and reliability of the information.

Consistent with relevant guidance, the procedures require that a configuration manager be assigned and that this individual be responsible for ensuring that the four elements are executed. However, after almost 4 years of architecture products development, a configuration manager has not been assigned. In addition, while the department established a configuration control board and chartered it to evaluate and decide whether to approve proposed product changes, this board is not fully functioning. Specifically, the board’s charter has yet to be approved, and its authority has been limited to a subset of BEA products. For example, its authority does not extend to the BEA transition plan.

With respect to configuration status accounting activities that have been conducted to ensure the integrity of product baselines, we were provided with two reports even though program officials, including the Configuration Control Board Chair, told us that no configuration status accounting reports exist and that neither do any auditing procedures and processes (e.g., audit checklists, agendas, or plans). However, one of these reports was missing key data, such as the date of the report, the submitter, and the version of the product currently being reviewed and, thus, was of limited use. It was also unclear as to which version of the product was referenced in the report, and these officials told us that the current baseline of approved configuration items, including the configuration management plan, is unknown. As a result, configuration items can be duplicated. For example, the Acting Assistant Deputy Director for Communications had a second communications plan prepared, and this official told us that he did not know that a prior draft plan existed. Because of this, the new strategy did not leverage any of the work that had previously been done, and duplicative plans exist.

Program officials, including the Configuration Control Board Chair, stated that they recognize the importance of effective configuration management. They attributed the absence of effective configuration management to a number of factors, including no policy or directive requiring it and the lack of a common understanding of effective configuration management practices.

The absence of effective configuration management raises questions about whether changes to the BEA and other relevant products have been justified and accounted for in a manner that maintains the integrity of the documentation. Unless this situation is remedied, the department will not have adequate assurance that it has maintained accountability and ensured the consistency of information among the products it is developing. In addition to the governance and planning weaknesses we previously discussed, the department’s lack of effective configuration management has also contributed to the state of the BEA discussed in the next section of this report.

DOD Has Yet to Develop a Well-Defined BEA to Guide Its Modernization Efforts

Despite six BEA releases and two updates, DOD still does not have a version of an enterprise architecture that can be considered well defined, meaning that the architecture, for example, has a clearly defined purpose that can be linked to the department’s goals and objectives and describes both the “As Is” and the “To Be” environments; consists of integrated and consistent architecture products; and has been approved by department leadership. According to program officials, the state of the BEA reflects the program’s focus on meeting arbitrary milestones rather than architecture content needs. Recognizing the need to develop well-defined architecture products that have utility and provide a foundation upon which to build, program officials have stated the department’s intent to change its BEA development approach, refocusing its efforts on fewer, higher quality products. Until a BEA development approach embodying architecture development and content best practices is clearly defined and implemented, it is not likely that DOD will produce an enterprise architecture that will provide needed content and utility.

“As Is” Description, Transition Plan, and Purpose of BEA Releases Are Missing

As we previously discussed, the various frameworks used to develop architecture products consistently provide for describing a given enterprise in both logical (e.g., business, performance, application, and information) and technical (e.g., hardware, software, and data) terms, and for doing so for the enterprise’s current or “As Is” environment and its target or “To Be” environment; these frameworks also provide for defining a capital investment sequencing plan to transition from the “As Is” to the “To Be” environment. However, the frameworks do not prescribe the degree to which the component parts should be described to be considered correct, complete, understandable, and usable—essential attributes of any architecture. This is because the depth and detail of the descriptive content depends on what the architecture is to be used for (i.e., its intended purpose and scope). Relevant architecture guidance states that a well- defined architecture should have a specific and commonly understood purpose and scope, and that it should be developed in incremental releases. Using this purpose and scope, the necessary content of architecture releases can then be defined.

In September 2003, we reported that Release 1.0 of the BEA was missing important content, and we made 62 recommendations to add this content. The latest releases of the BEA (see table 4) do not address the 32 of our 62 recommendations that are related to the “As Is” description and the transition plan. Specifically, the releases do not include products describing the “As Is” environment for those areas of the enterprise that are likely to change, and they do not include a sequencing plan that serves as a road map for transitioning from the “As Is” state to the “To Be” state. For example, the BEA releases do not contain a description of the “As Is” environment that would include current business operations in terms of the entities or people who perform the functions, processes, and activities, and the locations where the functions, processes, and activities are performed. It also does not describe the data or information being used by the functions, processes, and activities. As a result, DOD does not have a picture of its current environment to permit development of a meaningful and useful transition plan.

The BEA releases also do not contain a transition plan that would include information such as time frames for phasing out existing systems within DOD’s current inventory and resource requirements for implementing the “To Be” architecture. As a result, DOD does not yet have a meaningful and reliable basis for managing the disposition of its existing inventory of systems or for sequencing the introduction of modernized business operations and supporting systems. As we previously reported, not having defined the “As Is” operations and technology at this juncture is risky because it defers until too late in the architecture development cycle creation of sufficient descriptive content and context to develop an effective transition plan. (See app. II for our prior recommendations and their current status.) DOD’s architecture framework (DODAF), which the department is using to develop the BEA, does not require the development of an “As Is” architectural description or a transition plan, and thus neither has been the focus of the program. However, according to program officials, including the Chief Architect, the September 2005 BEA release will include both the “As Is” architectural description and a transition plan.

In addition, DOD has not clearly linked the purpose of any of the “To Be” architecture releases produced to date to the goals and objectives of increment 1. Further, these releases also do not have a clearly defined scope with respect to what business processes and supporting systems each release would focus on. According to program officials, the last five versions (i.e., Releases 2.1, 2.2, and 2.3 and the January and March 2005 Updates) support the objectives of increment 1 (see table 3). These objectives are broad-based strategic goals or outcomes that DOD proposed achieving through a series of architecture releases. However, DOD did not define how many releases were needed to achieve each objective and how the purpose of each release is associated with the objectives. Restated, while each incremental objective would describe the mission outcome that DOD intended to achieve via implementation of the series of releases that make up that increment, the purpose of the releases was not specified in terms of architectural questions that were related to the objectives. To illustrate, one objective of increment 1 is to achieve an unqualified audit opinion for consolidated DOD financial statements. Examples of the purpose questions that would support this objective include the following: 1. What changes need to be made to existing business processes and the supporting systems to address material internal control weaknesses affecting significant line items on the financial statements? 2. Where are gaps in IT support (systems functions) that need to be addressed in order to provide the business capabilities for ensuring that property, plant, and equipment are appropriately valued and recorded on the financial statements?

The department did not include architecture products that would answer these types of questions to support the increment 1 objective. As a result, the context needed to plan and measure content sufficiency was not established. Program officials, including the Special Assistant for Business Transformation and the Chief Architect agreed, stating that prior releases have not included a specific purpose and scope. Moreover, the Chief Architect told us that the architecture releases do not fully support increment 1’s objectives, nor do they describe the extent to which they do address the increment 1 objectives.

According to program officials, including the deputy program director and the Chief Architect, the prior releases were developed with the goal of producing as many architecture products as possible within a predefined schedule. The releases were not developed to provide the content needed to meet the defined purpose and scope of the release. Until the department defines the intended purpose and scope of its BEA, including its incremental releases, and ensures that the architecture products include adequate descriptions of the “As Is” and “To Be” environments, as well as a plan for transitioning between the two, it will not have a well-defined architecture to guide and constrain its systems modernization efforts.

BEA Products Are Incomplete, Inconsistent, and Not Integrated

Architecture frameworks provide for multiple products, each describing a particular aspect of the enterprise, such as data or systems. Moreover, each of these products is interdependent, meaning that they have relationships with one another that must be explicitly defined and maintained to ensure that the products form a coherent whole. In light of these relationships, it is important to develop the architecture products in a logical sequence that reflects this relationship. DODAF recognizes this need for integration of the products that make up its three “To Be” views—operational view (OV), systems view (SV), and technical standards view (TV). (See app. IV for a brief description of the products that comprise each of these views.) According to the framework, an architecture must be well structured so that the products can be readily assembled or decomposed into higher or lower levels of detail, as needed. It should also be consistent—that is, information elements should be the same throughout the architecture.

As noted in the previous section, we reported in September 2003, that Release 1.0 of the BEA was missing important content, and we made 62 recommendations to add this content. The latest releases of the BEA do not adequately address our 30 prior recommendations related to the “To Be” description. For example, these releases do not include descriptions of the actual systems to be developed or acquired to support future business operations and the physical infrastructure (e.g., hardware and software) that will be needed to support the business systems. As a result, the “To Be” environment lacks the detail needed to provide DOD with a common vision and frame of reference for defining a transition plan to guide and constrain capital investments and, thus, to effectively leverage technology to orchestrate logical, systematic change and to optimize enterprisewide mission performance. (See app. II for details on the status of our prior recommendations.)

In addition, the respective products of each of the latest BEA releases continue to be inconsistent and not integrated, because key architecture products were either not developed or not updated to reflect changes made in other products. Examples of where Releases 2.2 and 2.3 are not consistent and integrated follow: In Release 2.2, the department updated the system data exchange matrix (SV-6), which assigns attributes (e.g., timeliness) to the data to be exchanged (e.g., Performance Metrics) between system functions— “Manage Business Enterprise Reporting” and “Establish and Maintain Performance Information”—to support business operations. However, the OV-3 in Release 2.2, which shows the attributes of the information to be exchanged to support operations, is not consistent with the attributes defined in the SV-6. For example, in the OV-3, the attribute referred to as “timeliness” is defined in terms of either “hours,” “minutes,” or “seconds;” however, in the SV-6, the attribute referred to as “timeliness” is only defined in terms of either “high, medium, or basic.”

In Releases 2.2 and 2.3, the department updated the respective operational event-trace description product (OV-6c), which depicts when activities are to occur within operational processes. However, the department did not update, in either release, the operational activity model (OV-5), which shows the operational activities (or tasks) that are to occur and the input and output flows among these activities. For example, the OV-6c shows the sequence of the activities to occur for the process labeled “produce obligation reports;” however, the activities shown in the OV-5 were not associated with this process.

The latest releases also do not provide for traceability among the architecture products by clearly identifying the linkages and dependencies among the products, such as associating processes (e.g., produce obligation reports) with activities (e.g., compare expense to obligation) in the operational views and then associating these same processes to systems (e.g., financial reporting system) in the systems view. In addition, the linkage between the two functions (i.e., “Manage Business Enterprise Reporting” and “Establish and Maintain Performance Information”) previously discussed cannot be traced to the OV-3 in Release 2.2. This is because Release 2.2 did not include an SV-5, which would provide a traceability of system functions back to operational activities. The lack of an updated SV-5 also raises questions as to whether all operational requirements are satisfied by the system functions. In addition, the architecture products were not developed in a logical sequence, as called for in relevant guidance and standards (e.g., the OV-6c, which shows the timing or sequencing of activities, was built before the OV-5, which shows the activities that are to occur).

Further, according to the verification and validation contractor, the department has yet to address 242 of its 299 outstanding comments since Release 1.0. The verification and validation contractor also cited similar concerns, as previously described, for Releases 2.2 and 2.3. Specifically, the contractor reported that the BEA products were not integrated, and that key products were missing or had not been updated—such as the operational nodes connectivity description (OV-2) and the operational information exchange matrix (OV-3). In its report on the January 2005 Update, the contractor stated that the architecture products were developed in an order different from that recommended in DODAF, and that the dependency relationships between and among BEA products were not clearly depicted. For example, the contractor reported that the logical data model (OV-7) was to have been developed using the OV-3, OV-5, and OV-6c artifacts as inputs. However, this was not the case. Instead, the OV-7 was developed using information that may have been reverse engineered from existing systems and architectures external to the BEA. The contractor reported that unless these dependencies are clearly documented and depicted, new systems may be implemented without satisfying all operational requirements, with missing functions and interfaces or based on obsolete data models, recreating many of the problems the modernization is intended to resolve. The contractor also reported that the resulting technical problems in the OV-7 could interfere with the department’s achievement of the increment 1 objectives.

The March 2005 Update also did not have fully integrated products. Specifically, while some of the products were integrated, this integration occurred at the highest level only and could not be found at lower levels of decomposition (e.g., subprocesses and subactivities) within the architecture. For example, the level of integration does not enable the user to determine all information inputs for the activities at all levels, nor does it clearly reflect the dissemination or use of the information after it has been processed. In addition, this update did not include key architecture products that are recommended by DODAF—such as the system data exchange artifact (SV-6), which assigns attributes (e.g., timeliness) to the data to be exchanged between system functions and the system inventory (SV-8). The SV-8 provides a basis for portfolio investment decisions by depicting the evolution of systems, systems integration, and systems improvements over time. We also found that the architecture was not user friendly in that it was difficult to navigate. For example, the linkages among the architecture products did not always work, thereby, requiring manual navigation through the architecture to find the linkages. This would take hours to do, especially since it was complicated by the fact that certain artifacts (e.g., diagrams) could not be read online and had to be printed.

Relatedly, as shown in table 4, the latter BEA releases have not included all of the recommended DODAF products. DODAF recommends that the BEA include 23 out of 26 possible architecture products to meet the department’s stated intention to use the BEA as the basis for departmentwide business and systems modernization. However, Release 2.2 of the architecture included only 16 of the 23 recommended architecture products, and 6 of the 16 products (OV-1, OV-2, OV-3, OV-4, OV-5, and SV-9) were actually Release 2.0 products that had not been updated to align with the changes that had been made to the 10 products that were updated in this release. Similarly, Release 2.3 included only 4 products; the January 2005 Update included 6, and the March 2005 Update included 15. According to the Chief Architect, all prior architecture products that were not included in a specific release or update are obsolete. For example, Release 2.2 included 16 architecture products, of which 10 had been updated. The remaining 6 products had not been updated, but they were still considered to be valid because they were included in this release. This means that for all releases and updates, only those products included in the release or update are relevant. For example, DOD updated 15 products and included them in the March 2005 Update; therefore, as of March 2005, only these 15 products were considered to be valid artifacts of the BEA.

Program and contractor officials, including the Acting Assistant Deputy Director for Transition Planning, stated that although the department’s first release of its architecture included a fairly consistent and integrated set of architecture products, DOD’s current releases do not because the department did not update all the recommended architecture products. These officials, including the Chief Architect, also stated that, as a result, the utility of the architecture is limited. However, according to key program officials, including the Special Assistant for Business Transformation and the Chief Architect, the integration of the architecture products was not the focus; rather, DOD’s primary goal was to produce as many products as it could within a specified time period (see tables 3 and 4).

Recognizing these weaknesses, the Special Assistant for Business Transformation stated that the department intends to reduce the scope of the architecture and revise the development approach, which will be reflected in the September 30, 2005, architecture release. However, according to program officials, including the Special Assistant for Business Transformation, the September 2005 BEA release will not be comprehensive (i.e., it will not meet all the act’s requirements). Further, the department has yet to develop plans and a methodology to execute this new focus and vet it through the department. Program officials also stated that as a result of the new focus, they are trying to decide which products from prior releases could be salvaged and used.

Nevertheless, the department has spent almost 4 years and approximately $318 million in obligations to develop an architecture that is incomplete, inconsistent, and not integrated and, thus, has limited utility. Until the department develops an approved, well-defined architecture that includes a clear purpose and scope and integrated products, it remains at risk of not achieving its intended business modernization goals and of not having an architecture that the stakeholders can use to guide and constrain ongoing and planned business systems investments to prevent duplicative and noninteroperable systems.

BEA Releases Have Not Been Approved

Relevant architecture guidance state that architecture versions should be approved by the committee overseeing the development and maintenance of the architecture; the CIO; the chief architect; and senior management, including the department head. Such approval recognizes and endorses the architecture for what it is intended to be—a corporate tool for managing both business and technological change and transformation. Consistent with guidance, DOD has stated its intention to approve all BEA releases.

However, Release 1.0 of the BEA is the only release that DOD reports as having been approved. As we previously reported, DOD officials told us that Release 1.0 was approved by the former Executive Committee, the department’s CIO as a member of the Executive Committee, and the DOD Comptroller on behalf of the Secretary of Defense in May 2003, but they also said that documentation to verify these approvals did not exist.

Since Release 1.0, DOD has issued five additional releases and two updates. None of these have been approved by any individual or committee in the BEA governance structure. According to program officials, including the Special Assistant for Business Transformation and the Chief Architect, Release 3.0 of the BEA, which will be issued in September 2005, will be the next release of the architecture to be approved by the department. These officials stated that the architecture releases have not been approved because the department did not have a governance structure and process in place for doing so. Without the appropriate approvals, buy-in to and recognition of the BEA as an institutionally endorsed change management and transformation tool is not achievable.

DOD Has Yet to Fully Address Most of Our Other Recommendations

In addition to the governance, planning, and content issues previously discussed, we have made other recommendations relative to DOD’s ability to effectively develop, maintain, and implement an enterprise architecture for its business operations. To date, the department has fully addressed one of our other recommendations, which is to report every 6 months to the congressional committees on the status of the BEA effort, but it has yet to fully address the remaining recommendations. (See app. II for details on the status of these recommendations.) For example, the department has yet to address our recommendations to develop a position description for the Chief Architect that defines requisite duties and responsibilities, update policies to assign responsibility and accountability for approving BEA releases, update policies to address the issuance of waivers for business systems that are not compliant with the architecture but are nevertheless justified on the basis of documented analysis, and develop and implement a quality assurance plan.

According to the program director and deputy director, the current state of the BEA, including progress in addressing our recommendations, reflects the program’s prior focus on producing as many products as it could within a specific time period. The focus had not been on the content and quality of the releases, but rather on the timing of their delivery. In contrast, our recommendations have all focused on establishing the means by which to deliver a well-defined BEA and ensuring that delivered releases of the architecture contain this requisite content. Until DOD adopts the kind of approach embodied in our recommendations, it is unlikely that it will produce a well-defined BEA within reasonable time frames and at an affordable cost.

Conclusions

Having and using a well-defined enterprise architecture are essential for DOD to effectively and efficiently modernize its nonintegrated and duplicative business operations and systems environment. However, the department does not have such an architecture, and the architecture products that it has produced to date do not provide sufficient content and utility to effectively guide and constrain the department’s ongoing and planned business systems investments. This means that despite spending almost 4 years and about $318 million to develop its BEA, the department is not positioned to meet its legislative mandates. In our view, the state of the architecture is due largely to long-standing architecture management weaknesses that the recommendations we have made over the last 4 years are aimed at correcting, as well as the department’s prior focus on producing as many products as it could within a specific time period. To date, the department has not taken adequate steps to implement most of our recommendations.

While recent steps to begin revamping its BEA governance structure and to begin program planning are positive first steps and are consistent with some of the recommendations that we made to lay a foundation for architecture development, maintenance, and implementation, much more remains to be accomplished. Thus, it is imperative for the department to move swiftly to strengthen its BEA program in a manner that incorporates our prior recommendations and recognizes its current architecture management capabilities. Until it does, the department will continue to put billions of dollars at risk of being invested in systems that are duplicative, are not interoperable, cost more to maintain than necessary, and do not optimize mission performance and accountability.

Recommendations for Executive Action

We recommend that the Secretary of Defense direct the Deputy Secretary of Defense, as the chair of the DBSMC and in collaboration with DBSMC members, to immediately fully disclose the state of its BEA program to DOD’s congressional authorization and appropriations committees, including its limited progress and results to date, as well as specific plans and commitments for strengthening program management and producing measurable results that reflect the department’s capability to do so; ensure that each of our recommendations related to the BEA management and content are reflected in the above plans and commitments; and ensure that plans and commitments provide for effective BEA workforce planning, including assessing workforce knowledge and skills needs, determining existing workforce capabilities, identifying gaps, and filling these gaps.

Agency Comments

In written comments on a draft of this report signed by the Special Assistant for Business Transformation in the Office of the Under Secretary of Defense (Acquisition, Technology, and Logistics) and the Deputy Under Secretary of Defense (Financial Management) (reprinted in app. V), the department concurred with our recommendations and stated its intent to implement them. Specifically, DOD stated that it would (1) disclose plans, progress, and results of its BEA efforts to DOD’s congressional committees; (2) address our recommendations related to BEA management and content; and (3) assess its workforce needs and adjust its workforce to meet requirements.

We are sending copies of this report to interested congressional committees; the Director, Office of Management and Budget; the Secretary of Defense; the Deputy Secretary of Defense; the Under Secretary of Defense (Acquisition, Technology, and Logistics); the Under Secretary of Defense (Comptroller); the Assistant Secretary of Defense (Networks and Information Integration)/Chief Information Officer; the Under Secretary of Defense (Personnel and Readiness); and the Director, Defense Finance and Accounting Service. This report will also be available at no charge on our Web site at http://www.gao.gov.

If you or your staff have any questions on matters discussed in this report, please contact me at (202) 512-3439 or hiter@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI.

Objectives, Scope, and Methodology

Our objectives were to determine whether the Department of Defense (DOD) has (1) established an effective governance structure; (2) developed program plans, including supporting workforce plans; (3) performed effective configuration management; (4) developed well-defined business enterprise architecture (BEA) products; and (5) addressed our other recommendations.

To determine whether DOD has established an effective governance structure for its efforts, we reviewed program documentation—such as approved charters for the Executive and Steering Committees, the Domain Owners Integration Team (DO/IT), the Transformation Support Office, and the Defense Business Systems Management Committee—and the communications strategy and supporting documents. We compared these documents with the elements in our Enterprise Architecture Management Maturity Framework and federal Chief Information Officer (CIO) Council guidance.

To determine whether DOD has developed program plans, including supporting workforce plans, we interviewed the Director and deputy program director, and the assistant deputy directors for communications, strategic planning, and transition planning. We also reviewed draft plans that showed the department’s intent to address our prior recommendations for the content previously missing from the “As Is” architecture and the transition plan. We also reviewed the department’s March 15, 2005, annual report to Congress, briefing slides on the department’s BEA development approach, and the various statements of work for the contractor responsible for extending and evolving the architecture. For human capital, we reviewed program organization charts and position descriptions for key program officials. In addition, we interviewed key program officials, such as the assistant deputy directors for communications, strategic planning, and enterprise architecture, to discuss their roles and responsibilities.

To determine whether effective configuration management was being performed, we reviewed the configuration management plan and associated procedures, and the draft configuration control board charter. We compared these documents with best practices, including the federal CIO Council’s Practical Guide, to determine the extent to which DOD had adopted key management practices. In addition, we reviewed meeting minutes to determine whether the board was operating effectively and performing activities according to best practices. We also interviewed program officials, including the Chief Architect and the Configuration Control Board Chair, to discuss the process and its effect on the department’s ability to develop and maintain the BEA products.

To determine whether DOD had developed well-defined BEA products, we reviewed the latest BEA releases (i.e., Releases 2.2, 2.2.1, and 2.3 and the March 2005 Update) and the program’s verification and validation contractor’s reports documenting its assessment of Releases 2.2 and 2.3 and the January 2005 Update of the architecture. To determine whether these BEA releases addressed our prior recommendations on missing architecture content and inconsistencies, we requested contractual change requests related to our recommendations. Program officials, including the program director and Chief Architect, stated that change requests to address our recommendations do not exist. We also reviewed the verification and validation contractor’s assessment of DOD’s efforts to address its outstanding comments on prior versions of the BEA and DOD stakeholders’ comments on Release 2.2 of the BEA. Further, we reviewed DOD’s approach to developing the architecture products since Release 1.0 and compared it with relevant guidance, such as DOD’s architecture framework. We also observed architecture walk-through sessions held by program officials to discuss concerns and provide progress updates. In addition, we interviewed program officials, including the Special Assistant for Business Transformation, Deputy Under Secretary of Defense (Financial Management), Chief Architect, the Configuration Control Board Chair, and the verification and validation contractor to discuss the development and maintenance of the BEA products.

To determine the status of DOD’s efforts to address our other recommendations related to BEA development and maintenance, we reviewed program documentation, such as the draft quality assurance plan, and compared them with the elements in our Enterprise Architecture Management Maturity Framework. We requested updates to the Information Technology Portfolio Management Policy and the position description for the Chief Architect. We also interviewed program and contractor officials, such as the Director and deputy program director, and the assistant deputy directors for quality assurance and communications.

To augment our documentation reviews and analyses, we attended regularly scheduled meetings, such as the DO/IT meetings, the program execution status meetings, and configuration control board meetings. We also held monthly teleconferences with the program and deputy program directors to discuss any issues and to obtain explanations or clarification on the results of our audit work.

We did not independently validate cost and budget information provided by the department.

We conducted our work primarily at DOD headquarters in Arlington, Virginia, and we performed our work from July 2004 through May 2005, in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from the Secretary of Defense or his designee. Written comments from the Special Assistant for Business Transformation in the Office of the Under Secretary of Defense (Acquisition, Technology, and Logistics) and the Deputy Under Secretary of Defense (Financial Management) are addressed in the “Agency Comments” section of this report and are reprinted in appendix V.

Status of Prior Recommendations on DOD’s Development and Maintenance of Its Business Enterprise Architecture

Implemented?

GAO-01-525: Information Technology: Architecture Needed to Guide Modernization of DOD’s Financial Operations. May 17, 2001. (1) The Secretary immediately issue a Department of Defense (DOD) policy that directs the development, implementation, and maintenance of a business enterprise architecture (BEA).

DOD has developed the Information Technology Portfolio Management policy. While this policy, in conjunction with the overarching Global Information Grid policy, assigns responsibilities for the development, implementation, and maintenance of the BEA, it does not provide for having accountability for and approval of updates to the architecture processes for architecture oversight and control and architecture review and validation, and it does not address the issuance of waivers for business systems that are not compliant with the BEA but are nevertheless justified on the basis of documented analysis. Program officials stated that the department plans to revise this policy, but they did not provide a time frame for doing so. (2) The Secretary immediately modify the Senior Financial Management Oversight Council’s charter to designate the Deputy Secretary of Defense as the Council Chair and the Under Secretary of Defense (Comptroller) as the Council vice-Chair; and empower the council to serve as DOD’s BEA steering committee, giving it the responsibility and authority to ensure that a DOD BEA is developed and maintained in accordance with the DOD Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance (C4ISR) Architecture Framework.

We previously reported that DOD had established the Executive and Steering Committees, which were advisory in nature. The department also had established the Domain Owners Integration Team (DO/IT) and stated that these three bodies were responsible for governing the program. However, these groups had not been assigned responsibilities for directing, overseeing, and approving the BEA. According to key department officials, these three entities will be replaced. Specifically, in February 2005, DOD established the Defense Business Systems Management Committee (DBSMC), which replaced the Executive and Steering Committees. According to its charter, the DBSMC is accountable and responsible for the program. The department plans to establish an underlying management structure to support the DBSMC in carrying out its roles and responsibilities. In addition, program officials have stated the department’s intention to replace the DO/IT with the DOD Enterprise Transformation Integration Group whose roles and responsibilities and concept of operations have yet to be defined.

Implemented?

Yes Partial No DOD comments and our assessment (3) The Secretary immediately make the Assistant Secretary of Defense (Command, Control, Communications & Intelligence), in collaboration with the Under Secretary of Defense (Comptroller), accountable to the Senior Financial Management Oversight Council for developing and maintaining a DOD BEA.

The Assistant Secretary of Defense (Networks and Information Integration)/Chief Information Officer (ASD(NII)/CIO) is a member of the recently established DBSMC; however, it is not known how this committee will operate.

DOD established a program office in July 2001. DOD also appointed a Chief Architect, and, according to the department, it has adequate program funding and staff for developing and maintaining its architecture. However, DOD has yet to define the roles and responsibilities for the Chief Architect or provide a time frame for doing so. defines the architecture process and approach; develops the baseline architecture, the target facilitates the use of the architecture to guide business management modernization projects and investments; and (4) The ASD(NII)/CIO report at least quarterly to the Senior Financial Management Oversight Council on the Chief Architect’s progress in developing a BEA, including the Chief Architect’s adherence to enterprise architecture policy and guidance from the Office of Management and Budget (OMB), the CIO Council, and DOD.

The ASD(NII)/CIO is a member of the recently established DBSMC; however, it is not known how this committee will operate.

The Steering Committee was briefed monthly by the program office on various program activities until June 2004, when it held its last meeting. As a result, the Steering Committee was not updated on the content and status of Releases 2.2 and 2.3 and the January 2005 and March 2005 Updates of the BEA. According to program officials, the DBSMC held an executive session in February 2005 and its second meeting in April 2005, and the committee will initially hold monthly meetings. (5) The Senior Financial Management Oversight Council report to the Secretary of Defense every 6 months on progress in developing and implementing a BEA.

The Deputy Chief Financial Officer briefed the Secretary of Defense in November 2003 on behalf of DOD’s Comptroller, who chairs the Executive Committee. According to the program director, the Secretary of Defense has not been briefed since November 2003 on the department’s progress in developing and implementing the BEA.

Implemented?

Yes Partial No DOD comments and our assessment (6) The Secretary report every 6 months to the congressional defense authorizing and appropriating committees on progress in developing and implementing a BEA.

Senate Report 107-213 directs that the department report every 6 months on the status of the BEA effort. DOD submitted status reports on January 31 and July 31, 2003; January 31 and July 30, 2004; and March 15, 2005. The 2003 and 2004 reports were submitted by DOD’s Comptroller but were not signed by the members of the Executive or Steering Committees. The 2005 report was signed by the Acting Under Secretary of Defense for Acquisition, Technology, and Logistics, who is the vice-chair of the DBSMC.

GAO-03-458: DOD Business Systems Modernization: Improvements to Enterprise Architecture Development and Implementation Efforts Needed. February 28, 2003. (1) The Secretary of Defense ensure that the enterprise architecture executive committee members are singularly and collectively made explicitly accountable to the Secretary for the delivery of the enterprise architecture, including approval of each release of the architecture.

We previously reported that DOD had established the Executive and Steering Committees, which were advisory in nature. The department had also established the DO/IT and stated that these three bodies were responsible for governing the program. However, these groups had not been assigned responsibilities for directing, overseeing, and approving the BEA. According to key department officials, these three entities will be replaced. Specifically, in February 2005, DOD established the DBSMC, which replaced the Executive and Steering Committees. According to its charter, the DBSMC is accountable and responsible for the program. The department plans to establish an underlying management structure to support the DBSMC in carrying out its roles and responsibilities. In addition, program officials have stated the department’s intention to replace the DO/IT with the DOD Enterprise Transformation Integration Group, whose roles and responsibilities and concept of operations have yet to be defined. (2) The Secretary of Defense ensure that the enterprise architecture program is supported by a proactive marketing and communication program.

DOD has a strategic communications plan; however, the plan has yet to be implemented. According to the communications team, its activities have been limited to raising awareness because it lacks the authority to fully implement the other components of its plan, such as achieving buy-in. According to program officials, the department intends to revise the governance structure, including the communications strategy, in September 2005.

Implemented?

Yes Partial No DOD comments and our assessment (3) The Secretary of Defense ensure that the quality assurance function includes the review of adherence to process standards and reliability of reported program performance, is made independent of the program management is not performed by subject matter experts involved in the development of key architecture products.

DOD has established a quality assurance function; however, this function does not address process standards and program performance, nor is it an independent function. Further, DOD subject matter experts continue to be involved in the quality assurance function. Program officials stated that the department had yet to address our recommendation, and they could not provide a time frame for when they would begin addressing this recommendation.

GAO-03-1018: DOD Business Systems Modernization: Important Progress Made to Develop Business Enterprise Architecture, but Much Work Remains. September 19, 2003. (1) The Secretary of Defense or his appropriate designee implement the core elements in our Enterprise Architecture Framework for Assessing and Improving Enterprise Architecture Management that we identify in this report as not satisfied, including ensuring that minutes of the meetings of the executive body charged with directing, overseeing, and approving the architecture are prepared and maintained.

DOD has taken some actions, but these actions do not fully address our previous concerns. For example, DOD has begun to revise its governance structure to provide for improved management and oversight, such as establishing the DBSMC and assigning it accountability and responsibility for directing, overseeing, and approving the BEA; and developed a configuration management plan and the related procedures, and established a configuration control board.

However, the department has not established additional governance entities to support the DBSMC and outlined their roles and responsibilities; updated the policy for BEA development, maintenance, and implementation; included the missing scope and detail in the BEA; finalized, approved, and effectively implemented the plan, procedures, and charter governing the configuration management process; and metrics. (2) The Secretary of Defense or his appropriate designee update version 1.0 of the architecture to include the 29 key elements governing the “As Is” architectural content that our report identified as not being fully satisfied.

Of the 29 elements, program officials stated that 3 were not applicable and that it planned to address an additional 11 by January 2005. However, these officials did not provide any documentation to support this statement. Instead, they provided a draft plan that shows the department’s intent to develop a detailed action plan to guide the development of an “As Is” architecture. According to program officials, they plan to update the “As Is” architectural description in September 2005.

Implemented?

Yes Partial No DOD comments and our assessment (3) The Secretary of Defense or his appropriate designee update version 1.0 of the BEA to include the 30 key elements governing the “To Be” architectural content that our report identified as not being fully satisfied.

DOD officials have provided no evidence that this recommendation has been addressed or that it intends to implement this recommendation. (4) The Secretary of Defense or his appropriate designee update version 1.0 to ensure that “To Be” architecture artifacts are internally consistent, to include addressing the inconsistencies described in this report, as well as including user instructions or guidance for easier architecture navigation and use.

DOD officials have provided no evidence that this recommendation has been addressed or that it intends to implement this recommendation. (5) The Secretary of Defense or his appropriate designee update version 1.0 of the architecture to include (a) the 3 key elements governing the transition plan content that our report identified as not being fully satisfied and (b) those system investments that will not become part of the “To Be” architecture, including time frames for phasing out those systems.

DOD officials provided a draft plan that shows the department’s intent to develop a detailed action plan to guide the development of the transition plan; however, the draft plan does not provide time frames for doing so. According to program officials, the department will issue a revised transition plan in September 2005; but, this version will not fully address our recommendation. (6) The Secretary of Defense or his appropriate designee update version 1.0 of the architecture to address comments made by the verification and validation contractor.

According to program officials, of the 299 outstanding comments, 137 have been addressed in Release 2.3 and earlier releases, 100 were not applicable, and the remaining 62 will be addressed in future releases. These officials did not provide any documentation supporting their rationale for the 100 that they considered not applicable nor did they provide plans for addressing the 62 remaining comments. The verification and validation contractor stated that of the 137 comments that program officials stated had been addressed, 35 had been addressed, 22 were not applicable because they were either duplicate or no longer relevant based on updates to prior releases, 22 had yet to be addressed, and 58 were not assessed. The contractor has yet to provide its assessment on the 100 comments that DOD said were not applicable. (7) The Secretary of Defense or his appropriate designee develop a well-defined, near-term plan for extending and evolving the architecture and ensure that this plan includes addressing our recommendations, defining roles and responsibilities of all stakeholders involved in extending and evolving the architecture, explaining dependencies among planned activities, and defining measures of activity progress.

As discussed in this report, DOD has not developed explicit detailed plans to guide day-to-day program activities and to enable it to evaluate its progress. According to program officials, the department will develop a program baseline by September 30, 2005.

Summary of Several Architecture Frameworks

There are various enterprise architecture frameworks that an organization can follow. Although these frameworks differ in their nomenclatures and modeling approaches, they consistently provide for defining an enterprise’s operations in both (1) logical terms, such as interrelated business processes and business rules, information needs and flows, and work locations and users, and (2) technical terms, such as hardware, software, data, communications, and security attributes and performance standards. The frameworks also provide for defining these perspectives for both the enterprise’s current or “As Is” environment and its target or “To Be” environment, as well as a transition plan for moving from the “As Is” to the “To Be” environment.

For example, John A. Zachman developed a structure or framework for defining and capturing an architecture. This framework provides for six windows from which to view the enterprise, which Zachman terms “perspectives” on how a given entity operates: those of (1) the strategic planner, (2) the system user, (3) the system designer, (4) the system developer, (5) the subcontractor, and (6) the system itself. Zachman also proposed six models that are associated with each of these perspectives; these models describe (1) how the entity operates, (2) what the entity uses to operate, (3) where the entity operates, (4) who operates the entity, (5) when entity operations occur, and (6) why the entity operates. Zachman’s framework provides a conceptual schema that can be used to identify and describe an entity’s existing and planned components and their relationships to one another before beginning the costly and time- consuming efforts associated with developing or transforming the entity.

Since Zachman introduced his framework, a number of other frameworks has been proposed. In February 1998, DOD directed its components to use its C4ISR Architecture Framework, Version 2.0. In August 2003, the department released Version 1.0 of the DOD Architecture Framework (DODAF)—an evolution of the C4ISR Architecture Framework, which supersedes the C4ISR. The DODAF defines the type and content of the architectural artifacts, as well as the relationships among the artifacts that are needed to produce a useful architecture. Briefly, the framework decomposes an architecture into three primary views: operational, systems, and technical standards. See figure 3 for an illustration of these three views. According to DOD, the three interdependent views are needed to ensure that IT systems support operational needs, and that they are developed and implemented in an interoperable and cost-effective manner.

In September 1999, the federal CIO Council published the Federal Enterprise Architecture Framework (FEAF), which is intended to provide federal agencies with a common construct on which to base their respective architectures and to facilitate the coordination of common business processes, technology insertion, information flows, and system investments among federal agencies. FEAF describes an approach, including models and definitions, for developing and documenting architecture descriptions for multiorganizational functional segments of the federal government. Similar to most frameworks, FEAF’s proposed models describe an entity’s business, the data necessary to conduct the business, applications to manage the data, and technology to support the applications.

More recently, the Office of Management and Budget (OMB) established the Federal Enterprise Architecture (FEA) Program Management Office to develop a federated enterprise architecture in the context of five “reference models, and a security and privacy profile that overlays the five models.”

The Business Reference Model is intended to describe the federal government’s businesses, independent of the agencies that perform them. This model consists of four business areas: (1) services for citizens, (2) mode of delivery, (3) support delivery of services, and (4) management of government resources. It serves as the foundation for the FEA. OMB expects agencies to use this model, as part of their capital planning and investment control processes, to help identify opportunities to consolidate information technology (IT) investments across the federal government. Version 2.0 of this model was released in June 2003.

The Performance Reference Model is intended to describe a set of performance measures for major IT initiatives and their contribution to program performance. According to OMB, this model will help agencies produce enhanced performance information; improve the alignment and better articulate the contribution of inputs, such as technology, to outputs and outcomes; and identify improvement opportunities that span traditional organizational boundaries. Version 1.0 of this model was released in September 2003.

The Service Component Reference Model is intended to identify and classify IT service (i.e., application) components that support federal agencies and promote the reuse of components across agencies. This model is intended to provide the foundation for the reuse of applications, application capabilities, components (defined as “a self- contained business process or service with predetermined functionality that may be exposed through a business or technology interface”), and business services. According to OMB, this model is a business-driven, functional framework that classifies service components with respect to how they support business and/or performance objectives. Version 1.0 of this model was released in June 2003.

The Data Reference Model is intended to describe, at an aggregate level, the types of data and information that support program and business line operations and the relationships among these types. This model is intended to help describe the types of interactions and information exchanges that occur across the federal government. Version 1.0 of this model was released in September 2004.

The Technical Reference Model is intended to describe the standards, specifications, and technologies that collectively support the secure delivery, exchange, and construction of service components. Version 1.1 of this model was released in August 2003.

The Security and Privacy Profile is intended to provide guidance on designing and deploying measures that ensure the protection of information resources. OMB has released Version 1.0 of the profile.

Description of DOD Architecture Framework Products, Version 1.0

Executive-level summary information on the scope, purpose, and context of the architecture Architecture data repository with definitions of all terms used in all products High-Level Operational Concept Graphic High-level graphical/textual description of what the architecture is supposed to do, and how it is supposed to do it Graphic depiction of the operational nodes (or organizations) with needlines that indicate a need to exchange information Information exchanged between nodes and the relevant attributes of that exchange Command structure or relationships among human roles, organizations, or organization types that are the key players in an architecture Operations that are normally conducted in the course of achieving a mission or a business goal, such as capabilities, operational activities (or tasks), input and output flows between activities, and input and output flows to/from activities that are outside the scope of the architecture One of three products used to describe operational activity—identifies business rules that constrain operations One of three products used to describe operational activity—identifies business process responses to events One of three products used to describe operational activity—traces actions in a scenario or sequence of events Documentation of the system data requirements and structural business process rules of the operational view Identification of systems nodes, systems, and systems items and their interconnections, within and between nodes Specific communications links or communications networks and the details of their configurations through which systems interface Relationships among systems in a given architecture; can be designed to show relationships of interest (e.g., system-type interfaces, planned versus existing interfaces)

Mapping of relationships between the set of operational activities and the set of system functions applicable to that architecture Characteristics of the system data exchanged between systems Systems Performance Parameters Matrix Quantitative characteristics of systems and systems hardware/software items, their interfaces, and their functions Planned incremental steps toward migrating a suite of systems to a more efficient suite, or toward evolving a current system to a future implementation Emerging technologies and software/hardware products that are expected to be available in a given set of time frames and that will affect future development of the architecture One of three products used to describe system functionality—identifies constraints that are imposed on systems functionality due to some aspect of systems design or implementation One of three products used to describe system functionality—identifies responses of a system to events One of three products used to describe system functionality—lays out the sequence of system data exchanges that occur between systems (external and internal), system functions, or human role for a given scenario Physical implementation of the Logical Data Model entities (e.g., message formats, file structures, and physical schema)

Comments from the Department of Defense

GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Cynthia Jackson, Assistant Director; Barbara Collier; Joanne Fiorino; Neelaxi Lakhmani; Anh Le; Freda Paintsil; Randolph Tekeley; and William Wadsworth made key contributions to this report.