DOD Business Systems Modernization: Long-Standing Weaknesses in Enterprise Architecture Development Need to Be Addressed
Published: 2005-07-22T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 directed the Department of Defense (DOD) to develop, by September 2005, a well-defined business enterprise architecture (BEA) and a transition plan. GAO has made numerous recommendations to assist the department in successfully doing so. As part of ongoing monitoring of the architecture, GAO assessed whether the department had (1) established an effective governance structure; (2) developed program plans, including supporting workforce plans; (3) performed effective configuration management; (4) developed well-defined BEA products; and (5) addressed GAO's other recommendations.
section_title: What GAO Found
paragraphs: To effectively and efficiently modernize its nonintegrated and duplicative business operations and systems, it is essential for DOD to develop and use a well-defined BEA. However, it does not have such an architecture, and the products that it has produced do not provide sufficient content and utility to effectively guide and constrain ongoing and planned systems investments. As a result, despite spending almost 4 years and about $318 million, DOD does not have an effective architecture program. The current state of the program is due largely to long-standing architecture management weaknesses that GAO has made recommendations over the last 4 years to correct. In particular, DOD has not done the following: (1) established an effective governance structure, including an effective communications strategy, to achieve stakeholder buy-in. In particular, the structure that has been in place since 2001 lacks the requisite authority and accountability to be effective, and the key entities that made up this structure have not performed according to their respective charters; (2) developed program plans that explicitly identify measurable goals and outcomes to be achieved, nor has it defined the tasks to be performed to achieve the goals and outcomes, the resources needed to perform these tasks, or the time frames within which the tasks will be performed. DOD also has not assessed, as part of its program planning, the workforce capabilities that it needs in order to effectively manage its architecture efforts, nor does it have a strategy for doing so; (3) performed effective configuration management, which is a formal approach to controlling product parts to ensure their integrity. The configuration management plan and the charter for the configuration control board are draft; the board has limited authority; and, after 4 years of development, the department has not assigned a configuration manager; (4) developed a well-defined architecture. The latest versions of the architecture do not include products describing the "As Is" business and technology environments and a transition plan for investing in business systems. In addition, the versions that have been produced for the "To Be" environment have not had a clearly defined purpose and scope, are still missing important content, and contain products that are neither consistent nor integrated; and (5) fully addressed other GAO recommendations. DOD recognizes that these weaknesses need to be addressed and has recently assigned a new BEA leadership team. DOD also has either begun steps to or stated its intentions to (1) revise its governance structure; (2) develop a program baseline, by September 30, 2005, that will be used as a managerial and oversight tool to allocate resources, manage risks, and measure and report progress; and (3) revise the scope of the architecture and establish a new approach for developing it. However, much remains to be accomplished to establish an effective architecture program. Until it does, its business systems modernization effort will remain a high-risk program.

section_title: Letter
paragraphs: We first designated business systems modernization efforts within the  Department of Defense (DOD) as a high-risk program in 1995, and we  continue to designate it as such today. As our research on successful  public and private sector organizations has shown, attempting such large- scale systems modernization programs without a well-defined enterprise  architecture often results in systems that are duplicative; are not well  integrated; are unnecessarily costly to manage, maintain, and operate; and  do not effectively optimize mission performance. To help DOD transform  its operations, we recommended in 2001 that the department develop an  enterprise architecture to guide and constrain its multibillion dollar annual  investment in business systems. In July 2001, the department initiated a  business management modernization program to, among other things,  develop a business enterprise architecture (BEA).
Recognizing the importance of DOD’s efforts to transform its business  operations and systems through the use of an enterprise architecture,  Congress included provisions in the National Defense Authorization Act for  Fiscal Year 2003 aimed at having the department develop and effectively  implement a well-defined BEA. In addition, the Ronald W. Reagan National  Defense Authorization Act for Fiscal Year 2005 again directs DOD to  develop a well-defined BEA and a transition plan by September 2005.
As part of our ongoing monitoring of the BEA, we assessed whether DOD  had (1) established an effective governance structure; (2) developed  program plans, including supporting workforce plans; (3) performed  effective configuration management; (4) developed well-defined BEA  products; and (5) addressed our other recommendations.
We performed our work from July 2004 through May 2005, in accordance  with generally accepted government auditing standards. We conducted this  review under the Comptroller General’s authority and have addressed this  report to the committees of jurisdiction. Details on our objectives, scope,  and methodology are contained in appendix I.
subsections: section_title: Background
paragraphs: DOD is a massive and complex organization. In fiscal year 2004, the  department reported that its operations involved $1.2 trillion in assets, $1.7  trillion in liabilities, over 3.3 million military and civilian personnel, and  over $605 billion in net cost of operations. For fiscal year 2005, the  department received appropriations of about $417 billion. Moreover,  execution of its operations spans a wide range of defense organizations,  including the military services and their respective major commands and  functional activities, numerous large defense agencies and field activities,  and various combatant and joint operational commands, which are  responsible for military operations for specific geographic regions or  theaters of operations.
In executing these military operations, the department performs an  assortment of interrelated and interdependent business functions,  including logistics management, procurement, health care management,  and financial management. DOD reports that, in order to support these  business functions, it currently relies on about 4,200 systems—including  accounting, acquisition, finance, logistics, and personnel. As we have  previously reported, this systems environment is overly complex and error  prone and is characterized by (1) little standardization across the  department, (2) multiple systems performing the same tasks, (3) the same  data stored in multiple systems, and (4) manual data entry into multiple  systems. These problems continue despite the department’s spending  billions of dollars annually to operate, maintain, and modernize its business  systems. DOD received approximately $13.3 billion for fiscal year 2005 to  operate, maintain, and modernize this environment. In addition, our  reports continue to show that the department’s stovepiped and duplicative  systems contribute to fraud, waste, and abuse. Of the 25 areas on GAO’s  governmentwide “high-risk” list, 8 are DOD program areas, and the  department shares responsibility for 6 other high-risk areas that are  governmentwide in scope.
Because of the department’s size and complexity, modernizing its business  systems is a huge management challenge that we first designated as one of  the department’s high-risk areas in 1995, and we continue to do so today.  To help meet this challenge, DOD established its business systems  modernization program in 2001. As we testified in 2003, one of the seven  key elements that are necessary to successfully execute this modernization  program is to establish and implement an enterprise architecture.  Subsequently, in its Fiscal Year 2004 Performance and Accountability  Report, DOD acknowledged that deficiencies in its systems and business  processes hindered the department’s ability to collect and report financial  and performance information that was accurate, reliable, and timely. The  DOD report noted that to address its systemic problems and assist in the  modernization of its business operations, the department had undertaken  the development and implementation of a BEA.
subsections: section_title: Enterprise Architecture Is Critical to Achieving Successful Modernization
paragraphs: Effective use of an enterprise architecture, or a modernization blueprint, is  a trademark of successful public and private organizations. For more than  a decade, we have promoted the use of architectures to guide and constrain  systems modernization, recognizing them as a crucial means to a  challenging goal: agency operational structures that are optimally defined  in both business and technological environments. Congress, the Office of  Management and Budget (OMB), and the federal Chief Information Officer  (CIO) Council have also recognized the importance of an architecture- centric approach to modernization. The Clinger-Cohen Act of 1996  mandates that an agency’s CIO develop, maintain, and facilitate the  implementation of an information technology (IT) architecture. Further,  the E-Government Act of 2002 requires OMB to oversee the development  of enterprise architectures within and across agencies. In addition, OMB  has issued guidance that, among other things, requires system investments  to be consistent with these architectures.
section_title: What Is an Enterprise Architecture?
paragraphs: An enterprise architecture provides a clear and comprehensive picture of  an entity, whether it is an organization (e.g., a federal department) or a  functional or mission area that cuts across more than one organization  (e.g., financial maagement). This picture consists of snapshots of both the  enterprise’s current or “As Is” environment and its target or “To Be”  environment, as well as a capital investment road map for transitioning  from the current to the target environment. These snapshots consist of  “views,” which are one or more architecture products that provide logical  or technical representations of the enterprise.
The suite of products and their content that form a given entity’s enterprise  architecture are largely governed by the framework used to develop the  architecture. Since the 1980s, various architecture frameworks have  emerged and been applied. See appendix III for a discussion of these  various frameworks.
The importance of developing, implementing, and maintaining an  enterprise architecture is a basic tenet of both organizational  transformation and systems modernization. Managed properly, an  enterprise architecture can clarify and help to optimize the  interdependencies and relationships among an organization’s business  operations and the underlying IT infrastructure and applications that  support these operations. Employed in concert with other important  management controls, such as portfolio-based capital planning and  investment control practices, architectures can greatly increase the  chances that an organization’s operational and IT environments will be  configured to optimize its mission performance. Our experience with  federal agencies has shown that investing in IT without defining these  investments in the context of an architecture often results in systems that  are duplicative, not well integrated, and unnecessarily costly to maintain  and interface.
section_title: DOD’s Business Management Modernization Program: A Brief Description and Chronology
paragraphs: In July 2001, the Secretary of Defense established the Business  Management Modernization Program (BMMP) to improve the efficiency  and effectiveness of DOD’s business operations and provide the  department’s leaders with accurate and timely information through the  development and implementation of a BEA. At that time, the Secretary  tasked the Under Secretary of Defense (Comptroller), in coordination with  the Under Secretary of Defense for Acquisition, Technology, and Logistics  (USD(AT&L)) and the Assistant Secretary of Defense (Networks and  Information Integration)/Chief Information Officer (ASD(NII)/CIO), with  responsibility for overseeing the program. To accomplish this, in October  2001, the Comptroller established governance bodies and assigned them  responsibilities associated with developing, maintaining, and implementing  the BEA. These entities and their respective roles and responsibilities are  shown in table 1. DOD is currently revising its BEA governance structure,  including recently eliminating its long-standing governance entities. These  revisions are discussed later in this report.
Also in 2001, the BMMP program office was established to execute the  program’s day-to-day activities, including implementing internal  management controls and other mechanisms to provide reasonable  assurance that the office would develop and implement an effective BEA.  The office is led by a program director and comprised of seven program  divisions, each of which is headed by an assistant deputy director. Figure 1  is a simplified diagram of the organizational structure of the program  office, and table 2 shows the roles and responsibilities of the program  divisions.
Initially, DOD planned to develop the architecture in 1 year. Subsequently,  the department stated that it would develop its architecture in three  increments, each increment addressing a subset of objectives and  consisting of specific architecture releases. Table 3 shows these  increments, the corresponding architecture releases, and the planned  completion dates for the increments.
To develop the architecture, DOD entered into a 5-year, $95 million  contract with International Business Machines (IBM) in April 2002, under  which the department has issued a series of task orders aimed at  developing the architecture. In 2004, DOD increased the contract amount  to $250 million; however, the contract did not provide a reason for this  increase, and program officials have yet to provide an explanation. As of  September 2004, DOD reported that it had obligated approximately $318  million for the program, which is primarily for contractor support.
section_title: Prior Reviews of DOD’s Architecture Efforts Have Identified Many Weaknesses and Challenges
paragraphs: Over the last 4 years, we have identified the need for, and reviewed DOD’s  efforts to develop an enterprise architecture for modernizing its business  operations and systems, and we have made a series of recommendations to  assist the department in successfully developing the architecture and using  it to guide and constrain its ongoing and planned business systems  investments.
In particular, we reported in May 2001 that the department did not have an  architecture for its financial and financial-related business operations, nor  did it have the management structures, processes, and controls in place to  effectively develop and implement one. We concluded that continuing to  spend billions of dollars on new and modified systems would result in more  processes and systems that were duplicative, not interoperable,  unnecessarily costly to maintain and interface, and not optimizing mission  performance and accountability. We made eight recommendations to the  Secretary of Defense that were aimed at providing the means for effectively  developing and implementing an architecture and limiting DOD  components’ systems investments until it had a well-defined architecture  and the means to enforce it. In July 2001, DOD initiated the BMMP.
In February 2003, we reported that the department was following some  architecture best practices, such as establishing a program office to be  responsible for managing the enterprise architecture development effort.  We also reported challenges and weaknesses in DOD’s architecture efforts.  For example, we reported that DOD had not yet (1) established a  governance structure and the process controls needed to ensure ownership  of and accountability for the architecture across the department; (2) clearly  communicated to intended stakeholders its purpose, scope, and approach  for developing the architecture; and (3) defined and implemented an  independent quality assurance process. We reiterated our earlier  recommendations and made six new recommendations aimed at enhancing  DOD’s ability to develop its architecture and to guide and constrain its  business systems modernization investments.
In March 2003, we reported that DOD’s draft release of its architecture did  not include a number of items that were recommended by relevant  architectural guidance, and that DOD’s plans would not fully satisfy the  requirements of the National Defense Authorization Act for Fiscal Year  2003. For example, the draft architecture did not include a “To Be”  security view, which would define the security requirements—including  relevant standards to be applied in implementing security policies,  procedures, and controls. DOD officials generally agreed, and they stated  that subsequent releases of the architecture would provide these missing  details.
In July and September 2003, we reported that the initial release of the  department’s architecture, including the transition plan, did not adequately  address statutory requirements and other relevant architectural  requirements. For example, the description of the “As Is” environment did  not include (1) the current business operations in terms of entities and  people who perform the functions, processes, and activities and (2) the  locations where the functions, processes, and activities are performed. The  description of the “To Be” environment did not include actual systems to be  developed or acquired to support future business operations and the  physical infrastructure that would be needed to support the business  systems. The transition plan did not include time frames for phasing out  existing systems within DOD’s then reported inventory of about 2,300  business systems. We concluded that the department’s initial release of  the architecture did not contain sufficient scope and detail either to satisfy  the act’s requirements or to effectively guide and constrain  departmentwide business systems modernization. In our September 2003  report, we reiterated the open recommendations that we had made in our  May 2001 and February 2003 reports, and we made 10 new  recommendations to, among other things, improve DOD’s efforts for  developing the next release of the architecture.
In March and July 2004, we testified that DOD’s substantial long-standing  financial and business management problems adversely affected the  economy, effectiveness, and efficiency of its operations and had resulted in  a lack of adequate transparency and appropriate accountability across all  major business areas. Further, we said that substantial work remained  before the BEA would begin to have a tangible impact on improving the  department’s overall business operations. We concluded that until DOD  completed a number of actions, including developing a well-defined BEA,  its business systems modernization efforts would be at a high risk of  failure.
In May 2004, we reported that after 3 years of effort and over $203 million  in obligations, DOD had not made significant change in the content of the  BEA or in its approach to investing billions of dollars annually in existing  and new systems. We reported that few actions had been taken to address  the recommendations we had made in our September 2003 report, which  were aimed at improving the department’s plans for developing the next  release of the architecture and implementing the institutional means for  selecting and controlling both planned and ongoing business systems  investments. We also reported that DOD had still not adopted key  architecture management best practices that we had recommended, such  as assigning accountability and responsibility for directing, overseeing, and  approving the architecture and explicitly defining performance metrics to  evaluate the architecture’s quality, content, and utility. Further, DOD had  not added the scope and detail to its architecture that we had previously  identified as missing. For example, in the latest release of the BEA— Release 2.0—DOD did not provide sufficient descriptive content related to  future business operations and supporting technology to permit effective  acquisition and implementation of system solutions and associated  operational change. Moreover, the department had not yet explicitly  defined program plans, including milestones, detailing how it intended to  extend and evolve the architecture to incorporate this missing content. We  concluded that the future of DOD’s architecture development and  implementation activities was at risk, which in turn placed the  department’s business transformation effort in jeopardy of failing.  Therefore, we added that it was imperative that the department move  swiftly to implement our open recommendations. Because many of our  prior recommendations remained open, we did not make any new  recommendations in our May 2004 report, but we reiterated the open  recommendations that we had made in our May 2001, February 2003, and  September 2003 reports.
In November 2004 and April 2005, we testified that for DOD to  successfully transform its business operations, it would need a  comprehensive and integrated business transformation plan; people with  the skills, responsibility, and authority to implement the plan; an effective  process and related tools, such as a BEA; and results-oriented performance  measures that would link institutional, unit, and individual personnel goals  and expectations to promote accountability for results. We testified that  over the last 3 years, we had made a series of recommendations to DOD  and suggested legislative changes that, if implemented, could help the  department move forward in establishing the means to successfully  address the challenges it faces in transforming its business operations. We  also testified that, after about 3 years of effort and over $203 million in  reported obligations, the architecture’s content and the department’s  approach to investing billions of dollars annually in existing and new  systems had not changed significantly, and that the program had yielded  very little, if any, tangible improvements in DOD’s business operations.
section_title: DOD Has Yet to Implement Effective Governance and Communications, but Improvements Are Under Way
paragraphs: Long-standing weaknesses in DOD’s BEA governance structure and  communications strategy still remain. While DOD has established a new  senior committee to oversee its business transformation efforts, including  BEA development, much remains to be accomplished before proposed  governance and communications concepts are fully defined and  implemented. Until the department has made its intended governance and  communications concept operational, the success of DOD’s architecture  efforts will remain in doubt.
subsections: section_title: Long-standing Program Governance Weaknesses Remain, Although Recent Proposals Are Intended to Address Weaknesses
paragraphs: An enterprise architecture is a corporate asset that, among other things, is  intended to represent the strategic direction of the enterprise. Accordingly,  best practices recommend that to demonstrate commitment,  organizations should vest accountability and assign responsibility for  directing, overseeing, and approving the architecture to a committee or  group with representation from across the enterprise. Sustained support  and commitment by this committee to the architecture, as well as the  committee’s ownership of it, are critical to a successful enterprise  architecture development effort. We have previously recommended that  DOD establish this kind of architecture responsibility and accountability  structure. (See app. II for our prior recommendations and their current  status.)
During the last 4 years, DOD has relied on three primary management  entities to govern BEA development, maintenance, and implementation— the Executive Committee, the Steering Committee, and the Domain  Owners Integration Team (DO/IT). (See table 1 for their roles and  responsibilities.) This governance approach, however, does not assign  accountability and responsibility for directing, overseeing, and approving  the BEA to these entities either singularly or collectively. As we reported in  February 2003, the Executive and Steering Committees were advisory in  nature, and their responsibilities were limited to providing guidance to the  program office and advising the Comptroller and the Executive Committee.  Moreover, since they were established, neither the two committees nor the  DO/IT have adequately fulfilled their assigned responsibilities, as we  discuss below.
The Executive Committee was chartered to, among other things,  provide strategic direction to the Steering Committee, champion  program execution, and hold DOD components—including the military  services—responsible for program results. To accomplish these things,  the charter states that the committee should establish a meeting  schedule. However, no schedule was established, and the committee  met only four times in over 3½ years. Moreover, no minutes of the four  meetings were prepared, according to the program’s Acting Assistant  Deputy Director for Communications, and no other documentation  exists to demonstrate the committee’s performance of its chartered  functions. In fact, during numerous DO/IT meetings that we attended,  participants stated that the Executive Committee was not providing  strategic direction, nor was it championing program execution.
The Steering Committee was chartered to advise the Executive  Committee on program performance, serve as the forum for discussion  of component issues, and provide guidance to the program office.  According to the program’s Acting Assistant Deputy Director for  Communications, neither Executive Committee meeting minutes nor  any other documentation exists to demonstrate that the Steering  Committee advised the Executive Committee. Moreover, during the  Steering Committee meetings that we attended over the last 4 years, we  saw no evidence that this committee either planned to or actually did  advise the Executive Committee on program performance. While we did  observe in these meetings that issues were raised and discussed, which  is a chartered responsibility of the committee, we also observed that the  committee did not provide guidance and direction to the program office  during these meetings. The Steering Committee last met about 1 year  ago (June 2004).
The DO/IT was chartered to provide recommendations to the Steering  Committee; provide guidance regarding architecture updates and their  effects; and identify, address, and resolve domain and program issues.  The charter also states that the DO/IT was to ensure that its  representation included the military services. However, during the  Steering Committee meetings that we attended, DO/IT representatives  did not provide any recommendations to the Steering Committee, nor  did they provide guidance on architecture updates and their effects.  Moreover, the DO/IT did not include military service representatives,  and it did not establish any policies and procedures for how to address  and resolve issues. As a result, issues that were identified during  meetings were not resolved. For example, in July 2004, there were  discussions regarding the lack of involvement from the services, the  lack of detail in the architecture content, and the lack of clear  understanding of the roles and responsibilities among the domains and  the services. During this meeting, however, no decisions were made  about how these issues were to be resolved, and no actions were taken  to provide recommendations to the Steering Committee for resolving  the issues. As a result, we observed the same issues being discussed,  without resolution, 5 months later.
DOD has recently taken steps to improve the program’s governance  structure and, according to program officials, further steps are planned.  For example, the department has implemented the provisions in the Ronald  W. Reagan National Defense Authorization Act for Fiscal Year 2005, which  are aimed at increasing senior DOD leadership involvement in the program.  In particular, it includes DOD’s establishment of the Defense Business  Systems Management Committee (DBSMC) to replace both the Executive  and Steering Committees and to serve as the highest ranking governance  body overseeing business transformation. According to the DBSMC  charter, the committee is accountable and responsible for directing,  overseeing, and approving all program activities. Specific responsibilities  of the committee include  establishing strategic direction and plans for the business mission area  in coordination with the warfighting and enterprise information  environment mission areas;  approving business mission area transformation plans and initiatives  and coordinating transition planning in a documented program baseline  with critical success factors, milestones, metrics, deliverables, and  periodic program reviews;  establishing key metrics and targets by which to track business   establishing policies and approving the business mission area strategic  plan, the transition plan for implementation for business systems  modernization, the transformation program baseline, and the BEA; and  executing a comprehensive communications strategy.
Consistent with recent legislation, the DBSMC is chaired by the Deputy  Secretary of Defense, and the USD(AT&L) serves as the vice chair. Its  membership consists of senior leadership from across the department,  including the military service secretaries, the defense agency heads, and  the principal staff assistants. The department has also moved the program  office from the Comptroller to the USD(AT&L), reporting to the Special  Assistant for Business Transformation. According to DOD, this transfer of  functions and responsibilities will allow USD(AT&L) to establish the level  of activity necessary to support and coordinate the activities of the newly  established DBSMC.
Other entities may be established to support the DBSMC. According to  program officials, including the Program Director, the DO/IT has been  eliminated and may be replaced by a DOD Enterprise Transformation  Integration Group. Further, the DO/IT had identified the need for six  additional boards to assist the program office. These boards have not yet  been chartered, but potential board members have met to discuss the  boards’ roles, responsibilities, and operating procedures, as well as  program issues. However, the program director stated that not all of these  boards may be established under the new structure.
According to the Special Assistant for Business Transformation, a new  governance structure will be in place and operational by September 30,  2005. To this end, DOD officials told us that the DBSMC held its first  meeting in February 2005 to finalize its charter and member composition,  and that to move governance reforms forward, the committee will initially  hold monthly meetings.
The weaknesses in the BEA governance structure over the last 4 years,  according to program officials, are attributable to a lack of authority and  accountability in program leadership by the various management entities  (e.g., Executive and Steering Committees) and to the limited direction  provided by these entities. While DOD’s recent actions are intended to  address these root causes, almost 4 years and approximately $318 million  in obligations have been invested, and the department is still attempting to  establish an effective governance structure. Moreover, much remains to be  accomplished until DOD’s new governance structure becomes an  operational reality. Until it does, it is unlikely that the department will be  able to develop an effective BEA.
section_title: An Effective Communications Strategy Has Yet to Be Implemented, but Some Activities Are Under Way
paragraphs: Effective communications among architecture stakeholders are closely  aligned with effective governance. According to relevant guidance, once  initial stakeholder participation in an architecture program is achieved, a  communications strategy should be defined and implemented to facilitate  the exchange of information among architecture stakeholders about all  aspects of the program, such as program purpose, scope, methodologies,  progress, results, and key decisions. Such communication is essential to  executing an architecture program effectively, including obtaining  institutional buy-in for developing and using the architecture for corporate  decision making. Accordingly, in 2003 we recommended that DOD  provide for ensuring stakeholder commitment and buy-in through  proactive architecture marketing and communication activities. (See app.  II for our prior recommendation and its current status.)
In response, the program office defined and approved a strategic  communications plan in March 2004. According to the plan, its purpose is  to direct the flow of information to inform, collaborate with, and educate  DOD internal and external stakeholders about the program. To accomplish  this, the plan identifies categories of stakeholders and includes a five-phase  implementation approach. The five phases are as follows: 1. Plan Start-up: Conduct a formal tactical review, including an  assessment of current communication tools, communication  procedures, available resources, and agency or industry best practices. 2. Discovery and Assessment: Identify and verify all internal and external  stakeholders and begin defining the benchmarking and reporting  requirements. 3. Branding: Determine key messages to be communicated and select  tools for disseminating these messages. 4. Communications Planning and Execution: Execute and assess the  application of the strategy and continue to develop the communication  tools. 5. Evaluation: Evaluate the progress and overall success of the plan’s  implementation, using metrics.
The strategic communications plan states that the five phases were to have  been fully implemented by December 2004. Although 5 months have passed  since the plan was to be implemented, none of the five phases have been  completed, and communications activities that have been performed by  those responsible for implementing the plan have been limited in scope.  Specifically, communications activities have focused on raising program  awareness through the distribution of posters and press releases and the  establishment of a Web site that provides information about the program.  However, an assessment to evaluate the effectiveness of communication  tools and procedures, available resources, and agency or industry best  practices in communicating the program’s purpose to the various  stakeholders (e.g., the domains and the military services) has not been  performed. In addition, the program’s Acting Assistant Deputy Director for  Communications stated that a systematic approach, including metrics, to  measure the effectiveness of the communications strategy has not been  defined. According to this official, communications activities have been ad  hoc.
Further, DO/IT representatives told us that the program’s Web site does not  contain consistent and current information about the program; as a result,  stakeholders’ understanding of the BEA is limited. Moreover, the plan  focuses first on internal stakeholders, and it defers efforts to proactively  promote understanding and buy-in with external stakeholders, such as  Congress. The plan defines the internal stakeholders to exclude key  departmental components, such as the military services and the defense  agencies. Instead, it defines internal stakeholders to include only the  program office and the domains.
As a result, both the internal and external DOD stakeholders with whom  we spoke said that they did not have a clear understanding of the program,  including its purpose, scope, approach, activities, and status, as well as  stakeholders’ roles and responsibilities. For example, the Installation and  Environment domain representative stated that it was unclear how the  program would achieve one of the program’s original goals (i.e., achieving  an unqualified audit opinion by fiscal year 2007), and what the role of this  representative’s domain would be in achieving this goal. Similarly, the  Acquisition domain representative stated that the roles and responsibilities  of the domains for the program are confusing, because the domains should  be the entities that are developing the BEA, and not the program office.  According to this representative, the current approach will result in  redundancy and increased program costs.
In addition, the program office’s Acting Assistant Deputy Director for  Communications acknowledged that stakeholders are confused, stating  that they do not have a good understanding of either the BEA’s goals,  objectives, and intended outcomes or stakeholders’ roles and  responsibilities. This official also stated that cross-domain integration  issues remain that require strong DOD leadership to move the  communications efforts beyond conducting awareness activities to  achieving departmentwide buy-in.
The Special Assistant for Business Transformation has recently begun to  better communicate with key external stakeholders, such as Congress. For  example, this official stated that department officials have met with and  intend to hold future meetings with congressional staff to brief them on the  department’s plans and efforts to date. Without effective communication  with BEA stakeholders, the likelihood that DOD will be able to effectively  develop and implement its BEA is greatly reduced.
section_title: DOD Has Yet to Develop Program Plans and Supporting Workforce Plans, but Intends to Make Improvements
paragraphs: DOD does not have the program plans that it needs to effectively manage  the development, maintenance, and implementation of its BEA. In  particular, the department has yet to specify measurable goals and  outcomes to be achieved, nor has it defined the tasks to be performed to  achieve these goals and outcomes, the resources needed to perform the  tasks, or the time frames within which the tasks will be performed. DOD  has not assessed, as part of its program planning, the workforce  capabilities that it has in place and that it needs to manage its architecture  development, maintenance, and implementation efforts, nor does it have a  strategy for meeting its human capital needs. The absence of effective  program planning, including program workforce planning, has contributed  to DOD’s limited progress in developing a well-defined architecture and  clearly reporting on its progress to date. Unless its program planning  improves, it is unlikely that the department will be successful in its attempt  to develop, maintain, and implement its BEA. Recognizing this long- standing void in planning, DOD stated that it intends to complete, by  September 30, 2005, a transition plan that will include a program baseline  that will be used to oversee and manage program activities.
subsections: section_title: DOD Has Yet to Develop Effective Program Plans
paragraphs: Architecture management guidance states that organizations should  develop and execute program plans, and that these plans should provide an  explicit road map for accomplishing architecture development,  maintenance, and implementation goals. Among other things, effective  plans should specify tasks to be performed, resources needed to perform  these tasks (e.g., funding, staffing, tools, and training), program  management and contractor roles and responsibilities, time frames for  completing tasks, expected outcomes, performance measures, program  management controls, and reporting requirements. We have previously  reported on the program’s lack of effective planning and have  recommended that DOD develop BEA program plans. (See app. II for our  prior recommendation and its current status.)
Since the program was launched in 2001, DOD has operated without a  program plan. Instead, the department has set target dates for producing a  series of architecture releases as part of three generally defined  architecture increments (see table 3). However, DOD has not clearly  defined what the purpose of the respective increments are, either  individually or collectively, and it has not developed near-, mid-, or long- term plans for producing these increments. At a minimum, such plans  would identify specific tasks (i.e., provide a detailed work breakdown  structure) for producing the architecture releases that possess predefined  content and utility. These plans would also contain specific and reliable  estimates of the time and resources it will take to perform these tasks.
Also in lieu of program plans, the program office provided us with  documents in November 2004 that the deputy program director stated were  to address our open recommendations for (1) adding needed content and  consistency to the BEA’s “As Is” and “To Be” architectural products and (2)  developing a well-defined BEA transition plan. However, the documents  that we were provided were plans for developing plans to address our  recommendations and, thus, were not documents explaining how and  when our recommendations would be addressed.
DOD has recognized the need for program planning. According to the  Acting Assistant Deputy Director for Strategic Planning, the program office  had committed to developing a program baseline by December 2004.  According to the program director, this baseline was to include program  goals, objectives, and activities as well as performance, cost, and schedule  commitments. Further, the baseline was to establish program roles,  responsibilities, and accountabilities and was to be used as a managerial  and oversight tool to allocate resources, manage risk, and measure and  report progress. However, the department has yet to develop this program  baseline. Moreover, while this program baseline would be a useful tool for  strengthening program management, it nevertheless was not to have  included all of the elements of an effective program plan, such as a detailed  work breakdown structure and associated resource estimates. According  to senior officials, including the Special Assistant for Business  Transformation and the Deputy Under Secretary of Defense (Financial  Management), the department is drafting a transition plan that is to be  approved by the DBSMC and issued by September 30, 2005. According to  these officials, this plan will include a program baseline and a BEA  development approach.
The lack of well-defined program plans has contributed significantly to the  limited BEA progress that we have reported over the last several years.  Moreover, this absence of program plans has created a lack of transparency  and understanding about what is occurring on the program and what will  occur—both inside and outside the department. It has also inhibited BEA  governance entities’ ability to ensure that resources (e.g., program office,  domains, and contractors) are being effectively used to achieve worthwhile  outcomes and results. Although the contractor’s work statements have  provided some additional detail, these task descriptions lack the specificity  necessary to use them effectively to monitor the contractor’s progress and  performance. For example, the latest work statement includes the task  “develop business rules based on the available sources of information,”  which has been included in prior work statements. However, the latest  work statement does not define the scope of this effort, nor does it define  how this latest task will support prior efforts to develop business rules. As  a result, the extent to which business rules have been developed and the  work remaining to complete the development of these rules is unclear.  Further, the relationship of this task to DOD’s ability to satisfy the  objectives for increment 1 also has not been defined. These architecture  relationships need to be defined before the department can develop  explicit plans for effective BEA development.
Moreover, because there is no plan linking them together, it is not clear  how the contractor’s work statements and other BEA working groups’  efforts relate to or contribute to larger BEA goals and objectives. For  example, the program office has continued to task the contractor to  develop architecture releases, although the intended use, and thus the  explicit content, of the various releases has not been clearly linked to the  goals and objectives of increment 1. Representatives of the DOD business  domains raised similar concerns with the contractor’s work statements,  telling us that the work statements have been vague and have not been  linked to the specific architecture products. According to these  representatives, the department does not know if it is investing resources  on tasks that are needed and add value to the program.
According to the deputy program director, continuous changes in the  direction and scope of the program have hindered DOD’s ability to develop  effective program plans. Without such plans, DOD has been, and will  continue to be, limited in its ability to develop a well-defined architecture  on time and within budget.
section_title: DOD Has Not Performed Effective Workforce Planning
paragraphs: As we have previously reported, workforce planning is an essential  component of program management. Workforce planning enables an entity  to be aware of and prepared for its current and future human capital needs,  such as workforce size, knowledge and skills, and training. Such planning  involves assessing the knowledge and skills needed to execute the  program, inventorying existing staff knowledge and skills, identifying any  shortfalls, and providing for addressing these shortfalls. Through effective  workforce planning, programs and organizations can have the right people  with the right skills doing the right jobs in the right place at the right time.  Relevant architecture management guidance recognizes the importance  of planning for and having adequate human resources in developing,  maintaining, and implementing enterprise architectures.
DOD has yet to perform workforce planning for its BEA program.  Nevertheless, it has established a program office consisting of seven  program divisions (see fig. 1) and staffed the office with 60 government  employees and approximately 300 contractor staff. In addition, the  department has assigned other staff to support the various program  government entities, such as the domains and the DO/IT, and it has  established various formal and informal working groups. However, DOD  has not taken steps to ensure that the people assigned to the program have  the right skill sets or qualifications. In particular, DOD has not defined the  requisite workforce skills and abilities that the department needs in order  to develop, maintain, and implement the architecture. To illustrate, the  program’s Assistant Deputy Director for Administrative Support told us  that the position descriptions used to staff the program office were generic  and were not tailored to the needs of an enterprise architecture program.  This official added that, as a result, a person might satisfy the qualifications  contained in the position description, but still not meet the needs of the  BEA program. In addition to not defining its workforce needs, the program  also does not have an inventory of the capabilities of its currently assigned  program workforce. For example, the Assistant Deputy Director for  Administrative Support told us that the department did not know the  number of individuals assigned to support the various governance entities.
In the absence of an inventory of existing workforce knowledge and skills,  we requested any available information on the qualifications and training of  program staff in key leadership positions (e.g., assistant deputy directors).  In response, the Assistant Deputy Director for Administrative Support said  that such information was not readily available for all staff, and this official  provided us with résumés for 15 program officials, 4 of whose résumés we  were told were created in response to our request for key staff  qualifications and training.
Exacerbating the program’s lack of workforce planning is the fact that  several key program office positions are vacant. For example, four of the  seven program division leadership positions (i.e., assistant deputy  directors’ positions) are temporarily filled by persons “acting” in this  position (see fig. 2). In addition, key supporting positions within the  program divisions, such as the positions for performance management and  independent verification and validation/quality assurance, are vacant. As a  result, 1 program official is currently acting in three positions—strategic  planning/organization development (includes risk management),  performance management (earned value management system), and  independent verification and validation/quality assurance. Further, two of  the positions this person occupies are incompatible and do not allow for  appropriate separation of duties. Specifically, this individual is  responsible for both program performance management and independent  oversight of performance management.
In addition, significant staff turnover has occurred in key program  positions. For example, the program office has had three directors in 4  years, three transition planning directors since March 2004, and four  contracting officer technical representatives responsible for the prime  contract since January 2003.
The Assistant Deputy Director for Administrative Support stated that the  program lacks valid and reliable data about its human capital needs and  current capabilities. This official told us that plans are being developed to  begin addressing this situation. For example, to begin monitoring staff  turnover, the program office recently began maintaining a list of program  staff with start dates. However, this official also told us that the plans for  improving the program’s management of its human capital were not  complete, and dates for when the plans would be complete have not been  set.
The absence of effective workforce planning has contributed significantly  to DOD’s limited progress to date in developing its architecture. Unless the  program’s approach to human capital management improves, it is unlikely  that the department’s future efforts to develop, maintain, and implement  the BEA will be successful.
section_title: DOD Is Not Performing Effective Configuration Management
paragraphs: Relevant architecture guidance, including DOD guidance, recognizes the  importance of configuration management when developing and  maintaining an architecture. The purpose of configuration management is  to maintain integrity and traceability, and control modifications or changes  to the architecture products throughout their life cycles. Effective  configuration management, among other things, enables integration and  alignment among related architecture products. As we have previously  reported, an effective configuration management process comprises four  primary elements, each of which should be described in a configuration  management plan and implemented according to the plan. In addition,  responsibility, accountability, and authority for configuration management  should be assigned to a configuration manager. The four elements are:  Configuration identification: Procedures for identifying, documenting,  and assigning unique identifiers (e.g., serial number and name) to  product types generated for the architecture program, generally referred  to as configuration items.
Configuration control: Procedures for evaluating and deciding whether  to approve changes to a product’s baseline configuration, generally  accomplished through configuration control boards, which evaluate  proposed changes on the basis of costs, benefits, and risks and decide  whether to permit a change.
Configuration status accounting: Procedures for documenting and  reporting on the status of configuration items as a product evolves.  Documentation, such as historical change lists and original architecture  products, are generated and kept in a library, thereby allowing  organizations to continuously know the state of a product’s  configuration and to be in a position to make informed decisions about  changing the configuration.
Configuration auditing: Procedures for determining alignment  between the actual product and the documentation describing it,  thereby ensuring that the documentation used to support the  configuration control board’s decision making is complete and correct.  Configuration audits, both functional and physical, are performed when  a significant product change is introduced, and they help to ensure that  only authorized changes are being made.
DOD has a draft configuration management plan and related procedures  that address all four of these areas. However, the plan is not being  followed. For example, according to the plan and procedures, certain  product types should be placed under configuration management and be  assigned a unique identifier. However, in one case, the verification and  validation contractor reported that DOD had updated one of the BEA  products (i.e., All View-1) that was initially published in Release 2.3, but  that this updated product was not given a unique identifier and a new  release date, and no entry was made in the version history to enable  stakeholders to differentiate between the two versions.
Configuration naming conventions also have not been consistently  followed, resulting in the updates to a single document being given  different unique identifiers than the original document. For example, the  November 2003 configuration management plan had the unique identifier  “C0008_05_03_BMMP_Configuration_Management_Plan.doc,” which was  comprised of the call number, the task number, the subtask number, and  the name of the document. However, the department later assigned this  document the following unique identifier  “Configuration_Management_Plan.doc,” which did not include the call and  task numbers. Such inconsistencies could permit changes to be made to  the wrong version of a product, thereby compromising the integrity and  reliability of the information.
Consistent with relevant guidance, the procedures require that a  configuration manager be assigned and that this individual be responsible  for ensuring that the four elements are executed. However, after almost 4  years of architecture products development, a configuration manager has  not been assigned. In addition, while the department established a  configuration control board and chartered it to evaluate and decide  whether to approve proposed product changes, this board is not fully  functioning. Specifically, the board’s charter has yet to be approved, and its  authority has been limited to a subset of BEA products. For example, its  authority does not extend to the BEA transition plan.
With respect to configuration status accounting activities that have been  conducted to ensure the integrity of product baselines, we were provided  with two reports even though program officials, including the  Configuration Control Board Chair, told us that no configuration status  accounting reports exist and that neither do any auditing procedures and  processes (e.g., audit checklists, agendas, or plans). However, one of these  reports was missing key data, such as the date of the report, the submitter,  and the version of the product currently being reviewed and, thus, was of  limited use. It was also unclear as to which version of the product was  referenced in the report, and these officials told us that the current baseline  of approved configuration items, including the configuration management  plan, is unknown. As a result, configuration items can be duplicated. For  example, the Acting Assistant Deputy Director for Communications had a  second communications plan prepared, and this official told us that he did  not know that a prior draft plan existed. Because of this, the new strategy  did not leverage any of the work that had previously been done, and  duplicative plans exist.
Program officials, including the Configuration Control Board Chair, stated  that they recognize the importance of effective configuration management.  They attributed the absence of effective configuration management to a  number of factors, including no policy or directive requiring it and the lack  of a common understanding of effective configuration management  practices.
The absence of effective configuration management raises questions about  whether changes to the BEA and other relevant products have been  justified and accounted for in a manner that maintains the integrity of the  documentation. Unless this situation is remedied, the department will not  have adequate assurance that it has maintained accountability and ensured  the consistency of information among the products it is developing. In  addition to the governance and planning weaknesses we previously  discussed, the department’s lack of effective configuration management  has also contributed to the state of the BEA discussed in the next section of  this report.
section_title: DOD Has Yet to Develop a Well-Defined BEA to Guide Its Modernization Efforts
paragraphs: Despite six BEA releases and two updates, DOD still does not have a  version of an enterprise architecture that can be considered well defined,  meaning that the architecture, for example, has a clearly defined purpose  that can be linked to the department’s goals and objectives and describes  both the “As Is” and the “To Be” environments; consists of integrated and  consistent architecture products; and has been approved by department  leadership. According to program officials, the state of the BEA reflects the  program’s focus on meeting arbitrary milestones rather than architecture  content needs. Recognizing the need to develop well-defined architecture  products that have utility and provide a foundation upon which to build,  program officials have stated the department’s intent to change its BEA  development approach, refocusing its efforts on fewer, higher quality  products. Until a BEA development approach embodying architecture  development and content best practices is clearly defined and  implemented, it is not likely that DOD will produce an enterprise  architecture that will provide needed content and utility.
subsections: section_title: “As Is” Description, Transition Plan, and Purpose of BEA Releases Are Missing
paragraphs: As we previously discussed, the various frameworks used to develop  architecture products consistently provide for describing a given enterprise  in both logical (e.g., business, performance, application, and information)  and technical (e.g., hardware, software, and data) terms, and for doing so  for the enterprise’s current or “As Is” environment and its target or “To Be”  environment; these frameworks also provide for defining a capital  investment sequencing plan to transition from the “As Is” to the “To Be”  environment. However, the frameworks do not prescribe the degree to  which the component parts should be described to be considered correct,  complete, understandable, and usable—essential attributes of any  architecture. This is because the depth and detail of the descriptive content  depends on what the architecture is to be used for (i.e., its intended  purpose and scope). Relevant architecture guidance states that a well- defined architecture should have a specific and commonly understood  purpose and scope, and that it should be developed in incremental releases.  Using this purpose and scope, the necessary content of architecture  releases can then be defined.
In September 2003, we reported that Release 1.0 of the BEA was missing  important content, and we made 62 recommendations to add this content.  The latest releases of the BEA (see table 4) do not address the 32 of our 62  recommendations that are related to the “As Is” description and the  transition plan. Specifically, the releases do not include products  describing the “As Is” environment for those areas of the enterprise that are  likely to change, and they do not include a sequencing plan that serves as a  road map for transitioning from the “As Is” state to the “To Be” state. For  example, the BEA releases do not contain a description of the “As Is”  environment that would include current business operations in terms of the  entities or people who perform the functions, processes, and activities, and  the locations where the functions, processes, and activities are performed.  It also does not describe the data or information being used by the  functions, processes, and activities. As a result, DOD does not have a  picture of its current environment to permit development of a meaningful  and useful transition plan.
The BEA releases also do not contain a transition plan that would include  information such as time frames for phasing out existing systems within  DOD’s current inventory and resource requirements for implementing the  “To Be” architecture. As a result, DOD does not yet have a meaningful and  reliable basis for managing the disposition of its existing inventory of  systems or for sequencing the introduction of modernized business  operations and supporting systems. As we previously reported, not having  defined the “As Is” operations and technology at this juncture is risky  because it defers until too late in the architecture development cycle  creation of sufficient descriptive content and context to develop an  effective transition plan. (See app. II for our prior recommendations and  their current status.) DOD’s architecture framework (DODAF), which the  department is using to develop the BEA, does not require the development  of an “As Is” architectural description or a transition plan, and thus neither  has been the focus of the program. However, according to program  officials, including the Chief Architect, the September 2005 BEA release  will include both the “As Is” architectural description and a transition plan.
In addition, DOD has not clearly linked the purpose of any of the “To Be”  architecture releases produced to date to the goals and objectives of  increment 1. Further, these releases also do not have a clearly defined  scope with respect to what business processes and supporting systems  each release would focus on. According to program officials, the last five  versions (i.e., Releases 2.1, 2.2, and 2.3 and the January and March 2005  Updates) support the objectives of increment 1 (see table 3). These  objectives are broad-based strategic goals or outcomes that DOD proposed  achieving through a series of architecture releases. However, DOD did not  define how many releases were needed to achieve each objective and how  the purpose of each release is associated with the objectives. Restated,  while each incremental objective would describe the mission outcome that  DOD intended to achieve via implementation of the series of releases that  make up that increment, the purpose of the releases was not specified in  terms of architectural questions that were related to the objectives. To  illustrate, one objective of increment 1 is to achieve an unqualified audit  opinion for consolidated DOD financial statements. Examples of the  purpose questions that would support this objective include the following: 1. What changes need to be made to existing business processes and the  supporting systems to address material internal control weaknesses  affecting significant line items on the financial statements? 2. Where are gaps in IT support (systems functions) that need to be  addressed in order to provide the business capabilities for ensuring that  property, plant, and equipment are appropriately valued and recorded  on the financial statements?
The department did not include architecture products that would answer  these types of questions to support the increment 1 objective. As a result,  the context needed to plan and measure content sufficiency was not  established. Program officials, including the Special Assistant for Business  Transformation and the Chief Architect agreed, stating that prior releases  have not included a specific purpose and scope. Moreover, the Chief  Architect told us that the architecture releases do not fully support  increment 1’s objectives, nor do they describe the extent to which they do  address the increment 1 objectives.
According to program officials, including the deputy program director and  the Chief Architect, the prior releases were developed with the goal of  producing as many architecture products as possible within a predefined  schedule. The releases were not developed to provide the content needed  to meet the defined purpose and scope of the release. Until the department  defines the intended purpose and scope of its BEA, including its  incremental releases, and ensures that the architecture products include  adequate descriptions of the “As Is” and “To Be” environments, as well as a  plan for transitioning between the two, it will not have a well-defined  architecture to guide and constrain its systems modernization efforts.
section_title: BEA Products Are Incomplete, Inconsistent, and Not Integrated
paragraphs: Architecture frameworks provide for multiple products, each describing a  particular aspect of the enterprise, such as data or systems. Moreover, each  of these products is interdependent, meaning that they have relationships  with one another that must be explicitly defined and maintained to ensure  that the products form a coherent whole. In light of these relationships, it is  important to develop the architecture products in a logical sequence that  reflects this relationship. DODAF recognizes this need for integration of  the products that make up its three “To Be” views—operational view (OV),  systems view (SV), and technical standards view (TV). (See app. IV for a  brief description of the products that comprise each of these views.)  According to the framework, an architecture must be well structured so  that the products can be readily assembled or decomposed into higher or  lower levels of detail, as needed. It should also be consistent—that is,  information elements should be the same throughout the architecture.
As noted in the previous section, we reported in September 2003, that  Release 1.0 of the BEA was missing important content, and we made 62  recommendations to add this content. The latest releases of the BEA do not  adequately address our 30 prior recommendations related to the “To Be”  description. For example, these releases do not include descriptions of  the actual systems to be developed or acquired to support future business  operations and the physical infrastructure (e.g., hardware and software)  that will be needed to support the business systems. As a result, the “To Be”  environment lacks the detail needed to provide DOD with a common vision  and frame of reference for defining a transition plan to guide and constrain  capital investments and, thus, to effectively leverage technology to  orchestrate logical, systematic change and to optimize enterprisewide  mission performance. (See app. II for details on the status of our prior  recommendations.)
In addition, the respective products of each of the latest BEA releases  continue to be inconsistent and not integrated, because key architecture  products were either not developed or not updated to reflect changes made  in other products. Examples of where Releases 2.2 and 2.3 are not  consistent and integrated follow:  In Release 2.2, the department updated the system data exchange matrix  (SV-6), which assigns attributes (e.g., timeliness) to the data to be  exchanged (e.g., Performance Metrics) between system functions— “Manage Business Enterprise Reporting” and “Establish and Maintain  Performance Information”—to support business operations. However,  the OV-3 in Release 2.2, which shows the attributes of the information to  be exchanged to support operations, is not consistent with the attributes  defined in the SV-6. For example, in the OV-3, the attribute referred to as  “timeliness” is defined in terms of either “hours,” “minutes,” or  “seconds;” however, in the SV-6, the attribute referred to as “timeliness”  is only defined in terms of either “high, medium, or basic.”
In Releases 2.2 and 2.3, the department updated the respective  operational event-trace description product (OV-6c), which depicts  when activities are to occur within operational processes. However, the  department did not update, in either release, the operational activity  model (OV-5), which shows the operational activities (or tasks) that are  to occur and the input and output flows among these activities. For  example, the OV-6c shows the sequence of the activities to occur for the  process labeled “produce obligation reports;” however, the activities  shown in the OV-5 were not associated with this process.
The latest releases also do not provide for traceability among the  architecture products by clearly identifying the linkages and dependencies  among the products, such as associating processes (e.g., produce  obligation reports) with activities (e.g., compare expense to obligation) in  the operational views and then associating these same processes to  systems (e.g., financial reporting system) in the systems view. In addition,  the linkage between the two functions (i.e., “Manage Business Enterprise  Reporting” and “Establish and Maintain Performance Information”)  previously discussed cannot be traced to the OV-3 in Release 2.2. This is  because Release 2.2 did not include an SV-5, which would provide a  traceability of system functions back to operational activities. The lack of  an updated SV-5 also raises questions as to whether all operational  requirements are satisfied by the system functions. In addition, the  architecture products were not developed in a logical sequence, as called  for in relevant guidance and standards (e.g., the OV-6c, which shows the  timing or sequencing of activities, was built before the OV-5, which shows  the activities that are to occur).
Further, according to the verification and validation contractor, the  department has yet to address 242 of its 299 outstanding comments since  Release 1.0. The verification and validation contractor also cited similar  concerns, as previously described, for Releases 2.2 and 2.3. Specifically, the  contractor reported that the BEA products were not integrated, and that  key products were missing or had not been updated—such as the  operational nodes connectivity description (OV-2) and the operational  information exchange matrix (OV-3). In its report on the January 2005  Update, the contractor stated that the architecture products were  developed in an order different from that recommended in DODAF, and  that the dependency relationships between and among BEA products were  not clearly depicted. For example, the contractor reported that the logical  data model (OV-7) was to have been developed using the OV-3, OV-5, and  OV-6c artifacts as inputs. However, this was not the case. Instead, the OV-7  was developed using information that may have been reverse engineered  from existing systems and architectures external to the BEA. The  contractor reported that unless these dependencies are clearly  documented and depicted, new systems may be implemented without  satisfying all operational requirements, with missing functions and  interfaces or based on obsolete data models, recreating many of the  problems the modernization is intended to resolve. The contractor also  reported that the resulting technical problems in the OV-7 could interfere  with the department’s achievement of the increment 1 objectives.
The March 2005 Update also did not have fully integrated products.  Specifically, while some of the products were integrated, this integration  occurred at the highest level only and could not be found at lower levels of  decomposition (e.g., subprocesses and subactivities) within the  architecture. For example, the level of integration does not enable the user  to determine all information inputs for the activities at all levels, nor does it  clearly reflect the dissemination or use of the information after it has been  processed. In addition, this update did not include key architecture  products that are recommended by DODAF—such as the system data  exchange artifact (SV-6), which assigns attributes (e.g., timeliness) to the  data to be exchanged between system functions and the system inventory  (SV-8). The SV-8 provides a basis for portfolio investment decisions by  depicting the evolution of systems, systems integration, and systems  improvements over time. We also found that the architecture was not user  friendly in that it was difficult to navigate. For example, the linkages among  the architecture products did not always work, thereby, requiring manual  navigation through the architecture to find the linkages. This would take  hours to do, especially since it was complicated by the fact that certain  artifacts (e.g., diagrams) could not be read online and had to be printed.
Relatedly, as shown in table 4, the latter BEA releases have not included all  of the recommended DODAF products. DODAF recommends that the BEA  include 23 out of 26 possible architecture products to meet the  department’s stated intention to use the BEA as the basis for  departmentwide business and systems modernization. However, Release  2.2 of the architecture included only 16 of the 23 recommended  architecture products, and 6 of the 16 products (OV-1, OV-2, OV-3, OV-4,  OV-5, and SV-9) were actually Release 2.0 products that had not been  updated to align with the changes that had been made to the 10 products  that were updated in this release. Similarly, Release 2.3 included only 4  products; the January 2005 Update included 6, and the March 2005 Update  included 15. According to the Chief Architect, all prior architecture  products that were not included in a specific release or update are  obsolete. For example, Release 2.2 included 16 architecture products, of  which 10 had been updated. The remaining 6 products had not been  updated, but they were still considered to be valid because they were  included in this release. This means that for all releases and updates, only  those products included in the release or update are relevant. For example,  DOD updated 15 products and included them in the March 2005 Update;  therefore, as of March 2005, only these 15 products were considered to be  valid artifacts of the BEA.
Program and contractor officials, including the Acting Assistant Deputy  Director for Transition Planning, stated that although the department’s first  release of its architecture included a fairly consistent and integrated set of  architecture products, DOD’s current releases do not because the  department did not update all the recommended architecture products.  These officials, including the Chief Architect, also stated that, as a result,  the utility of the architecture is limited. However, according to key program  officials, including the Special Assistant for Business Transformation and  the Chief Architect, the integration of the architecture products was not the  focus; rather, DOD’s primary goal was to produce as many products as it  could within a specified time period (see tables 3 and 4).
Recognizing these weaknesses, the Special Assistant for Business  Transformation stated that the department intends to reduce the scope of  the architecture and revise the development approach, which will be  reflected in the September 30, 2005, architecture release. However,  according to program officials, including the Special Assistant for Business  Transformation, the September 2005 BEA release will not be  comprehensive (i.e., it will not meet all the act’s requirements). Further, the  department has yet to develop plans and a methodology to execute this  new focus and vet it through the department. Program officials also stated  that as a result of the new focus, they are trying to decide which products  from prior releases could be salvaged and used.
Nevertheless, the department has spent almost 4 years and approximately  $318 million in obligations to develop an architecture that is incomplete,  inconsistent, and not integrated and, thus, has limited utility. Until the  department develops an approved, well-defined architecture that includes  a clear purpose and scope and integrated products, it remains at risk of not  achieving its intended business modernization goals and of not having an  architecture that the stakeholders can use to guide and constrain ongoing  and planned business systems investments to prevent duplicative and  noninteroperable systems.
section_title: BEA Releases Have Not Been Approved
paragraphs: Relevant architecture guidance state that architecture versions should be  approved by the committee overseeing the development and maintenance  of the architecture; the CIO; the chief architect; and senior management,  including the department head. Such approval recognizes and endorses the  architecture for what it is intended to be—a corporate tool for managing  both business and technological change and transformation. Consistent  with guidance, DOD has stated its intention to approve all BEA releases.
However, Release 1.0 of the BEA is the only release that DOD reports as  having been approved. As we previously reported, DOD officials told us  that Release 1.0 was approved by the former Executive Committee, the  department’s CIO as a member of the Executive Committee, and the DOD  Comptroller on behalf of the Secretary of Defense in May 2003, but they  also said that documentation to verify these approvals did not exist.
Since Release 1.0, DOD has issued five additional releases and two updates.  None of these have been approved by any individual or committee in the  BEA governance structure. According to program officials, including the  Special Assistant for Business Transformation and the Chief Architect,  Release 3.0 of the BEA, which will be issued in September 2005, will be the  next release of the architecture to be approved by the department. These  officials stated that the architecture releases have not been approved  because the department did not have a governance structure and process in  place for doing so. Without the appropriate approvals, buy-in to and  recognition of the BEA as an institutionally endorsed change management  and transformation tool is not achievable.
section_title: DOD Has Yet to Fully Address Most of Our Other Recommendations
paragraphs: In addition to the governance, planning, and content issues previously  discussed, we have made other recommendations relative to DOD’s ability  to effectively develop, maintain, and implement an enterprise architecture  for its business operations. To date, the department has fully addressed one  of our other recommendations, which is to report every 6 months to the  congressional committees on the status of the BEA effort, but it has yet to  fully address the remaining recommendations. (See app. II for details on  the status of these recommendations.) For example, the department has  yet to address our recommendations to  develop a position description for the Chief Architect that defines  requisite duties and responsibilities,  update policies to assign responsibility and accountability for approving  BEA releases,  update policies to address the issuance of waivers for business systems  that are not compliant with the architecture but are nevertheless  justified on the basis of documented analysis, and  develop and implement a quality assurance plan.
According to the program director and deputy director, the current state of  the BEA, including progress in addressing our recommendations, reflects  the program’s prior focus on producing as many products as it could within  a specific time period. The focus had not been on the content and quality of  the releases, but rather on the timing of their delivery. In contrast, our  recommendations have all focused on establishing the means by which to  deliver a well-defined BEA and ensuring that delivered releases of the  architecture contain this requisite content. Until DOD adopts the kind of  approach embodied in our recommendations, it is unlikely that it will  produce a well-defined BEA within reasonable time frames and at an  affordable cost.
section_title: Conclusions
paragraphs: Having and using a well-defined enterprise architecture are essential for  DOD to effectively and efficiently modernize its nonintegrated and  duplicative business operations and systems environment. However, the  department does not have such an architecture, and the architecture  products that it has produced to date do not provide sufficient content and  utility to effectively guide and constrain the department’s ongoing and  planned business systems investments. This means that despite spending  almost 4 years and about $318 million to develop its BEA, the department is  not positioned to meet its legislative mandates. In our view, the state of the  architecture is due largely to long-standing architecture management  weaknesses that the recommendations we have made over the last 4 years  are aimed at correcting, as well as the department’s prior focus on  producing as many products as it could within a specific time period. To  date, the department has not taken adequate steps to implement most of  our recommendations.
While recent steps to begin revamping its BEA governance structure and to  begin program planning are positive first steps and are consistent with  some of the recommendations that we made to lay a foundation for  architecture development, maintenance, and implementation, much more  remains to be accomplished. Thus, it is imperative for the department to  move swiftly to strengthen its BEA program in a manner that incorporates  our prior recommendations and recognizes its current architecture  management capabilities. Until it does, the department will continue to put  billions of dollars at risk of being invested in systems that are duplicative,  are not interoperable, cost more to maintain than necessary, and do not  optimize mission performance and accountability.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Defense direct the Deputy Secretary  of Defense, as the chair of the DBSMC and in collaboration with DBSMC  members, to  immediately fully disclose the state of its BEA program to DOD’s  congressional authorization and appropriations committees, including  its limited progress and results to date, as well as specific plans and  commitments for strengthening program management and producing  measurable results that reflect the department’s capability to do so;   ensure that each of our recommendations related to the BEA  management and content are reflected in the above plans and  commitments; and  ensure that plans and commitments provide for effective BEA  workforce planning, including assessing workforce knowledge and  skills needs, determining existing workforce capabilities, identifying  gaps, and filling these gaps.
section_title: Agency Comments
paragraphs: In written comments on a draft of this report signed by the Special  Assistant for Business Transformation in the Office of the Under Secretary  of Defense (Acquisition, Technology, and Logistics) and the Deputy Under  Secretary of Defense (Financial Management) (reprinted in app. V), the  department concurred with our recommendations and stated its intent to  implement them. Specifically, DOD stated that it would (1) disclose plans,  progress, and results of its BEA efforts to DOD’s congressional committees;  (2) address our recommendations related to BEA management and  content; and (3) assess its workforce needs and adjust its workforce to  meet requirements.
We are sending copies of this report to interested congressional  committees; the Director, Office of Management and Budget; the Secretary  of Defense; the Deputy Secretary of Defense; the Under Secretary of  Defense (Acquisition, Technology, and Logistics); the Under Secretary of  Defense (Comptroller); the Assistant Secretary of Defense (Networks and  Information Integration)/Chief Information Officer; the Under Secretary of  Defense (Personnel and Readiness); and the Director, Defense Finance and  Accounting Service. This report will also be available at no charge on our  Web site at http://www.gao.gov.
If you or your staff have any questions on matters discussed in this report,  please contact me at (202) 512-3439 or hiter@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to this  report are listed in appendix VI.
section_title: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine whether the Department of Defense  (DOD) has (1) established an effective governance structure; (2) developed  program plans, including supporting workforce plans; (3) performed  effective configuration management; (4) developed well-defined business  enterprise architecture (BEA) products; and (5) addressed our other  recommendations.
To determine whether DOD has established an effective governance  structure for its efforts, we reviewed program documentation—such as  approved charters for the Executive and Steering Committees, the Domain  Owners Integration Team (DO/IT), the Transformation Support Office, and  the Defense Business Systems Management Committee—and the  communications strategy and supporting documents. We compared these  documents with the elements in our Enterprise Architecture Management  Maturity Framework and federal Chief Information Officer (CIO) Council  guidance.
To determine whether DOD has developed program plans, including  supporting workforce plans, we interviewed the Director and deputy  program director, and the assistant deputy directors for communications,  strategic planning, and transition planning. We also reviewed draft plans  that showed the department’s intent to address our prior recommendations  for the content previously missing from the “As Is” architecture and the  transition plan. We also reviewed the department’s March 15, 2005, annual  report to Congress, briefing slides on the department’s BEA development  approach, and the various statements of work for the contractor  responsible for extending and evolving the architecture. For human capital,  we reviewed program organization charts and position descriptions for key  program officials. In addition, we interviewed key program officials, such  as the assistant deputy directors for communications, strategic planning,  and enterprise architecture, to discuss their roles and responsibilities.
To determine whether effective configuration management was being  performed, we reviewed the configuration management plan and  associated procedures, and the draft configuration control board charter.  We compared these documents with best practices, including the federal  CIO Council’s Practical Guide, to determine the extent to which DOD had  adopted key management practices. In addition, we reviewed meeting  minutes to determine whether the board was operating effectively and  performing activities according to best practices. We also interviewed  program officials, including the Chief Architect and the Configuration  Control Board Chair, to discuss the process and its effect on the  department’s ability to develop and maintain the BEA products.
To determine whether DOD had developed well-defined BEA products, we  reviewed the latest BEA releases (i.e., Releases 2.2, 2.2.1, and 2.3 and the  March 2005 Update) and the program’s verification and validation  contractor’s reports documenting its assessment of Releases 2.2 and 2.3  and the January 2005 Update of the architecture. To determine whether  these BEA releases addressed our prior recommendations on missing  architecture content and inconsistencies, we requested contractual change  requests related to our recommendations. Program officials, including the  program director and Chief Architect, stated that change requests to  address our recommendations do not exist. We also reviewed the  verification and validation contractor’s assessment of DOD’s efforts to  address its outstanding comments on prior versions of the BEA and DOD  stakeholders’ comments on Release 2.2 of the BEA. Further, we reviewed  DOD’s approach to developing the architecture products since Release 1.0  and compared it with relevant guidance, such as DOD’s architecture  framework. We also observed architecture walk-through sessions held by  program officials to discuss concerns and provide progress updates. In  addition, we interviewed program officials, including the Special Assistant  for Business Transformation, Deputy Under Secretary of Defense  (Financial Management), Chief Architect, the Configuration Control Board  Chair, and the verification and validation contractor to discuss the  development and maintenance of the BEA products.
To determine the status of DOD’s efforts to address our other  recommendations related to BEA development and maintenance, we  reviewed program documentation, such as the draft quality assurance plan,  and compared them with the elements in our Enterprise Architecture  Management Maturity Framework. We requested updates to the  Information Technology Portfolio Management Policy and the position  description for the Chief Architect. We also interviewed program and  contractor officials, such as the Director and deputy program director, and  the assistant deputy directors for quality assurance and communications.
To augment our documentation reviews and analyses, we attended  regularly scheduled meetings, such as the DO/IT meetings, the program  execution status meetings, and configuration control board meetings. We  also held monthly teleconferences with the program and deputy program  directors to discuss any issues and to obtain explanations or clarification  on the results of our audit work.
We did not independently validate cost and budget information provided by  the department.
We conducted our work primarily at DOD headquarters in Arlington,  Virginia, and we performed our work from July 2004 through May 2005, in  accordance with generally accepted government auditing standards. We  requested comments on a draft of this report from the Secretary of Defense  or his designee. Written comments from the Special Assistant for Business  Transformation in the Office of the Under Secretary of Defense  (Acquisition, Technology, and Logistics) and the Deputy Under Secretary of  Defense (Financial Management) are addressed in the “Agency Comments”  section of this report and are reprinted in appendix V.
section_title: Status of Prior Recommendations on DOD’s Development and Maintenance of Its Business Enterprise Architecture
paragraphs: Implemented?
GAO-01-525: Information Technology: Architecture  Needed to Guide Modernization of DOD’s Financial  Operations. May 17, 2001. (1) The Secretary immediately issue a Department of  Defense (DOD) policy that directs the development,  implementation, and maintenance of a business  enterprise architecture (BEA).
DOD has developed the Information Technology  Portfolio Management policy. While this policy, in  conjunction with the overarching Global Information  Grid policy, assigns responsibilities for the  development, implementation, and maintenance of  the BEA, it does not provide for having accountability  for and approval of updates to the architecture  processes for architecture oversight and control and  architecture review and validation, and it does not  address the issuance of waivers for business systems  that are not compliant with the BEA but are  nevertheless justified on the basis of documented  analysis. Program officials stated that the department  plans to revise this policy, but they did not provide a  time frame for doing so. (2) The Secretary immediately modify the Senior  Financial Management Oversight Council’s charter to  designate the Deputy Secretary of Defense as the  Council Chair and the Under Secretary of Defense  (Comptroller) as the Council vice-Chair; and  empower the council to serve as DOD’s BEA steering  committee, giving it the responsibility and authority to  ensure that a DOD BEA is developed and maintained in  accordance with the DOD Command, Control,  Communications, Computers, Intelligence, Surveillance,  and Reconnaissance (C4ISR) Architecture Framework.
We previously reported that DOD had established the  Executive and Steering Committees, which were  advisory in nature. The department also had  established the Domain Owners Integration Team  (DO/IT) and stated that these three bodies were  responsible for governing the program. However,  these groups had not been assigned responsibilities  for directing, overseeing, and approving the BEA.  According to key department officials, these three  entities will be replaced. Specifically, in February  2005, DOD established the Defense Business  Systems Management Committee (DBSMC), which  replaced the Executive and Steering Committees.  According to its charter, the DBSMC is accountable  and responsible for the program. The department  plans to establish an underlying management  structure to support the DBSMC in carrying out its  roles and responsibilities. In addition, program  officials have stated the department’s intention to  replace the DO/IT with the DOD Enterprise  Transformation Integration Group whose roles and  responsibilities and concept of operations have yet to  be defined.
Implemented?
Yes Partial No DOD comments and our assessment (3) The Secretary immediately make the Assistant  Secretary of Defense (Command, Control,  Communications & Intelligence), in collaboration with the  Under Secretary of Defense (Comptroller), accountable to  the Senior Financial Management Oversight Council for  developing and maintaining a DOD BEA.
The Assistant Secretary of Defense (Networks and  Information Integration)/Chief Information Officer  (ASD(NII)/CIO) is a member of the recently  established DBSMC; however, it is not known how this  committee will operate.
DOD established a program office in July 2001. DOD  also appointed a Chief Architect, and, according to  the department, it has adequate program funding and  staff for developing and maintaining its architecture.  However, DOD has yet to define the roles and  responsibilities for the Chief Architect or provide a  time frame for doing so. defines the architecture process and approach;  develops the baseline architecture, the target   facilitates the use of the architecture to guide business  management modernization projects and investments;  and (4) The ASD(NII)/CIO report at least quarterly to the  Senior Financial Management Oversight Council on the  Chief Architect’s progress in developing a BEA, including  the Chief Architect’s adherence to enterprise architecture  policy and guidance from the Office of Management and  Budget (OMB), the CIO Council, and DOD.
The ASD(NII)/CIO is a member of the recently  established DBSMC; however, it is not known how this  committee will operate.
The Steering Committee was briefed monthly by the  program office on various program activities until June  2004, when it held its last meeting. As a result, the  Steering Committee was not updated on the content  and status of Releases 2.2 and 2.3 and the January  2005 and March 2005 Updates of the BEA. According  to program officials, the DBSMC held an executive  session in February 2005 and its second meeting in  April 2005, and the committee will initially hold  monthly meetings. (5) The Senior Financial Management Oversight Council  report to the Secretary of Defense every 6 months on  progress in developing and implementing a BEA.
The Deputy Chief Financial Officer briefed the  Secretary of Defense in November 2003 on behalf of  DOD’s Comptroller, who chairs the Executive  Committee. According to the program director, the  Secretary of Defense has not been briefed since  November 2003 on the department’s progress in  developing and implementing the BEA.
Implemented?
Yes Partial No DOD comments and our assessment (6) The Secretary report every 6 months to the  congressional defense authorizing and appropriating  committees on progress in developing and implementing  a BEA.
Senate Report 107-213 directs that the department  report every 6 months on the status of the BEA effort.  DOD submitted status reports on January 31 and July  31, 2003; January 31 and July 30, 2004; and March  15, 2005. The 2003 and 2004 reports were submitted  by DOD’s Comptroller but were not signed by the  members of the Executive or Steering Committees.  The 2005 report was signed by the Acting Under  Secretary of Defense for Acquisition, Technology, and  Logistics, who is the vice-chair of the DBSMC.
GAO-03-458: DOD Business Systems Modernization:  Improvements to Enterprise Architecture  Development and Implementation Efforts Needed.  February 28, 2003. (1) The Secretary of Defense ensure that the enterprise  architecture executive committee members are singularly  and collectively made explicitly accountable to the  Secretary for the delivery of the enterprise architecture,  including approval of each release of the architecture.
We previously reported that DOD had established the  Executive and Steering Committees, which were  advisory in nature. The department had also  established the DO/IT and stated that these three  bodies were responsible for governing the program.  However, these groups had not been assigned  responsibilities for directing, overseeing, and  approving the BEA. According to key department  officials, these three entities will be replaced.  Specifically, in February 2005, DOD established the  DBSMC, which replaced the Executive and Steering  Committees. According to its charter, the DBSMC is  accountable and responsible for the program. The  department plans to establish an underlying  management structure to support the DBSMC in  carrying out its roles and responsibilities. In addition,  program officials have stated the department’s  intention to replace the DO/IT with the DOD  Enterprise Transformation Integration Group, whose  roles and responsibilities and concept of operations  have yet to be defined. (2) The Secretary of Defense ensure that the enterprise  architecture program is supported by a proactive  marketing and communication program.
DOD has a strategic communications plan; however,  the plan has yet to be implemented. According to the  communications team, its activities have been limited  to raising awareness because it lacks the authority to  fully implement the other components of its plan, such  as achieving buy-in. According to program officials,  the department intends to revise the governance  structure, including the communications strategy, in  September 2005.
Implemented?
Yes Partial No DOD comments and our assessment (3) The Secretary of Defense ensure that the quality  assurance function  includes the review of adherence to process standards  and reliability of reported program performance,  is made independent of the program management   is not performed by subject matter experts involved in  the development of key architecture products.
DOD has established a quality assurance function;  however, this function does not address process  standards and program performance, nor is it an  independent function. Further, DOD subject matter  experts continue to be involved in the quality  assurance function. Program officials stated that the  department had yet to address our recommendation,  and they could not provide a time frame for when they  would begin addressing this recommendation.
GAO-03-1018: DOD Business Systems Modernization:  Important Progress Made to Develop Business  Enterprise Architecture, but Much Work Remains.  September 19, 2003. (1) The Secretary of Defense or his appropriate designee  implement the core elements in our Enterprise  Architecture Framework for Assessing and Improving  Enterprise Architecture Management that we identify in  this report as not satisfied, including ensuring that  minutes of the meetings of the executive body charged  with directing, overseeing, and approving the architecture  are prepared and maintained.
DOD has taken some actions, but these actions do  not fully address our previous concerns. For example,  DOD has  begun to revise its governance structure to provide  for improved management and oversight, such as  establishing the DBSMC and assigning it  accountability and responsibility for directing,  overseeing, and approving the BEA; and  developed a configuration management plan and  the related procedures, and established a  configuration control board.
However, the department has not  established additional governance entities to  support the DBSMC and outlined their roles and  responsibilities;  updated the policy for BEA development,  maintenance, and implementation;  included the missing scope and detail in the BEA;  finalized, approved, and effectively implemented the  plan, procedures, and charter governing the  configuration management process; and metrics. (2) The Secretary of Defense or his appropriate designee  update version 1.0 of the architecture to include the 29  key elements governing the “As Is” architectural content  that our report identified as not being fully satisfied.
Of the 29 elements, program officials stated that 3  were not applicable and that it planned to address an  additional 11 by January 2005. However, these  officials did not provide any documentation to support  this statement. Instead, they provided a draft plan that  shows the department’s intent to develop a detailed  action plan to guide the development of an “As Is”  architecture. According to program officials, they plan  to update the “As Is” architectural description in  September 2005.
Implemented?
Yes Partial No DOD comments and our assessment (3) The Secretary of Defense or his appropriate designee  update version 1.0 of the BEA to include the 30 key  elements governing the “To Be” architectural content that  our report identified as not being fully satisfied.
DOD officials have provided no evidence that this  recommendation has been addressed or that it  intends to implement this recommendation. (4) The Secretary of Defense or his appropriate designee  update version 1.0 to ensure that “To Be” architecture  artifacts are internally consistent, to include addressing  the inconsistencies described in this report, as well as  including user instructions or guidance for easier  architecture navigation and use.
DOD officials have provided no evidence that this  recommendation has been addressed or that it  intends to implement this recommendation. (5) The Secretary of Defense or his appropriate designee  update version 1.0 of the architecture to include (a) the 3  key elements governing the transition plan content that  our report identified as not being fully satisfied and (b)  those system investments that will not become part of the  “To Be” architecture, including time frames for phasing out  those systems.
DOD officials provided a draft plan that shows the  department’s intent to develop a detailed action plan  to guide the development of the transition plan;  however, the draft plan does not provide time frames  for doing so. According to program officials, the  department will issue a revised transition plan in  September 2005; but, this version will not fully  address our recommendation. (6) The Secretary of Defense or his appropriate designee  update version 1.0 of the architecture to address  comments made by the verification and validation  contractor.
According to program officials, of the 299 outstanding  comments, 137 have been addressed in Release 2.3  and earlier releases, 100 were not applicable, and the  remaining 62 will be addressed in future releases.  These officials did not provide any documentation  supporting their rationale for the 100 that they  considered not applicable nor did they provide plans  for addressing the 62 remaining comments. The  verification and validation contractor stated that of the  137 comments that program officials stated had been  addressed, 35 had been addressed, 22 were not  applicable because they were either duplicate or no  longer relevant based on updates to prior releases, 22  had yet to be addressed, and 58 were not assessed.  The contractor has yet to provide its assessment on  the 100 comments that DOD said were not applicable. (7) The Secretary of Defense or his appropriate designee  develop a well-defined, near-term plan for extending and  evolving the architecture and ensure that this plan  includes addressing our recommendations, defining roles  and responsibilities of all stakeholders involved in  extending and evolving the architecture, explaining  dependencies among planned activities, and defining  measures of activity progress.
As discussed in this report, DOD has not developed  explicit detailed plans to guide day-to-day program  activities and to enable it to evaluate its progress.  According to program officials, the department will  develop a program baseline by September 30, 2005.
section_title: Summary of Several Architecture Frameworks
paragraphs: There are various enterprise architecture frameworks that an organization  can follow. Although these frameworks differ in their nomenclatures and  modeling approaches, they consistently provide for defining an enterprise’s  operations in both (1) logical terms, such as interrelated business  processes and business rules, information needs and flows, and work  locations and users, and (2) technical terms, such as hardware, software,  data, communications, and security attributes and performance standards.  The frameworks also provide for defining these perspectives for both the  enterprise’s current or “As Is” environment and its target or “To Be”  environment, as well as a transition plan for moving from the “As Is” to the  “To Be” environment.
For example, John A. Zachman developed a structure or framework for  defining and capturing an architecture. This framework provides for six  windows from which to view the enterprise, which Zachman terms  “perspectives” on how a given entity operates: those of (1) the strategic  planner, (2) the system user, (3) the system designer, (4) the system  developer, (5) the subcontractor, and (6) the system itself. Zachman also  proposed six models that are associated with each of these perspectives;  these models describe (1) how the entity operates, (2) what the entity uses  to operate, (3) where the entity operates, (4) who operates the entity,  (5) when entity operations occur, and (6) why the entity operates.  Zachman’s framework provides a conceptual schema that can be used to  identify and describe an entity’s existing and planned components and their  relationships to one another before beginning the costly and time- consuming efforts associated with developing or transforming the entity.
Since Zachman introduced his framework, a number of other frameworks  has been proposed. In February 1998, DOD directed its components to use  its C4ISR Architecture Framework, Version 2.0. In August 2003, the  department released Version 1.0 of the DOD Architecture Framework  (DODAF)—an evolution of the C4ISR Architecture Framework, which  supersedes the C4ISR. The DODAF defines the type and content of the  architectural artifacts, as well as the relationships among the artifacts that  are needed to produce a useful architecture. Briefly, the framework  decomposes an architecture into three primary views: operational,  systems, and technical standards. See figure 3 for an illustration of these  three views. According to DOD, the three interdependent views are needed  to ensure that IT systems support operational needs, and that they are  developed and implemented in an interoperable and cost-effective manner.
In September 1999, the federal CIO Council published the Federal  Enterprise Architecture Framework (FEAF), which is intended to provide  federal agencies with a common construct on which to base their  respective architectures and to facilitate the coordination of common  business processes, technology insertion, information flows, and system  investments among federal agencies. FEAF describes an approach,  including models and definitions, for developing and documenting  architecture descriptions for multiorganizational functional segments of  the federal government. Similar to most frameworks, FEAF’s proposed  models describe an entity’s business, the data necessary to conduct the  business, applications to manage the data, and technology to support the  applications.
More recently, the Office of Management and Budget (OMB) established  the Federal Enterprise Architecture (FEA) Program Management Office to  develop a federated enterprise architecture in the context of five “reference  models, and a security and privacy profile that overlays the five models.”
The Business Reference Model is intended to describe the federal  government’s businesses, independent of the agencies that perform  them. This model consists of four business areas: (1) services for  citizens, (2) mode of delivery, (3) support delivery of services, and  (4) management of government resources. It serves as the foundation  for the FEA. OMB expects agencies to use this model, as part of their  capital planning and investment control processes, to help identify  opportunities to consolidate information technology (IT) investments  across the federal government. Version 2.0 of this model was released in  June 2003.
The Performance Reference Model is intended to describe a set of  performance measures for major IT initiatives and their contribution to  program performance. According to OMB, this model will help agencies  produce enhanced performance information; improve the alignment and  better articulate the contribution of inputs, such as technology, to  outputs and outcomes; and identify improvement opportunities that  span traditional organizational boundaries. Version 1.0 of this model  was released in September 2003.
The Service Component Reference Model is intended to identify and  classify IT service (i.e., application) components that support federal  agencies and promote the reuse of components across agencies. This  model is intended to provide the foundation for the reuse of  applications, application capabilities, components (defined as “a self- contained business process or service with predetermined functionality  that may be exposed through a business or technology interface”), and  business services. According to OMB, this model is a business-driven,  functional framework that classifies service components with respect to  how they support business and/or performance objectives. Version 1.0  of this model was released in June 2003.
The Data Reference Model is intended to describe, at an aggregate level,  the types of data and information that support program and business  line operations and the relationships among these types. This model is  intended to help describe the types of interactions and information  exchanges that occur across the federal government. Version 1.0 of this  model was released in September 2004.
The Technical Reference Model is intended to describe the standards,  specifications, and technologies that collectively support the secure  delivery, exchange, and construction of service components. Version 1.1  of this model was released in August 2003.
The Security and Privacy Profile is intended to provide guidance on  designing and deploying measures that ensure the protection of  information resources. OMB has released Version 1.0 of the profile.
section_title: Description of DOD Architecture Framework Products, Version 1.0
paragraphs: Executive-level summary information on the scope, purpose, and context of the  architecture Architecture data repository with definitions of all terms used in all products High-Level Operational Concept Graphic High-level graphical/textual description of what the architecture is supposed to  do, and how it is supposed to do it Graphic depiction of the operational nodes (or organizations) with needlines that  indicate a need to exchange information Information exchanged between nodes and the relevant attributes of that  exchange Command structure or relationships among human roles, organizations, or  organization types that are the key players in an architecture  Operations that are normally conducted in the course of achieving a mission or a  business goal, such as capabilities, operational activities (or tasks), input and  output flows between activities, and input and output flows to/from activities that  are outside the scope of the architecture One of three products used to describe operational activity—identifies business  rules that constrain operations One of three products used to describe operational activity—identifies business  process responses to events One of three products used to describe operational activity—traces actions in a  scenario or sequence of events Documentation of the system data requirements and structural business process  rules of the operational view Identification of systems nodes, systems, and systems items and their  interconnections, within and between nodes Specific communications links or communications networks and the details of  their configurations through which systems interface Relationships among systems in a given architecture; can be designed to show  relationships of interest (e.g., system-type interfaces, planned versus existing  interfaces)
Mapping of relationships between the set of operational activities and the set of  system functions applicable to that architecture Characteristics of the system data exchanged between systems Systems Performance Parameters Matrix Quantitative characteristics of systems and systems hardware/software items,  their interfaces, and their functions Planned incremental steps toward migrating a suite of systems to a more efficient  suite, or toward evolving a current system to a future implementation Emerging technologies and software/hardware products that are expected to be  available in a given set of time frames and that will affect future development of  the architecture One of three products used to describe system functionality—identifies  constraints that are imposed on systems functionality due to some aspect of  systems design or implementation One of three products used to describe system functionality—identifies  responses of a system to events One of three products used to describe system functionality—lays out the  sequence of system data exchanges that occur between systems (external and  internal), system functions, or human role for a given scenario Physical implementation of the Logical Data Model entities (e.g., message  formats, file structures, and physical schema)
section_title: Comments from the Department of Defense
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Cynthia Jackson, Assistant  Director; Barbara Collier; Joanne Fiorino; Neelaxi Lakhmani; Anh Le;  Freda Paintsil; Randolph Tekeley; and William Wadsworth made key  contributions to this report.