Business Systems Modernization: Strategy for Evolving DOD's Business Enterprise Architecture Offers a Conceptual Approach, but Execution Details Are Needed
Published: 2007-04-16T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 1995, we first designated the Department of Defense's (DOD) business systems modernization program as "high risk," and we continue to designate it as such today. To assist in addressing this high-risk area, Congress passed legislation consistent with prior GAO recommendations for Defense to develop a business enterprise architecture (BEA). In September 2006, DOD released version 4.0 of its BEA, which despite improvements over prior versions, was not aligned with component architectures. Subsequently, Defense issued a strategy for extending its BEA to the component military services and defense agencies. To support GAO's legislative mandate to review DOD's BEA, GAO assessed DOD's progress in defining this strategy by comparing it with prior findings and recommendations relevant to the strategy's content.
section_title: What GAO Found
paragraphs: DOD's Business Mission Area federation strategy for extending its BEA to the military departments and defense agencies provides a foundation on which to build and align the department's parent business architecture (the BEA) with its subordinate architectures (i.e., component- and program-level architectures). In particular, the strategy, which was released in September 2006, states the department's federated architecture goals; describes federation concepts that are to be applied; and explains high-level activities, capabilities, products, and services that are intended to facilitate implementation of the concepts. However, the strategy does not adequately define the tasks needed to achieve the strategy's goals, including those associated with executing high-level activities and providing related capabilities, products, and services. Specifically, it does not adequately address how strategy execution will be governed, including assignment of roles and responsibilities, measurement of progress and results, and provision of resources. Also, the strategy does not address, among other things, how the component architectures will be aligned with the latest version of the BEA and how it will identify and provide for reuse of common applications and systems across the department. According to program officials, the department intends to develop more detailed plans to execute the strategy. This means that much remains to be decided and accomplished before DOD will have the means in place to create a federated BEA that satisfies GAO's prior recommendations and legislative requirements. Without one, the department will remain challenged in its ability to minimize duplication and maximize interoperability among its thousands of business systems.

section_title: Letter
paragraphs: For decades, the Department of Defense (DOD) has been challenged in  repeated attempts to modernize its timeworn business systems. In 1995,  we designated DOD’s business systems modernization program as “high  risk,” and we continue to designate it as such today. As our research on  public and private sector organizations has shown, one essential  ingredient to a successful systems modernization program is having and  using a well-defined enterprise architecture (EA).
Accordingly, we made recommendations to the Secretary of Defense in  May 2001 that included the means for effectively developing an EA. In July  2001, the department initiated a business management modernization  program to, among other things, develop one. Between 2001 and 2005, we  reported that the department’s business management modernization  program was not being effectively managed, concluding in 2005 that  hundreds of millions of dollars had been spent on a business enterprise  architecture (BEA) that had limited use. Accordingly, we made additional  architecture-related recommendations.
To assist DOD in addressing its modernization management challenges,  Congress included provisions in the Ronald W. Reagan National Defense  Authorization Act for Fiscal Year 2005 that were consistent with our  recommendations, including those for developing a BEA and associated  enterprise transition plan (ETP). In 2006, we reported that, among other  things, DOD had made important progress in developing its BEA, but that  much remained to be accomplished relative to the act’s requirements and  relevant guidance. For example, we reported that the BEA was not  adequately linked to the military departments and defense agency  architectures. To address these shortfalls, DOD issued a strategy for  “federating” or extending its architecture.
To support GAO’s legislative mandate to review DOD’s BEA and as agreed  with your offices, the objective of this review was to determine DOD’s  progress in defining its Business Mission Area (BMA) federation strategy.  To accomplish our objective, we compared the federation strategy and  associated plans with prior findings and recommendations relative to the  content of the strategy, and interviewed DOD officials regarding the  strategy’s executability. We performed our work at DOD headquarters in  Arlington, Virginia, from August 2006 through March 2007 in accordance  with generally accepted government auditing standards. Additional details  on our objective, scope, and methodology are contained in appendix I.
subsections: section_title: Background
paragraphs: DOD is a massive and complex organization. To illustrate, the department  reported that its fiscal year 2006 operations involved approximately   $1.4 trillion in assets and $2.0 trillion in liabilities; more than 2.9 million in  military and civilian personnel; and $581 billion in net cost of operations.  Organizationally, the department includes the Office of the Secretary of  Defense, the Chairman of the Joint Chiefs of Staff, the military  departments, numerous defense agencies and field activities, and various  unified combatant commands that are responsible for either specific  geographic regions or specific functions.
In support of its military operations, the department performs an  assortment of interrelated and interdependent business functions,  including logistics management, procurement, health care management,  and financial management. As we have previously reported, the DOD  systems environment that supports these business functions is overly  complex and error-prone, and is characterized by (1) little standardization  across the department, (2) multiple systems performing the same tasks,  (3) the same data stored in multiple systems, and (4) the need for data to  be entered manually into multiple systems. Moreover, DOD recently  reported that this systems environment is comprised of approximately  3,100 separate business systems. For fiscal year 2006, Congress  appropriated approximately $15.5 billion to DOD, and for fiscal year 2007,  DOD has requested about $16 billion in appropriated funds to operate,  maintain, and modernize these business systems and associated  infrastructure.
As we have previously reported, the department’s nonintegrated and  duplicative systems contribute to fraud, waste, and abuse. In fact, DOD  currently bears responsibility, in whole or in part, for 15 of our 27 high-risk  areas. Eight of these areas are specific to DOD and the department  shares responsibility for 7 other governmentwide high-risk areas. DOD’s  business systems modernization is one of the high-risk areas, and it is an  essential enabler to addressing many of the department’s other high-risk  areas. For example, modernized business systems are integral to the  department’s efforts to address its financial, supply chain, and information  security management high-risk areas.
subsections: section_title: Enterprise Architecture Is Critical to Achieving Successful Business Systems Modernization
paragraphs: Effective use of an enterprise architecture—a modernization blueprint—is  a hallmark of successful public and private organizations. For more than a  decade, we have promoted the use of architectures to guide and constrain  systems modernization, recognizing them as a crucial means to this  challenging goal: optimally defined operational and technological  environments. Congress, the Office of Management and Budget (OMB),  and the federal Chief Information Officer’s (CIO) Council have also  recognized the importance of an architecture-centric approach to  modernization. The Clinger-Cohen Act of 1996 mandates that an agency’s  CIO develop, maintain, and facilitate the implementation of an information  technology (IT) architecture. Furthermore, the E-Government Act of 2002  requires OMB to oversee the development of enterprise architectures  within and across agencies. In addition, we, OMB, and the CIO Council  have issued guidance that emphasizes the need for system investments to  be consistent with these architectures.
subsections: section_title: Enterprise Architecture: A Brief Description
paragraphs: An enterprise architecture provides a clear and comprehensive picture of  an entity, whether it is an organization (e.g., a federal department) or a  functional or mission area that cuts across more than one organization  (e.g., financial management). This picture consists of snapshots of both  the enterprise’s current (“As Is”) environment and its target (“To Be”)  environment. These snapshots consist of “views,” which are one or more  interdependent and interrelated architecture products (e.g., models,  diagrams, matrices, and text) that provide logical or technical  representations of the enterprise. The architecture also includes a  transition or sequencing plan, which is based on an analysis of the gaps  between the “As Is” and “To Be” environments. This plan provides a  temporal road map for moving between the two environments and  incorporates such considerations as technology opportunities,  marketplace trends, fiscal and budgetary constraints, institutional system  development and acquisition capabilities, legacy and new system  dependencies and life expectancies, and the projected value of competing  investments.
The suite of products produced for a given entity’s enterprise architecture,  including its structure and content, is largely governed by the framework  used to develop the architecture. Since the 1980s, various architecture  frameworks have been developed, such as John A. Zachman’s “A  Framework for Information Systems Architecture” and the DOD  Architecture Framework.
The importance of developing, implementing, and maintaining an  enterprise architecture is a basic tenet of both organizational  transformation and systems modernization. Managed properly, an  enterprise architecture can clarify and help optimize the  interdependencies and relationships among an organization’s business  operations (and the underlying IT infrastructure and applications) that  support these operations. Moreover, when an enterprise architecture is  employed in concert with other important management controls, such as  portfolio-based capital planning and investment control practices,  architectures can greatly increase the chances that an organization’s  operational and IT environments will be configured to optimize mission  performance. Our experience with federal agencies has shown that  investing in IT without defining these investments in the context of an  architecture often results in systems that are duplicative, not well  integrated, and unnecessarily costly to maintain and interface.
One approach to structuring an enterprise architecture is referred to as a  federated enterprise architecture. Such a structure treats the architecture  as a family of coherent but distinct member architectures that conform to  an overarching architectural view and rule set. This approach recognizes  that each member of the federation has unique goals and needs as well as  common roles and responsibilities with the levels above and below it.  Under a federated approach, member architectures are substantially  autonomous, although they also inherit certain rules, policies, procedures,  and services from higher-level architectures. As such, a federated  architecture enables component organization autonomy, while ensuring  enterprisewide linkages and alignment where appropriate. Where  commonality among components exists, there are also opportunities for  identifying and leveraging shared services.
A service-oriented architecture (SOA) is an approach for sharing business  capabilities across the enterprise by designing functions and applications  as discrete, reusable, and business-oriented services. As such, service  orientation permits sharing capabilities that may be under the control of  different component organizations. As we have previously reported, such  capabilities or services need to be, among other things, (1) self-contained,  meaning that they do not depend on any other functions or applications to  execute a discrete unit of work; (2) published and exposed as self- describing business capabilities that can be accessed and used; and   (3) subscribed to via well-defined and standardized interfaces. A SOA  approach is thus not only intended to reduce redundancy and increase  integration, but also to provide the kind of flexibility needed to support a  quicker response to changing and evolving business requirements and  emerging conditions.
section_title: DOD’s Efforts to Adopt a Federated Approach to Architecting All of Its Mission Areas
paragraphs: The Office of the Assistant Secretary of Defense (Networks and  Information Integration)/Chief Information Officer (ASD(NII)/CIO),  reports that it is developing a strategy for federating the many and varied  architectures across the department’s four mission areas—Warfighting,  Business, DOD Intelligence, and Enterprise Information Environment.  According to ASD(NII)/CIO officials, they are drafting a yet-to-be-released  strategy for evolving DOD’s Global Information Grid architecture, so that  it provides a comprehensive architectural description of the entire DOD  enterprise, including all mission areas and the relationships between and  among all levels of the enterprise (e.g., mission areas, components, and  programs). Figure 1 provides a simplified depiction of DOD’s EA  federation strategy.
ASD(NII)/CIO officials stated that the goal of this strategy is to improve  the ability of DOD’s mission areas, components, and programs to share  architectural information. In this regard, officials stated that the DOD EA  federation strategy will define (1) federation and integration concepts,   (2) alignment (i.e., linking and mapping) processes, and (3) shared  services.
The BMA federation strategy, according to these officials, is the first  mission area federation strategy, and it is their expectation that the other  mission areas will develop their own respective federation strategies.
section_title: DOD Roles and Responsibilities for BEA Development and Use
paragraphs: In 2005, the department reassigned responsibility for directing, overseeing,  and executing its business transformation and systems modernization  efforts to the Defense Business Systems Management Committee  (DBSMC) and the Business Transformation Agency (BTA). At that time, it  also adopted a tiered accountability approach to business transformation.  Under tiered accountability, responsibility and accountability for business  architectures and systems investment management was allocated among  the DOD enterprise, component, and program levels, depending on such  factors as the scope, size, and complexity of each investment.
The DBSMC is chaired by the Deputy Secretary of Defense and serves as  the highest-ranking governance body for business systems modernization  activities. According to its charter, the DBSMC provides strategic direction  and plans for the BMA in coordination with the Warfighting and Enterprise  Information Environment Mission Areas. The DBSMC is also responsible  for reviewing and approving the BEA and the ETP. In addition, the DBSMC  recommends policies and procedures required to integrate DOD business  transformation and attain cross-department, end-to-end interoperability of  business systems and processes.
The BTA operates under the authority, direction, and control of the  DBSMC and reports to the Under Secretary of Defense for Acquisition,  Technology, and Logistics in the incumbent’s capacity as the vice chair of  the DBSMC. Oversight for this agency is provided by the Deputy Under  Secretary of Defense for Business Transformation, and day-to-day  management is provided by the director. The BTA’s primary responsibility  is to lead and coordinate business transformation efforts across the  department. Regarding the BEA, the BTA is responsible for   (1) maintaining and updating the department’s architecture; (2) ensuring  that functional priorities and requirements of various defense components,  such as the Department of the Army and Defense Logistics Agency (DLA),  are reflected in the architecture; and (3) ensuring the adoption of DOD- wide information and process standards as defined in the architecture.
Under DOD’s tiered accountability approach to systems modernization,  components are responsible for defining their respective component  architectures and transition plans while complying with BEA and ETP  policy and requirements. Similarly, program managers are responsible for  developing program-level architectures and transition plans and ensuring  integration with the architectures and transition plans developed and  executed at the DOD enterprise and component levels.
section_title: Summary of GAO’s Prior Reviews on DOD’s Architecture Efforts
paragraphs: Between May 2001 and July 2005, we reported on DOD’s efforts to develop  an architecture and identified serious problems and concerns with the  department’s architecture program, including the lack of specific plans  outlining how DOD plans to extend and evolve the architecture to include  the missing scope and detail. To address these concerns, in September  2003 we recommended that DOD develop a well-defined near-term plan  for extending and evolving the architecture and ensure that this plan  includes addressing our recommendations, defining roles and  responsibilities of all stakeholders involved in extending and evolving the  architecture, explaining dependencies among planned activities, and  defining measures of progress for the activities.
In response to our recommendations, in 2005, DOD adopted a 6-month  incremental approach to developing its enterprise architecture and  released version 3.0 of the BEA and the ETP in September 2005, describing  them as the initial baselines. DOD further released version 3.1 on March  15, 2006, and version 4.0 on September 28, 2006. As we have previously  reported, these incremental versions have provided additional content  and clarity and resolved limitations that we identified in the prior versions.  For example, DOD reports that version 4.0 begins to define a key business  process area missing from prior versions—the planning, programming,  and budgeting process area. In this regard, according to DOD, the  architecture includes departmental and other federal planning,  programming, and budgeting guidance (e.g., OMB Circular A-11) and some  high-level activities associated with this area. In addition, DOD reports  that version 4.0 included restructured business process models to reduce  data redundancy and ensure adherence to process modeling standards  (e.g., eliminated numerous process modeling standards violations and  stand-alone process steps with no linkages).
We concluded, however, that these incremental versions were still not  sufficiently complete to effectively and efficiently guide and constrain  business system investments across the department. In particular, we  reported that the BEA was not yet adequately linked to the component  architectures and transition plans, which is important given that the  department (1) had previously announced that it had adopted a federated  approach to developing and implementing the architecture and (2) had yet  to address our recommendation from September 2003 for developing an  architecture development management plan that defined how it intended  to extend and evolve its BEA.
Accordingly, in May 2006 we recommended that DOD submit an  enterprise architecture development management plan to defense  congressional committees. We stated that at a minimum, the plan should  define what the department’s incremental improvements to the  architecture and transition plan would be and how and when they would  be accomplished, including what (and when) architecture and transition  plan scope and content and architecture compliance criteria would be  added into which versions. In addition, we stated that the plan should  include an explicit purpose and scope for each version of the architecture,  along with milestones, resource needs, and performance measures for  each planned version, with particular focus and clarity on the near-term  versions. In response, DOD stated that, in the future, the ETP and annual  report to Congress would provide additional high-level milestones for BTA  activities, including the additional detail for the capability improvements  to be addressed by the BEA.
Our August 2006 report on the maturity of federal agency enterprise  architecture programs, including those of the military departments,  reemphasized the importance of DOD having an effective plan for  federating its BEA. Specifically, the August report showed that the  Departments of the Air Force, Army, and Navy had not satisfied about 30,  55, and 30 percent, respectively, of the 31 core elements in our Enterprise  Architecture Management Maturity Framework, which is a five-stage  model for effectively managing architecture governance, content, use, and  measurement. In addition, the Army had only fully satisfied 1 of the 31  core elements. (See table 1 for the number of elements that were fully,  partially, and not satisfied by each of the military departments.)
By comparison, the other major federal departments and agencies that we  reviewed had as a whole fully satisfied about 67 percent of the  framework’s core elements. Among the key elements that all three military  departments had not fully satisfied were developing architecture products  that describe their respective target architectural environments and  developing transition plans for migrating to a target environment.
Furthermore, while the military departments had partially satisfied  between 8 and 13 core elements in our framework, we reported that  partially satisfied elements are not necessarily easy to satisfy fully, such as  those that address architecture content and thus have important  implications for the quality and usability of an architecture. To assist the  military departments in addressing enterprise architecture challenges and  managing their architecture programs, we recommended that the military  departments develop and implement plans for fully satisfying each of the  conditions in our framework. The department generally agreed with our  findings and recommendations.
section_title: DOD Is in the Early Stages of Federating Its BEA and Much Remains to Be Decided and Accomplished
paragraphs: DOD’s BMA federation strategy provides a foundation on which to build  and align DOD’s parent business architecture (the BEA) with its  subordinate architectures (i.e., component- and program-level  architectures). In particular, this strategy (1) states the department’s  federated architecture goals; (2) describes federation concepts that are to  be applied; and (3) includes high-level activities, capabilities, products,  and services that are intended to facilitate implementation of the concepts.  However, DOD has yet to define the details needed to execute the strategy,  such as how the architecture federation will be governed; how alignment  with the DOD EA federation strategy and other potential mission area  federation strategies will be achieved; how component architectures’  alignment with incremental versions of the BEA will be achieved; how  shared services will be identified, exposed, and subscribed to; and what  milestones will be used to measure progress and results. According to BTA  program officials, including the chief technical officer, the department is  in the early stages of defining and implementing its strategy and intends to  develop more detailed plans. As a result, much remains to be decided and  accomplished before DOD will have in place the means to create a  federated architecture and thus be able to satisfy both our prior  recommendations and legislative requirements aimed at adopting an  architecture-centric approach to departmentwide business systems  investment management.
subsections: section_title: BMA Federation Strategy Describes Goals, Concepts, and High-level Activities
paragraphs: BTA released the BMA federation strategy in September 2006. According  to the strategy, its purpose is to expand on the DOD EA federation strategy  and provide details on how various aspects of the federation will be  applied within the department’s BMA. In this regard, the BMA strategy  cites the following four goals:    establish a capability to search for data in member architectures that may  be relevant for analysis, reference, or reuse;   develop a consistent set of standards for architecture configuration  management that will enable users to determine the development status  and quality of data in various architectures;   establish a standard methodology for specifying linkages among existing  component architectures that were developed using different tools and  that are maintained in independent repositories; and   develop a standard methodology to reuse capabilities described by various  architectures.
To assist in accomplishing these goals, the strategy describes three  concepts that are to be applied.  1.  Tiered accountability, which provides for architecture development at  each of the department’s organizational levels. Under this concept,  each level or tier—enterprise, component, and program—has its own  unique goals as well as responsibilities to the tiers above and below it.  More specifically, the BTA has responsibility for the enterprise tier,  including common, DOD-wide requirements and standards, while  components and programs are responsible for defining component-  and program-level architecture requirements and standards for their  respective tiers of responsibility that are aligned with the  departmentwide requirements and standards. As such, this concept  introduces the need for autonomy, while also seeking to ensure  linkages and alignment from the program level through the component  level to the enterprise level.  2.  Net-centricity, which provides for seamless and timely accessibility to  information where and when needed via the department’s  interconnected network environment. This concept includes  infrastructure, systems, processes, and people and is intended to  ensure that users (i.e., people, applications, and platforms) of  information at any level can both take what they need and contribute  what they know.  3.  Federating DOD architectures, which provides for linking or aligning  different architectures via the mapping of common architectural  information. This concept advocates subordinate architecture  alignment to the parent architecture(s).
Figure 2 shows a simplified version of DOD’s BMA federated architecture.
To support the achievement of its goals and implementation of its  concepts, the strategy also describes three categories of high-level  activities, capabilities, products, and services—governance, federating  architecture operational views, and federating architecture systems  views. Table 2 shows the strategy’s operational and systems view related  activities, capabilities, products, and services.
section_title: The BMA Federation Strategy Is Missing Executable Details
paragraphs: Relevant architecture management guidance states that organizations  should develop executable architecture development management plans  and that these plans should specify, among other things, tasks to be  performed, resources needed to perform these tasks (e.g., funding,  staffing, tools, and training), roles and responsibilities, time frames for  completing tasks, and performance measures. As previously stated, we  have recommended that DOD develop such an architecture development  plan to govern the evolution and extension of the BEA. We also have  previously reported that a SOA approach needs to ensure that shared  systems and applications (i.e., services) are, among other things, defined,  developed, exposed, and subscribed to.
The high-level construct of DOD’s BMA federation strategy and the yet-to- be-issued DOD EA federation strategy reinforces the need to implement  our recommendation. In particular, the strategy defines the department’s  federated architecture goals; describes federation concepts that are to be  applied; and explains high-level activities, capabilities, products, and  services intended to facilitate implementation of the concepts. However, it  does not adequately define the tasks needed to achieve the strategy’s  goals, including those associated with executing high-level activities and  providing related capabilities, products, and services. Specifically, the  strategy does not adequately address how strategy execution will be  governed, including assignment of roles and responsibilities, measurement  of progress and results, and provision of resources. In addition, while the  BMA strategy refers to several activities that are to be provided by the yet- to-be-issued DOD EA federation strategy, it does not clearly describe the  relationships, dependencies, and touch points between the two strategies.  Also, the strategy does not address, among other things, how the  architectures of the military departments will align with the latest version  of the BEA and how DOD will identify and provide for sharing of common  applications and systems across the department. Moreover, the strategy  does not include milestones for executing the activities and related  capabilities, products, and services.
subsections: section_title: Governance Structure Is Not Clearly Defined
paragraphs: According to ASD(NII)/CIO officials, each mission area will be responsible  for establishing its own governance structures, to include defined roles  and responsibilities of its members (i.e., components and programs), and  such governance disciplines as measurement of progress and results and  provision of resources. Moreover, officials from DOD components, such as  the DLA and the Defense Information Systems Agency (DISA), told us that  clearly defined and understood federation roles and responsibilities are  critical to successfully executing the BMA strategy.
However, the BMA strategy does not clearly define the respective roles  and responsibilities of each member of the federation (i.e., enterprise,  component, and program). It also does not identify the resource  commitments (e.g., funding, staffing, tools, and training) needed to  execute the strategy’s activities and deliver capabilities, products, and  services, or identify how fundamental governance disciplines will be  performed, including performance and progress measurement. For  example:    The strategy states that the DBSMC, which is currently responsible for the  approval and maintenance of the BEA, will receive updates on how  component (e.g., the military departments) architectures are aligning to  the BEA. However, it does not describe which organizational entities are  to be responsible for providing these updates or for aligning component  and program architectures to the BEA.
The strategy states that in conjunction with the DOD investment review  boards, the DBSMC will set the business priorities at the enterprise level  through the identification of gaps in business capabilities. By establishing  these priorities, the DBSMC is to determine where and when specific  capabilities are addressed within the different architectures (i.e., from  BEA to program-level architectures) and is to approve recommended  solutions to business capability needs. However, the strategy does not  provide information on who is responsible for ensuring that component  priorities fit with the overall enterprise priorities, or how the DBSMC will  otherwise be provided the information it needs to fulfill its stated decision- making role.
The strategy states that BMA stakeholders will need to be trained to  understand the concepts presented in the strategy and begins to identify  topics, such as SOA and the overall federation strategy. However, the  strategy does not identify time frames and the entity responsible for  providing and overseeing such training. In addition, the strategy does not  address how it will be funded and staffed.
The strategy identifies categories of high-level activities, capabilities,  products, and services intended to facilitate implementation of the  concepts, but it does not provide for metrics that can be used to gauge the  progress and ensure that expected results are realized.
section_title: Relationship to DOD EA Federation Strategy Effort Is Unclear
paragraphs: According to the BMA federation strategy, the DOD EA federation strategy  outlines an approach for linking the repositories of all of the department’s  various architectures and enabling search and navigation across them. In  addition, it states that the DOD EA federation strategy outlines a series of  pilot efforts that will demonstrate this approach. However, the BMA  federation strategy does not clearly define how its various activities will  integrate with the activities and concepts described in the yet-to-be-issued  DOD EA federation strategy, or other potential mission area federation  strategies, nor does it discuss how these activities will be carried out or  who will be responsible for accomplishing them. For example:    ASD(NII)/CIO officials told us that the DOD EA federation strategy will  establish new responsibilities for components and programs for making  architecture information understandable and accessible across the  department. However, these responsibilities are not explicitly discussed in  the BMA federation strategy. Therefore, it is unclear how these new  responsibilities are relevant to federating the BEA. Moreover, it is unclear  how the BMA roles and responsibilities relate to the yet-to-be-released EA  federation strategy roles and responsibilities.
The BMA federation strategy does not define how linkages among the BEA  and the various component and program architectures will be established,  including whether program architectures will be linked to component  architectures as well as the BEA, or if program architectures will be linked  to the BEA, as is currently the case. Moreover, it is not clear if establishing  these linkages will be the responsibility of the programs, components, the  BTA, or ASD(NII)/CIO.
section_title: Operational View Federation Does Not Address Component Architecture Alignment
paragraphs: According to the BMA federation strategy, it builds on the DOD EA  federation strategy by proposing new tools and procedures to both  identify overlaps and gaps in capabilities and ensure the compliance of all  component and program architectures with the BEA. In this regard, it  describes the following two tools: the Investment Management  Framework, which is a spreadsheet that aligns program architectures’  capabilities (and activities) with the BEA, and the Architecture  Compliance and Requirements Traceability tool, which is an automated  tool that provides programs with an interface to the BEA so that they can  assess their alignment with the BEA’s operational view content (e.g.,  business capabilities, activities, processes, rules, and standards).
However, the strategy does not address how alignment of component  architectures with the BEA is to be achieved, including what, if any,  component architecture alignment guidance, criteria, and tools are to be  developed and who will develop them. Specifically, while the strategy  states that it provides for demonstration of operational view linkages (e.g.,  activities, process, and capabilities) between the BEA and both  component and program architectures, the tools cited do not provide the  capability to either align program architectures to component  architectures or to align component architectures to the BEA. According  to officials from the Air Force, Navy, and DLA, they are using the  traceability tool to assess compliance of their programs with the BEA.  However, this tool does not allow them to assess their programs’  compliance with their component architectures. In contrast, Army and  U.S. Transportation Command officials told us that they do not require the  use of the traceability tool to assess compliance of their programs to the  BEA or their component architectures. According to BTA officials, they  are currently working with the Air Force and Navy to expand this tool to  include component architecture alignment capabilities.
section_title: Systems View Federation Lacks Key Execution Details
paragraphs: According to the BMA strategy, the systems view federation is the  application of principles, standards, services, and infrastructure to create  interoperable and reusable applications and systems. The strategy states  that this will be accomplished through the delivery of services within a  SOA construct, including an IT infrastructure that will expose reusable  functionality to federation members and enable interoperation and  interconnection of the business systems and applications that provide this  functionality. The strategy notes that this operating environment will be  comprised of applications, systems, metadata, and a unifying portal.  According to the strategy, this environment will build on existing  Enterprise Information Environment Mission Area capabilities and provide  the standards, policies, and technology needed to permit BMA services to  be shared with the other DOD mission areas.
However, the strategy does not describe how this will be accomplished,  including respective roles and responsibilities of those involved, the range  of services to be shared and developed, and the standards to be used.  Moreover, component officials told us that the details behind the strategy’s  SOA concepts need to be defined before a systems view federation can be  achieved. More specifically:    The strategy does not clearly describe how interoperable services will be  defined, developed, exposed, and subscribed to. For example, it does not  delineate the specific roles and responsibilities of the military departments  and defense agencies relative to defining, providing, and employing shared  systems and applications. As a result, the military departments and  defense agencies may pursue duplicative efforts. This is of particular  concern due to the various service orientation activities already under way  in the military departments and defense agencies. For example, the Air  Force has chartered a Transparency Integrated Product Team to guide  their SOA initiatives, and the Navy has established a Transformation  Group to support its service orientation activities. This is important  because a key aspect of the BMA federation strategy is reusing and  leveraging both enterprise-level and component-level systems and  applications.
The strategy does not relate system federation activities and capabilities to  its existing ETP. In particular, while the strategy describes a number of  “leave-in-place” pilots (systems and applications) that will be implemented  during the next year to demonstrate the use of shared services, it does not  describe how these relate to programs in the ETP. This is important  because the chief technical officer told us that many of the enterprise-level  programs being managed by the BTA and included in the ETP are to  evolve into shared services.
The strategy does not describe how interface standards will be established  and used for obtaining and delivering shared services. Defining and  enforcing such standards are important aspects of having services that are  interoperable and reusable. According to the BTA chief technical officer,  these standards will need to align with the yet-to-be-issued Enterprise  Information Environment Mission Area standards. Officials from the Air  Force and DISA agreed that more needs to be done to define the  infrastructure standards that will enable user subscription to reusable  systems and applications, particularly since the military departments and  DOD are moving ahead with their own SOA initiatives.
section_title: Strategy Does Not Include Milestones
paragraphs: The strategy outlines what it refers to as a high-level road map by listing  activities, capabilities, products, and services that are to be produced. (See  table 2 for this high-level road map.)
However, the strategy does not specify the milestones or provide specific  completion dates for the activities and related capabilities, products, and  services listed in its high-level road map. Instead, the strategy states that  the road map began in October 2006 and that milestones will occur at  approximately 3-month increments, without identifying, for example,  which steps have begun and what is to be accomplished over 3 months for  each of the steps.
section_title: Conclusions
paragraphs: DOD is in the early, formative stage of federating its BEA, with much  remaining to be decided and accomplished before it achieves its goals.  While the goals, concepts, and related activities; capabilities; products;  and services discussed in the strategy have merit and hold promise, the  strategy lacks sufficient specificity for it to be executed and, therefore,  must be viewed as a beginning. To the department’s credit, it recognizes  the need for greater detail surrounding how it will extend (federate) its  BEA. One key to making this happen is for the department to implement  our prior recommendation for having a BEA development management  plan. However, the department has yet to address this recommendation.  Until it does, the likelihood of effectively extending the BEA to include the  military departments and defense agencies is greatly reduced.
section_title: Recommendation for Executive Action
paragraphs: To further assist the department in evolving its BEA, we are reiterating our  prior recommendation for a BEA development management plan, and  augmenting it by recommending that the Secretary of Defense direct the  Deputy Secretary of Defense, as the chair of the DBSMC, to task the  appropriate DOD organizations, to ensure that this plan describes, at a  minimum, how the BMA architecture federation will be governed; how the  BMA federation strategy alignment with the DOD EA federation strategy  will be achieved; how component business architectures’ alignment with  incremental versions of the BEA will be achieved; how shared services  will be identified, exposed, and subscribed to; and what milestones will be  used to measure progress and results.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, signed by the DOD Deputy  Chief Information Officer and the Deputy Under Secretary of Defense  (Business Transformation) and reprinted in appendix II, the department  stated that it largely disagrees with our recommendation and added that  while the BMA played a leading role in defining the department’s approach  to architecture federation and a service-oriented architecture, the impact  of the issues discussed in this report goes beyond the scope of the  business systems modernization. DOD also stated that any analysis of  architecture federation should begin with the department’s approach and  not the BMA, since the BMA federation strategy was written as an  addendum to an enterprise approach. However, DOD added that it  recognizes that our analysis was complicated by the fact that many of the  enterprise-level strategy and governance documents, to which the BMA  must comply, have yet to be issued.
The department also made the following specific comments on the five  elements in our recommendation.
First, DOD stated that it partially concurs with the element relating to  architecture federation. According to DOD, responsibility for developing  the policy and guidance regarding how architectures are to be managed  within its federated environment lies with the ASD(NII)/CIO; officials  acknowledge the current lack of such guidance and stated that this will be  addressed with the issuance of the DOD EA federation strategy. As such,  the department recommends that we address our recommendation to  ASD(NII)/CIO. We agree on the current lack of and the need to develop  policies and guidance describing how the federation will be governed;  however, our recommendation is not intended to dictate who should  develop the policies or guidance for managing architectures within a  federated environment. Rather, it is focused on developing plans that  describe how the BMA will adopt and implement the policies and guidance  relating to federation governance.
Second, the department stated that it nonconcurs with the element  relating to ensuring alignment with other federation strategies. According  to DOD, there is a single architecture federation strategy for the  department—the DOD EA federation strategy—and other architecture  federation strategies supplement this overarching strategy. As such, it  stated that this element of our recommendation is not needed. We  disagree. While we do not question the department’s comment about the  relationships among the strategies, we believe that this element of our  recommendation is needed because its intent is to recognize these  relationships by promoting collaboration and ensuring linkages among the  various strategies.
Third, DOD stated that it nonconcurs with the element relating to  component architecture alignment with incremental versions of the BEA.  According to DOD, this element has been implemented both in policy and  execution to comply with legislative requirements, to include DOD’s  development and use of the Architecture Compliance and Requirements  Traceability tool. It also added that the Departments of the Air Force,  Army, and Navy have mandated the use of this tool to assess compliance  of their systems and architectures with the BEA. We disagree. The  National Defense Authorization Act for Fiscal Year 2005 includes a  requirement for ensuring that all business systems in excess of $1 million  be certified as being in compliance with the BEA; the architecture  traceability tool provides a mechanism for asserting only system  compliance and not component architecture compliance. In addition,  according to officials from the Air Force and Army, while they are  encouraging the use of the tool for assessing compliance of their systems  with the BEA, they have not mandated its use and are not using it to assess  compliance of their architectures with the BEA. Moreover, officials from  the Air Force further stated that they have not mandated the use of this  tool because it does not provide the capability to map the Air Force  architecture with the BEA. While we recognize DOD’s efforts to align  programs to the BEA, our recommendation focuses on the lack of a  discussion in the BMA federation strategy on how component  architectures (military departments and defense agencies) will be linked  to the BEA, including the lack of component architecture alignment  guidance, criteria, and tools.
Fourth, the department stated that it partially concurs with the element  relating to the identification and management of shared services.  According to DOD, each mission area or component is responsible for  identifying its own services requirements, and the ASD(NII)/CIO is  responsible for defining the overall approach to how these services will be  managed. As such, the department recommends that our recommendation  be directed to the ASD(NII)/CIO. We agree on the need for guidance  describing how shared services will be identified and managed; however,  our recommendation is not intended to dictate who should develop the  policies or guidance for managing shared services within a federated  environment. Rather, it is focused on developing plans that describe how  the BMA will adopt and implement the policies and guidance relating to  service orientation. As stated in the report, this is important because a key  aspect of the BMA federation strategy is to reuse and leverage both  enterprise-level and component-level systems and applications.
Fifth, DOD stated that it nonconcurs with the element relating to  milestones. According to DOD, milestones for gauging progress are and  will continue to be monitored in the department’s enterprise transition  plan. As such, it stated that it is unclear how the need to describe what  milestones will be used relates to the topics in the report. While we have  previously recognized that the transition plan provides information on  progress on major investments over the last 6 months—including key  accomplishments and milestones attained, this element of our  recommendation is intended to address the lack of measures (e.g., return  on investment of service-oriented architecture service reuse) or specific  completion dates for the activities and related capabilities, products, and  services that are to be produced for federating the Business Mission Area.
To further ensure that our recommendation is properly interpreted and  implemented, and to address DOD’s comments about directing the  recommendation to the appropriate parties, we have slightly modified our  recommendation.
We are sending copies of this report to interested congressional  committees; the Director, Office of Management and Budget; the Secretary  of Defense; the Deputy Secretary of Defense; the Under Secretary of  Defense for Acquisition, Technology, and Logistics; the Under Secretary of  Defense (Comptroller); the Assistant Secretary of Defense (Networks and  Information Integration)/Chief Information Officer; the Under Secretary of  Defense (Personnel and Readiness); and the Director, Defense Finance  and Accounting Service. We will also make copies available to others on  request. In addition, this report will also be available at no charge on our  Web site at http://www.gao.gov.
If you have any questions concerning this information, please contact me  at (202) 512-3439 or by e-mail at hiter@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. Key contributors to this report are listed in  appendix III.
section_title: Appendix I: Objective, Scope, and Methodology
paragraphs: Our objective was to determine what progress the Department of Defense  (DOD) has made in defining its Business Mission Area federation strategy.  To accomplish our objective, we reviewed DOD’s Business Mission Area  Federation Strategy and Road Map released in September 2006, comparing  the strategy and any associated implementation plans with prior findings  and recommendations relative to the content of the strategy.
In particular, we compared the strategy with our prior recommendations  for developing an architecture development management plan to define  how the department intends to extend and evolve its business enterprise  architecture. In addition, we compared the strategy with our prior findings  and the need to ensure that shared systems and applications (i.e., services)  are, among other things, defined, developed, exposed, and subscribed to  via well-defined and standardized interfaces. Furthermore, we reviewed  available information on activities, capabilities, products, and services  associated with the federation strategy, such as the Investment  Management Framework and the Architecture Compliance and  Requirements Traceability User’s Guide. In addition, we interviewed key  program officials, including the director of the Business Transformation  Agency’s Investment Management Directorate and the chief technical  officer and representatives from the Office of the Assistant Secretary of  Defense (Networks and Information Integration)/Chief Information  Officer, and the Departments of the Air Force, Army, and Navy; the  Defense Logistics Agency and Defense Information Systems Agency; and  the United States Transportation Command, to obtain an understanding of  the steps taken and required to develop and execute the federation  strategy.
We conducted our work at DOD headquarters offices in Arlington,  Virginia, from August 2006 through March 2007 in accordance with  generally accepted government auditing standards.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact person named above, key contributors to this  report were Neil Doherty, Nancy Glover, Michael Holland, Neelaxi  Lakhmani (Assistant Director), Anh Le, Jacqueline Mai, and Jennifer  Stavros-Turner.