DOD Business Systems Modernization: Governance Mechanisms for Implementing Management Controls Need to Be Improved
Published: 2012-06-01T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: For decades, DOD has been challenged in modernizing its business systems. Since 1995, GAO has designated DODs business systems modernization program as high risk, and it continues to do so today. To assist in addressing DODs business system modernization challenges, the National Defense Authorization Act for Fiscal Year 2005 requires the department to take certain actions prior to obligating funds for covered systems. It also requires DOD to annually report to the congressional defense committees on these actions and for GAO to review each annual report. In response, GAO performed its annual review of DODs actions to comply with the act and related federal guidance. To do so, GAO reviewed, for example, the latest version of DODs business enterprise architecture, fiscal year 2013 budget submission, investment management policies and procedures, and certification actions for its business system investments.
section_title: What GAO Found
paragraphs: The Department of Defense (DOD) continues to take steps to comply with the provisions of the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, as amended, and to satisfy relevant system modernization management guidance. While the department has initiated numerous activities aimed at addressing the act, it has been limited in its ability to demonstrate results. Specifically, the department
released its most recent business enterprise architecture version, which continues to address the acts requirements and is consistent with the departments future vision for developing its architecture. However, the architecture has not yet resulted in a streamlined and modernized business systems environment, in part, because DOD has not fully defined the roles, responsibilities, and relationships associated with developing and implementing the architecture.
included a range of information for 1,657 business system investments in its fiscal year 2013 budget submission; however, it does not reflect about 500 business systems, due in part to the lack of a reliable, comprehensive inventory of all defense business systems.
has not implemented key practices from GAOs Information Technology Investment Management framework since GAOs last review in 2011. In addition, while DOD has reported its intent to implement a new organizational structure and guidance to address statutory requirements, this structure and guidance have yet to be established. Further, DOD has begun to implement a business process reengineering review process but has not yet measured and reported results.
continues to describe certification actions in its annual report for its business system investments as required by the actDOD approved 198 actions to certify, decertify, or recertify defense business system modernizations, which represented a total of $2.2 billion in modernization spending. However, the basis for these actions and subsequent approvals is supported with limited information, such as unvalidated architectural compliance assertions.
lacks the full complement of staff it identified as needed to perform business systems modernization responsibilities. Specifically, the office of the Deputy Chief Management Officer, which took over these responsibilities from another office in September 2011, reported that 41 percent of its positions were unfilled.
DODs progress in modernizing its business systems is limited, in part, by continued uncertainty surrounding the departments governance mechanisms, such as roles and responsibilities of key organizations and senior leadership positions. Until DOD fully implements governance mechanisms to address these long-standing institutional modernization management controls provided for under the act, addressed in GAO recommendations, and otherwise embodied in relevant guidance; its business systems modernization will likely remain a high-risk program.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Secretary of Defense take steps to strengthen the departments mechanisms for governing its business systems modernization activities. DOD concurred with two of GAOs recommendations and partially concurred with one, but did not concur with the recommendation that it report progress on staffing the office responsible for business systems modernization to the congressional defense committees. GAO maintains that including staffing progress information in DODs annual report will facilitate congressional oversight and promote departmental accountability.

section_title: Letter
paragraphs: For decades, the Department of Defense (DOD) has been challenged in  modernizing its business systems. In 1995, we designated the  department’s business systems modernization program as high risk  because of its vulnerability to fraud, waste, abuse, and mismanagement,  and because of opportunities to achieve greater efficiencies and free up  resources for higher-priority needs; and we continue to designate it as  such today. In addition, we have reported that significant potential exists  for identifying and avoiding costs associated with duplicative functionality  across these business system investments, which account for billions of  dollars in annual expenditures and, according to the department, include  about 2,200 systems. Moreover, the systems that comprise DOD’s  business systems environment contribute to many DOD initiatives,  including improving departmentwide financial management and military  personnel health care.
Since May 2001, we have recommended that the Secretary of Defense  establish the means for effectively developing an enterprise architecture  and a corporate, architecture-centric approach to investment control and  decision making—two essential ingredients to a successful systems  modernization program. Further, Congress has included provisions in the  Ronald W. Reagan National Defense Authorization Act (NDAA) for Fiscal  Year 2005, as amended, that were consistent with our  recommendations. More specifically, section 332 of the act, as amended,  requires the department to, among other things, (1) develop a business  enterprise architecture (BEA) and a transition plan for implementing the  architecture, (2) identify systems information in its annual budget  submission, (3) establish a systems investment approval and  accountability structure along with an investment review process, and   (4) certify and approve any business system program costing in excess of  $1 million. The act further requires that the Secretary of Defense submit  an annual report to the congressional defense committees on DOD’s  compliance with certain requirements of the act not later than March 15 of  each year, through 2016. Additionally, the act directed us to submit to  these congressional committees—within 60 days of DOD’s report  submission—an assessment of the department’s actions to comply with  the requirements of the act.
As agreed with your offices, the objective of our review was to assess the  actions by DOD to comply with the act and related federal guidance. To  address the enterprise architecture and investment management  provisions, we focused on progress that has been made relative to  developing the federated BEA and establishing investment management  structures and processes, using our prior reports as a baseline. To  address the budgetary disclosure and certification provisions of the act,  we reviewed the department’s report to Congress, which was submitted  on April 3, 2012, and evaluated the information used to satisfy the budget  submission and investment review, certification, and approval aspects of  the act. We did not evaluate the department’s updated enterprise  transition plan because an updated plan was not issued during the time  period covered by our audit.
We conducted this performance audit at DOD and military department  offices in Arlington and Alexandria, VA, from September 2011 to June  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives. Details on our objective,  scope, and methodology are contained in appendix I.
subsections: section_title: Background
paragraphs: DOD is one of the largest and most complex organizations in the world,  and is entrusted with more taxpayer dollars than any other federal  department or agency. For fiscal year 2013, the department requested  approximately $613.9 billion—$525.4 billion in spending authority for its  base operations and an additional $88.5 billion to support overseas  contingency operations, such as those in Iraq and Afghanistan.
In support of its military operations, DOD performs an assortment of  interrelated and interdependent business functions, such as logistics  management, procurement, health care management, and financial  management. As we have previously reported, the DOD systems  environment that supports these business functions is overly complex and  error prone, and is characterized by (1) little standardization across the  department, (2) multiple systems performing the same tasks, (3) the  same data stored in multiple systems, and (4) the need for data to be  entered manually into multiple systems. The department recently  requested about $17.2 billion for its business systems environment and IT  infrastructure investments for fiscal year 2013.department’s systems inventory, this environment is composed of about  2,200 business systems and includes 310 financial management, 724  human resource management, 580 logistics, 254 real property and  installation, and 287 weapon acquisition management systems.
DOD currently bears responsibility, in whole or in part, for 14 of the 30  areas across the federal government that we have designated as high   risk. Seven of these areas are specific to the department, and 7 other  high-risk areas are shared with other federal agencies. Collectively,  these high-risk areas relate to DOD’s major business operations that are  inextricably linked to the department’s ability to perform its overall  mission. Furthermore, the high-risk areas directly affect the readiness and  capabilities of U.S. military forces and can affect the success of a  mission. In particular, the department’s nonintegrated and duplicative  systems impair its ability to combat fraud, waste, and abuse. As such,  DOD’s business systems modernization is one of the department’s  specific high-risk areas and is an essential enabler in addressing many of  the department’s other high-risk areas. For example, modernized  business systems are integral to the department’s efforts to address its  financial, supply chain, and information security management high-risk  areas.
subsections: section_title: DOD’s Approach to Business Systems Modernization
paragraphs: The department’s approach to modernizing its business systems  environment includes developing and using a BEA and associated  enterprise transition plan, improving business systems investment  management, and reengineering the business processes supported by its  defense business systems. These efforts are guided by DOD’s Chief  Management Officer and Deputy Chief Management Officer (DCMO). The  Chief Management Officer’s responsibilities include developing and  maintaining a departmentwide strategic plan for business reform and  establishing performance goals and measures for improving and  evaluating overall economy, efficiency, and effectiveness, and monitoring  and measuring the progress of the department. The DCMO’s  responsibilities include recommending to the Chief Management Officer  methodologies and measurement criteria to better synchronize, integrate,  and coordinate the business operations to ensure alignment in support of  the warfighting mission. The DCMO is also responsible for developing  and maintaining the department’s enterprise architecture for its business  mission area.
The DOD Chief Management Officer and DCMO are to interact with  several entities to guide the direction, oversight, and execution of DOD’s  business transformation efforts, which include business systems  modernization. These entities include the Defense Business Systems  Management Committee, which is intended to serve as the department’s  highest-ranking investment review and decision-making body for business  systems programs and is chaired by the Deputy Secretary of Defense.  The committee’s composition includes the principal staff assistants,  defense agency directors, DOD Chief Information Officer (CIO), and  military department Chief Management Officers. Table 1 describes key  DOD business systems modernization governance entities and their  composition.
Since 2005, DOD has employed a “tiered accountability” approach to  business systems modernization. Under this approach, responsibility and  accountability for business architectures and systems investment  management are assigned to different levels in the organization. For  example, the DCMO is responsible for developing the corporate BEA (i.e.,  the thin layer of DOD-wide policies, capabilities, standards, and rules)  and the associated enterprise transition plan. Each component is  responsible for defining a component-level architecture and transition  plan associated with its own tiers of responsibility and for doing so in a  manner that is aligned with (i.e., does not violate) the corporate BEA.  Similarly, program managers are responsible for developing program- level architectures and plans and for ensuring alignment with the  architectures and transition plans above them. This concept is to allow for  autonomy while also ensuring linkages and alignment from the program  level through the component level to the corporate level.
Consistent with the tiered accountability approach, the NDAA for Fiscal  Year 2008 required the Secretaries of the military departments to  designate the department Under Secretaries as Chief Management  Officers with primary responsibility for business operations. Moreover,  the Duncan Hunter NDAA for Fiscal Year 2009 required the military  departments to establish business transformation offices to assist their  Chief Management Officers in the development of comprehensive  business transformation plans.departments have designated their respective Under Secretaries as the  Chief Management Officers. In addition, the Department of the Navy  (DON) and Army have issued business transformation plans. Air Force  officials have stated that the department’s corporate Strategic Plan also  serves as its business transformation plan.
DOD’s BEA is intended to serve as a blueprint for DOD business  transformation. In particular, the BEA is to guide and constrain  implementation of interoperable defense business systems by, among  other things, documenting the department’s business functions and  activities, the information needed to execute its functions and activities,  and the business rules, laws, regulations, and policies associated with its  business functions and activities. According to DOD, the BEA is being  developed using an incremental approach, where each new release  addresses business mission area gaps or weaknesses based on priorities  identified by the department. The department considers its current  approach to developing the BEA both a “top-down” and “bottom-up”  approach. Specifically, it focuses on developing content to support  investment management and strategic decision making and oversight  (“top-down”) while also responding to department needs associated with  supporting system implementation, system integration, and software  development (“bottom-up”).
The department’s most recent BEA version (version 9.0), released in  March 2012, focuses on documenting information associated with its 15  end-to-end business process areas. (See table 2 for a list and description  of these business process areas.) In particular, the department’s most  recent Strategic Management Plan has identified the Hire-to-Retire and  Procure-to-Pay business process areas as its priorities. According to the  department, the process of documenting the needed architecture  information also includes working to refine and streamline each of the  associated end-to-end business processes.
In addition, DOD’s approach to developing its BEA involves the  development of a federated enterprise architecture. Such an approach  treats the architecture as a family of coherent but distinct member  architectures that conform to an overarching architectural view and rule  set. This approach recognizes that each member of the federation has  unique goals and needs, as well as common roles and responsibilities  with the levels above and below it. Under a federated approach, member  architectures are substantially autonomous, although they also inherit  certain rules, policies, procedures, and services from higher-level  architectures. As such, a federated architecture gives autonomy to an  organization’s components while ensuring enterprisewide linkages and  alignment where appropriate. Where commonality among components  exists, there are also opportunities for identifying and leveraging shared  services. Figure 1 provides a conceptual overview of DOD’s federated  BEA approach.
The certification of business system investments is a key step in DOD’s  IT investment selection process that the department has aimed to model  after GAO’s Information Technology Investment Management (ITIM)  framework. While defense business systems with a total cost over $1  million are required, as of June 2011, to use the Business Capability  Lifecycle,are also subject to the formal review and certification process through the  IRBs before funds are obligated for them.   a streamlined process for acquiring systems, these systems  Under DOD’s current approach to certifying investments, there are  several types of certification actions as follows:
Certify or certify with conditions: An IRB certifies the modernization as  fully meeting criteria defined in the act and IRB investment review  guidance (certify) or imposes specific conditions to be addressed by a  certain time (certify with conditions).
Recertify or recertify with conditions: An IRB certifies the obligation of  additional modernization funds for a previously-certified modernization  investment (recertify) or imposes additional related conditions to the  action (recertify with conditions).
Decertify: An IRB may decertify or reduce the amount of  modernization funds available to an investment when (1) a component  reduces funding for a modernization by more than 10 percent of the  originally certified amount, (2) the period of certification for a  modernization is shortened, or (3) the entire amount of funding is not  to be obligated as previously certified. An IRB may also decertify a  modernization after development has been terminated or if previous  conditions assigned by the IRB are not met.
section_title: Summary of NDAA Requirements
paragraphs: Congress included provisions in the act, as amended, that are aimed at  ensuring DOD’s development of a well-defined BEA and associated  enterprise transition plan, as well as the establishment and  implementation of effective investment management structures and  processes. The act requires DOD to develop a BEA and an enterprise transition plan for implementing the  identify each business system proposed for funding in DOD’s fiscal  year budget submissions, delegate the responsibility for business systems to designated establish an investment review structure and process, and not obligate appropriated funds for a defense business system  program with a total cost of more than $1 million unless the approval  authority certifies that the business system program meets specified  conditions.
The act also requires that the Secretary of Defense annually submit to the  congressional defense committees a report on the department’s  compliance with the above provisions.
In addition, the act sets forth the following responsibilities:  the DCMO is responsible and accountable for developing and  maintaining the BEA, as well as integrating business operations; the CIO is responsible and accountable for the content of those  portions of the BEA that support DOD’s IT infrastructure or information  assurance activities; the Under Secretary of Defense for Acquisition, Technology, and  Logistics is responsible and accountable for the content of those  portions of the BEA that support DOD’s acquisition, logistics,  installations, environment, or safety and occupational health activities; the Under Secretary of Defense (Comptroller) is responsible and  accountable for the content of those portions of the BEA that support  DOD’s financial management activities or strategic planning and  budgeting activities; and the Under Secretary of Defense for Personnel and Readiness is  responsible and accountable for the content of those portions of the  BEA that support DOD’s human resource management activities.
section_title: Prior GAO Reviews of DOD’s Business Systems Modernization
paragraphs: Between 2005 and 2008, we reported that DOD had taken steps to  comply with key requirements of the NDAA relative to architecture  development, transition plan development, budgetary disclosure, and  investment review, and to satisfy relevant systems modernization  management guidance. However, each report also concluded that much  remained to be accomplished relative to the act’s requirements and  relevant guidance.areas.
We made recommendations to address each of the  In May 2009, we reported that the pace of DOD’s efforts in defining and  implementing key institutional modernization management controls had  slowed compared with progress made in each of the previous 4 years,  leaving much to be accomplished to fully implement the act’s  requirements and related guidance. In addition, between 2009 and  2011, we found that long-standing challenges we previously identified  remained to be addressed.
The corporate BEA had yet to be extended (i.e., federated) to the  entire family of business mission area architectures, and the military  departments had yet to address key enterprise architecture  management practices and develop important content.
Budget submissions included some, but omitted other, key information  about business system investments, in part because of the lack of a  reliable, comprehensive inventory of all defense business systems.
The business system information used to support the development of  the transition plan and DOD’s budget requests, as well as certification  and annual reviews, was of questionable reliability.
DOD and the military departments had not fully defined key practices  (i.e., policies and procedures) related to effectively performing both  project-level (Stage 2) and portfolio-based (Stage 3) investment  management as called for in the ITIM.
Business system modernizations costing more than $1 million  continued to be certified and approved, but these decisions were not  always based on complete information. Further, we concluded that  certification and approval decisions may not be sufficiently justified  because investments were certified and approved without conditions  even though our prior reports had identified program weaknesses that  were unresolved at the time of certification and approval.
Accordingly, we reiterated existing recommendations and made additional  recommendations to address each of these areas. DOD partially agreed  with our recommendations and described actions being planned or under  way to address them. Nonetheless, DOD’s business systems  modernization efforts remain on our high-risk list due in part to issues  such as those described above.
Furthermore, in 2011, we reported that none of the military department  enterprise architecture programs had fully satisfied the requirements of  our Enterprise Architecture Management Maturity Framework and  recommended that they each develop a plan to do so. Our  recommendation further stated that if any department did not plan to  address any element of our framework, that department should include a  rationale for determining why the element was not applicable. DOD and  Army concurred, and Air Force and DON did not. In this regard, DOD  stated that Air Force and DON did not have a valid business case that  would justify the implementation of all of our framework elements.  However, Air Force and DON did not address why the elements called for  by our recommendation should not be developed. Further, Army officials  stated that the department had not yet issued a plan. To date, none of the  military departments have addressed our recommendation.
GAO-11-278. management processes. These management controls are vital to  ensuring that DOD can effectively and efficiently manage an undertaking  with the size, complexity, and significance of its business systems  modernization, and minimize the associated risks.
section_title: DOD Lacks Governance Mechanisms for Institutionalizing Modernization Management Controls
paragraphs: DOD continues to take steps to comply with the provisions of the Ronald  W. Reagan NDAA for Fiscal Year 2005, as amended, and to satisfy  relevant system modernization management guidance. However, despite  undertaking activities to address NDAA requirements and its future vision;  the department has yet to demonstrate significant results. Specifically,  DOD has updated its BEA and is beginning to modernize its corporate  business processes, but the architecture is still not federated through  development of aligned subordinate architectures for each of the  military departments, and it still does not include common definitions  for key terms and concepts to help ensure that the respective portions  of the architecture will be properly linked and aligned. has not included all business system investments in its fiscal year  2013 budget submission, due in part to an unreliable inventory of all  defense business systems. has made limited progress regarding investment management policies  and procedures and has not yet established the new organizational  structure and guidance that DOD has reported will address statutory  requirements. In addition, while DOD implemented a business  process reengineering (BPR) review process, the department is not  measuring and reporting its results.  continues to describe certification actions for its business system  investments based on limited information. has fewer staff than it identified as needed to execute its  responsibilities for business systems modernization. Specifically, the  office of the DCMO, which took over these responsibilities from  another office that was disestablished in 2011, reported that it had  filled only 82 of its planned 139 positions, with 57 positions vacant.
DOD’s limited progress in developing and implementing its federated  BEA, investment management policies and procedures, and our related  recommendations is due in part, to the roles and responsibilities of key  organizations and senior leadership positions being largely undefined.  Furthermore, the impact of DOD’s efforts to reengineer its end-to-end  business processes has yet to be measured and reported, and efforts to  execute needed activities are limited by challenges in staffing the office of  the DCMO. Until the long-standing institutional modernization  management controls provided for under the act, addressed in our  recommendations, and otherwise called for in best practices are fully  implemented, it is likely that the department’s business systems  modernization will continue to be a high-risk program.
subsections: section_title: DOD Has Made Progress in Developing Its BEA but Has Not Developed Important Architecture Content or Fully Defined Roles and Responsibilities
paragraphs: Among other things, the act requires DOD to develop a BEA that would  cover all defense business systems and their related functions and  activities and that would enable the entire department to (1) comply with  all federal accounting, financial management, and reporting requirements  and (2) routinely produce timely, accurate, and reliable financial  information for management purposes. The BEA should also include  policies, procedures, data standards, and system interface requirements  that are to be applied throughout the department. In addition, the NDAA  for Fiscal Year 2012 added requirements that the BEA include, among  other things, performance measures that are to apply uniformly  throughout the department and a target defense business systems  computing environment for each of DOD’s major business processes.  Furthermore, the act requires a BEA that extends to (i.e., federates) all  defense organizational components and requires that each military  department develop a well-defined enterprisewide business architecture  and transition plan.
According to DOD, achieving its vision for a federated business  environment requires, among other things, creating an overarching   that can effectively map the  taxonomy and associated ontologiescomplex interactions and interdependencies of the department’s business  environment. Such a taxonomy and ontologies will provide the various  components of the federated BEA with the structure and common  vocabularies to help ensure that their respective portions of the  architecture will be properly aligned and coordinated. In April 2011, DOD  provided additional guidance that calls for the use of ontologies for  federating the BEA and asserting systems compliance. In addition, DOD  guidance states that, because of the interrelationship among models and  across architecture efforts, it is useful to define an overarching taxonomy  with common definitions for key terms and concepts in the development  of the architecture. The need for such a taxonomy and associated  ontologies was derived from lessons learned from federation pilots  conducted within the department that showed that federation of  architectures was made much more difficult because of the use of  different definitions to represent the same architectural data.
In addition, we have previously reported that defining and documenting  roles and responsibilities is critical to the success of enterprise  architecture efforts. More specifically, our Enterprise Architecture  Management Maturity Framework calls for a corporate policy that  identifies the major players associated with enterprise architecture  development, maintenance, and use and provides for a performance and  accountability framework that identifies each player’s roles,  responsibilities, and relationships and describes the results and outcomes  for which each player is responsible and accountable.
In 2009, we reported that the then-current version of the BEA (version  6.0) addressed, to varying degrees, missing elements, inconsistencies,  and usability issues that we previously identified, but that gaps still  remained. In March 2012, DOD released BEA version 9.0, which  continues to address the act’s requirements. For example, version 9.0 organizes BEA content around its end-to-end business processes and  adds additional content associated with these processes. For  example, version 9.0 added the “Accept Purchase Request”  subprocess and placed this subprocess in the context of its Procure- to-Pay end-to-end business process. In addition, the Hire-to-Retire  end-to-end business process includes the subprocess “Manage  Benefits,” which is linked to over 1,200 laws, regulations, and policies,  as well as 11 subordinate business activities, such as “Manage  Retirement Benefits.” As a result, users can navigate the BEA to  identify relevant subprocesses for each end-to-end business process  and determine important laws, regulations, and policies, business  capabilities, and business rules associated with a given business  process.  includes enterprise data standards for the Procure-to-Pay and Hire-to- Retire end-to-end business processes. Specifically, as part of the  Procure-to-Pay end-to-end business process, enterprise standards for  Procurement Data and Purchase Request Data were added. In  addition, for the Hire-to-Retire end-to-end business process, DOD  updated the Common Human Resources Information Standards,  which is a standard for representing common human resources  management data concepts and requirements within the defense  business environment. As a result, stakeholders can accelerate   coordination and implementation of the high priority end-to-end  business processes and related statutory requirements. uses a standardized business process modeling approach to  represent BEA process models. For example, the BEA uses the  business process modeling notation standard to create a graphical  representation of the “Accept Goods and Services” business process.  Using a modeling approach assists DOD in its effort to eventually  support automated queries of architecture information, including  business models and authoritative data, to verify investment  compliance and validate system solutions. includes performance measures and milestones for initiatives in  DOD’s Strategic Management Plan and relates the end-to-end  business processes and operational activities documented in the BEA  with the plan’s initiatives and performance measures. For example,  the BEA identifies that the Procure-to-Pay end-to-end business  process is related to the Strategic Management Plan’s measure to  determine the percentage of contract obligations competitively  awarded. This is important for meeting the act’s new requirements   associated with performance measures and to enable traceability of  BEA content to the Strategic Management Plan.
DOD has defined a federated approach to its BEA that is to provide  overarching governance across all business systems, functions, and  activities within the DOD. This approach involves the use of semantic web  technologies to provide visibility across its respective business  architecture efforts. Specifically, this approach calls for the use of non- proprietary, open standards and protocols to develop DOD architectures  to allow users to, among other things, locate and analyze needed  architecture information across the department. Among other things,  DOD’s approach calls for the corporate BEA, each end-to-end business  process area (e.g., Procure-to-Pay), and each DOD organization (e.g.,  Army) to establish a common vocabulary and for the programs and  initiatives associated with these areas to use this vocabulary when  developing their respective system and architecture products.
However, in 2011, we reported that each of the military departments had  taken steps to develop architectural content, but that none had   well-defined architectures to guide and constrain its business  transformation initiatives. Further, since May 2011, the BEA has yet to  be federated through development of aligned subordinate architectures  for each of the military departments. Specifically, DON reported that it has  not made any significant changes to its BEA content. Army reported that it  has adopted the end-to-end processes as the basis of the Army BEA, and  Air Force reported that it has added additional architecture business  process content and mapped some of this content to the end-to-end  processes. However, each has yet to fully satisfy the requirements of our  Enterprise Architecture Management Maturity Framework.
In addition, the BEA does not include other important content that will be  needed for achieving the office of the DCMO’s vision for BEA federation.  For example,
While DOD has begun to develop a taxonomy that provides a  hierarchical structure for classifying BEA information into categories, it  has yet to develop an overarching taxonomy that identifies and  describes all of the major terms and concepts for the business  mission area. Further, version 9.0 does not include a systematic  mechanism for evaluating and adding new taxonomy terms and rules  for addressing ambiguous terms and descriptions. This is important  since federation relies heavily on the use of taxonomy to provide the  structure to link and align enterprise architectures across the business  mission area, thus enabling architecture federation. Without an  overarching taxonomy, there is an increased risk of not finding the  most relevant content, thereby making the BEA less useful for making  informed decisions regarding portfolio management and  implementation of business systems solutions.
DOD has begun to define corporate BEA ontologies and is developing  ontologies in the human resources management area and for the U.S.  Transportation Command. However, BEA 9.0 does not include  ontologies for all business mission domains and organizations.  According to DOD officials, each domain and organization will develop  its own ontology. This is important since ontologies promote a  comprehensive understanding of data and their relationships. In  addition, they enable DOD to implement automated queries of  information and integrate information across the department.  However, DOD has yet to describe how military departments will be  held accountable for executing tasks needed to be accomplished for  establishing domain ontologies for their respective BEAs or whether  these ontologies are also to be used for their respective corporate  enterprise architecture efforts. Without these ontologies, there is an  increased risk of not fully addressing the act’s requirements relating to  integrating budget, accounting, and program information and systems  and achieving DOD’s vision for a federated architecture.
DOD officials acknowledged these issues and stated that future versions  of the BEA will leverage semantic technologies to create and document a  common vocabulary and associated ontology. However, the department  has yet to describe how each of the relevant entities will work together in  developing the needed taxonomy and ontology.
In addition to describing certain content required to be in the BEA, as  described earlier, the act assigns responsibility for developing portions of  the BEA to various entities. The department has developed strategies that  begin to document certain responsibilities associated with architecture  federation. For example, the Global Information Grid Architecture  Federation Strategy states that the DOD enterprise is responsible for  establishing a governance structure for DOD architecture federation. The  strategy also states that each mission area, such as the business mission  area, is to develop and maintain mission area architectures, such as the  BEA. However, given the many entities involved in BEA and DOD  architecture federation, officials from the office of the DCMO have  expressed concerns over who is accountable for achieving specific  federation tasks and activities and how the new vision for BEA federation  will be enforced.
section_title: Fiscal Year 2013 Budget Submission Did Not Include Key Information on All Business Systems
paragraphs: Another requirement of the NDAA for Fiscal Year 2005, as amended, is  that DOD’s annual IT budget submission must include key information on  each business system for which funding is being requested, such as the  system’s precertification authority and designated senior official, the  appropriation type and amount of funds associated with modernization  and current services (i.e., operation and maintenance), and the  associated Defense Business Systems Management Committee approval  decisions.
The department’s fiscal year 2013 budget submission includes a range of  information for 1,657 business system investments, including the  system’s name, approval authority, and appropriation type.submission also identifies the amount of the fiscal year 2013 request that  is for development and modernization versus operations and  maintenance and notes the certification status (e.g., approved, approved  with conditions, not applicable, and withdrawn) and the Defense Business  Systems Management Committee approval date, where applicable.
However, similar to prior budget submissions, the fiscal year 2013 budget  submission does not reflect all business system investments. To prepare  the submission, DOD relied on business system investment information  (e.g., funds requested, mission area, and system description) that the  components entered into the department’s system used to prepare its  budget submission (SNAP-IT). In accordance with DOD guidance and  according to DOD CIO officials, the business systems listed in SNAP-IT  should match the systems listed in the Defense Information Technology  Portfolio Repository (DITPR)—the department’s authoritative business  systems inventory. However, the DITPR data provided by DOD in March  2012 included 2,179 business systems. Therefore, SNAP-IT did not reflect about 500 business systems that were identified in DITPR.
In 2009, we reported that the information between SNAP-IT and DITPR  data repositories were not consistent and, accordingly, recommended  that DOD develop and implement plans for reconciling and validating the  completeness and reliability of information in its two repositories, and to  include information on the status of these efforts in the department’s fiscal  year 2010 report in response to the act. DOD agreed with the need to reconcile information between the two repositories and stated that it had  begun to take actions to address this. In 2011, we reported that,  according to the office of the DOD CIO, efforts to provide automated  SNAP-IT and DITPR integration work were delayed due to increased  SNAP-IT requirements in supporting the fiscal year 2012 budget  submission and ongoing reorganization efforts within the department.  DOD officials also told us that the department planned to restart the  process of integrating the two repositories beginning in the third quarter of  fiscal year 2011.
Since that time, DOD CIO officials have reiterated the department’s  commitment to integrating the two repositories and taken steps toward  achieving this end. For example, the officials stated that they have added  a field to the DITPR repository that allows components to identify an  individual system as a defense business system. These officials added  that this change, once fully implemented, will be a key to providing  automated DITPR and SNAP-IT integration. The Deputy DOD CIO  (Resources) has also sent memoranda to specific DOD components  identifying systems listed in DITPR that are not properly associated with  systems identified in SNAP-IT and requesting that the components take  action to address these inconsistencies. Nevertheless, DOD CIO officials  responsible for the DITPR and SNAP-IT repositories stated that efforts to  integrate them continue to be limited by ongoing organizational changes  and the time required to address new system requirements unrelated to  integrating the repositories. For example, these officials cited slowdowns  resulting from the recent disestablishment of DOD’s Networks and  Information Integration organization, as well as time spent making  adjustments to the SNAP-IT repository to accommodate new Office of  Management and Budget reporting requirements.data are owned by the components and therefore it is ultimately the  responsibility of the components to update their respective data.  However, DOD has not established a deadline by which it intends to  complete the integration of the repositories and validate the completeness  and reliability of information.
They added that all  Until DOD has a reliable, comprehensive inventory of all defense  business systems, it will not be able to ensure the completeness and  reliability of the department’s IT budget submissions. Moreover, the lack  of current and accurate information increases the risk of oversight  decisions that are not prudent and justified.
section_title: DOD Has Not Yet Redefined Its Investment Management Process
paragraphs: DOD has made limited progress in defining and implementing investment  management policies and procedures as required by the act and  addressed in our ITIM framework since our last review in 2011. In  addition, while the department has reported its intent to implement a new  organizational structure and guidance to address statutory requirements,  this structure and guidance have yet to be established. DOD also  continues to approve investments on the basis of BEA compliance  assessments that have not been validated. Further, while DOD has  conducted various BPR activities related to its business system  investments and underlying business processes, the department has not  yet begun to measure associated results. Thus, the extent to which these  efforts have streamlined and improved the efficiency of the underlying  business processes remains uncertain.
The act requires DOD to establish an IRB and investment management  processes that are consistent with the investment management  provisions of the Clinger-Cohen Act of 1996. As we have previously  reported, organizations that satisfy Stages 2 and 3 of our ITIM  framework have the investment selection, control, and evaluation  governance structures, and the related policies, procedures, and  practices that are consistent with the investment management provisions  of the Clinger-Cohen Act. We have used the framework in many of our  evaluations, and a number of agencies have adopted it.
In 2011, we reported that DOD had continued to establish investment  management processes described in our ITIM framework but had not fully  defined all key practices. For example, we reported that DOD had fully  implemented two critical processes associated with capturing investment  information and meeting business needs, and partially completed the  Stage 2 critical process associated with instituting an investment board.  However, the department had yet to address other critical processes,  including those associated with selecting investments and providing  investment oversight.
Since 2011, DOD has not fully implemented any additional key  practices.progress in addressing elements of our ITIM framework that we  previously reported as unsatisfied. For example,   Furthermore, the military departments have made very little  In 2011, we reported that Air Force had implemented four key  practices related to effectively managing investments as individual  business system programs (Stage 2). The Air Force had also  addressed a key practice associated with portfolio-level investment  management (Stage 3) — assigning responsibility for the  development and modification of IT portfolio selection criteria.  However, it has not implemented any additional practices since that  time. The Air Force has described its intent to change its IT  investment management structure and form a new branch to lay the  foundation for integrated, efficient IT portfolio management processes;  however, according to Air Force officials, this office is not yet fully  established and faces competing personnel issues within the  department. Further, Air Force officials stated that they are working to  update the department’s IT portfolio management and IT investment  guidance, but the updates are not expected to be issued until  November 2012.
In 2011, we reported that DON had implemented four key practices  related to effectively managing investments as individual business  system programs (Stage 2) and one key practice related to managing  IT investments as a portfolio of programs (Stage 3). Since that time,  DON has not fully implemented any additional key practices. While  the department demonstrated that it has documented policies and  procedures related to establishing assessment standards to describe  a program’s health (e.g., cost, schedule, and performance), these  policies and procedures do not describe the enterprisewide IT  investment board’s role in reviewing and making decisions based on  this information. Such a description is important because the   investment board has ultimate responsibility for making decisions  about IT investments.
In 2011, we reported that Army had implemented two key practices  associated with capturing investment information. Specifically, it had  established policies and procedures for collecting information about  the department’s investments and had assigned responsibility for  investment information collection and accuracy. These are activities  associated with effectively managing investments as individual  business system programs (Stage 2). However, with regard to  managing IT investments as a portfolio of programs (Stage 3), the  Army had not fully defined any of the five key practices. Further, since  that time, the Army has not fully implemented any additional Stage 2  or Stage 3 practices. Army officials stated that the department has  been focused on performing extensive portfolio reviews that are  intended to inform many of the ITIM key practices and lead to updates  of its investment management policies and procedures. As of April  2012, Army officials stated that the department had completed its first  round of portfolio reviews. According to Army officials, the department  has also worked to release its Business Systems Information  Technology Implementation Plan, which is to provide details for its  investment management strategy, due as part of the 2012 Army  Campaign Plan; however, this plan has not yet been released.
According to the department, the slow progress made on the investment  management process at DOD and the military departments in the past  year is due, in part, to the department’s activities to address the new  NDAA for Fiscal Year 2012 requirements. Specifically, in April 2012,  DOD reported that it was in the process of constituting a single IRB.
According to DOD, this IRB is to replace the existing governance  structure and is to be operational by October 2012. In addition, DOD  reported that it intends to incrementally implement an expanded  investment review process that analyzes business system investments  using common decision criteria and establishes investment priorities while  ensuring integration with the department’s budgeting process. The  department has stated its intention to use our ITIM model to assess its  ability to comply with its related investment selection and control  requirements. Further, DOD officials stated that this new investment  review process will encompass a portfolio-based approach to investment  management that is to employ a structured methodology for classifying  and assessing business investments in useful views across the  department. DOD officials stated that an initial review of all systems  requiring certification under the new NDAA requirements is also planned  to be completed by the start of the new fiscal year.
While the department has reported its intent to implement this new  organizational structure and guidance to address statutory requirements  and redefine the process by which the department selects, evaluates, and  controls business systems investments, this structure and guidance have  yet to be established. DOD officials stated that the process has not yet  been completed because they want to make sure they consider the best  approach for investment management going forward. Accordingly, DOD is  taking a phased approach as described in the department’s congressional  report, which it intends to fully implement by October 2012.
While it is too soon to evaluate the department’s updated approach to  business system investment management, we will further evaluate DOD’s  progress in defining and implementing its updated investment review  processes in our fiscal year 2013 report on defense business systems  modernization. Until DOD redefines and implements its investment  management processes by the established deadline and until the military  departments make additional progress on their own investment  management processes, it is unlikely that the thousands of DOD business  system investments will be managed in a consistent, repeatable, and  effective manner.
Since 2005, DOD has been required to certify and approve all business  system modernizations costing more than $1 million to ensure that they  meet specific conditions defined in the act. This process includes  asserting that an investment is compliant with the BEA.
The department continues to approve investments on the basis of  architecture compliance. However, the department’s policy and guidance  associated with architecture compliance still does not call for compliance  assertions to be validated and officials agreed that not all of the  compliance information has been validated. Department officials stated  that some information associated with the compliance process has been  validated, such as information associated with complying with DOD’s  Standard Financial Information Structure. recommendations that the department amend existing policy and  requirements to explicitly call for such validation to occur. DOD agreed  with our findings and recommendations and stated that it planned to  assign validation responsibilities and issue guidance that described the  methodology for performing validation activities. Nonetheless, the  department has not yet addressed our recommendation.
The Standard Financial Information Structure is intended to provide a standard financial  management data structure and uniformity throughout DOD in reporting on the results of  operations.
Among other things, BEA compliance is important for helping to ensure  that DOD programs have been optimized to support DOD operations.  However, as we have reported, without proper validation of compliance  assertions, there is an increased risk that DOD will make business  system investment decisions based on information that is inaccurate and  unreliable. Under DOD’s vision for a semantic BEA, described previously  in this report, officials have stated that compliance validations will be  conducted automatically using specialized software tools as program  architecture artifacts are developed. However, until DOD achieves its  semantic BEA vision and addresses our prior recommendation,  compliance assertions will continue to be unvalidated.
In addition to the requirement that covered business systems be certified  and approved to be in compliance with the BEA, the act requires that the  Chief Management Officer certify that these business systems have  undergone appropriate BPR activities. BPR is an approach for  redesigning the way work is performed to better support an organization’s  mission and reduce costs. After considering an organization’s mission,  strategic goals, and customer needs, reengineering focuses on improving  an organization’s business processes. We have issued BPR guidance  that, among other things, discusses the importance of having meaningful  performance measures to assess whether BPR activities actually achieve   In this regard, the act, as amended, identifies  the intended results.intended results of BPR reviews such as ensuring that the business  process to be supported by the defense business system will be as  streamlined and efficient as practicable and the need to tailor commercial- off-the-shelf systems to meet unique requirements or incorporate unique  interfaces has been eliminated or reduced to the maximum extent  practicable.
While DOD has conducted various BPR activities, including preparing  BPR assessment guidance; conducting assessments to meet the act’s  requirements; and performing other BPR efforts including refining its end- to-end business processes, the department has not yet begun to measure  associated results. The department’s BPR activities are summarized as  follows:
DOD issued interim guidance in April 2010 and final guidance in April  2011 to assist programs in addressing the act’s BPR requirement.This guidance describes the types of documentation required for  systems seeking certification, including a standardized BPR  assessment form, and illustrates the process for submitting  documentation for review and approval. DOD’s final BPR guidance  related to system certification generally comports with key practices  described in our guidance. For example, DOD’s guidance recognizes  the importance of developing a clear problem statement and business  case, analyzing the as-is and to-be environments, and developing a  change management approach for implementing the new business  process.
Consistent with its guidance, DOD has begun to implement its BPR  review process in an effort to meet the act’s requirements.  Specifically, all systems in fiscal year 2011 submitted BPR  assessment forms for review. In addition, the DCMO and military  department Chief Management Officers are in the process of signing  formal determinations that sufficient BPR was conducted with respect  to each program.
The department has also performed BPR to respond to specific needs  that have been identified by departmental components and to refine  its end-to-end business processes. For example, the Defense  Commissary Agency, in cooperation with the Business Transformation  Agency and now the office of the DCMO, used BPR to help formulate  a future enterprise transition plan for the agency. In addition, DOD  officials described activities to refine DOD’s debt management  business process, which is part of the Budget-to-Report end-to-end  process. The standardization of related business process models  related to debt management led to updates in the latest BEA, which  now provide tools that can be used to guide and constrain  investments.
While DOD has performed the BPR activities described above, the extent  to which these efforts have streamlined and improved the efficiency of the  underlying business processes remains uncertain because the  department has yet to establish specific measures and report outcomes  that align with the department’s efforts. For example, the department  does not track information, such as the number of systems that have  undergone material process changes or the number of interfaces reduced  or eliminated as a result of BPR reviews. DOD officials noted that  addressing these requirements has been challenging and measuring  progress, such as the number of interfaces reduced, has not been a  priority. However, until the department develops and reports on  performance measures associated with the development of its end-to-end  processes and their related BPR activities, the department and its  stakeholders will not know the extent to which BPR is effectively  streamlining and improving its end-to-end business processes as  intended.
section_title: DOD’s Annual Report Continues to Describe Certification Actions for Its Business System Investments
paragraphs: Among other things, the act requires DOD to include, in its annual report  to congressional defense committees, a description of specific actions the  department has taken on each business system submitted for  certification. As applicable in fiscal year 2011, the act required that  modernization investments involving more than $1 million in obligations  be certified by a designated approval authority as meeting specific  criteria, such as whether or not the system is in compliance with DOD’s  BEA and appropriate BPR efforts have been undertaken. Further, the act  requires that the Defense Business Systems Management Committee  approve each of these certifications.
DOD’s annual report identifies that the Defense Business Systems  Management Committee approved 198 actions to certify, decertify, or  recertify defense business system modernizations. These 198 IRB  certification actions represented a total of about $2.2 billion in  modernization spending. Specifically, the annual report states that during  fiscal year 2011, the Defense Business Systems Management Committee  approved 58 unique certifications, 102 recertifications, and 38  decertifications—101 with and 97 without conditions. Examples of  conditions associated with individual systems include conditions related to  business process engineering and BEA compliance.
While DOD has continued to report its certification actions, these actions  have been based on limited information, such as unvalidated architecture  compliance assertions, as discussed in the previous section. Until DOD  addresses our prior recommendations, the department faces increased  risk that it will not effectively be able to oversee its extensive business  systems investments.
section_title: DCMO Lacks Staff It Identified as Needed to Support Departmentwide Business Systems Modernization
paragraphs: Among other things, the act calls for the DCMO to be responsible and  accountable for developing and maintaining the BEA, as well as  integrating defense business operations. Although responsibility for these  activities previously resided with the Business Transformation Agency,  DOD announced the disestablishment of this agency in August 2010. In  June 2011, we recommended that DOD expeditiously complete the  implementation of the announced transfer of functions of the agency and  provide specificity as to when and where these functions will be  transferred.structure consisting of a front office and six directorates and identified the  staff resources it would need to fulfill its new responsibilities, which  became effective in September 2011.
Subsequently, the DCMO defined an organizational  However, the office reported that it has not yet filled many of the positions  needed to execute these responsibilities. In particular, as of April 2012,  the office reported that it had filled only 82 of its planned 139 positions,  with 57 positions (41 percent) remaining unfilled. had filled only 12 of 43 positions within its Technology, Innovation, and  Engineering Directorate; which, among other things, is responsible for  developing the BEA. Further, only 10 of 19 positions within the Planning  and Performance Management Directorate, 14 of 22 positions within its  Business Integration Directorate, and 16 of 23 positions within its  Investment and Acquisition Management Directorate had been filled.  Table 3 identifies the key responsibilities of each DCMO organizational  component as well as planned and actual staffing.
section_title: Conclusions
paragraphs: Establishing a well-defined, federated BEA and modernizing DOD’s  business systems and processes are critical to effectively improving the  department’s business systems environment. The department is taking  steps to establish such a business architecture and modernize its  business systems and processes, but long-standing challenges remain.  Specifically, while DOD had made progress in developing its corporate  enterprise architecture, it has yet to be federated through the  development of aligned subordinate architectures for each of the military  departments. The department has also taken effective steps to establish  an infrastructure for establishing a federated BEA, including documenting  a vision for the BEA and developing content around its end-to-end  business processes. However, the department’s ability to achieve its  federated BEA vision is limited by the lack of common definitions for key  terms and concepts to help ensure that each of the respective portions of  the architecture will be properly linked and aligned, as well as by the  absence of a policy that clarifies roles, responsibilities, and accountability  mechanisms. In addition, information used to support the development of  the DOD’s budget requests continues to be of questionable reliability and  no deadline for validating reliable information has been set. DOD has also  not implemented key practices from our ITIM framework since our last  review in 2011. Further, while the department has begun taking steps to  reengineer its business systems and processes, and has issued sound  guidance for conducting BPR associated with individual business  systems, it has yet to measure and report on the impact these efforts  have had on streamlining and simplifying its corporate business  processes. Finally, the efforts of the office of the DCMO have been  impacted by having fewer staff than the office identified as needed to  support departmentwide business systems modernization.
Collectively, these limitations continue to put the billions of dollars spent  annually on about 2,200 business system investments that support DOD  functions, such as departmentwide financial management and military  personnel health care at risk. Our previous recommendations to the  department have been aimed at accomplishing these and other important  activities related to its business systems modernization. While the  department has agreed with these recommendations, its progress in  addressing the act’s requirements, its vision for a federated architecture,  and our related recommendations is limited, in part, by continued  uncertainty surrounding the roles and responsibilities of key organizations  and senior leadership positions. In light of this, it is essential that the  Secretary of Defense issue a policy that resolves these issues, as doing  so is necessary for the department to establish the full range of  institutional management controls needed to address its business  systems modernization high-risk area. It is equally important that DOD  measure the impact of its BPR efforts and include information on the  results of these efforts and its efforts to fully staff the office of the DCMO  in the department’s annual report in response to the act.
section_title: Recommendations for Executive Action
paragraphs: Because we have existing recommendations that address many of the  institutional management control weaknesses discussed in this report, we  reiterate those recommendations.
In addition, to ensure that DOD continues to implement the full range of  institutional management controls needed to address its business  systems modernization high-risk area, we recommend that the Secretary  of Defense ensure that the Deputy Secretary of Defense, as the  department’s Chief Management Officer, establish a policy that clarifies  the roles, responsibilities, and relationships among the Chief  Management Officer, Deputy Chief Management Officer, DOD and  military department Chief Information Officers, Principal Staff Assistants,  military department Chief Management Officers, and the heads of the  military departments and defense agencies, associated with the  development of a federated BEA. Among other things, the policy should  address the development and implementation of an overarching  taxonomy and associated ontologies to help ensure that each of the  respective portions of the architecture will be properly linked and aligned.  In addition, the policy should address alignment and coordination of  business process areas, military department and defense agency  activities associated with developing and implementing each of the  various components of the BEA, and relationships among these entities.
To ensure that annual budget submissions are based on complete and  accurate information, we recommend that the Secretary of Defense direct  the appropriate DOD organizations to establish a deadline by which it  intends to complete the integration of the repositories and validate the  completeness and reliability of information.
To facilitate congressional oversight and promote departmental  accountability, we recommend that the Secretary of Defense ensure that  the Deputy Secretary of Defense, as the department’s Chief Management  Officer, direct the Deputy Chief Management Officer to include in DOD’s  annual report to Congress on compliance with 10 U.S.C. § 2222,  the results of the department’s BPR efforts. Among other things, the  results should include the department’s determination of the number  of systems that have undergone material process changes, the  number of interfaces eliminated as part of these efforts (i.e., by  program, by name), and the status of its end-to-end business process  reengineering efforts, and an update on the office of the DCMO’s progress toward filling staff  positions and the impact of any unfilled positions on the ability of the  office to conduct its work.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, signed by the Deputy Chief  Management Officer and reprinted in appendix II, the department partially  concurred with our first recommendation, concurred with our second and  third recommendations, and did not concur with the remaining  recommendation.
The department partially concurred with our first recommendation to  establish a policy that clarifies the roles, responsibilities, and relationships  among its various management officials associated with the development  of a federated BEA. In particular, the department stated its belief that  officials’ roles, relationships, and responsibilities are already sufficiently  defined through statute, policy, and practice, and that additional guidance  is not needed. However, the department added that it will continue to look  for opportunities to strengthen and expand guidance, to include the new  investment management and architecture processes. We do not agree  that officials’ roles, relationships, and responsibilities are sufficiently  defined in existing policy. For example, we found that DOD has not  developed a policy that fully defines the roles, responsibilities, and  relationships associated with developing and implementing the BEA.  Moreover, in our view, responsibility and accountability for architecture  federation will not be effectively addressed with additional guidance  because guidance cannot be enforced. Rather, we believe a policy, which  can be enforced, will more effectively establish responsibility and  accountability for architecture federation. Without a policy, the department  risks not moving forward with its vision for a federated architecture. Thus,  we continue to believe our recommendation is warranted.
The department concurred with our second recommendation, to establish  a deadline by which it intends to complete the integration of the  repositories and validate the completeness and reliability of information,  and described commitments and actions being planned or under way. We  support the department’s efforts to address our recommendation and  reiterate the importance of following through in implementing the  recommendation within the stated time frame.
DOD also concurred with our third recommendation that the Deputy  Secretary of Defense, as the department’s Chief Management Officer,  direct the Deputy Chief Management Officer to include the results of the  department’s BPR efforts in its annual report to Congress. However, the  department stated that given the passage of the NDAA for Fiscal Year  2012, BPR authority now rests with the military department Chief  Management Officers. As such, DOD stated that it would be appropriate  for the recommendation to be directed to the BPR owners. We agree that  the act requires the appropriate precertification authority for each covered  business system to determine that appropriate BPR efforts have been  undertaken. However, we disagree that our recommendation should be  directed to the BPR owners. The recommendation is not intended to be  prescriptive as to who should measure the impact of the BPR efforts.  Rather, it calls for the reporting of the results of such efforts in the  department’s annual report to Congress, which is prepared by the office  of the DCMO under the department’s Chief Management Officer.
The department did not concur with our fourth recommendation to provide  an update on the office of the DCMO’s progress toward filling staff  positions and the impact of any unfilled positions in its annual report to  Congress. DOD stated that it does not believe that the annual report is  the appropriate communication mechanism; however, it offered to provide  us with an update. While we support the department’s willingness to  provide us with an update, we, nonetheless, stand by our  recommendation. The purpose of the annual report is to document the  department’s progress in improving its business operations through  defense business systems modernization. Thus, the potential for staffing  shortfalls in the office of the DCMO to adversely impact the department’s  progress should be communicated to the department’s congressional  stakeholders as part of the report. Including information about the  department’s progress in staffing the office that was recently established  to be responsible for business systems modernization would not only  facilitate congressional oversight, but also promote departmental  accountability.
We are sending copies of this report to the appropriate congressional  committees; the Director, Office of Management and Budget; the  Secretary of Defense; and other interested parties. This report also is  available at no charge on the GAO website at http://www.gao.gov.
If you or your staff members have any questions on matters discussed in  this report, please contact me at (202) 512-6304 or melvinv@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix III.
section_title: Appendix I: Objective, Scope, and Methodology
paragraphs: As agreed with the congressional defense committees, our objective was  to assess the Department of Defense’s (DOD) actions to comply with key  aspects of section 332 of the National Defense Authorization Act (NDAA)  for Fiscal Year 2005 (the act), as amended, 10 U.S.C. § 2222 and  related federal guidance. These include (1) developing a business  enterprise architecture (BEA) and a transition plan for implementing the  architecture, (2) identifying systems information in its annual budget  submission, (3) establishing a system investment approval and  accountability structure along with an investment review process, and (4)  certifying and approving any business system program costing in excess  of $1 million. (See the background section of this report for additional  information on the act’s requirements.) Our methodology relative to each  of the four provisions is as follows:  To address the architecture, we analyzed version 9.0 of the BEA, which  was released on March 15, 2012, relative to the act’s specific  architectural requirements and related guidance that our previous annual  reports in response to the act identified as not being fully implemented. Specifically, we interviewed office of the Deputy Chief Management  Officer (DCMO) officials and reviewed written responses and related  documentation on steps completed, under way, or planned to address  these weaknesses. We then reviewed architectural artifacts in BEA 9.0 to  validate the responses and identify any discrepancies. We also  determined the extent to which BEA 9.0 addressed 10 U.S.C. § 2222, as  amended by the NDAA for Fiscal Year 2012. In addition, we analyzed  documentation and interviewed knowledgeable DOD officials about  efforts to establish a federated business mission area enterprise  architecture. Further, we reviewed the military departments’ responses  regarding actions taken or planned to address our previous  recommendations on the maturity of their respective enterprise  architecture programs. We did not determine whether the DOD  Enterprise Transition Plan addressed the requirements specified in the  act, because an updated plan was not released during the time we were  conducting our audit work.
See, for example, GAO-09-586 and GAO-11-684.
To determine whether DOD’s fiscal year 2013 IT budget submission was  prepared in accordance with the criteria set forth in the act, we reviewed  and analyzed the Report on Defense Business System Modernization  Fiscal Year 2005 National Defense Authorization Act, Section 332, dated  March 2012, and compared it with the specific requirements in the act.  We also compared information contained in the department’s system that  is used to prepare its budget submission (SNAP-IT) with information in  the department’s authoritative business systems inventory (DITPR) to  determine if DOD’s fiscal year 2013 budget request included all business  systems and assessed the extent to which DOD has made progress in  addressing our related recommendation. In addition, we reviewed DOD’s  budget submission to determine the extent to which it addresses 10  U.S.C. § 2222, as amended by the NDAA for Fiscal Year 2012. We also  analyzed selected business system information contained in DITPR, such  as system life cycle start and end dates, to validate the reliability of the  information. We also interviewed officials from the office of DOD’s Chief  Information Officer (CIO) to discuss the accuracy and  comprehensiveness of information contained in the SNAP-IT system, the  discrepancies in the information contained in the DITPR and SNAP-IT  systems, and efforts under way or planned to address these  discrepancies.
To assess the establishment of DOD enterprise and component  investment management structures and processes, we followed up on  related weaknesses that our previous reports in response to the act have  identified as not being fully implemented. Specifically, we interviewed the  office of the DCMO and military department officials and reviewed written  responses and related documentation on steps completed, under way, or  planned to address these weaknesses. We also met with cognizant  officials on steps taken to address new investment management  requirements of the NDAA for Fiscal Year 2012. Further, we reviewed  DOD’s most recent BEA compliance guidance to determine the extent to  which it addressed our related open recommendations. Finally, we  reviewed business process reengineering documentation provided to  support assertions that modernization programs had undergone business  process reengineering assessments.
To determine whether the department was certifying and approving  business system investments with annual obligations exceeding $1  million, we reviewed and analyzed all Defense Business Systems  Management Committee certification approval memoranda. We also  reviewed IRB certification memoranda issued prior to the Defense  Business Systems Management Committee’s final approval decisions for  fiscal year 2011. We contacted officials from the office of the DCMO and  investment review boards to discuss any discrepancies. In addition, we  discussed with officials from the office of the DCMO its plans for updating  the investment review process consistent with requirements of the NDAA  for Fiscal Year 2012 and obtained related documentation. To assess the  office of the DCMO’s progress toward filling staff positions, we compared  the number of authorized positions with the staff on board as of late April  2012; reviewed and analyzed related staffing documentation; and  interviewed office of the DCMO officials about staffing.
We did not independently validate the reliability of the cost and budget  figures provided by DOD because the specific amounts were not relevant  to our findings. We conducted this performance audit at DOD offices in  Arlington and Alexandria, Virginia, from September 2011 to June 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objective.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Neelaxi Lakhmani and Mark  Bird, Assistant Directors; Debra Conner; Rebecca Eyler; Michael Holland;  Anh Le; Donald Sebers; and Jennifer Stavros-Turner made key  contributions to this report.