DOD Financial Management: Ineffective Risk Management Could Impair Progress toward Audit-Ready Financial Statements
Released: 2013-09-03T00:00:00

Letter

Objective, Scope, and Methodology

Given DOD’s difficulties in achieving audit readiness and addressing its long-standing financial management deficiencies, you asked us to assess DOD’s risk management process for implementing its FIAR Plan. Our objective was to determine the extent to which DOD has established an effective process for identifying, analyzing, and addressing risks that could impede its progress in achieving audit readiness.

To address this objective, we identified relevant guiding principles and leading practices of risk management used by the private sector and GAO. Based on our analysis, we found commonalities and identified five basic guiding principles governing effective risk management: (1) identify risks, (2) analyze risks, (3) plan for risk mitigation, (4) implement a risk mitigation plan, and (5) monitor risks and mitigation plans. Using these guiding principles as criteria, we analyzed DOD documents related to risk management, such as the May 2012 and November 2012 FIAR Plan Status Reports, which identified DOD’s program risks and mitigation plans, and FIAR oversight committee meeting minutes, which documented the results of DOD’s efforts to prioritize and manage these risks. We interviewed the FIAR Director and other officials responsible for the FIAR Plan in the Office of the Under Secretary of Defense (Comptroller) and the Office of the Deputy Chief Management Officer (DCMO) to obtain an understanding of DOD’s risk management process.

We also inquired about coordinated risk management efforts and about DOD’s plans to revisit identified risks, identify new risks, and mitigate those risks.

Although the FIAR Directorate is responsible for DOD-wide risk management activities to implement the FIAR Plan, FIAR Directorate officials told us that some of DOD’s component entities may have risk management activities under way. Accordingly, we made inquiries of the military components and two of the largest defense agencies—the Defense Finance and Accounting Service (DFAS) and the Defense Logistics Agency (DLA)—to identify those that had risk management efforts under way for implementing the FIAR Plan. Of these, the Department of the Navy (Navy) and DLA had risk management practices being implemented at the time of our review, and we included them for comparison purposes to the DOD-wide efforts. Using the five risk management guiding principles as criteria, we reviewed and analyzed the Navy’s and DLA’s risk management plans and supporting documents that identified, described, and prioritized risks to audit readiness as well as progress or status reports related to their efforts to address and monitor those risks. We also interviewed the Navy’s and DLA’s Financial Improvement Plan directors and other knowledgeable officials about their risk management processes and coordination with DOD’s FIAR Director and the Office of the DCMO.

We conducted this performance audit from October 2011 to August 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.

Background

In 2005, the DOD Comptroller established the FIAR Directorate, consisting of the FIAR Director and his staff, to develop, manage, and implement a strategic approach for addressing financial management deficiencies, achieving audit readiness, and integrating those efforts with other initiatives. Also in 2005, DOD first issued the FIAR Plan—a strategic plan and management tool for guiding, monitoring, and reporting on the department’s ongoing financial management improvement efforts and for communicating the department’s approach to addressing its financial management weaknesses and achieving financial statement audit readiness.

In August 2009, the DOD Comptroller sought to focus FIAR efforts by giving priority to improving processes and controls that support the financial information most often used to manage the department. Accordingly, the DOD Comptroller revised the FIAR Plan strategy to focus on two priorities—budgetary information and asset accountability. The first priority was to strengthen processes, controls, and systems that produce DOD’s budgetary information. The second priority was to improve the accuracy and reliability of management information pertaining to the department’s mission-critical assets, including military equipment, real property, and general equipment. In May 2010, the DOD Comptroller first issued the FIAR Guidance, which provided the standard methodology for the components—including the Departments of the Army, Navy, and Air Force and DLA—to implement the FIAR Plan. According to DOD, the components’ successful implementation of this methodology is essential to the department’s ability to achieve full financial statement auditability.

In recent years, legislation has reinforced certain DOD financial improvement goals and initiatives and has strengthened the role of DOD’s Chief Management Officer (CMO). For example, the NDAA for Fiscal Year 2010 tasked the CMO, in consultation with the DOD Comptroller, with the responsibility for developing and maintaining the FIAR Plan, and required the plan to describe the specific actions to be taken and the costs associated with validating audit readiness by the end of fiscal year 2017. This act also mandated that the department provide semiannual reports—no later than May 15 and November 15—on the status of its implementation of the FIAR Plan. In October 2011, the Secretary of Defense directed the department to achieve audit readiness for its SBR for general fund activities by the end of fiscal year 2014, and the NDAA for Fiscal Year 2012 required that the next FIAR Plan update include a plan to support this goal. Most recently, the NDAA for Fiscal Year 2013 made the 2014 target for SBR auditability an ongoing component of the FIAR Plan by amending the NDAA for Fiscal Year 2010 such that it now explicitly refers to describing the actions and costs associated with validating as audit ready both DOD’s SBR by the end of fiscal year 2014 and DOD’s complete set of financial statements by the end of fiscal year 2017.

Key Oversight Entities for the FIAR Effort

The department has established a FIAR governance hierarchy to oversee the FIAR Directorate’s management and implementation of the FIAR Plan. At the top is the CMO, who approves the vision, goals, and priorities of the FIAR Plan, which are provided by the DOD Comptroller, in coordination with stakeholders within the department (e.g., military departments) as well as external stakeholders (e.g., the Office of Management and Budget and Congress). The CMO chairs the Deputy Management Action Group, which (1) provides advice and assistance to the CMO on matters pertaining to DOD enterprise management, business transformation, and operations and (2) reviews DOD component FIAR Plans and monitors their progress. To manage and oversee FIAR Plan implementation efforts, a number of committees and working groups, beginning with the FIAR Governance Board, have been established, as shown in table 1. The FIAR Governance Board engages the department’s most senior leaders from the functional and financial communities and oversees DOD component progress. The FIAR Committee and Subcommittee oversee the management of the FIAR Plan. Descriptions of these key FIAR oversight bodies are presented below.

In the November 2012 FIAR Plan Status Report, DOD reported the following:

Fifteen percent of the department’s reported general fund budgetary resources were undergoing audits, including the Marine Corps’ budgetary resources. The military departments, defense agencies, and other components were preparing the remaining budgetary resources to be ready for audit by the end of September 2014.

For mission-critical assets, DOD reported that 4 percent of these assets were undergoing audits, 37 percent had been validated as audit ready, 12 percent had been asserted as audit ready by the respective component, and the remaining 47 percent were being prepared for audit readiness assertions.

DOD’s projected funding for the FIAR effort for fiscal years 2012 through 2018 is shown in table 2.

Risk management is a strategy for helping program managers and stakeholders make decisions about assessing risk, allocating resources, and taking actions under conditions of uncertainty. Risk management can be applied to an entire organization, at its many levels, or to specific functions, projects, and activities. While risk management does not provide absolute assurance regarding the achievement of an organization’s objectives, an effective risk management strategy can be particularly useful in a decentralized organization—such as DOD—to help top management identify potential problems and reasonably allocate resources to address them. Leading risk management practices recommend that organizations develop, implement, and continuously improve a process for managing risk and integrate it into the organization’s overall governance, strategy, policies, planning, management, and reporting processes. When planning for risk, an organization determines the methodology, strategies, scope, and parameters for managing risks to the objective.

In researching risk management principles, we identified five basic guiding principles of risk management, as shown in figure 1.

Identify risks. The goal of risk identification is to generate a comprehensive list of risks, regardless of whether those risks are under the control of the organization, based on events that could significantly affect the achievement of objectives. Risk identification involves continuous and iterative communication and consultation with internal and external stakeholders to identify new risks, sources of risk, areas these risks affect, events (including changes in circumstances), their causes (root causes), and potential consequences to the objective. This can be performed through additional inquiry with subject matter experts, surveying and interviewing experienced executives, high-level and detailed documentation reviews, checklists based on historical information, and diagramming processes.

Analyze risks. Risk analysis involves developing an understanding of identified risks to assist management in determining the most appropriate methods and strategies in prioritizing and responding to risk. It requires risks to be analyzed to determine the impact of interdependencies between the overall program risks and program component risks. According to guiding principles, risk analysis is a vital part of the entire risk management process as it helps managers determine where to focus their attention and allocate resources to maximize the likelihood of achieving objectives. This requires management to consult with key stakeholders, project managers, and experts to discuss, analyze, and rank risks based on their expert analysis. Suggested techniques for risk analysis include the following: o Risk categorization. Risks can be categorized by sources of risk, the area of the program affected, or other useful categories to determine the areas of the program most exposed to the effects of uncertainty. Grouping risks by common root causes can lead to developing effective risk responses. o Risk urgency assessment. Risks requiring near-term responses may be considered more urgent to address. Indicators of priority can include time to affect a risk response, symptoms and warning signs, and the risk rating. o Modeling. This includes techniques that can be used to assess the effect of risk interdependencies (i.e., one risk is dependent on another risk being resolved) with specific attention to life cycle program costs. Examples include (1) sensitivity analysis, which helps to determine which risks have the most potential impact on the program, and (2) financial analysis methods, such as life cycle program costs, return on investment, or cost benefit analysis, which helps to determine the viability, stability, and profitability of a program.

Plan for risk mitigation. Planning for risk mitigation entails selecting the most appropriate and timely action to address risks while balancing the costs and efforts of implementation against the benefits derived. The mitigating actions must also be realistic, achievable, measurable, and documented. Among other things, the plan should include the point of contact responsible for addressing each risk, the root causes of the risk, the options for mitigation, risk status, contingency actions or fallback approach, and resource needs.

Implement risk mitigation plan. Implementing the risk mitigation plan determines what planning, budget, requirements, contractual changes, or a combination of these is needed; provides a coordination vehicle for management and other stakeholders; directs the team to execute the defined and approved risk mitigation plans; outlines the risk reporting requirements for ongoing monitoring; and documents the history of changes.

Monitor risks and mitigation plan implementation. Effective tracking of risk mitigation implementation (risk monitoring) provides information that assists managers with making effective decisions before problems occur by continually monitoring mitigation plans for new and changing risks. Risk monitoring is the process of identifying, analyzing, and planning for new risks; tracking identified risks; and reanalyzing existing risks throughout the life of the program. Monitoring is also intended to help management determine whether program assumptions are still valid and whether proper risk management policies and procedures are being followed.

Risk management is an iterative process and these guiding principles are interdependent such that deficiencies in implementing one guiding principle will cause deficiencies in performing other guiding principles. For example, if the procedures for identifying risks are not comprehensive and not all significant risks are identified, then the other guiding principles for risk management will not be carried out for any risks not identified. Similarly, if identified risks are not sufficiently analyzed, then it is less likely that effective risk mitigation plans will be developed.

DOD Has Not Established an Effective Risk Management Process

DOD carried out some risk management practices centrally with respect to implementing the FIAR Plan, but did not follow many risk management principles necessary for effective risk management and did not document its risk management policies and procedures. Specifically, DOD identified some risks to its FIAR effort, but its risk identification procedures were not comprehensive or documented. In addition, its procedures for analyzing, mitigating, and monitoring risks were also undocumented and did not adhere to guiding principles. We found, however, that two DOD components—the Navy and DLA—had documented risk management processes that were consistent with many of the guiding principles for effective risk management.

Identification of Risks to FIAR Implementation Was Incomplete

Although DOD has identified several risks that could hinder its efforts to achieve financial statement auditability, it did not identify or address additional key risks that were reported in external audit reports. In January 2012, DOD identified six risks that if not mitigated, could impede its efforts to achieve auditability. The department included these risks in its May 2012 semiannual FIAR Plan Status report. The following is DOD’s summary of the six risks it identified. 1. A lack of DOD-wide commitment. Stakeholders must be committed to improving controls and providing supporting documentation. 2. Insufficient accountability. Leaders and managers must be incentivized to achieve FIAR goals. 3. Poorly defined scope and requirements. Financial improvement plans should address accounting requirements important to audit success. 4. Unqualified or inexperienced personnel. DOD must ensure that personnel are capable of making and supporting judgments that auditors will agree meet accounting standards. 5. Insufficient funding. Resources must be aligned to the scope and scale of the FIAR effort. 6. Information system control weaknesses. Many processes and controls reside entirely in software applications, and therefore these systems and interfaces must support complete and accurate records.

DOD did not have written policies and procedures or a documented process for identifying these risks; however, DOD officials told us that they held internal management meetings, brainstormed with internal and external stakeholders, and reviewed prior GAO and DOD Inspector General (IG) reports. While DOD’s identification of risks was a positive step, DOD did not identify sufficient information about these risks, such as the source, root cause, the audit area(s) the risk will impact, and potential consequences to the program if the risk is not effectively mitigated—all critical to properly analyze and prioritize risk. Further, DOD’s risk identification process did not identify all significant risks to achieving its auditability goals.

DOD officials told us that risk management practices were embedded throughout the FIAR process and that these six risks were identified in whole or in part through this process. Specifically, they said that monthly and quarterly meetings of the various FIAR oversight committees included ongoing discussions with DOD components regarding their progress in meeting FIAR goals and milestones. According to guiding principles, agencies should generate a comprehensive list of risks based on those events that might create, enhance, prevent, degrade, accelerate, or delay the achievement of objectives. In addition, guiding principles state that risk identification is an iterative process where program stakeholders continually forecast the outcomes of current strategies, plans, and activities and exercise their best judgment to identify new risks as the program progresses throughout its life cycle. Although DOD indicated that risks are discussed on an ongoing basis during various meetings, the risks it initially identified were not comprehensive, and it did not provide evidence of efforts to identify additional risks.

We identified additional risks based on prior audit work. For example, DOD did not identify risks related to (1) the components’ reliance on service providers for significant aspects of their financial operations, such as processing and recording financial transactions, and (2) the lack of a department-wide effort to follow documentation retention standards to ensure that required audit support can be provided to auditors. We did not attempt to identify all significant risks to DOD’s audit readiness effort, but these two examples indicate that DOD did not identify all significant risks to the FIAR effort. Conducting a risk identification process in accordance with guiding principles would have increased the likelihood of DOD identifying additional risks that could impede the department’s ability to achieve its auditability goals. As noted previously, the guiding principles are interdependent, and deficiencies in the identification of risks will hinder implementation of other guiding principles, such as risk mitigation.

Reliance on service providers: The Marine Corps received a disclaimer of opinion on its fiscal years 2010 and 2011 SBRs because of its inability to provide timely and complete responses to audit documentation requests. Specifically, the DOD IG reported that DFAS—the service provider responsible for performing accounting, disbursing, and financial reporting services for the Marine Corps—did not have effective procedures in place to ensure that supporting documentation for transactions was complete and readily available to support basic audit transaction testing. In December 2011, we reported that the Navy and Marine Corps could not reconcile their Fund Balance with Treasury accounts in large part because they depend on DFAS to maintain the data necessary for the reconciliation, and DFAS did not maintain reliable data or the documentation necessary to complete the reconciliation.

DOD officials stated that although they did not identify the reliance on service providers as a risk, they recognized it as a challenge and, as a result, developed requirements in the FIAR Guidance. The FIAR Guidance requires the service providers to have their control activities and supporting documentation examined by the DOD IG or an independent auditor in accordance with Statement on Standards for Attestation Engagements (SSAE) 16 so that reporting entities (components) have a basis for relying on the service provider’s data for their financial statement audits. To prepare for an SSAE 16 examination, the FIAR Guidance requires a service provider first to evaluate its control activities and supporting documentation, take corrective actions as necessary, and then assert audit readiness to the FIAR Directorate. Once the FIAR Directorate validates that the service provider has sufficient controls and supporting documentation, the service provider can then engage an auditor to conduct an SSAE 16 audit examination.

The FIAR Guidance states that service providers should identify the reporting components’ audit readiness assertion dates so that they can complete SSAE 16 examinations in time to meet the components’ needs. However, the November 2012 FIAR Plan Status Report indicates that key service providers will not have SSAE 16 examinations completed until sometime in fiscal year 2014. DOD components need to rely on the results of SSAE 16 examinations of key service providers so that the components can effectively assess their own controls in accordance with the FIAR Guidance. In light of the expected completion dates of SSAE 16 examinations, it is not clear if components will have sufficient time to carry out the activities necessary to test and validate their own controls and assert audit readiness for their SBRs by September 2014.

For these reasons, the requirements in the FIAR Guidance have not fully mitigated the risk associated with the reliance on DOD’s service providers. Although DOD recognized this issue as a challenge, the reliance on service providers was not identified by DOD management as a significant risk to DOD achieving audit readiness. If DOD formally identified the reliance on service providers as a risk, it is more likely to manage and monitor this risk in accordance with risk management guiding principles.

Need for supporting documentation: Document retention and the ability to provide supporting documentation for transactions have been pervasive problems throughout DOD. For example, during the Marine Corps audits, the DOD IG found that DFAS had only retained selected pages of the documents supporting payment vouchers, such as the voucher cover sheet, and did not have critical items, such as the purchase order, receiving report, and invoice, to support that payments were made as required. In addition, we reported in March 2012 that the Army did not have an efficient or effective process or system for providing supporting documentation for its military payroll expenses and, as a result, was unable to locate or provide supporting personnel documents for our statistical sample of fiscal year 2010 Army military pay accounts.

DOD officials told us that they recognized document retention as a challenge, and that this issue was addressed in the FIAR Guidance as well as in DOD’s Financial Management Regulation (FMR) and requirements established by the National Archives and Records Administration (NARA). Both the FIAR Guidance and the FMR refer to NARA for guidance on record retention, and the FIAR Guidance also refers to Standards for Internal Control in the Federal Government.However, neither the FIAR Guidance nor the FMR was specific enough to ensure that the documents needed to support audit readiness were retained and available in a timely manner. For example, the FIAR Guidance and the FMR did not address which types of documentation to retain and the required time frames for retaining these documents, thus leaving these decisions to the judgment of DOD component personnel responsible for preparing for audit readiness. DOD officials informed us that they were in the process of updating the FMR to address documentation types and retention periods; however, the updated guidance was not yet available at the time of our review. As a result, we could not determine how and to what extent a revised FMR would address document retention issues.

Continuous and comprehensive risk identification is critical because, if a risk is not formally identified, it is less likely to be managed effectively and in accordance with risk management guiding principles. The first step to managing and mitigating risks is to identify them. For example, if DOD had identified the reliance on service providers and the need for document retention standards as risks, it might have implemented actions to address these risks sooner so that they would not have been major impediments to Navy, Marine Corps, and Army audit readiness efforts. If risks to the FIAR effort are not comprehensively identified, DOD is less likely to take the actions necessary to mitigate or minimize the risks and therefore less likely to meet its audit readiness goals.

Both the Navy and DLA employed techniques that are consistent with guiding principles for risk identification. For example, they collaborated with stakeholders, experts, support personnel, and project managers on a weekly or monthly basis to discuss potential new risks to the audit effort using techniques such as brainstorming, interviewing key stakeholders, diagramming, and SWOT (strengths, weaknesses, opportunities, and threats) analysis, and documented the results in risk registers or risk databases. Both the Navy’s and DLA’s identified risks included the reliance on service providers and the need for better document retention.

Risk Analysis of FIAR Implementation Was Incomplete

DOD did not follow guiding principles for performing risk analysis. The FIAR Director plotted the six risks DOD identified on graphs that were intended to show the likelihood of the risks occurring (or probability) and the effect (or impact) on the overall implementation of the FIAR Plan (see fig. 2).

The FIAR Director said that he did not consult with key stakeholders, project managers, and experts to analyze these risks as suggested by guiding principles. He also stated that he did not use recommended analytical techniques, such as (1) risk categorization, (2) risk urgency assessment, or (3) sensitivity analysis. In addition, the FIAR Director did not perform an assessment to determine the individual DOD components’ ability to achieve audit readiness. For example, if one DOD component has significantly more information technology system control weaknesses or fewer skilled personnel than another DOD component, it is likely to have a higher risk of not achieving audit readiness. Performing effective risk analysis could enable DOD to develop appropriate risk mitigation plans to address such concerns, including resource allocation among the components.

A probability and impact matrix is generally used for both communication and prioritization. Guiding principles state that risk analysis is a vital part of the risk management process because it helps management determine the most appropriate methods and strategies for mitigating risks. In addition, it allows management to better allocate resources to maximize the likelihood of achieving objectives. By not analyzing risks in accordance with guiding principles, DOD increased the likelihood that it would not adequately address the most critical risks in a timely manner.

Navy and DLA officials generally followed guiding principles for risk analysis. For example, at both the Navy and DLA, project management teams worked together to determine who was primarily responsible for managing each identified risk. The Navy and DLA employed analytical techniques to assess risk and documented the results of their analyses— such as the impact each risk has or could have on the objectives and the risk’s priority—in risk registers. In addition, both the Navy and DLA documented their risk analysis processes to allow for consistent implementation. As a result of these analyses, Navy identified the following as its three highest risks to audit readiness efforts: (1) reliance on service providers, (2) internal resources in information technology operations, and (3) tracking unmatched disbursements, while DLA identified (1) data access limitations, (2) standard accounting and financial management functions, and (3) audit response capabilities as its three highest risks. The Navy and DLA each considered its respective risks to have a high impact on audit readiness and a high probability of occurrence.

Planning for Risk Mitigation Actions Was Not Detailed or Sufficient

The DOD FIAR Directorate developed risk mitigation plans first published in the May 2012 FIAR Plan Status Report. However, DOD did not have documented policies and detailed procedures for planning risk mitigation actions. As a result, its plans did not have most of the elements recommended by guiding principles. For example, the plans did not include (1) assignment of responsibility or ownership of the risk mitigation actions, (2) information about DOD’s or the components’ roles and responsibilities in executing these plans, (3) deadlines or milestones for individual mitigation actions, and (4) resource needs.

The lack of details makes it difficult to determine whether the planned risk mitigation actions are sufficient to address the risks. For example, the risk mitigation plan for addressing the risk of unqualified or inexperienced personnel did not provide sufficient information as recommended by guiding principles. According to the plan, DOD intends to hire experienced individuals who are certified public accountants hire independent public accounting firms to help the department provide FIAR training to the appropriate functional and financial modify existing military department training and education programs to include FIAR objectives, and conduct limited-scope audits of portions of the financial statements to provide firsthand experience in preparation for future financial statement audits.

However, the mitigation plan did not provide further details, such as the following:

DOD’s actions to comply with the mandate, included in the NDAA for Fiscal Year 2010, to prepare a strategic workforce plan and conduct a gap analysis for mission-critical skills in its civilian workforce, including those in its financial management community. As we recently reported,gap analyses for financial management.

DOD has not completed any of its competency

How many CPAs DOD plans to hire, in what capacity these CPAs will be utilized, what components will be involved, and at what cost.

The relevant criteria for determining which employees should attend new FIAR training, whether training is mandatory, and how many employees are affected.

How DOD’s financial management certification program would coincide with the current mandatory training.

How or which existing training and education programs would be modified, the time frames for doing so, the intent of the modifications (i.e., how this training would differ from FIAR training), and which employees will be attending these classes.

DOD FIAR officials stated that their mitigation plans were straightforward and did not require additional detail for implementation purposes. However, as discussed earlier, guiding principles state that effective planning ensures that the activities to be performed to achieve the objectives are realistic, known, and understood by those who are responsible for performing them, including the milestones and available resources. Without sufficiently detailed plans for risk mitigation, achieving the program’s overall objectives—financial management improvements and auditability—is at increased risk of failure.

The Navy and DLA included risk mitigation plans for each of their identified risks in their risk registers. The plans documented the mitigation strategy, assignment of responsibility or ownership of the risk mitigation actions, status updates, and the potential impact of the risk on the objectives.

Information on Implementation of Risk Mitigation Actions Was Limited

The DOD FIAR Directorate did not maintain documentation of specific mitigation actions taken or who performed them. Specifically, evidence of risk mitigation actions provided by the FIAR Directorate consisted of metrics reported each month and each quarter to the key oversight entities, such as the FIAR Governance Board and FIAR Committee. According to FIAR Directorate officials, they compiled these metrics— related to such matters as the total attendance at FIAR training classes and the number of information technology systems assessed—based largely on information self-reported by the components. The FIAR Directorate did not independently validate this information for reliability as suggested by guiding principles.

We found that the reported metrics did not provide a complete picture of the status of the department’s efforts to implement its risk mitigation action plan. Specifically, the metrics did not provide the details needed to determine what actions had been taken, their status and impact, who performed the work, the resources used, the remaining resource needs, and the actions still to be taken. The FIAR Director did not provide an explanation for how these particular metrics were selected for reporting or why more information about mitigation actions was not reported. DOD did not have policies and procedures requiring DOD to (1) document the implementation of mitigation actions, (2) develop appropriate metrics, and (3) validate reported metrics. If DOD does not effectively measure its progress in the implementation of risk mitigation plans, it cannot sufficiently manage risk mitigation actions and monitor the extent to which they are or are not succeeding. Without such information, DOD is limited in its ability to make informed decisions about ongoing mitigation efforts, adjust course as necessary, and identify and mitigate any new risks. This, in turn, could adversely affect DOD’s ability to meet the mandated deadlines of an audit-ready SBR by fiscal year 2014 and audit-ready consolidated financial statements by fiscal year 2017.

The following are examples of two of DOD’s identified risks wherein the reported risk management metrics did not adequately measure DOD’s progress in implementing its risk mitigation plans.

Unqualified or inexperienced personnel. To address this risk, the November 2012 FIAR Plan Status Report stated that DOD is hiring experienced individuals who are CPAs, modifying existing training programs, and providing FIAR training to employees. DOD reported one metric for this risk, which relates to attendance at FIAR training classes. However, DOD’s metrics did not address the number of CPAs hired or to be hired, who is responsible for the hiring, or the progress to date in hiring experienced personnel. Moreover, the reported metric related to FIAR training classes did not provide key information for assessing progress. As of January 2013, DOD components reported that approximately 7,000 of their financial management personnel had attended FIAR training classes. However, DOD acknowledged that this metric likely included some individuals who were counted multiple times. For example, an individual who attended each of the six FIAR training courses would be counted six times. As a result, it was unclear how many staff members had taken the training courses. In addition, the metrics did not identify the total number of DOD’s approximately 58,000 financial management personnel who are required or expected to take these training courses. As a result, DOD’s “Total Attendance at FIAR Training” metric did not provide a meaningful measure of progress against the identified risk of unqualified and inexperienced personnel.

Information system control weaknesses. DOD engaged the DCMO to oversee development and implementation for enterprise resource planning (ERP) business system modernization and has required ERP deployment plans to be integrated with components’ financial improvement plans to mitigate risks of information system control weaknesses. However, DOD’s metric for information systems control weaknesses focuses on the number of information technology systems that have been assessed against the Federal Information System Controls Audit Manual (FISCAM) requirements. As of January 2013, DOD components reported that only 18 of 140 information technology systems had been assessed against FISCAM requirements. This metric does not provide needed details, such as the number of systems assessed that were found to be noncompliant with FISCAM requirements, the number of system change requests identified or completed, and the status of corrective actions. Moreover, the 140 systems identified in DOD’s metric may not constitute the total universe of relevant financial management systems; as we recently reported, DOD had identified 310 financial management systems.

GAO, DOD Financial Management: Reported Status of Department of Defense’s Enterprise Resource Planning Systems, GAO-12-565R (Washington, D.C.: Mar. 30, 2012).

ERP systems incurring additional cost increases and schedule delays that could affect its ability to achieve an auditable SBR by 2014 and a complete set of auditable financial statements by 2017. As noted previously, if DOD had been more specific in its identification of risks related to its information systems, it would have been in a better position to analyze these risks and develop effective mitigation plans to address them.

Navy and DLA officials document their risk implementation efforts by including status updates on a weekly or monthly basis for each risk in their risk registers. The Navy’s risk register had detailed status updates for each risk that included the current status of mitigation efforts and any updates or additional comments that need to be addressed. DLA’s risk register indicated whether risk mitigation efforts were under way (active) for each risk. DLA also identified events (triggers) for each risk that provided an alert as to when a certain risk was close to being realized or imminent, which could then initiate the next course of action.

Monitoring of Mitigation Actions to Address FIAR Implementation Risk Was Not Sufficient

DOD officials, including those in the FIAR Directorate and key FIAR oversight entities such as the FIAR Governance Board and the FIAR Committee, were monitoring risk mitigation efforts using the metrics previously discussed. However, these metrics do not provide the information that managers need to (1) track identified risks and assess the effectiveness of implemented mitigation actions, (2) make effective decisions, and (3) identify and plan for new risks. Further, our review of oversight committee meeting minutes did not find evidence that the metrics were discussed in any greater detail or that decisions were made based on these metrics.

If DOD is not effectively monitoring risks, it may be unaware of deficiencies in risk mitigation action plans or implementation that may weaken the effectiveness of its risk mitigation. Guiding principles state that risk monitoring reduces the impact of risk by identifying, analyzing, reporting, and managing risks on a continuous basis for the life of the program. Moreover, if DOD management does not follow guiding principles for monitoring risks to the FIAR effort, it lacks assurance that the department is doing all it can to ensure the success of its audit readiness efforts. Also at risk are the substantial resources that DOD estimates it will need to become audit ready. Based on the data in table 2, DOD’s reported audit readiness resources will average approximately $515 million annually over 7 years. Without the awareness gained through effective monitoring, DOD will not have the information it needs to proactively respond to new risks or adjust its plans based on lessons learned in a manner that can benefit the entire department. For example, as we have previously reported, the Marine Corps’ unsuccessful attempts to have its SBR audited for fiscal years 2010 and 2011 have resulted in lessons learned that may be helpful to other components in preparing for audit readiness.

Navy and DLA officials told us that they monitor their risk management efforts during their weekly and monthly meetings that include risk owners and their internal financial management oversight teams. Those meetings are used to discuss new risks, update risk registers, and provide status updates and feedback to component managers about the status of audit readiness efforts.

Conclusions

In light of current budget constraints and fiscal pressures throughout the federal government and particularly at DOD, it is more important than ever for DOD to have reliable information with which to manage its resources effectively and efficiently. This necessity and DOD’s estimated costs for the FIAR effort make the successful implementation of its FIAR Plan even more imperative. DOD has taken some actions to manage its department-level risks associated with preparing auditable financial statements through its FIAR Plan. However, DOD had not followed most risk management guiding principles, and had not designed and implemented written policies and procedures to fully identify and manage risks affecting implementation of the FIAR Plan. DOD identified some risks to the FIAR effort, but its risk identification process was not comprehensive. Moreover, DOD did not sufficiently analyze the risks, plan and implement mitigation actions, and monitor the results.

To improve management of the risks to the FIAR effort throughout the department, the risk management processes established by two DOD components—the Navy and DLA—could serve as a starting point.

Ineffective management of the risks to successful implementation of the FIAR Plan increases the likelihood that DOD will not achieve its audit readiness goals.

Recommendations for Executive Action

We recommend that the Secretary of Defense direct the Under Secretary of Defense, in his capacity as the Chief Management Officer and in consultation with the Under Secretary of Defense (Comptroller), to take the following two actions:

Design and implement department-level policies and detailed procedures for FIAR Plan risk management that incorporate the five guiding principles for effective risk management. The following are examples of key features of each of the guiding principles that DOD should, at a minimum, address in its policies and procedures.

Identify risks. Generate a comprehensive and continuously updated list of risks that includes the root cause of each risk, audit area(s) each risk will affect, and the potential consequences if a risk is not effectively mitigated. o Analyze risks. Consult with key stakeholders, including program managers; use analytical techniques, such as risk categorization, risk urgency assessment, or sensitivity analysis; and determine the impact of the identified risks on individual DOD components’ abilities to achieve audit readiness. o Plan for risk mitigation. Assign responsibility or ownership of the risk mitigation actions, define roles and responsibilities in executing mitigation plans, establish deadlines or milestones for individual mitigation actions, and estimate resource needs.

Implement risk mitigation plan. Document the implementation of mitigation actions, develop appropriate metrics that allow for tracking of progress, and validate reported metrics. o Monitor risks. Track identified risks and assess the effectiveness of implemented mitigation actions on a continuous basis, including identifying and planning for new risks.

Consider and incorporate, as appropriate, the Navy’s and DLA’s risk management practices in department-level policies and procedures.

Agency Comments and Our Evaluation

DOD officials provided written comments on a draft of this report, which are reprinted in appendix I. DOD acknowledged that it does not have a written risk management policy specifically related to the FIAR effort, but did not concur with our assessment of the department’s overall risk management of the FIAR initiative. However, DOD cited planned actions that are consistent with our recommendations and findings, including (1) improving the documentation related to FIAR risk management activities, (2) reinforcing the importance of more detailed risk management activity within each DOD component executing its detailed FIAR Plan, (3) reinstating the DOD probability and impact matrix for risk analysis for the FIAR initiative, and (4) reevaluating all metrics used to monitor progress and risk for audit readiness and developing new measures as appropriate. DOD’s planned actions, if implemented effectively and efficiently, would help address some aspects of the five guiding principles of risk management that are the basis for our recommendations. While these are good first steps, we continue to believe additional action is warranted. Consequently, we reaffirm our recommendations.

DOD stated that its risk management processes and activities were embedded into the design of the FIAR initiative. DOD also stated that all common risk management activities were occurring, including identification, evaluation, remediation, and monitoring of enterprise-wide risks for the FIAR initiative, and these activities were effectively managing risk. As stated in our report, while DOD does have some aspects of risk management activities under way in each of these areas, these activities do not go far enough in addressing most risk management guiding principles, nor has DOD designed and implemented written policies and procedures to fully identify and manage risks affecting implementation of the FIAR Plan. For example, although DOD identified six enterprise-wide risks through its risk identification process, DOD did not provide any evidence that the six identified risks were reevaluated on a continuous basis or that new risks were identified or discussed. Additionally, DOD did not identify sufficient details about these risks, such as the root cause, areas the risks will affect, and consequences to the program if a risk is not effectively mitigated nor did it develop a comprehensive list of risks.

As noted in our report, we identified at least two additional risks that could impede DOD’s ability to achieve audit readiness—reliance on service providers and lack of documentation standards. DOD’s response noted that it did not label these as risks but as challenges and had actions under way to address them. While we commend DOD for taking some actions to address these two issues, by not adding them to the formal list of risks during the risk identification process, they may not undergo the same level of risk analysis, mitigation, and monitoring as the six formally identified risks.

In addition, DOD did not agree with our finding that its planned mitigation actions lacked details and made it difficult to determine whether the planned actions were sufficient to address the risk. For example, we reported that DOD’s mitigation plans did not address specific details related to its mitigating actions for the risk of unqualified or inexperienced personnel, such as the number of CPAs or experienced personnel the department planned to hire, relevant criteria to determine which personnel would attend FIAR training, timing of the DOD financial management certification program, and how existing training and education programs will be modified. In response to our report, DOD provided additional details related to its mitigating actions to address this risk, which were not previously provided to us or reported in the FIAR Plan status updates, including time frames for implementing some of its mitigating actions. However, DOD’s additional details still do not address the findings in our report or issues related to the timing for implementing planned mitigation actions, as many actions are to be implemented beginning in fiscal years 2013 and 2014. This raises concerns about whether DOD can effectively manage and mitigate risks in time to meet its audit readiness goals, beginning with achieving an audit-ready Statement of Budgetary Resources by September 30, 2014, as mandated. Given these concerns, we continue to believe that DOD could improve its risk management processes by designing and implementing department-level policies and detailed procedures that reflect the five guiding principles of effective risk management, as we recommended.

DOD also provided one general comment, suggesting that we delete reference to our prior reports on our reviews of the Navy’s Civilian Pay and Air Force’s Military Equipment audit readiness efforts, in which we identified significant deficiencies in the components’ execution of the FIAR Guidance. Although DOD provided an update of progress made since we issued those reports, we have not reviewed those results, and in any case, we included these examples to demonstrate the difficulties encountered by the components in successfully executing the FIAR Guidance effectively and consistently. The examples also show that the components’ initial attempts to assert audit readiness may not be successful and that additional time and mitigating actions may be needed to address components’ deficiencies in implementing the FIAR Guidance.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, Secretary of Defense, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-9869 or khana@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff members who made key contributions to this report are listed in appendix II.

Appendix I: Comments from the Department of Defense

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Cindy Brown Barnes (Assistant Director), Francine DelVecchio, Kristi Karls, and Carroll M. (CJ) Warfield, Auditor-in-Charge, made key contributions to this report. Also contributing to this report were Cynthia Jackson, Maxine Hattery, and Jason Kirwan.