DOD Financial Management: Improvement Needed in DOD Components' Implementation of Audit Readiness Effort
Released: 2011-09-15T00:00:00

Letter

Background

For over a decade, DOD has dominated GAO’s list of federal programs and operations at high risk of being vulnerable to fraud, waste, abuse, and mismanagement. All of DOD’s programs on GAO’s High-Risk List relate to its business operations, including systems and processes related to the management of contracts, finances, the supply chain, and support infrastructure, as well as weapons system acquisition. Long-standing and pervasive weaknesses in DOD’s financial management and related business processes and systems have (1) resulted in a lack of reliable information needed to make sound decisions and report on the financial status and cost of DOD activities to Congress and DOD decision makers, (2) adversely affected its operational efficiency in business areas such as major weapons system acquisition and support and logistics, and (3) left the department vulnerable to fraud, waste, and abuse.

DOD is required by various statutes to improve its financial management processes, controls, and systems to ensure that complete, reliable, consistent, and timely information is prepared and responsive to the financial information needs of agency management and oversight bodies, and to produce annual audited financial statements. As noted above, DOD has been required since 1997 to prepare and issue annual departmentwide audited financial statements and, pursuant to various statutes, certain DOD components, including the military departments, are required to prepare and issue annual audited financial statements. DOD consolidates the component financial statements to prepare its departmentwide financial statements. As reflected in OMB requirements and DOD policy for preparation and audit of agency financial statements, DOD and its components must prepare their financial statements consistent with generally accepted accounting principles (GAAP) and establish and maintain effective internal control over financial reporting and compliance with laws and regulations. The DOD Office of Inspector General (OIG) is responsible for auditing the financial statements in accordance with GAGAS, which, among other things, requires the auditor to determine if the financial statements are fairly presented and properly supported by the agencies’ accounting records.

DOD has undertaken a number of initiatives over the years, such as its Financial Improvement Initiative in 2003, to improve the department’s business operations, including financial management, and achieve unqualified (or “clean”) financial statement audit opinions. In 2005, the DOD Comptroller established the DOD FIAR Directorate to manage DOD-wide financial management improvement efforts and to integrate those efforts with transformation activities, such as those outlined in the department’s Enterprise Transition Plan, across the department.

The FIAR Plan, which was first prepared in 2005, is DOD’s strategic plan and management tool for guiding, monitoring, and reporting on the department’s financial management improvement efforts. As such, the plan communicates incremental progress in addressing the department’s financial management weaknesses and achieving financial statement auditability. The plan focuses on three goals: (1) achieve and sustain unqualified assurance on the effectiveness of internal controls through the implementation of sustained improvements in business processes and controls addressing the material internal control weaknesses, (2) develop and implement financial management systems that support effective financial management, and (3) achieve and sustain financial statement audit readiness. The NDAA for Fiscal Year 2010 requires DOD to update and report on the FIAR Plan twice a year—no later than May 15 and November 15—and provide each update to OMB and Congress, among others.

Consistent with prior GAO recommendations and the NDAA for Fiscal Year 2010, the DOD Comptroller issued the FIAR Guidance in May 2010 to provide standardized guidance for DOD components to follow in developing their FIPs. DOD components are expected to prepare a FIP in accordance with the FIAR Guidance for each of their assessable units. The FIPs are intended to both guide and document financial improvement efforts. While the name “FIP” indicates that it is a plan, as a component implements that plan, it must document the steps performed and the results of those steps, and retain that documentation within the FIP. Therefore, a FIP includes plans for testing controls and data, and documentation of the testing conducted, results of the testing, and any actions taken based on the results. When a component determines that it has completed sufficient financial improvement efforts for an assessable unit so that it is ready for audit, the FIP documentation is used to support the conclusion of audit readiness.

A summary of the purpose of each DOD financial improvement document is provided below.

The department intends to progress toward achieving financial statement auditability in five “waves” of concerted improvement activities within groups of end-to-end business processes, which are broken down into discrete units, called assessable units. Under the FIAR Plan Strategy, execution of the FIAR Guidance methodology for groups of assessable units across the five waves is intended to result in the preparation of various components’ financial statements through fiscal year 2017.

The first three waves of the FIAR Plan Strategy focus on achieving the DOD Comptroller’s interim budgetary and asset accountability priorities, while the remaining two waves are intended to complete actions needed to achieve full financial statement auditability. However, the department has not yet fully defined its strategy for completing waves 4 and 5, which focus on the auditability of most of the department’s financial statements such as the Balance Sheet and the Statement of Net Cost, and significant audit areas such as equipment valuation. The FIAR Guidance states that an analysis of significant reporting entities relevant to waves 4 and 5 will be included in a future version of the FIAR Guidance. For example, in relation to wave 4, the DOD Comptroller has identified weaknesses in DOD valuation of military equipment as an obstacle to achieving auditable balances in DOD financial statements, and the May 2011 FIAR Status Plan Report states that DOD plans to resolve the matter through the deployment of new business and financial management systems, a revised definition of military equipment, and through a change to GAAP related to accounting for the cost of military equipment. DOD plans to seek a change to GAAP and implement the change by September 30, 2017, but DOD has not yet prepared a detailed time line for implementing the changes within affected assessable units (assuming that any proposed changes to GAAP would be approved).

The focus and scope of each wave include the following: Wave 1—Appropriations Received Audit focuses efforts on assessing and strengthening, as necessary, internal controls and business systems involved in the appropriations receipt and distribution process, including funding appropriated by Congress for the current fiscal year and related apportionment/reapportionment activity by OMB, as well as allotment and suballotment activity within the department.

Wave 2—Statement of Budgetary Resources (SBR) Audit focuses efforts on assessing and strengthening, as necessary, the internal controls, processes, and business systems supporting the budgetary- related data (e.g., status of funds received, obligated, and expended) used for management decision making and reporting, including the SBR. In addition to fund balance with Treasury reporting and reconciliation, significant end-to-end business processes in this wave include procure- to-pay, hire-to-retire, order-to-cash, and budget-to-report.

Wave 3—Mission-Critical Assets Existence and Completeness Audit focuses efforts on assessing and strengthening, as necessary, internal controls and business systems involved in ensuring that all assets (including military equipment, general equipment, real property, inventory, and operating materials and supplies) are recorded in the department’s accountable property systems of record, all of the reporting entities’ assets are recorded in those systems of record, reporting entities have the right (ownership) to report these assets, and the assets are consistently categorized, summarized, and reported.

Wave 4—Full Audit Except for Legacy Asset Valuation focuses efforts on assessing and strengthening, as necessary, internal controls, processes, and business systems involved in the proprietary side of budgetary transactions covered by the Statement of Budgetary Resources effort of wave 2, including accounts receivable, revenue, accounts payable, expenses, environmental liabilities, and other liabilities. This wave also includes efforts to support valuation and reporting of new asset acquisitions.

Wave 5—Full Financial Statement Audit focuses efforts on assessing and strengthening, as necessary, processes, internal controls, and business systems involved in supporting the valuations reported for legacy assets once efforts to ensure controls over the valuation of new assets acquired and the existence and completeness of all mission assets are deemed effective on a go-forward basis. Given the lack of documentation to support the values of the department’s legacy assets, federal accounting standards allow for the use of alternative methods to provide reasonable estimates for the cost of these assets.

To guide the components in executing the strategy, the FIAR Guidance provides a set of mandatory, standardized phases and steps that the components must follow to develop and implement their FIPs to achieve audit readiness. This step-by-step methodology delineates FIP responsibilities between the components’ management and the auditors. For each assessable unit, management’s responsibilities focus on identifying, implementing, and documenting necessary financial management improvements during the first four phases of the FIAR Methodology, and sustaining those improvements through the fifth phase. For phases six and seven, after the DOD Comptroller’s initial review and approval of a FIP supporting audit readiness, the component engages an independent auditor to first perform an examination of the FIP and if validated, then an audit of the assessable unit and finally, the entity’s financial statements.

After a component concludes that an assessable unit is ready for audit, a component continues to maintain the FIP to document the validation of audit readiness by an independent auditor and the sustainability of audit readiness through ongoing efforts by the component. Throughout this report, however, we use the term “FIP” to mean the record of FIP implementation and related documentation that was used to support the conclusion of audit readiness because that is the point at which we reviewed the two selected FIPs. In instances where additional documentation was provided to us later, we indicate that such information was provided subsequent to the audit readiness conclusion.

A description of each of the phases of the FIAR Methodology as set forth in the FIAR Guidance follows: Phase 1 – Evaluation and Discovery: Management documents its business and financial environment, defines and prioritizes its processes into assessable units, assesses risk and tests controls, evaluates supporting documentation, identifies weaknesses and deficiencies, and defines its audit readiness environment.

Phase 2 – Corrective Action: Management develops and executes corrective action plans that identify the specific steps a reporting entity will take to resolve deficiencies, the resources required and committed, and targeted milestones and completion dates.

Phase 3 – Evaluation: Management evaluates its corrective action effectiveness through testing and determines whether the entity is ready for audit.

Phase 4 – Assertion: Management prepares all relevant supporting documentation and asserts audit readiness to the Office of the Under Secretary of Defense (Comptroller) (OUSD(C)) and the DOD OIG.

Phase 5 – Sustainment: Management maintains audit readiness through risk-based periodic testing of internal controls utilizing the OMB Circular No. A-123, Appendix A processes and procedures, and resolves any identified weaknesses in a timely manner.

Phase 6 – Validation: The DOD Comptroller and DOD OIG conduct an initial review of the FIP including management’s assertion. If the DOD Comptroller determines that management’s assertion is supported by the FIP, then an independent auditor performs an examination for the audit readiness assertion.

Phase 7 – Audit: The component (reporting entity) engages an independent auditor and supports the audit of the assessable unit or the financial statements.

Each wave of the FIAR Plan Strategy is comprised of numerous assessable units identified by the components, and each assessable unit must go through the seven phases of the FIAR Methodology. As shown in table 2, the DOD military components reported 73 assessable units for waves 1-3 in the May 2011 FIAR Plan Status Report. No assessable units had yet been reported for waves 4 and 5. While our focus was on the military components, the FIAR Plan Status Report also provided general information on the status of assessable units for other DOD components for waves 1-3.

FIAR Guidance Provides a Reasonable Methodology

The May 2010 FIAR Guidance provides a reasonable methodology for the DOD components to use to develop and implement their FIPs. It details the roles and responsibilities of the DOD components, and prescribes a standard, systematic process components should follow to assess processes, controls, and systems, and identify and correct weaknesses in order to achieve auditability. The FIAR Guidance requires components to fully document the procedures carried out as they implement the FIPs, and the results, which will allow for an independent assessment of audit readiness. Overall, the procedures required by the FIAR Guidance are consistent with selected procedures for conducting financial statement audits, such as reconciling the population of transactions to be tested, conducting tests of information systems controls, and conducting internal control and substantive testing. The FIAR Guidance also requires the components to correct the deficiencies identified during testing and document the results, which is consistent with federal internal control standards and OMB guidance. While the audit strategy for waves 4 and 5 has not been completely defined yet, the same overall FIAR Methodology will likely apply to these waves as well. DOD’s ability to achieve audit readiness is highly dependent on the components’ ability to effectively develop and implement FIPs in compliance with the FIAR Guidance.

The DOD Comptroller has identified critical elements of financial reporting which DOD components are expected to carry out during phases 1 through 3 of the FIAR Methodology and which closely align with procedures that are performed during an audit. Following are more details about some of the critical elements required by the FIAR Guidance.

Internal control and substantive testing. DOD components are required to perform both internal control and substantive testing as part of the process to assess audit readiness. Internal control tests are performed to obtain evidence about the achievement of specific control objectives, while substantive tests are performed to obtain evidence on whether amounts reported on the financial statements are reliable. Both types of testing generally involve determining whether appropriate supporting documentation exists and is readily available. Internal control testing focuses on assessing the effectiveness of controls that would prevent or detect potential misstatements in the financial statements. For example, to test controls over operating expenses, a component would review a sample of invoices to determine if they had been properly approved for payment, typically indicated by a signature of an authorized official. Substantive testing, on the other hand, is conducted to obtain evidence on whether the amounts reported on the financial statements are reliable. For example, to test an operating expense balance, a component’s procedures would include examining invoices to determine if the amounts of the invoices matched the transaction amounts recorded in the general ledger and determining if the purchased item or service was actually received. As discussed below, a key step in testing involves reconciling the population of transactions to be tested.

Reconciliation of population of transactions. To conduct both internal control and substantive testing, a sample of the data transactions is typically selected for testing. An organization must ensure that a sample is taken from, or represents, the complete population of transactions that needs to be tested. According to the FIAR Guidance, to ensure the completeness of the population, the DOD components are required to identify the population of transactions for an assessable unit, compare the total amount of those transactions to the amounts recorded in the general ledger and the financial statements, and then research and resolve any differences between the amounts prior to selecting a sample.

Tests of information systems controls. Because most financial information is maintained in computer systems, the controls over how those systems operate are integral to the reliability of financial data. The components are required to identify, document, and test both general and application controls for key systems that process transactions. General controls are the policies and procedures that apply to all or a large segment of an entity’s information systems and help ensure their proper operation. The objectives of general controls include safeguarding data, protecting application programs, and ensuring continued computer operations in case of unexpected interruptions. For example, general controls include logical access controls that prevent or detect unauthorized access to sensitive data and programs that are stored, processed, and transmitted electronically. Application controls, sometimes referred to as business controls, are incorporated directly into computer applications to help ensure the validity, completeness, accuracy, and confidentiality of data during application processing and reporting. For example, a system edit used to prevent or detect a duplicate entry is an application control.

Corrective action plans. The components are required to develop and execute corrective action plans to remediate any deficiencies that indicate that controls are not working and/or transaction amounts are not supported. The corrective action plans should include the solutions to be implemented to resolve the deficiencies, the identification of resources required and committed to carry out the solutions, and the targeted milestones and completion dates. Further, the FIAR Guidance states that after corrective actions have been taken, the components should perform additional testing to determine whether the deficiencies were in fact remediated.

Navy Civilian Pay and Air Force Military Equipment FIPs Were Not Adequately Developed and Implemented

The Navy and Air Force did not adequately develop and implement their respective FIPs for Civilian Pay and Military Equipment in accordance with the FIAR Guidance. Our review of these FIPs found similar deficiencies in both of them. Specifically, our review of the FIPs found that the Navy and Air Force did not conduct sufficient control and substantive testing, and reached conclusions that were not supported by the testing results, did not complete reconciliations of the population of transactions, and did not fully test information systems controls. Also, neither component had fully developed and implemented corrective action plans to address deficiencies identified during implementation of the FIPs. In addition, the Navy did not accurately report the status of certain metrics in the November 2010 FIAR Plan Status Report. As a result of these deficiencies, neither FIP provided sufficient support for the components’ conclusions that the assessable units were ready to be audited. Navy officials stated that they were taking action to address the issues identified and planned to submit a revised FIP by March 2012. Air Force officials also indicated that they were taking action to address the issues identified. In July 2011, Air Force officials provided updates on the status of several actions that were underway or completed but, for the most part, did not provide supporting documentation for our review. Further, the actions they identified did not address all of the deficiencies that we noted.

Navy’s Civilian Pay FIP Was Not Adequately Developed and Implemented

The Navy did not adequately develop and implement its FIP for civilian pay in accordance with the FIAR Guidance. Specifically, our review of this FIP found that the Navy did not (1) conduct sufficient control and substantive testing, and reached conclusions that were not supported by the testing results; (2) reconcile the population of transactions recorded in the payroll system to those in the general ledger prior to testing; (3) fully test information systems controls; (4) adequately develop and implement corrective action plans; and (5) accurately assess and report the status of its FIP work in terms of specific FIAR Plan metrics. As such, the FIP documentation did not support audit readiness for civilian pay as asserted by the Navy on March 31, 2010.

Both the DOD Comptroller and the DOD OIG found many of the same issues that we did during their reviews of the FIP. Navy officials said that they are performing additional analysis and testing to address identified deficiencies and plan to submit a revised FIP by March 2012.

The following paragraphs provide more details about the deficiencies we found in the Navy’s Civilian Pay FIP.

Testing Was Insufficient and Did Not Support Conclusions. The Navy did not conduct sufficient internal control and substantive testing for civilian pay, as required by the FIAR Guidance. We found instances in which documentation of the Navy’s testing results was not included in the FIP and other instances in which the documentation was included but did not support the conclusion reached. For example, while the Navy concluded that internal controls were designed and operating effectively, the results of the control testing indicated that 9 of 17 controls tested were ineffective. The Navy reported that its civilian pay was ready for audit because its control environment mitigated the deficiencies of its ineffective controls; however, it did not explain how this was accomplished. Based on our analysis and discussions with Navy officials, we determined the control environment did not mitigate these deficiencies. Both the DOD Comptroller and OIG had similar comments about the control environment. In addition, the Navy concluded that system exception and change reports, which are computerized input and edit controls in the form of reports, were in general operating effectively. However, the detailed test results showed that 5 of 14 reports would not always run properly, and that one report was in fact not running properly as it was producing false positives and negatives (i.e., information that was not always accurate).

The FIP indicated that no exceptions were identified as a result of the substantive testing performed. However, during our review of the testing results, we identified substantive exceptions. For example, the Navy was unable to verify the reasonableness of payroll amounts for a sample of employees because of incomplete and missing documentation, such as time and attendance reports and schedules with approved pay rates. Navy officials said that they did not pursue the missing documents because the related control test for this process had failed; therefore, they did not believe the substantive test needed to be completed nor did they consider the incomplete or missing documents to be substantive exceptions. Navy officials said that they later tested payroll transactions for another sample of employees, but did not retain the documentation because it contained personally identifiable information (e.g., social security numbers). The officials also said that they are performing additional substantive testing and will include that documentation in the revised FIP.

In addition, there was no evidence in the FIP that control and substantive tests were performed for personnel benefits (e.g., payments for retirement plans and health insurance). Navy officials said that they had performed testing of personnel benefits, but that they did not retain the documentation as it contained personally identifiable information. However, as stated earlier, the FIAR Guidance requires that all FIP procedures and the results be fully documented. The officials also said that the additional payroll testing that they are performing includes personnel benefits, and that this will be documented in the revised FIP.

Population of Transactions Was Not Reconciled. The Navy did not reconcile the population of transactions for civilian pay prior to conducting testing as required by the FIAR Guidance.

Specifically, the Navy did not ensure that it selected samples from the complete population of payroll transactions recorded in the Defense Civilian Pay System (DCPS) because it did not reconcile all DCPS data to the Navy’s general ledger systems. Instead, it used a subset of the payroll transactions to conduct a reconciliation. The Navy stated that the transactions excluded from the reconciliation were immaterial but the rationale and support for this conclusion was not documented in the FIP at the time the Navy concluded that civilian pay was audit-ready. In addition, the Navy identified discrepancies during its efforts but did not clearly document what these discrepancies were or their resolution. For example, the Navy documented in the FIP that there were instances of missing data and high variances, but did not indicate the nature of these issues or how pervasive they were, the actions taken to resolve them, and/or whether the issues were resolved. In response to these concerns we raised, Navy officials said that they subsequently performed a reconciliation using more recent payroll transactions that resulted in insignificant unreconciled discrepancies. The officials stated that they believed these results were sufficient to ensure that a complete population was identified and that they used this population to select a sample of transactions and are performing detailed testing. Navy officials said that the results of this work will be included in the revised FIP.

In addition, the Navy documented in the FIP that it was unable to reconcile the payroll accounts in its general ledgers to the DOD-wide general ledger that is used to generate the components’ and department’s financial statements. In their attempt to perform this reconciliation, Navy officials noted that they were unable to extract reliable payroll data from the DOD-wide general ledger and that the payroll account balance in the DOD-wide general ledger was greater than the total of the account balances in the Navy’s general ledgers. As a result, the Navy was unable to reconcile the population of payroll transactions to the system that ultimately produces its financial statements; in other words, it could not identify the information needed to test the civilian pay amounts included in the financial statements. The Navy officials stated they plan to address these issues as part of the SBR Financial Statement Compilation and Reporting assessable unit, which they plan to have audit-ready by the end of fiscal year 2012.

Information Systems Controls Were Not Fully Tested. The FIAR Guidance requires DOD components to test system controls to ensure that they are operating as intended. However, the methodology—the Defense Information Assurance Certification and Accreditation Process (DIACAP)—that the Navy used to assess information systems controls did not address all the elements of a general controls assessment. For example, the DIACAP did not address the periodic review of (1) users’ access authorizations or privileges to ensure that they are appropriate based on assigned roles and responsibilities, and (2) automated logs of changes to security access authorizations to ensure that management is aware of any unusual activity. In addition, the Navy did not review the results of general controls assessments to ensure that the assessments covered the overall operating environment in which the systems operated. For example, any general control weaknesses in a mainframe or network that were not included in the scope of the assessment could possibly negate the effectiveness of the controls for the individual system reviewed. Because the Navy did not include all elements of a general controls assessment in its testing, it was unable to demonstrate the effectiveness of the Navy’s overall general control environment.

The Navy reported in the FIP that it relied on a SAS 70 report to obtain assurance over key payroll processing controls (i.e., application controls) for the DCPS. However, that report identified significant weaknesses. Specifically, it indicated that several control activities were found to be ineffective and as a result, certain control objectives were not met for DCPS. In the FIP, the Navy stated that the particular control activities identified as ineffective in the SAS 70 report did not significantly affect Navy Civilian Pay, but it did not adequately document the rationale for this assessment. Nevertheless, we determined that several of these control activities, and related control objectives, were significant to Navy Civilian Pay and, as such, the Navy did not have reasonable assurance that its personnel and payroll data were complete, accurate, and timely processed.

Corrective Actions Were Not Adequately Developed and Implemented. The Navy’s FIP did not include the information needed for corrective action plans as required by the FIAR Guidance. For the most part, the FIP did not include (1) the deficiencies to be corrected (the root cause of the exceptions), (2) the solutions to be implemented to resolve the identified deficiencies, (3) the resources needed to carry out those solutions, and (4) a schedule for timely completion of corrective actions. Navy officials stated that they are currently developing corrective action plans; however, their first priority is to develop and implement the revised FIP which will include evidence of audit readiness based on substantive procedures rather than reliance on internal controls. In effect, the Navy plans to assert audit readiness based on its testing of account balances without addressing the identified internal control deficiencies. However, the development and implementation of a corrective action plan to address such deficiencies is a requisite for improving financial management, which is one of the goals of the FIAR effort.

FIP Status Was Not Accurately Reported. The FIAR Guidance requires the components to report the status of Key Control Objectives (KCO) and Key Supporting Documents (KSD) for their assessable units. However, the status for these metrics that the Navy reported in the November 2010 FIAR Plan Status Report to demonstrate audit readiness was not supported by the results of the Navy FIP work. For example,  The Status Report indicated that 95 percent of KCOs pertaining to Navy Civilian Pay were found to be effective; however, the Navy’s control testing results reflected mostly ineffective internal controls.  The Status Report indicated that 100 percent of KSDs related to Navy Civilian Pay (e.g., documentation evidencing the operation of an internal control such as properly approved time and attendance records) were found to exist. However, as discussed earlier, Navy Civilian Pay testing results indicated that incomplete and missing documentation was one of the more prevalent findings.

Navy officials said that the differences we noted were due to the fact that the KCOs and KSDs required for the FIAR Plan did not exactly match or align with the Navy’s actual work. Therefore, the Navy tried to estimate the appropriate level of progress to report for the KCOs and KSDs listed in the FIAR Plan based on testing that it conducted. In the May 2011 FIAR Plan, the Navy revised its reported progress for these metrics. For example, instead of reporting that 95 percent of KCOs were effective, the May 2011 FIAR Plan indicates that 60 percent of KCOs were effective for Navy’s civilian pay. We did not assess the accuracy of these revised metrics.

Air Force’s Military Equipment FIP Was Not Adequately Developed and Implemented

The Air Force did not adequately develop and implement its FIP for military equipment in accordance with the FIAR Guidance. See table 3 for the types of Air Force military equipment, and their reported quantities and values. In our review of this FIP, we found that the Air Force did not (1) conduct sufficient control and substantive testing, and reached conclusions that were not supported by the testing results; (2) reconcile the population of transactions recorded in its accountable property system to the general ledger; (3) fully test information systems controls; and (4) adequately develop and implement corrective action plans. As a result of these deficiencies, the FIP documentation did not support the Air Force’s December 2010 assertion that the military equipment assessable unit was ready to be audited.

The DOD Comptroller provided initial comments based on its review of the FIP which indicated that it had found issues similar to those we identified and concluded that the Air Force had not demonstrated audit readiness for its military equipment. In addition, the DOD OIG identified similar issues and concluded that the Air Force had not complied with the FIAR Guidance in developing and implementing this FIP. Air Force officials acknowledged that they had more work to do to address the identified deficiencies and indicated that they planned to complete these corrective actions by the end of June 2011. In July, the Air Force provided updates on the status of several actions underway or completed but did not provide any supporting documentation for our review. Further, these actions did not address all of the deficiencies that we identified.

The following paragraphs provide more details about the deficiencies we found in the Air Force’s FIP.

Testing Was Insufficient and Did Not Support Conclusions. As described earlier, DOD components are required to conduct both internal control and substantive testing for each assessable unit. The Air Force did not perform sufficient testing to support audit readiness for the existence and completeness of various types of its military equipment. For aircraft, the Air Force judgmentally selected five sites at which to perform the testing but did not provide evidence that the conditions at these five sites were representative of all Air Force locations. Selecting sites judgmentally could be an acceptable method if the Air Force could demonstrate that the processes and controls at the selected sites were representative of all other locations not tested.

For the other four categories of military equipment—ICBMs, RPAs, satellites, and pods—the FIP did not include any documentation of internal control or inventory testing for the existence and completeness of these assets. Instead, the FIP described the routine monitoring activities over these assets that are conducted for operational purposes. With regard to ICBMs, Air Force officials said that the Air Force Audit Agency will be performing inventory testing of ICBMs in fiscal year 2011.

Population of Transactions Was Not Reconciled. The Air Force did not reconcile the population of transactions for military equipment prior to conducting testing as required by the FIAR Guidance. Specifically, the Air Force did not ensure that it selected testing samples from the complete population of transactions because it did not complete a reconciliation of the military equipment data recorded in its accountable property systems of record to its general ledger. When it compared the data in these systems, it found discrepancies that it did not resolve. For example, there was an unresolved difference of about $2 billion that was largely attributed to differences in both the recorded costs and accumulated depreciation of satellites. We also found that the FIP included documentation that reported different balances for aerospace vehicles. As shown in table 3, the Air Force reported a net book value of $89.5 billion for its aerospace vehicles, which was about $11 billion more than the balance used to perform the reconciliation to the general ledger. Air Force officials said that the balance for aerospace vehicles shown in table 3 is inaccurate and that the balance used to perform the reconciliation is likely more reasonable; however, there was no documentation in the FIP to support this statement. Because of the unresolved reconciling items and the discrepancies in the balances reported for aerospace vehicles, the Air Force does not have assurance that the testing done to determine audit readiness covered the complete population of its military equipment.

Information Systems Controls Were Not Fully Tested. The FIAR Guidance requires DOD components to test system controls to ensure that they are operating as intended. However, the FIP did not provide support to indicate that general and application systems controls were operating effectively for the two systems that maintain accountability for the Air Force’s military equipment. For REMIS, the FIP included information about the Air Force’s conclusions regarding the effectiveness of specific controls. However, the FIP did not include any evidence of the testing performed that would allow for an independent evaluation of its work. For RAMPOD, the FIP included a list of controls to be tested, but did not provide any conclusions or any other evidence that any testing had been done.

The DOD Comptroller expressed concerns similar to ours and as a result, Air Force officials stated that they would be performing additional testing of general and application controls. In July 2011, Air Force officials reported that this testing was completed but they did not provide supporting documentation for our review.

Corrective Actions Were Not Adequately Developed and Implemented. The Air Force had not developed corrective action plans to address all of the exceptions identified during testing, and had not implemented any corrective actions to address these exceptions as of December 31, 2010, when it submitted its FIP, as required by the FIAR Guidance. For example, in reviewing the status of the corrective actions, we found the following:  Capital Modifications—The Air Force’s testing determined that it did not have the necessary controls in place to ensure that equipment modifications were capitalized when appropriate. The Air Force’s corrective action plan had identified the nature of this deficiency, the solution, the required resources, and targeted milestone. However, the targeted milestone noted in the FIP was January 2011—1 month after the Air Force had indicated that the assessable until was audit ready. Air Force officials indicated that they did not expect to complete this corrective action until December 2011.  Accumulated Depreciation—The Air Force’s initial testing identified discrepancies with the accumulated depreciation balance reported in REMIS. The errors included both over- and underdepreciation of assets, which, in some instances, resulted in accumulated depreciation amounts in excess of the acquisition cost of the assets. However, as of the date it indicated audit readiness, the Air Force had not identified the cause of this problem, the solution, or a time frame for implementation. Subsequent to indicating audit readiness, the Air Force said that it analyzed the issue further and that it had resolved the issue by July 2011. However, based on our review of documentation provided, the Air Force’s actions did not fully address this weakness.  REMIS ICBM Records—The FIP stated that 555 complete ICBMs were recorded in REMIS, but that only approximately 450 complete ICBMs exist at any time. The approximately 100 remaining ICBMs pertain to unassembled missile components which, according to the Air Force, should be classified as operating materials and supplies rather than military equipment. The FIP identified the solution needed to address this deficiency and stated that it must be corrected before military equipment can be ready for audit. However, Air Force officials indicated that they did not expect to complete this corrective action until the fourth quarter of fiscal year 2011, and that testing the effectiveness of the corrective action would be incorporated into the fiscal year 2012 testing efforts.

FIP Monitoring and Oversight Needs Improvement

DOD and its military components have established senior executive committees as well as designated officials at the appropriate levels to monitor and oversee their financial improvement efforts. These committees and individuals have also generally been assigned appropriate roles and responsibilities. (Figure 1 depicts the key organizations and positions involved in the overall FIP process and table 4 in app. II outlines their roles and responsibilities.) According to relevant criteria, monitoring should be performed continually and includes regular management and supervisory activities such as assigning qualified people with the appropriate roles and responsibilities, carrying out assigned oversight duties, and documenting the results of oversight activities. We found that Navy and Air Force officials as well as oversight committees did not effectively carry out their monitoring responsibilities for the FIPs that we reviewed. However, once the components indicated audit readiness, we found that the DOD OIG and the DOD Comptroller appropriately carried out their responsibilities for reviewing the FIPs.

Based on our reviews of the Navy Civilian Pay and Air Force Military Equipment FIPs discussed earlier, the Navy and Air Force officials responsible for monitoring and oversight did not effectively ensure that the FIP work was performed in accordance with the FIAR Guidance. The FIP Directors did not ensure that their respective FIPs provided sufficient evidence to support the conclusions of audit readiness before providing the FIPs to the Assistant Secretaries—Financial Management and Comptroller (FM&C) for their signature. Neither Assistant Secretary ensured that the FIPs were sufficient before signing them to indicate audit readiness. For example, the Air Force’s FIP was signed even though the FIP stated that the deficiency in ICBM reporting, as discussed earlier, “must be addressed before the military equipment line item can be ready for audit.”

In addition, committees at both the component and DOD levels did not effectively carry out their responsibilities for FIP oversight. Minutes of the components’ senior executive committee meetings did not indicate that these committees thoroughly reviewed the progress of the FIPs in addressing financial management weaknesses. With respect to the FIPs that we reviewed, the FIAR Governance Board’s activities consisted primarily of receiving status update briefings. Because neither the individual FIP managers nor the oversight committees adequately reviewed and monitored the FIPs, each of the assessable units we reviewed was deemed audit-ready even though the results did not support these conclusions.

Once the components indicated audit readiness, both the DOD OIG and the DOD Comptroller carried out their responsibilities for reviewing the FIPs. The DOD OIG, which reviews FIPs concurrently with the DOD Comptroller after a component indicates audit readiness, provided comments to the DOD Comptroller on each of the FIPs. It identified many of the same, or similar, issues that we did, as discussed earlier, and concluded that the FIPs did not comply with the FIAR Guidance nor demonstrate audit readiness for the assessable units.

The DOD Comptroller, which makes the final determination as to whether an assessable unit is ready for audit, also identified issues for the Navy Civilian Pay FIP similar to those discussed earlier and concluded that the Navy had not demonstrated audit readiness for its civilian pay. For the Air Force Military Equipment FIP, the DOD Comptroller provided initial comments indicating that it had found issues similar to those we identified and concluded that the Air Force had not demonstrated audit readiness for its military equipment, but it had not yet issued final comments.

Recognizing that additional actions were needed to assist the components in developing and implementing their FIPS, the DOD Comptroller established a quality assurance team in January 2011 to review the components’ FIPs as they are being developed and implemented. The intent is for the quality assurance team to provide detailed feedback on the FIPs before they are formally submitted for review and validation. In addition, the DOD Comptroller developed a series of training courses to help component personnel understand and execute the FIAR Methodology. Officials from the DOD Comptroller’s Office said that the components need additional training and assistance with their FIPs because they do not necessarily have staff with the appropriate skills and qualifications to adequately carry out the procedures required by the FIAR Guidance. We believe that the DOD Comptroller’s efforts to review the FIPs as they are being developed and implemented, and to provide additional training and ongoing feedback, will improve the FIPs and thus, the components’ ability to demonstrate that assessable units are audit-ready.

When the components report the progress of their FIPs inaccurately and submit FIPs to DOD that do not adequately support audit readiness, DOD—both the Comptroller and the OIG—must use resources to review unreliable or incomplete information, and components must then perform rework to reach audit readiness. Thus, the lack of adequate oversight results in an inefficient FIP process and can impact the ability of components to meet established milestones.

Conclusions

DOD’s FIAR Guidance provides a reasonable and systematic process that DOD components can follow in their efforts to achieve audit readiness. It establishes clear priorities for the components and a road map for reaching auditability for each assessable unit. However, we found that the components did not adequately carry out the procedures required by the FIAR Guidance for the two FIPs we reviewed. Top managers involved in FIP oversight also did not properly monitor and assess the status of FIP efforts in order to make accurate decisions regarding audit readiness. As a result, both the Navy’s and Air Force’s conclusions of audit readiness for civilian pay and military equipment, respectively, were unsupported. Both the Navy and the Air Force indicated that they have initiated additional actions to address the identified deficiencies, but they did not provide supporting documentation for us to verify their actions. To achieve departmentwide audit readiness, DOD leaders will need to ensure that the components develop, implement, and document their FIPs in compliance with the FIAR Guidance. Considering the deficiencies identified in this report can help inform DOD leaders and the components as they develop and implement other FIPs to better utilize resources by minimizing rework. If the DOD components are unable to achieve interim FIAR milestones, DOD will need to consider the effect on its ability to achieve departmentwide audit readiness by September 30, 2017.

Recommendations for Executive Action

We are making 13 recommendations to the Secretary of Defense to improve the development, implementation, documentation, and oversight of the department’s financial management improvement efforts.

To ensure that the Navy develops and implements its Financial Improvement Plan in accordance with the FIAR Guidance, we recommend that the Secretary of Defense direct the Secretary of the Navy to put procedures in place to help ensure that the Navy’s Financial Improvement Plans include documentation that the Navy performed the following:  Sufficient control and substantive testing.  A reconciliation of the complete population of transactions for an assessable unit to the relevant general ledger(s) and to the amount(s) reported in the financial statements, including researching and resolving reconciling items.  An assessment of information systems controls that (1) addresses all relevant critical elements, and (2) for any deficiencies identified in a SAS 70 report that is relied upon, show that either mitigating controls exist or actions have been taken to address the deficiencies.  Preparation and execution of corrective action plans to address significant control weaknesses.  Assessments of the metrics (e.g., key control objectives and key supporting documents) to ensure that they are consistent with, and supported by, testing results.

To ensure that the Air Force develops and implements its Financial Improvement Plan in accordance with the FIAR Guidance, we recommend that the Secretary of Defense direct the Secretary of the Air Force to ensure that the Air Force’s Financial Improvement Plans include documentation that the Air Force performed the following:  Sufficient control and substantive testing.  A reconciliation of the complete population of transactions for an assessable unit to the relevant general ledger(s) and to the amount(s) reported in the financial statements, including researching and resolving reconciling items.  An assessment of information systems controls that includes documentation of both the testing and the results.  Preparation and execution of corrective action plans to address significant control weaknesses.

To ensure that other FIPs from DOD components comply with the requirements in the FIAR Guidance, we recommend that the Secretary of Defense direct the Secretary of the Army and the heads of other DOD components to consider the weaknesses identified in this report when preparing their FIPs.

To improve DOD’s monitoring and oversight of FIP activities, we recommend that the Secretary of Defense direct:  The Co-Chairs of the FIAR Governance Board to ensure that the board carries out its responsibilities for identifying risks that could prevent the department from achieving its goals and ensuring sufficient documentation of FIP assessment results.  The Secretary of the Navy to ensure that all responsible parties within the Navy, including the Assistant Secretary of the Navy (Financial Management and Comptroller), carry out their responsibilities for ensuring that FIP development and implementation complies with the FIAR Guidance and that the FIP contains sufficient information to indicate audit readiness before it is signed.  The Secretary of the Air Force to ensure that all responsible parties within the Air Force, including the Assistant Secretary of the Air Force (Financial Management and Comptroller) carry out their responsibilities for ensuring that FIP development and implementation complies with the FIAR Guidance and that the FIP contains sufficient information to indicate audit readiness before it is signed.

Agency Comments and Our Evaluation

We provided a draft of this report to the Secretary of Defense and received written comments from the Under Secretary of Defense (Comptroller), which are reprinted as appendix III. Overall, DOD concurred with 10 recommendations and partially concurred with three, and identified some specific actions that are completed, underway, or planned.

DOD commented that its approach of prioritizing its efforts on improving information related to budgetary resources and the existence and completeness of its assets and achieving auditability in those areas has garnered more participation and attention in the FIAR effort than in the past, and DOD described some initiatives underway to speed progress on this effort. DOD recognized that there is room for improvement in implementation of the FIP process by the military components as well as in governance and management of the process. In that regard, DOD concurred with 10 of our recommendations, and said that it is critical to continue to review how DOD applies lessons learned across the department and changes business processes to reflect those lessons. We agree that identifying and incorporating lessons learned into the FIAR process will be an important part of effective implementation, and we look forward to seeing how DOD develops a mechanism to capture and disseminate the lessons.

DOD partially concurred with three other recommendations specifically related to the Navy and Air Force FIPs. DOD explained that the Navy and Air Force FIPs that we reviewed were prepared before issuance of the May 2010 FIAR Guidance and may have proceeded with a strategy that was not sufficiently supported, but that corrective actions are underway. As we discussed with DOD, although the FIPs we selected were initiated prior to issuance of the final FIAR Guidance, the issues we identified are consistent with draft FIAR Guidance as well as standard procedures for conducting a financial statement audit that we found to be incorporated into the final FIAR Guidance. On our recommendation related to improving the Navy’s process for reconciling transactions with its general ledgers, DOD partially concurred, but noted that the Navy will be unable to reconcile transaction populations until it completes its Financial Statement Compilation Process. As we report, performing reconciliations is key to properly testing financial statement amounts and therefore should be done prior to asserting audit readiness.

On our recommendation related to improving the sufficiency of the Air Force’s control and substantive testing, DOD partially concurred, but noted that the Air Force based the extent of its testing procedures on its assessment of the inherent risk of the asset category, stating that ICBMs were not tested due to their limited number and extensive controls for these assets. Although the Air Force’s FIP noted that ICBMs are subject to extensive controls, the FIP did not document those controls or any tests conducted to validate the controls. Also, for RPAs, satellites, and pods, no evidence of controls was provided and no testing was done. While the FIAR Guidance allows components to use a substantive approach (versus a controls approach) when there are a limited number of items, it does not allow no testing if an area is significant or material. Also as we report, the Air Force’s basis for its judgmental selection of the locations to test aircraft was inadequate since it did not demonstrate that the processes and controls at the five selected sites were representative of all other locations not tested. In addition to partial concurrence with this recommendation, DOD commented it will further review this issue and take action as appropriate.

On our recommendation related to improving the Air Force’s corrective action plans, DOD partially concurred, but noted that when the Air Force asserted auditability it did not believe that the issues being addressed by corrective actions were significant control weaknesses and, therefore, the Air Force was allowed under the FIAR Guidance to assert auditability. We believe that the Air Force reported significant control weaknesses that would have precluded an assertion of auditability; the FIAR Guidance does not allow for auditability assertions when there are unresolved material deficiencies. For example, as we report, the Air Force documented that it was unable to reconcile its population of transactions to its financial statements, and as a result, did not have assurance that the testing done covered the population of military equipment. Also, the Air Force reported errors in its depreciation amounts and that accurate depreciation amounts were “essential” and Air Force military equipment “will not be ready for audit” if not corrected. The Air Force also reported that the cause of errors was “unknown.” Although the Air Force subsequently withdrew part of its FIP related to some of these issues, we evaluated the content of the FIP that the Air Force asserted was audit- ready. In addition to partial concurrence with this recommendation, DOD commented it will further review this issue and take action as appropriate.

As agreed with your offices, we plan no further distribution of this report until two days from its date, unless you publicly announce its contents earlier. At that time, we will send copies of this report to the Secretary of Defense; the Under Secretary of Defense (Comptroller); the Secretary of the Navy; the Secretary of the Air Force; the Deputy Chief Management Officer; the Chief Management Officer of the Navy; and the Chief Management Officer of the Air Force. This report also is available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact Asif A. Khan at (202) 512-9095 or khana@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to determine whether (1) the Financial Improvement and Audit Readiness (FIAR) Guidance provided a reasonable methodology for the Department of Defense (DOD) components to develop Financial Improvement Plans (FIP), (2) the DOD components had adequately developed and implemented selected FIPs in accordance with the FIAR Guidance, and (3) DOD is adequately monitoring and overseeing the FIP process.

To address the first objective, we analyzed the FIAR Guidance using relevant criteria such as the GAO and President’s Council on Integrity and Efficiency (PCIE) Financial Audit Manual (FAM); the Federal Information System Controls Audit Manual (FISCAM); and Office of Management and Budget (OMB) Circular No. A-123, Management’s Responsibility for Internal Control, Appendix A, Internal Control over Financial Reporting. The FAM provides a methodology to perform financial statement audits of federal entities in accordance with professional auditing and attestation standards and OMB guidance. The FISCAM provides a methodology for performing information system control audits in accordance with generally accepted government auditing standards. The OMB Circular No. A-123 provides guidance to Federal managers on improving the accountability and effectiveness of Federal programs and operations by establishing, assessing, correcting, and reporting on internal control. Appendix A of OMB Circular No. A-123 provides a methodology to assess and report on agencies’ internal controls over financial reporting. We also interviewed agency officials at the Office of the Under Secretary of Defense (Comptroller) (OUSD(C)) FIAR Directorate’s office, which developed the FIAR Guidance, and at the Navy and Air Force to obtain explanations and clarifications as a result of our analysis of selected FIPs.

To address the second objective, we selected FIPs for two assessable units (Navy Civilian Pay and Air Force Military Equipment) that were scheduled to assert audit readiness in 2010 and were within wave 2 (i.e., Statement of Budgetary Resources) and wave 3 (i.e., Existence and Completeness of Mission Critical Assets) since these waves reflect DOD’s priority focus areas. Using the FIAR Guidance, we analyzed the documentation included in the FIPs, such as process flows, control assessments, test plans, test results, and corrective action plans. We did not perform separate audit procedures to assess the effectiveness of the controls or the completeness or accuracy of the Navy civilian pay amounts or the Air Force military equipment. We interviewed the Navy and Air Force FIP directors to obtain explanations and clarifications as a result of our evaluation of the documentation.

To address the third objective, we analyzed relevant documentation, such as the FIAR Guidance, FIAR Plan Status Reports, and committee charters and meeting minutes, to identify the entities and officials responsible for monitoring and oversight as well as their roles and responsibilities. We also interviewed officials that play a key role in the monitoring and oversight process, such as Army, Navy, and Air Force officials from the offices of Financial Management and Comptroller and the Deputy Chief Management Officers and DOD officials from the Office of the Under Secretary of Defense (Comptroller), to clarify our understanding of these entities and officials’ roles and responsibilities. We then analyzed this information using elements of monitoring discussed in the FAM; the Implementation Guide for OMB Circular No. A-123, Appendix A; Standards for Internal Control in the Federal Government; the Internal Control Management and Evaluation Tool; the COSO Guidance on Monitoring Internal Control Systems; and the FIAR Guidance.

We conducted this performance audit from May 2010 to September 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Key Oversight Entities and Their Roles and Responsibilities

The following table summarizes the roles and responsibilities of key individuals and committees involved in monitoring and overseeing the FIP process.

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, the following individuals made key contributions to this report: Abe Dymond, Assistant Director; Paul Foderaro, Assistant Director; J. Mark Yoder; Kristi Karls; Michael Bingham; Carmen Harris; Adrienne Walker; David Yoder; Jason Kelly; and Jason Kirwan.