DCAA Audits: Widespread Problems with Audit Quality Require Significant Reform
Released: 2009-09-23T00:00:00

Letter

Widespread Management Environment and Audit Quality Problems

We found audit quality problems at DCAA offices nationwide. Of the 69 audits and cost-related assignments we reviewed, 65 exhibited serious GAGAS or other deficiencies similar to those found in our investigation, including compromise of auditor independence, insufficient audit testing, and inadequate planning and supervision. Although not as serious, the remaining four audits also had GAGAS compliance problems. In addition, while DCAA did not consider 26 of the 32 cost-related assignments we reviewed to be GAGAS audits, DCAA did not perform sufficient testing to support reported conclusions on that work. According to DCAA officials, DCAA rescinded 80 audit reports related to our prior investigation as well as the audit leading to today’s report because the audit evidence was outdated, insufficient, or inconsistent with reported conclusions and opinions and reliance on the reports for contracting decisions could pose a problem. About one third of the rescinded reports relate to unsupported opinions on contractor internal controls and were used as the basis for risk-assessments and planning on subsequent internal control and cost- related audits. Other rescinded reports relate to Cost Accounting Standards (CAS) compliance and contract pricing decisions. Because the conclusions and opinions in the rescinded reports were used to assess risk in planning subsequent audits, they impact the reliability of hundreds of other audits and contracting decisions covering billions of dollars in DOD expenditures.

A management environment and agency culture that focused on facilitating the award of contracts and an ineffective audit quality assurance structure are at the root of the agencywide audit failures we identified. DCAA’s focus on a production-oriented mission led DCAA management to establish policies, procedures, and training that emphasized performing a large quantity of audits to support contracting decisions and gave inadequate attention to performing quality audits. An ineffective quality assurance structure, whereby DCAA gave passing scores to deficient audits compounded this problem.

Although the reports for all 37 audits of contractor internal controls that we reviewed stated that the audits were performed in accordance with GAGAS, we found GAGAS compliance issues with all of these audits. The issues or themes are consistent with those identified in our prior investigation.

Lack of independence. In seven audits, independence was compromised because auditors provided material nonaudit services to a contractor they later audited; experienced access to records problems that were not fully resolved; and significantly delayed report issuance, which allowed the contractors to resolve cited deficiencies so that they were not cited in the audit reports. GAGAS state that auditors should be free from influences that restrict access to records or that improperly modify audit scope.

Insufficient testing. Thirty-three of 37 internal control audits did not include sufficient testing of internal controls to support auditor conclusions and opinions. GAGAS for examination-level attestation engagements require that sufficient evidence be obtained to provide a reasonable basis for the conclusion that is expressed in the report. For internal control audits, which are relied on for 2 to 4 years and sometimes longer, the auditors would be expected to test a representative selection of transactions across the year and not transactions for just one day, one month, or a couple of months. However, we found that for many controls, the procedures performed consisted of documenting the auditors’ understanding of controls, and the auditors did not test the effectiveness of the implementation and operation of controls.

Unsupported opinions. The lack of sufficient support for the audit opinions on 33 of the 37 internal control audits we reviewed rendered them unreliable for decision making on contract awards, direct-billing privileges, the reliability of cost estimates, and reported direct cost and indirect cost rates. For example, we found that: For many controls, DCAA did not perform any testing at all. For example, audits of contractor accounting systems do not include any transaction testing. Instead, these audits focus on a review of the adequacy of contractor policies and procedures. At least six of the nine accounting audits we reviewed did not include procedures for confirming contractor segregation of allowable and unallowable cost.

DCAA issued an adequate opinion on the accounting system for a major DOD contractor after performing a walkthrough of the accounting process and interviewing two employees.

In billing system audits we reviewed, DCAA auditors often tested only two, three, or sometimes five transactions to support audit conclusions on contractor systems and related internal controls. Further, the auditors performed limited procedures such as determining whether the vouchers were mathematically correct and included current and cumulative billed amounts. Twenty of the 22 billing system audits we reviewed did not include tests to identify duplicate invoices.

DCAA auditors reported on the adequacy of a contractor’s billing system based on tests of four vouchers, all issued on the same day.

In an audit of controls over indirect and other direct cost for a business segment of one of the top five DOD contractors, DCAA auditors tested 12 out of about 22,000 transactions processed from May through July 2005.

We also found that reports did not adequately disclose the criteria used in performing individual audits. According to GAGAS, audit reports should, among other matters, identify the subject matter being reported and the criteria used to evaluate the subject matter. Criteria identify the required or desired state or expectation with respect to the program or operation and provide a context for evaluating evidence and understanding the findings. None of the 37 internal control audit reports we reviewed cited specific criteria used in individual audits. Instead, the reports uniformly used boilerplate language to state that DCAA audited for compliance with the “FAR, CAS, DFARS, and contract terms.” As a result the user of the report does not know the specific Federal Acquisition Regulation (FAR), Cost Accounting Standards (CAS), or contract terms used as criteria to test contractor controls. This makes it difficult for users of the reports to determine whether the reports provide the level of assurance needed to make contracting decisions. Similarly, the 32 cost-related assignments we reviewed did not contain sufficient testing to provide reasonable assurance that overpayments and billing errors that might have occurred were identified. As a result, there is little assurance that any such errors, if they occurred, were corrected and that related improper contract payments, if any, were refunded or credited to the government. Contractors are responsible for ensuring that their billings reflect fair and reasonable prices and contain only allowable costs, and taxpayers expect DCAA to review these billings to provide reasonable assurance that the government is not paying more than it should for goods and services. We identified the following problems with these assignments.

Paid voucher reviews. DCAA performs annual testing of paid vouchers (invoices) to determine if contractor voucher preparation procedures are adequate for continued contractor participation in the direct-bill program. Under the direct-bill program, contractors may submit their invoices directly to the DOD disbursing officer for payment without further review. Although DCAA does not consider its reviews of contractor paid vouchers to be GAGAS engagements, it has not determined what standards, if any, apply to these assignments. In addition, for the 16 paid voucher assignments we reviewed, we found that DCAA auditors failed to comply with DCAA Contract Audit Manual (CAM) guidance. Rather than documenting the population of vouchers, preparing sampling plans, and testing a random (statistical) sample, auditors generally did not identify the population of vouchers, did not create sampling plans, and made a small, nonrepresentative selection of as few as one or two invoices for testing to support conclusions on their work. The auditors performed limited procedures such as determining whether the vouchers were mathematically correct and included current and cumulative billed amounts. Based on this limited work, the auditors concluded that controls over invoice preparation were sufficient to support approval of the contractors’ direct billing privileges. This is of particular concern because we determined that Defense Finance and Accounting Service (DFAS) certifying officers rely on DCAA voucher reviews, and they do not repeat review procedures they believe to be performed by DCAA.

Overpayment assignments. DCAA performs overpayment assignments to verify that contractors have billing procedures and internal controls in place to identify and resolve contractor billing errors and overpayments in a timely manner. DCAA guidance states that these engagements should be conducted in accordance with GAGAS to the extent applicable under the circumstances. However, none of the 10 overpayment assignments we reviewed were performed or reported as GAGAS engagements. We found that auditor judgments about the population and selection of transactions for these assignments did not provide a representative universe for testing and concluding on contractor controls over billings and payments received. For example, for the 10 assignments we assessed, the auditors selectively reviewed an accounts receivable aging report to identify overpayments and determine if they had been resolved. As a result, this work does not provide reasonable assurance that contractors have adequate controls in place to identify and correct overpayments and billing errors and make appropriate, timely refunds and adjustments.

Incurred cost audits. The purpose of incurred cost audits is to examine contractors’ cost representations and opine on whether the costs are allowable, allocable to government contracts, and reasonable in accordance with the contract and applicable government acquisition regulations. DCAA performs these audits as GAGAS attestation engagements. For the four incurred cost audits we reviewed, we found that the auditors did not adequately document their judgments about control risk or the sampling and test methodologies used. In addition, we found that the auditors traced claimed pool and base costs (indirect costs) to the contractors’ accounting books and records to determine their accuracy and allowability. However, the auditors did not perform sufficient, detailed testing of claimed indirect and direct costs. For example, the auditors traced and reconciled indirect costs to contractor accounting system data, but did not test a representative selection of direct costs. As a result, the scope of work performed was not sufficient to identify claimed costs, if any, that were not adequately supported or unallowable costs, if any, that should have been questioned.

Production environment and audit quality issues. DCAA’s mission statement, strategic plan, and metrics all focused on producing a large number of audit reports and provided little focus on assuring quality audits. For example, DCAA’s current approach of performing 30,000 to 35,000 audits and issuing over 22,000 audit reports with 3,600 auditors substantially contributed to the widespread audit quality problems we identified. Within this environment, DCAA’s audit quality assurance program was not properly implemented, resulting in an ineffective quality control process that accepted audits with significant deficiencies and noncompliance with GAGAS and DCAA policy. Moreover, even when DCAA’s quality assurance documentation showed evidence of serious deficiencies within individual offices, those offices were given satisfactory ratings. Considering the large number of DCAA audit reports issued annually and the reliance the contracting and finance communities have placed on DCAA audit conclusions and opinions, an effective quality assurance program is key to protecting the public interest. Such a program would report review findings along with recommendations for any needed corrective actions; provide training and additional policy guidance, as appropriate; and perform follow-up reviews to assure that corrective actions were taken. GAGAS require that each audit organization performing audits and attestation engagements in accordance with GAGAS should have a system of quality control that is designed to provide the audit organization with reasonable assurance that the organization and its personnel comply with professional standards and applicable legal and regulatory requirements, and have an external peer review at least once every 3 years.

DCAA officials advised us that going forward, DCAA will no longer rate an FAO’s overall compliance with GAGAS and DOD policy. The officials told us that instead, DCAA headquarters plans to (1) report the detailed results of the audit quality reviews, (2) make recommendations to FAOs for any needed corrective actions, (3) conduct follow-up reviews for all FAOs with identified audit deficiencies to ensure that corrective actions are taken, and (4) provide training and policy guidance, as appropriate. If properly implemented, these procedures would help to assure an effective audit quality assurance program.

In addition, the DOD IG reported an adequate (“clean”) opinion on DCAA’s most recent peer review results although the reported evidence indicated that numerous audits had serious deficiencies in audit quality. In conducting DOD’s audit oversight review of DCAA audits, DOD IG audit oversight reviewers considered the same results of DCAA’s internal audit quality assurance reviews that we analyzed and reviewed numerous additional audits, which also identified significant GAGAS noncompliance as evidenced by DOD IG peer review findings and recommendations. Although the DOD IG report contained evidence of significant, systemic noncompliance with professional standards throughout DCAA audits that OIG staff reviewed and the IG report included numerous findings and recommendations related to those issues, the DOD IG gave DCAA a “clean” peer review opinion, concluding that for audits and attestation engagements performed during fiscal year 2006, “…the internal quality control system was operating effectively to provide reasonable assurance that DCAA personnel were following established policies, procedures, and applicable auditing standards….”

The overall report conclusion in the DOD IG report is inconsistent with the detailed observations in the report, which indicate numerous significant deficiencies in DCAA’s system of quality control. Furthermore, of the 80 audit reports that DCAA rescinded, 39 of the rescinded reports were issued during fiscal year 2006—the period covered by the last DOD IG peer review. Therefore, we have concluded that DCAA’s quality control system for the period covered by the last DOD IG peer review was not effectively designed and implemented to provide assurance that DCAA and its personnel comply with professional standards.

DCAA Is Making Progress, but Sustained Leadership and Oversight Is Needed

Although DCAA has taken several positive steps, much more needs to be done to address widespread audit quality problems. DCAA’s production- oriented culture is deeply imbedded and will likely take several years to change. Under DCAA’s decentralized management environment, there had been little headquarters oversight of DCAA regions, as demonstrated by the nationwide audit quality problems. DCAA’s mission focused primarily on producing reports to support procurement and contracting community decisions with no mention of quality audits that serve taxpayer interest. Further, DCAA’s culture has focused on hiring at the entry level and promoting from within the agency and most training has been conducted by agency staff, which has led to an insular culture where there are limited perspectives on how to make effective organizational changes.

DCAA corrective actions. DCAA initiated a number of actions to address findings in our July 2008 report, the DOD Comptroller/CFO August 2008 “tiger team” review, and the Defense Business Board study, which was officially released in January 2009. Examples of key DCAA actions to date include the following.

Eliminating production metrics and implementing new metrics intended to focus on achieving quality audits.

Establishing an anonymous Web site to address management and hotline issues. DCAA’s Assistant Director for Operations has been proactive in handling internal DCAA Web site hotline complaints.

Revising policy guidance to address auditor independence, assure management involvement in key decisions, and address audit quality issues. DCAA also took action to halt auditor participation in nonaudit services that posed independence concerns.

Further, DCAA has enlisted assistance from other agencies to develop a human capital strategic plan, assist in cultural transformation, and conduct a staffing study. In March 2009, the new DCAA Comptroller/CFO established a DCAA Oversight Committee to monitor and advise on DCAA corrective actions.

While these are positive actions, other DCAA actions have focused on process improvements, and DCAA has not yet addressed the fundamental weaknesses in its mission, strategic plan, audit approach, and human capital practices.

Although DCAA is making progress, we are concerned that DCAA actions to date evidence some of the past cultural problems that could limit their success. For example, DCAA identified the following six new performance metrics as focusing on the intended outcome-related goal of achieving quality audits that comply with GAGAS. 1. Obtaining an unqualified DOD IG peer review opinion. 2. DCAA’s internal quality assurance program results show that 100 percent of the audits reviewed reflected professional judgment. 3. Checklist confirmation that issued reports did not include serious deficiencies. 4. A goal that 45 percent of audit reports will have findings as an indication of the tangible value of the audit work performed. 5. A goal that 15 percent of the audits will use quantitative methods to measure the extent to which advanced level audit techniques are used. 6. A goal that auditors will meet 100 percent of their continuing professional education requirements on time.

Given the problems with DOD IG peer review results and DCAA’s ineffective quality assurance program, for these metrics to achieve the intended audit quality goal, significant changes will be needed. DCAA also retained three performance metrics that address issuing reports within specified times to support contract awards and closeouts, for example: 1. A forward-pricing audit timeliness goal of 95 percent based on agreement with requesters. 2. Incurred cost audit timeliness goals of 90 percent of corporate audits completed within 12 months, 90 percent of major contractor audits completed in 15 months, and 95 percent of non-major contractor audits completed in 24 months. 3. An efficiency goal of cost per direct audit hour of less than $113.45 to be monitored at the agency level only.

It is critical that agreements with the contracting community on timeliness goals for forward-pricing and incurred cost audits allow performance of sufficient audit procedures to help contracting officers ensure that prices paid by the government are fair and reasonable, and that contract costs comply with applicable laws, regulations, cost accounting standards, and contract terms.

Risk-based audit approach. DCAA did not agree to develop a risk-based audit approach, as recommended by the Defense Business Board. DCAA lacks a risk-based audit approach to address how it will perform required audits with available audit resources, reassess the need to perform 30,000 or more audits annually and the appropriate level of audit resources, and establish priorities for performing quality audits that meet GAGAS within available resources. While resources are a key element of a risk-based planning approach, DCAA is performing the Defense Business Board recommended staffing study as a stand-alone effort rather than performing this study in concert with an effort to establish a risk-based planning process.

DCAA policy guidance. DCAA’s new policy guidance on adequate audit documentation and testing does not contain sufficient instruction to assure that auditors (1) adequately document significant decisions affecting the audit objectives, scope and methodology, findings, conclusions, and recommendations and (2) perform sufficient work to support decisions to approve contractors for direct-bill status. For example, DCAA’s new policy on “Workpaper Documentation of Judgmental Selections”—requires a description of the universe (population) from which items are selected for testing, identification of items and attributes to be tested, and an explanation to support that the judgmental selection will result in adequate audit coverage.

Emphasizing the requirement that audit documentation include a description of the population used for sampling and identification of items and attributes to be tested is appropriate. However, the requirement for an explanation in the audit documentation that the judgmental selection will result in adequate audit coverage needs to be sufficiently justified. GAGAS and AICPA standards require that auditors document significant decisions affecting the audit objectives, scope and methodology, findings, conclusions, and recommendations resulting from professional judgment.

DCAA’s new policy on “Audit Guidance for Annual Testing of Contractor Eligibility for Direct Bill” is intended to determine whether continued reliance can be placed on the contractor’s procedures for preparation of interim vouchers. This policy change clarified and consolidated audit steps related to the contractor’s compliance with contract provisions, added audit steps for reviewing vouchers under time-and-material and labor-hour contracts, and removed the requirement to verify that the contractor’s Central Contractor Registration is current. The policy memorandum states that this scope of work performed does not constitute an audit or attestation engagement under GAGAS.

While it is within DCAA’s purview to determine whether these procedures constitute an audit, because direct-bill decisions present a risk of undetected improper contract payments, prudent decisions to continue a contractor’s direct-bill authorization would necessarily be based on testing a statistical sample of invoices and include a review of supporting documentation, including documentation to confirm the government received goods and services noted on the billing invoice. We confirmed that Defense Finance and Accounting Service certifying officers rely on DCAA reviews and that they do not repeat review procedures they believe to be performed by DCAA.

In addition, DCAA’s policy to eliminate the “inadequate-in-part” opinion for contractor internal control systems audits does not recognize different levels of severity of control deficiencies and weaknesses and could unfairly penalize contractors whose systems have less severe deficiencies by giving them the same opinion—”inadequate”—as contractors having material weaknesses or significant deficiencies that in combination would constitute a material weakness. DCAA would benefit from outside expertise to develop effective audit policy guidance and training on auditing standards.

Legislative and Other Actions Could Further Improve DCAA

In addition to correcting the fundamental weaknesses in DCAA’s mission and overall management environment, we believe certain legislative measures as well as other actions could enhance DCAA’s effectiveness and independence. For example, granting DCAA certain authorities and protections—similar to those offered to presidentially appointed inspectors general (IG) under the IG Act—could enhance DCAA’s independence. The IG Act contains provisions that enhance the independence of presidentially appointed IGs, including protections from removal without congressional notification, access to independent legal counsel, public reporting of audit results, rights to take statements from contractor and other personnel, and budget visibility. These provisions would enhance the important DCAA initiatives currently under way. In the longer term, Congress could consider changes in organizational placement after DCAA has had sufficient opportunity to effectively implement current reform efforts. However, moving DCAA as an organization would require careful analysis and planning before implementation. Continued monitoring and oversight will be essential to assuring the successful implementation of DCAA’s management initiatives.

Our Recommendations and DOD’s Response

Our report contains several recommendations to DOD as well as matters for congressional consideration intended to strengthen DCAA in fulfilling its contract audit responsibilities. Our report also discusses matters for congressional consideration that could enhance DCAA’s effectiveness and independence. These recommendations and matters are discussed below.

We made 17 recommendations to improve DCAA’s management environment, audit quality, and oversight, including 15 recommendations to DOD and 2 recommendations to the DOD IG regarding DCAA’s last peer review. DOD fully agreed with 13 of the 15 recommendations, partially concurred on one recommendation and did not concur with one other recommendation. We view DOD comments as being generally responsive to the intent of our recommendations.

DOD partially concurred with our recommendation that DCAA consult with DOD stakeholders and engage outside experts to develop a risk- based contract audit approach that identifies resource requirements and focuses on performing quality audits that meet GAGAS. DOD stated that DCAA already has a risk-based contract audit approach that identifies resource requirements and focuses on performing quality audits that meet GAGAS. However, DOD stated that DCAA will coordinate with the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD (AT&L)) to assess DCAA audit requirements. DOD expects to complete its assessment of stakeholder needs based on regulatory and statutory requirements by December 2010.

We appreciate these steps; however, we remain concerned that DCAA’s current approach of performing 30,000 to 35,000 audits and issuing over 22,000 audit reports with 3,600 auditors substantially contributed to the widespread audit quality problems we identified. Generating that many reports and doing that many audits with 3,600 auditors leaves very little time to perform in-depth, complex audits of contractors.

DOD did not concur on our recommendation to develop policies and procedures related to direct-billing decisions, stating that (1) the department believes that a review of the contractor’s interim public vouchers is an integral function of DCAA’s continued assessment of a contractor’s billing system, (2) DCAA is in the best position to review and approve contract interim billings based on its thorough understanding of the contractor’s system, (3) DOD believes that our concerns are mitigated based on comprehensive supervisory and audit manager reviews, and (4) DCAA does not believe that the approval of interim vouchers along with the approval for contractors to be on direct billing results in a lack of auditor objectivity.

We continue to believe that DCAA’s management (nonaudit) responsibility to perform prepayment reviews of contractor vouchers for DOD and the auditor’s decision-making role of approving contractors for direct-billing privileges based on its audit conclusions about the strength of the contractor’s system of internal controls, create audit objectivity issues. Under normal circumstances, DCAA auditors must review contractor vouchers prior to payment—a management support function for DOD. By obtaining direct-billing privileges, however, contractors can receive payment for goods and services without a voucher review by DCAA prior to payment. Because we found that this situation created an incentive for DCAA to reduce its workload by recommending that contractors are placed on direct billing, we recommended that DCAA develop new policies and procedures to ensure a separation between staff reviewing vouchers and staff making direct-bill decisions. DCAA did not explain the basis for its belief that DCAA administrative staff have a thorough understanding of the contractors’ systems. Further, we disagree with DOD’s statement that our concerns are mitigated based on the comprehensive supervisory and audit manager reviews because this is not supported by our findings. The fact that DCAA approvals of contractor direct-bill privileges were not based on sufficient audit procedures as demonstrated by our work and DCAA’s removal of over 200 contractors from the direct-bill program since our July 2008 report support our concern that the existence of such an incentive presents an objectivity impairment.

With regard to our two recommendations to the DOD IG, the IG concurred on our recommendation to reconsider the overall conclusions in its May 2007 peer review report on its audit of DCAA’s system of quality control. However, the IG did not agree with our recommendation to determine whether the report should be rescinded or modified and did not take action to do so. The IG comments stated that the IG took alternative action that conformed to the intent of our recommendation, including notification of DCAA on August 24, 2009, that the May 2007 “adequate” opinion on DCAA’s system of quality control would expire on August 26, 2009. In addition, the IG stated, “We have determined that it is not prudent to allow the adequate opinion from our May 2007 report to carry forward.” However, peer review opinions neither “expire” nor “carry forward” beyond the period covered by the peer review. Based on the significant audit quality deficiencies identified in the IG peer review report, DCAA’s decision to rescind 80 audit reports—39 of which relate to the period of the IG’s peer review—and the findings in our audit, we concluded that DCAA’s quality control system for the period covered by the DOD IG peer review was not effectively designed and implemented to provide assurance that DCAA and its personnel comply with professional standards.

DOD also provided comments on our matters for congressional consideration. Although DOD disagreed with the matters we discussed, we continue to believe these are valid matters for congressional consideration. The IG Act provides many important authorities and protections for IG’s that could enhance DCAA’s independence and effectiveness. Further, if DCAA is unsuccessful in addressing our recommendations for resolving fundamental weaknesses in its mission and the overall management environment under the current organizational placement, additional options would need to be considered.

Mr. Chairman and Members of the Committee, this concludes my statement. We would be pleased to answer any questions that you may have at this time.

Contacts and Acknowledgments

For further information about this testimony, please contact Gregory D. Kutz at (202) 512-6722 or kutzg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. Major contributors to our testimony include F. Abe Dymond, Assistant General Counsel; Richard Cambosos; Jeremiah Cockrum; Andrew McIntosh; and Lerone Reid.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.