**EXTERNAL PEER REVIEW GUIDE**

#### **PREFACE**

This document presents the guidance for conducting external peer reviews
of the audit operations of Offices of Inspector General (OIG). This
guide was developed to ensure the adequacy and consistency of the
reviews in accordance with the 2005 policy statement issued by the Audit
Committee. The guidance contained herein is not intended to supplant the
review team\'s professional judgment as to what approach to take or what
specific procedures need to be performed. The Audit Committee welcomes
any suggestions for further improving the external review program**.**

##### Table of Contents

**PREFACE** i

**ACRONYMS** iii

**GENERAL CONSIDERATIONS**

Definitions 1

Objective of the External Peer Review Program 1

Independence of the External Peer Review Team 1

Professional Judgment of the External Peer Review Team 2

External Peer Review Staff 2

Documentation Requirements 2

Initiation of the Review and Administrative Records 3

Memorandum of Understanding (MOU) 3

PLANNING AND PERFORMING THE EXTERNAL PEER REVIEW

Scope of the Review 5

Presite Review Steps 5

Review Approach 6

Understanding the Internal Quality Control Policies and Procedures 6

Review of the Internal Quality Assurance Program 7

Selection of Offices and Individual Audits 7

Nature and Extent of Tests 9

Review of Individual Audits 9

**REPORTING REVIEW RESULTS**

General 11

Rendering an Opinion 11

Reports on External Peer Reviews 11

Agency Response 12

Report Distribution and Follow-Up 13

APPENDICES

A OIG's Audit Quality Control Policies and Procedures

B Checklist for Review of Adherence to Selected General Standards

C Checklist for Assessment of the Internal Quality Assurance Program

D Checklist for Review of Individual Financial Statement Audits
Performed by Inspectors General

E Checklist for Review of Individual Performance Audits

F Checklist for Review of OIG Monitoring of Audits Performed by
Independent Public Accountants

G Checklist for Review of Attestation Engagements

##### 

  -----------------------------------------------------------------------
  **ACRONYMS**

  -----------------------------------------------------------------------

  ---------------- ------------------------------------------------------
  AICPA            American Institute of Certified Public Accountants

                   

  AT               Attestation Standards

                   

  AU               Codification of Statements on Auditing Standards

                   

  CPE              Continuing Professional Education

                   

  ECIE             Executive Council on Integrity and Efficiency

                   

  GAO              Government Accountability Office

                   

  GAGAS            Generally Accepted Government Auditing Standards

                   

  IPA              Independent Public Accountant

                   

  IT               Information Technology

                   

  OIG              Office of Inspector General

                   

  OMB              Office of Management and Budget

                   

  PCIE             President's Council on Integrity and Efficiency

                   

  SAS              Statements on Auditing Standards

                   

  SFFAS            Statements of Federal Financial Accounting Standards

                   
  ---------------- ------------------------------------------------------

##### [GENERAL CONSIDERATIONS]{.underline}

###### DEFINITIONS

1.  **Internal Quality Control System.** The methods and measures
    designed and implemented to collectively provide reasonable
    assurance that the applicable auditing standards, policies, and
    procedures were met. The system should generally reflect four
    characteristics: (1) policies and procedures that establish internal
    guidance and requirements; (2) supervisory review processes; (3) a
    quality control process for each audit (such as independent
    referencing); and, (4) a quality assurance program.

2.  **Quality Assurance Program.** An internal evaluation program that
    is performed by Office of Inspector General (OIG) personnel (either
    by personnel in the audit office or personnel in another component
    of the OIG) and is designed to assess whether the OIG carries out
    its work in accordance with established policies and procedures,
    generally accepted government auditing standards (GAGAS), Office of
    Management and Budget (OMB) and Government Accountability Office
    (GAO) guidance, and statutory provisions applicable to the OIG.

3.  **External Peer Review.** A review of an OIG's audit function by a
    Federal audit organization not affiliated with the OIG being
    reviewed that satisfies the objective specified below.

4.  **Audits.** For the purpose of providing guidance on the external
    peer review process, the term audits pertains to both audit and
    attestation engagements performed in accordance with GAGAS.

5\. Nonaudit Services. Tasks performed by an OIG at the request of
agency management that directly supports the entity's operations or
gathering of information or data at the request of a party without
providing verification, analysis, or evaluation of the information or
data.

######### OBJECTIVE OF THE EXTERNAL PEER REVIEW PROGRAM

**6.** The objective of the PCIE/ECIE external peer review program is to
determine whether the reviewed audit organization's internal quality
control system was adequate and complied with to provide reasonable
assurance that applicable auditing standards, policies, and procedures
were met. The program is intended to be positive and constructive and
should be carried out in that spirit.

###### INDEPENDENCE OF THE EXTERNAL PEER REVIEW TEAM

**7.** The reviewing OIG and review team members should meet the GAGAS
independence criteria.

######### PROFESSIONAL JUDGMENT OF THE EXTERNAL PEER REVIEW TEAM 

**8.** The review team should exercise professional judgment in all
matters relating to planning, performing, and reporting the results of
the external peer review.

**[EXTERNAL PEER REVIEW STAFF]{.underline}**

**9.** The number of staff assigned to the review team is dependent on a
number of factors including, but not limited to, the size and geographic
dispersion of the OIG being reviewed, and the nature and extent of its
audit universe. The review should be adequately staffed to complete the
review in a timely manner.

**10.** The team leader should be an experienced manager with the
appropriate audit background. The team should consist of senior auditors
or audit managers. The staff assigned to conduct the review collectively
should possess adequate professional proficiency for the tasks required.
At a minimum, each team member should have current knowledge of GAGAS,
the government environment relative to the work being reviewed, and
knowledge of how to perform a peer review. Members of the review team
should be selected from one OIG, or an ad hoc review team may be
assembled, comprised of representatives from various OIG audit
organizations.

**11.** Team members should be selected based on the types and
complexity of audits to be reviewed and any specialized skills that may
be needed (for example, information technology (IT) specialists,
statisticians, or auditors with financial statement skills). Also, when
OIGs use electronic audit documentation to support their audits, review
teams should be capable of reviewing such work and plan accordingly, to
include obtaining training and any software needed. Because of these
considerations, final decisions on team composition may be deferred
until after some preliminary planning and presite procedures have been
completed. Nothing in this section should be construed to limit the
flexibility of the team leader in planning and performing the
review**.**

######### DOCUMENTATION REQUIREMENTS

**12.** Documentation should be prepared to support the work performed
and the conclusions reached during the course of the review. The
checklists included in this guide are available in electronic format on
the IGNet [(http://www.ignet.gov/)]{.underline}.

**13.** The documentation should be retained by the reviewing OIG at
least until the subsequent external review is completed. The external
review documentation should be subject to the same custody and physical
security policies that the reviewing OIG applies to its audit
documentation. At a minimum, these policies should include safeguards
against unauthorized use or access to the documentation.

**[INITIATION OF THE REVIEW AND ADMINISTRATIVE RECORDS]{.underline}**

**14.** An engagement letter should be forwarded by the reviewing OIG
announcing the initiation of the review and requesting a formal entrance
conference. The engagement letter should also contain requests for the
information delineated in section 20 to be provided at the entrance
conference (sufficient time should be allotted for the compilation of
the data). The entrance conference should be used to bring the parties
together, establish the ground rules of the review, and facilitate the
conduct of the review. At that time, the OIG under review should brief
the review team on organizational issues and work practices (e.g., roles
and responsibilities of the audit divisions, the use of electronic audit
documentation, etc.), the level of clearance/access and any training
that may be required at the outset of the review to facilitate
preparation and planning. The proposed elements of the Memorandum of
Understanding (MOU) (see section 17) should be discussed during the
entrance conference as well.

**15.** Adequate work space should be provided for the review team. If
travel is necessary to accomplish the objectives of the review, the
reviewing OIG should pay its own travel expenses. If the team is made up
of members of different OIGs, the team members' OIG should pay their
travel expenses.

**16.** The reviewing OIG should maintain administrative records on the
staff days and calendar days it takes to do the review and the travel
and other costs incurred.

**[MEMORANDUM OF UNDERSTANDING (MOU)]{.underline}**

**17.** A MOU should be prepared to ensure mutual agreement regarding
the fundamental aspects of the review and to avoid any
misunderstandings. The MOU should be issued subsequent to the entrance
conference and signed off on by both parties prior to the initiation of
fieldwork. The following topics should be covered in the MOU:

> a\. **Scope of the Review.** See Section 19 of this Addendum.
>
> b\. **Staffing and Timeframe.** The MOU should set forth the planned
> staffing and time frames. The review should be scheduled and conducted
> to ensure a report is issued within 3 years of the date of the
> previous peer review report and be designed to be performed
> efficiently in order to minimize the impact on the reviewed OIG's
> operations.
>
> c\. **Nonaudit Services.** The MOU should state that the reviewed OIG
> will provide, in writing, a description and a listing of all nonaudit
> services rendered within the prior 3 years that related to individual
> audits selected for review during the current peer review. The
> reviewed OIG should also provide related audit documentation required
> as independence safeguards by GAGAS 3.17a through 3.17e.
>
> d\. **Preliminary Findings.** The MOU should provide for timely
> interim discussion of preliminary findings. A commitment to open and
> ongoing communication between the parties will be stressed.
>
> e\. **Reporting Results.** The MOU should establish the guidelines for
> the reporting process, i.e., (1) designating the report's addressee
> and signer (e.g., draft to and from the respective Assistant
> Inspectors General for Audit and final to and from the respective
> Inspectors General), (2) providing a draft report, (3) scheduling the
> exit conference, (4) designating a period of time for a response to
> the draft report, and (5) issuing the final report.

**18.** Other topics may be covered, as needed. When preparing MOUs,
care should be taken not to limit the review team's ability to conduct
the work necessary to accomplish the objectives of the review.

#########  

######### PLANNING AND PERFORMING THE EXTERNAL PEER REVIEW

######### SCOPE OF THE REVIEW

19\. The scope of the external peer review typically consists of the
audit operations conducted over the period of time covered by the two
most recent semiannual reports to Congress, but may be expanded as
deemed necessary by the review team. The peer review may also include
other areas, with mutual agreement between the OIG being reviewed and
the external peer review team. GAGAS recognizes that the nature and
extent of an audit organization's internal quality control system
depends on a number of factors, such as its size. OIG organizations must
nonetheless have a system in place to provide reasonable assurance that
GAGAS is met on an ongoing basis. It is recognized, however, that the
system may be composed of a structure of compensating controls to
fulfill these requirements within the restrictions imposed by staffing
sizes. The guidance contained herein should be modified and adapted as
appropriate to conform to this structure.

**[PRESITE REVIEW STEPS]{.underline}**

**20.** The following steps should be performed prior to the site visit.

> a\. **Audit Quality Control Policies and Procedures.** Reviewed OIGs
> should be requested to complete section 1 of Appendix A, OIG's Audit
> Quality Control Policies and Procedures and provide a complete set of
> its policies and procedures, prior to the initiation of the review.
>
> b\. **Semiannual Reports to Congress.** The review team should request
> a copy of the two most recent semiannual reports to Congress. The
> semiannual reports provide information regarding the nature and volume
> of completed audit work as well as other matters that may help the
> review team understand the environment in which the reviewed OIG
> operates. The reports should also serve as a source for selecting
> individual audits for review.
>
> c\. **Other Information or Documentation.** The review team should
> obtain and review the annual audit plan(s) for the period covered, a
> printout of the audit tracking system of the specific information
> needed (e.g., audits scheduled, cancelled, terminated, or completed
> during the period), an organization chart, a roster of staff
> (including series and grades), professional designations, and a
> continuing education summary for all staff for the most recent 2-year
> reporting period. If it is readily available, obtain information
> regarding the staff's advanced degrees or special skills (if the
> information is not readily available, request this data, as needed,
> after the individual audits to be reviewed have been selected).
>
> d\. **Prior External Peer Review.** Copies of the final report and
> letter of comments (for external peer reviews performed prior to 2005)
> and the documentation of the review performed, to include copies of
> documents, as needed, should be obtained from the prior external peer
> review team. A written description of the corrective action taken in
> response to the prior external peer review should be obtained from the
> reviewed organization.
>
> e\. **Prior Internal Quality Assurance Review Reports.** The review
> team should obtain the internal quality assurance review reports
> issued during and subsequent to the external peer review period.

######### 

######### REVIEW APPROACH

**21.** The approach advocated by this guide is to:

> a\. Gain an understanding of the reviewed OIG's audit operation and
> its internal quality control system.
>
> b\. Evaluate the reviewed OIG's policies and procedures designed to
> provide reasonable assurance that GAGAS and other pertinent
> requirements are met and conduct the examination accordingly (test for
> compliance if adequate as prescribed by modifying the checklists
> provided herein, as needed).

c.  Interview various levels of the professional staff to assess their
    understanding of and compliance with relevant quality control
    policies and procedures.

d.  Gain an understanding of the OIG's internal quality assurance
    program, evaluate its design, and assess internal quality assurance
    reports to determine the adequacy of the program and the degree of
    control provided in the OIG's overall internal quality control
    system.

e.  Review individual audits, emphasizing the quality control measures
    in place and adherence thereto for each audit.

f.  Maintain open communication to ensure understanding of the issues
    evaluated and to keep the reviewed OIG fully informed of potential
    issues as they arise.

**[UNDERSTANDING THE INTERNAL QUALITY CONTROL POLICIES AND
PROCEDURES]{.underline}**

**22.** Based on a review and evaluation of policies and procedures,
supplemented as necessary by an inquiry of management, the review team
should complete section 2 of Appendix A. The purpose of this analysis is
twofold:

a.  To scope the review, if the organization's policies and procedures
    are determined to be adequate as prescribed, specific review
    procedures should be designed as needed to test compliance (to
    include modifying the checklists provided, if necessary). If not,
    the checklists provided herein should be used, and

b.  To gain insight as to the adequacy of the organization's overall
    internal quality control system.

**[REVIEW OF THE INTERNAL QUALITY ASSURANCE PROGRAM]{.underline}**

**23.** The purpose of reviewing the internal quality assurance program
is to determine whether the program is adequately designed and
implemented and serves as a segment of the overall internal quality
control system. The PCIE "Silver Book" provides a detailed framework as
to how this critical quality control system feature should be organized
and carried out. Appendix C contains guidance on assessing the design of
the internal quality assurance program and on reviewing internal quality
assurance reports. A representative sample of the quality assurance
reports issued covering the external peer review period should be
assessed. The results of this assessment should be considered in
determining the adequacy of the overall internal quality control system.

######### SELECTION OF OFFICES AND INDIVIDUAL AUDITS

**24.** In selecting offices and reports for review, the team should
consider the following:

a.  The assignments listed in the audit tracking system.

```{=html}
<!-- -->
```
a.  Audits appearing (described) in the Semiannual Reports to Congress.

b.  The degree of centralized controls in place.

c.  The number of OIG offices.

```{=html}
<!-- -->
```
e.  The selection of a sufficient number of audits to enable the
    external peer review team to reach a defendable conclusion as to the
    adequacy of the internal quality control system.

f.  Findings/comments from the prior external peer review report.

g.  At least one audit that had been reviewed by the internal quality
    assurance review team.

h.  At least one financial statement audit.

i.  Audits which were related to nonaudit services.

j.  The selection of additional audits for review to substantiate the
    pervasiveness and impact of the deficiencies on the system as a
    whole if, during the course of the review, deficiencies in the
    organization's quality control system were noted which could result
    in a less than unmodified opinion.

k.  A reasonable cross section of the assignments performed.

**25.** The external peer review team will apply a "no advance notice"
policy in advising the OIG of the specific individual audits selected
for review. The review team should advise the OIG of the specific audits
selected for examination only when it is ready to initiate the review of
the individual audits. The OIG should provide immediate access to all
audit documentation requested by the review team. When the review team
plans to conduct field visits (regional offices, sub-offices, etc.), the
field offices should be advised of the specific audits selected for
review upon the review team\'s arrival.

> If the OIG cannot provide the requested documentation within 2 working
> days, it should complete the following attestation of the completeness
> of the audit documentation upon its delivery.

**CERTIFICATION OF AUDIT DOCUMENTATION**

**Agency:
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_Audit
Number: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

**Audit Title:
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

**Certification:** I certify that:

**\[ \]** To the best of my knowledge, the audit documentation provided
to the \[Name of OIG Office\] external peer review team is complete and
no changes were made since the report was issued.

**\[ \]** Documentation is not complete and/or changes or additions have
been made since the report was issued. Attached is an explanation as to
why the audit documentation is incomplete or what and why the changes
and/or additions were made, with the effected audit documentation
specifically identified.

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

**Assistant Inspector General for Audit Date**

**[NATURE AND EXTENT OF TESTS]{.underline}**

**26.** The nature and extent of tests employed throughout should be
sufficiently comprehensive to provide a reasonable basis for concluding
whether the OIG's overall internal quality control system is adequate.
The extent of work performed should be limited, or expanded as
necessary, to achieve that level of assurance.

######### REVIEW OF INDIVIDUAL AUDITS

**27.** The purpose of reviewing individual audits is to determine
whether established policies, procedures, and applicable professional
standards were followed. Appendix D contains a checklist for the review
of individual financial audits, Appendix E contains a checklist for the
review of individual performance audits, and Appendix G contains a
checklist for the review of individual attestation engagements.

**28.** For audits performed by IPAs under direct contract with the OIG,
or the agency where the OIG has assumed responsibility for the quality
of the audit, the review should determine whether the OIG has issued and
implemented quality control policies and procedures for assuring the
work meets standards and contractual requirements. Appendix F contains a
checklist for reviewing the OIG's monitoring of these audits.

**29.** The review of individual audits should include a review of the
auditors' reports, audit documentation, and discussions with
professional staff of the OIG.

**30.** Ordinarily, the review of individual audits should be conducted
onsite (at the office which performed the audit).

**31.** After completing a checklist for each audit reviewed, findings
should be developed and conclusions formulated.

a\. Summarize the checklists' results.

> b\. Identify significant deficiencies (noncompliance with GAGAS and/or
> the OIG's quality control policies and procedures) in the individual
> audits reviewed which could impact the external peer review report's
> opinion.

c.  Identify any other matters that warrant disclosure to the officials
    of the reviewed agency.

d.  Assess the overall adequacy of the implementation of the
    organization's internal quality control system.

**32.** After all evidence has been compiled, the adequacy of the scope
of the external peer review should be re-assessed and expanded upon, if
necessary, to ensure the sufficiency criterion has been met.

##### [REPORTING REVIEW RESULTS]{.underline}

######### GENERAL

**33.** The process for reporting must be discussed and agreed to prior
to the start of the review. At a minimum, the process should provide for
the reviewed OIG to comment on the draft report prior to the issuance of
the final report. The review team must consider the comments before
finalizing the report.

**34.** A written report should be issued at the completion of the
review. The external peer review report should contain the reviewing
OIG's opinion as to whether the OIG's overall internal quality control
system provided reasonable assurance of compliance with professional
auditing standards, principles, and procedures in the conduct of its
audits.

**[RENDERING AN OPINION]{.underline}**

**35. Types of Opinions.** There are four types of opinions that may be
rendered: unmodified, modified, adverse, and disclaimer. The opinions
must be supported by strong and convincing evidence. In forming its
opinion, the review team should consider the nature and extent of the
evidence compiled taken as a whole to provide a reasonable basis for
formulating the opinion. Foremost, however, the determination of what
opinion should be rendered is a matter of professional judgment.

## [REPORTS ON EXTERNAL PEER REVIEWS]{.underline}

**36. Report Content.** If a modified or adverse opinion is issued, the
report must include a description of the findings which support the
opinion. As noted above, conclusions should be reached based upon the
cumulative impact of all of the findings. Issues may be disclosed,
however, that did not affect the opinion, neither singularly nor in the
aggregate, but nonetheless warrant reporting (in that formal corrective
action needs to be taken). In these cases, the findings should be
segregated by the subheading "Reportable Conditions Not Affecting the
Opinion." This subheading is not required when an unmodified opinion is
rendered but there are findings nonetheless. Generally, findings should
include the elements of criteria, condition, cause, and effect. Findings
should also be supported by sufficient, competent, and relevant
evidence. Recommendations should be made to rectify the causal factors
noted in the findings. Recommendations should be constructive, action
oriented, specific, achievable, and cost effective. Presentation of
findings should be complete, fair and conveyed in a positive and
constructive manner. Matters of lesser importance that are nonetheless
of interest to management should be conveyed orally.

**37.** The draft and final written report should contain:

a\. A transmittal letter.

> b\. An opinion as to whether the OIG's internal quality control system
> was designed and was complied with during the period under review to
> provide reasonable assurance that applicable auditing standards,
> policies, and procedures were met in the conduct of its audits.
>
> c\. Findings and recommendations (if applicable) to include emphasis
> on deficiencies that recurred from the OIG's previous external review.
>
> d\. An exhibit that describes the external peer review scope and
> methodology, including a list of the audit reports reviewed and the
> OIG offices visited. Include any limitations and/or expansions of the
> scope, if applicable.
>
> e\. In keeping with the constructive nature of the external peer
> review program, the team should report any noteworthy accomplishments
> or best practices found during the review such as particularly
> creative, efficient, and effective audit approaches, in an exhibit.
>
> f\. An exhibit which contains the reviewed organization's response, in
> its entirety, to the external peer review report.

(Addendum 3 contains instructions on the report format, guidance on
formulating opinions, standardized report language, and illustrations of
findings and recommendations).

**[AGENCY RESPONSE]{.underline}**

**38. Views of Responsible Officials.** To ensure the objectivity,
accuracy, and completeness of the findings, the review team should
obtain the views of responsible officials. When deficiencies are found
during the course of the review, the team must discuss the issues with
senior field office audit management and staff, or the responsible
official(s) designated by the reviewed OIG. All preliminary draft
findings and conclusions must be presented during the review to the
official(s) designated by the reviewed OIG to avoid any
misunderstandings and to help assure that all material facts are
considered before a draft report is prepared. These disclosures may be
conveyed informally but should be in writing to facilitate agreement
regarding the conditions noted. Upon issuance of the initial draft
report, an exit conference should be requested and held, modifications
made to the report as necessary, and then a formal draft conveyed with a
request for written comments. The final report should be revised, or the
response rebutted as necessary, throughout the text. The written reply
should be included in the final report as an attachment in its entirety.

## [REPORT DISTRIBUTION AND FOLLOW-UP]{.underline}

39\. The decision to implement any recommendations in the report rests
solely with the reviewed OIG; for those recommendations implemented, the
reviewed OIG is responsible for the resolution and follow-up of
corrective actions. Follow-up on implemented recommendations should also
occur during the course of the subsequent external peer review.

**40.** The reviewed OIG must provide copies of the final review report
to the head of its agency, the Chair and Vice Chair of the PCIE, and the
Chair of the PCIE's Audit Committee.[^1] In addition, the ECIE
organizations should provide a copy of the report to the Vice-Chair of
the ECIE. It is recommended that, upon request, the peer review report
be made available to the Government Accountability Office and the public
in a timely manner.

[^1]: The Vice Chair of the PCIE and the ECIE will prepare schedules of
    completed peer reviews, and advise the appropriate congressional
    committees that the opinion report is available on request from the
    reviewed OIGs.