**Qualitative Assessment Review Guidelines**

**for**

**Federal Offices of Inspector General**

**President's Council**

**on Integrity and Efficiency**

**Executive Council**

**on Integrity and Efficiency**

**January 2008**

**[ACKNOWLEDGEMENTS]{.underline}**

These guidelines originally were developed by representatives from over
fifteen Offices of Inspector General. This version was spearheaded by:

> \- Department of Agriculture
>
> \- Department of Energy
>
> \- Environmental Protection Agency
>
> \- National Science Foundation
>
> \- Department of State
>
> \- Veterans Affairs

The following references were reviewed during the development of this
document:

1.  1\. The Inspector General Act of 1978, as amended.

    2\. PCIE/ECIE Quality Standards for Investigations (PCIE/ECIE QSI)
    (September 1997) issued by the President's Council on Integrity and
    Efficiency (PCIE) and the Executive Council on Integrity and
    Efficiency (ECIE).

    3\. "Investigative Peer Review" (January 1997) issued by the
    Inspector General Criminal Investigator Academy.

    4\. Quality Standards for Federal Offices of Inspector General
    (October 2003) issued by the President's Council on Integrity and
    Efficiency (PCIE) and the Executive Council on Integrity and
    Efficiency (ECIE).

    5\. "Attorney General Guidelines for Offices of Inspector General
    with Statutory Law Enforcement Authority" (December 8, 2003) issued
    by the Office of the Attorney General, United States Department of
    Justice, which state, in part, "the purpose of the \[peer\] review
    is to ascertain whether adequate internal safeguards and management
    procedures exist to ensure that the law enforcement powers conferred
    by the 2002 amendments to the Inspector General Act are properly
    exercised."

    6\. The Memorandum of Understanding between the Offices of Inspector
    General, the Criminal Division of the Department of Justice, and the
    Federal Bureau of Investigation.

    7\. "Quality Assurance Review Guide for Investigations" (July 1995)
    issued by the Department of Agriculture Office of Inspector General.

    8\. "Guide for Conducting External Quality Control Reviews of the
    Audit Operation of Offices of Inspector General" (February, 2002)
    issued by the PCIE Audit Committee.

**An electronic version of this guide can be located on the web at:**

[**http://www.ignet.gov/**](http://www.ignet.gov/)

(under 7 Committees/Investigations)

**[Table of Contents]{.underline}**

**[Page]{.underline}**

**ACKKNOWLEDGMENTS** 2

**PREFACE** 5

**GENERAL CONSIDERATIONS** 5

1\. Applicability 5

2\. Background 6

3\. Objectives of the Investigative Qualitative Assessment Review (QAR)
Program 7

4\. Management and Oversight of PCIE/ECIE QAR Program 7

5\. Review Team Staffing and Qualifications 7

6\. Independence 8

7\. Confidentiality and Security 8

8\. Due Professional Care 9

9\. Self-Inspection Programs 9

**PLANNING AND PERFORMING THE INVESTIGATIVE QUALITATIVE ASSESSMENT
REVIEW** 10

1\. Scope 10

2\. Approach 10

3\. Pre-Site Review Steps 11

4\. Working Environment 11

5\. Review Schedule 12

6\. Entrance Briefing 12

7\. Sample Selection 13

8\. Assessment Options 14

8\. Views of Responsible Officials 15

10\. Exit Conference 15

**REPORTING REVIEW RESULTS** 16

1\. Opinion Letter 16

2\. Letter of Observations 16

3\. Views of responsible Officials 17

4\. Dispute Resolution 17

5\. Letter Distribution 18

6\. Files Maintenance 18

**APPENDICES**

-   Qualitative Assessment Review Organizational Profile A-1

-   Questionnaire for Review of Law Enforcement Powers Implementation
    B-1

-   Questionnaire for Review of Compliance with the PCIE/ECIE Quality
    Standards for Investigations C-1

-   Questionnaire for Review of Conformity with Quality Standards for
    Investigations (Computer Forensics) C-2

-   Individual Closed Case Review Checklist D-1

-   Case Review Summary Checklist D-2

-   Sample Formats for Quality Assessment Review Reports E-1

-   Attorney General Guidelines for Offices of Inspector General with
    Statutory Law Enforcement Authority F-1

-   Attorney General's Guidelines on General Crimes, Racketeering
    Enterprise and Terrorism Enterprise Investigations G-1

-   Attorney General's Guidelines Regarding the Use of Confidential
    Informants H-1

-   Attorney General's Memorandum on Procedures for Lawful, Warrentless

> Monitoring of Verbal Communications I-1

-   PCIE/ECIE Quality Standards for Investigations J-1

**[PREFACE]{.underline}**

This document includes standards and guidance for conducting external
quality assessment reviews of the investigative operations of Offices of
Inspector General (OIG). It was developed by the PCIE Investigations
Committee to establish an independent external review process to:

1.  Ensure general and qualitative standards adopted by OIGs comply with
    the requirements of the Quality Standards for Investigations
    (PCIE/ECIE QSI) adopted by the PCIE/ECIE. This item will be assessed
    for all PCIE/ECIE organizations.

2.  Ascertain whether adequate internal safeguards and management
    procedures exist to ensure that the law enforcement powers conferred
    by the 2002 amendments to the Inspector General Act are properly
    exercised pursuant to Section 6(e) of the Inspector General Act (as
    amended) and the United States Attorney General Guidelines for
    Offices of Inspector General with Statutory Law Enforcement
    Authority. This item will be assessed during peer reviews only for
    agencies with statutory law enforcement authority under the
    Inspector General Act of 1978, as amended.

This Guide may be adapted for organizations' internal reviews (self
assessments) within the PCIE/ECIE communities. It also provides
guidelines for reviewing investigative processes and records maintenance
in any OIG investigative operation.

**[GENERAL CONSIDERATIONS]{.underline}**

**1.** **Applicability.** This guide applies to PCIE/ECIE Qualitative
Assessment Reviews (QAR), also known as peer reviews, for investigative
operations of the OIGs. The following questionnaires and checklists were
developed to assist in conducting the review of an organization.

-   Appendix A is a profile sheet of administrative data about the
    organization being reviewed.

-   Appendix B is a questionnaire to assess whether adequate internal
    safeguards and management procedures exist within those Offices of
    Inspector General that exercise law enforcement powers pursuant to
    Section 6(e) of the Inspector General Act (as amended) and the
    United States "Attorney General's Guidelines for Offices of
    Inspector General with Statutory Law Enforcement Authority."

-   Appendix C1 is a questionnaire to assess compliance with the general
    and qualitative standards outlined in the PCIE/ECIE QSI. Appendix C2
    is a questionnaire to assess conformity with computer forensics
    standards.[^1]

-   Appendix D1 and D2 are checklists used to sample closed
    investigative case files when testing the degree of compliance with
    the Attorney General Guidelines and/or the Quality Standards
    mentioned above.

-   Appendix E includes sample formats for reporting PCIE/ECIE QAR
    findings.

-   Appendix F is a copy of the "Attorney General Guidelines for Offices
    of Inspector General with Statutory Law Enforcement Authority."

-   Appendix G is a copy of the "Attorney General's Guidelines on
    General Crimes, Racketeering Enterprise and Terrorism Enterprise
    Investigations."

-   Appendix H is a copy of the "Attorney General's Guidelines Regarding
    the Use of Confidential Informants."

-   Appendix I is a copy of the Memorandum from the Attorney General on
    "Procedures for Lawful, Warrentless Monitoring of Verbal
    Communications."

-   Appendix J is a copy of the "PCIE/ECIE Quality Standards for
    Investigations, December 2003."

**2. Background.** This guide is based on the Inspector General Act of
1978 (as amended), the PCIE/ECIE QSI (December 2003), and the "Attorney
General Guidelines for Offices of Inspector General with Statutory Law
Enforcement Authority" (December 8, 2003).

> The Inspector General Act of 1978 (as amended) and the Inspector
> General Act Amendments of 1988 established statutory Offices of
> Inspector General (OIGs) in nearly 60 Federal establishments and
> entities, including all cabinet departments and the largest Federal
> agencies, smaller boards, commissions, corporations, and foundations.
>
> The PCIE/ECIE QSI were developed by the PCIE/ECIE. The Standards
> contain three general standards and four qualitative standards. The
> general standards (Qualifications, Independence, and Due Professional
> Care) apply to investigators and the organizational environment in
> which they perform. The qualitative standards (Planning, Execution,
> Reporting, and Information Management) apply to the management
> functions and processes that investigators perform.
>
> The "Attorney General Guidelines for Offices of Inspector General with
> Statutory Law Enforcement Authority" govern the exercise of statutory
> police powers by Inspectors General and eligible employees and the
> role of Federal prosecutors in providing guidance in the use of
> sensitive criminal investigative techniques.

**3.** **Objectives of the Investigative Qualitative Assessment Review
Program.** The overall objective of a PCIE/ECIE QAR, or peer review, is
to determine whether internal control systems are in place and operating
effectively to provide reasonable assurance that professional
investigative standards are being followed. This assessment program is
intended to be positive and constructive rather than negative or
punitive. With this in mind, the review team is encouraged to identify
"best practices" or similar notable positive attributes of the
organization. Additionally, the review team should view favorably
on-the-spot corrections to non-systemic potential weaknesses. Further,
the team must consider the extent to which the reviewed OIG had/has
control over a potential weakness (e.g., agency is responsible for a
particular process such as inventory control, encryption, background
investigations, etc.).

> These Guidelines are applicable to a diverse set of Federal and
> non-Federal organizations including all cabinet departments and the
> largest Federal agencies, smaller boards, commissions, corporations,
> and foundations. Reviewing OIGs should be cognizant of the structure
> of the organization they are reviewing and how each has adapted to the
> unique circumstances of that respective department or agency.
> Reviewing OIG's should adapt the Guidelines as appropriate.

**4.** **Management and Oversight of PCIE/ECIE QAR Program.** The PCIE
Investigations Committee has responsibility for overall management and
oversight of the PCIE/ECIE QAR process. This committee will resolve all
issues that cannot be mutually agreed upon by the PCIE/ECIE QAR team and
any office being reviewed.

> The Chairperson of the PCIE Investigations Committee is responsible
> for arranging a matching process to ensure that an OIG is subject to a
> PCIE/ECIE QAR no less than once every three years. The selection of
> assessment partners should be made through a random process such as a
> lottery or a rotating roster of participants. The PCIE/ECIE QAR
> schedule should be distributed on an annual or biennial basis to allow
> appropriate lead-time for OIGs to plan their participation.
>
> The function of the PCIE/ECIE QAR is considered inherently
> governmental. The process should be handled within the Inspector
> General (IG) community and not contracted externally.

**5. Review Team Staffing and Qualifications.** Conducting a PCIE/ECIE
QAR review requires considerable professional judgment and leadership.
The PCIE/ECIE QAR team will consist of a team leader with appropriate
investigative background and experience. It is recommended (but not
mandated) that the team leader be at or above the GS-15 grade level (or
equivalent). The rest of the team will consist of OIG investigators and
an administrative support staff from one or more OIGs, as deemed
necessary.

> The team size and composition may vary depending on a number of
> factors including, but not limited to: the size and geographic
> dispersion of the OIG being reviewed; changes in organizational
> structure, control, and leadership; and the number, type, and
> importance of reports issued at each field location or satellite
> office.
>
> If the organization under review handles classified information,
> members of the assessment team must have the appropriate level of
> security clearance(s) to permit a complete PCIE/ECIE QAR without undue
> impact on the quality of the review.

**6.** **Independence.** The review team members and their senior
management should meet the independence standards in the Quality
Standards for Federal Offices of Inspector General and the PCIE/ECIE
QSI. To avoid any appearance of bias, care should be taken to ensure
that the PCIE/ECIE QAR team members do not have personal or professional
relationships with the officials in the OIG being reviewed.
Specifically, the PCIE/ECIE QAR team members should not be former
employees of the OIG organization. The OIG managing a PCIE/ECIE QAR
cannot review an office that conducted its most recent PCIE/ECIE QAR.
Questions or concerns related to the composition of a particular QAR
team should first be raised with the IG of the review team. If these
issues cannot be resolved they can be raised with the PCIE
Investigations Committee.

1.  

**7. Confidentiality and Security.** The PCIE/ECIE QAR team should
safeguard all privileged, confidential, and national security or
classified information in compliance with applicable laws, regulations,
and professional standards.

> All matters discussed, materials assembled, documents prepared, and
> reports generated through an external PCIE/ECIE QAR should, at a
> minimum, be treated as proprietary information and maintained
> appropriately. To the extent possible, privileged and confidential
> information such as names and other personally identifying information
> should not be recorded in reports issued by the PCIE/ECIE QAR team.
> The team leader must ensure that the team complies with relevant
> professional guidance on the use, protection, and reporting of
> information such as classified material, Internal Revenue Service tax
> information, and protection of grand jury material and information.
>
> It is possible that the review team may not be granted access to
> sensitive material because of legal restrictions. If this situation
> occurs, the review team should review the system related to the
> maintenance and protection of information to determine the adequacy of
> established procedures. Discussion among review team members of any
> information obtained during an external review is limited to a
> need-to-know basis.

**8. Due Professional Care.** The review team should strive to achieve
quality performance by exercising due professional care and sound
professional judgment in planning, performing, and reporting the results
of the review.

**9.** **Self-Inspection Programs.** Some OIGs have an internal
self-inspection program. If so, the OIG being reviewed will furnish a
copy of any internal self-inspection reports that have been completed
since the last peer review to the new PCIE/ECIE QAR team. The QAR team
will review the self-inspection report onsite and focus on those
portions that relate to areas covered by the peer review. Removal and/or
copying of the internal report may be restricted by the reviewed OIG.

**[PLANNING AND PERFORMING THE INVESTIGATIVE PCIE/ECIE QAR
REVIEW]{.underline}**

As stated above, the objective of a qualitative assessment review is to
determine whether internal safeguards and management procedures are in
place and operating effectively to provide reasonable assurance that
established policies, procedures, and applicable investigative standards
are being followed. In making this determination, the PCIE/ECIE QAR team
will analyze existing policies and procedures, conduct interviews with
selected management officials and the investigative staff, and sample
closed investigative files and other administrative records, as
warranted.

Completion of the PCIE/ECIE QAR process using Appendix A, Appendix B (if
applicable), Appendix C, and Appendix D1/D2 constitutes the requirements
for a full peer review. For agencies not governed by the law enforcement
powers conferred by the 2002 amendments to the Inspector General Act
(Section 6e), the scope of the review may be limited or expanded based
on the agreement of the reviewed organization and the PCIE/ECIE QAR team
leader.

1.  **1. Scope.**

> [Appendix A]{.underline} -- This section is an organizational profile
> of the office being reviewed.
>
> [Appendix B]{.underline} -- If applicable, this section of the
> PCIE/ECIE QAR assesses whether an organization meets the requirement
> of statutory law enforcement implementation. The scope of a review
> using Appendix B [cannot]{.underline} be limited. (Note: An OIG that
> received statutory law enforcement powers under legislation other than
> Section 6 of the IG Act may be reviewed in accordance with its
> criteria.)
>
> [Appendix C1/C2]{.underline} -- This portion of the PCIE/ECIE QAR
> process tests an office's general conformity with the PCIE/ECIE QSI.
>
> [Appendix D1/D2]{.underline} -- This portion of the PCIE/ECIE QAR
> includes checklists for sampling closed investigative files for their
> compliance with applicable law enforcement standards and the PCIE/ECIE
> QSI.

**2.** **Approach.** Review team members should be knowledgeable of all
facets of an investigation and use prudent judgment when evaluating
compliance with the Inspector General Act, the PCIE/ECIE QSI**,**
applicable law enforcement guidelines, and OIG policies and procedures.
To the extent possible teams will review offices with similar law
enforcement authorities and structures.

Generally, review teams will be assessing the following:

-   Does the organization have policies, procedures, or programs in
    place to facilitate compliance with the Attorney General's
    Guidelines and/or the PCIE/ECIE QSI?

-   Does the organization comply with the above policies, procedures, or
    programs?

-   Does the organization have policies, procedures, or programs in
    place to facilitate the identification of non-compliance?

-   Does the organization have policies, procedures, or programs in
    place to facilitate correction of non-compliance?

**3.** **Pre-Site Review Steps.** The organization being reviewed will
complete Appendix A in its entirety and only the "Reviewed Agency
Policy/Manual Reference" column of Appendix B (if applicable) and
Appendix C. Hyper linking is optional but encouraged. It is preferable
that this documentation be furnished electronically to the PCIE/ECIE QAR
team for analysis before a site visit begins. The review team should
always consider obtaining and reviewing relevant policy and procedural
documentation to save time on-site.

> Examples of references and other documentation that should be
> available for the review team to examine when the on-site review is
> conducted include:
>
> a\. **Manuals, Policy Statements, and Handbooks** -- pertinent
> documents describing the operational policies and procedures.
>
> b\. **Semiannual Reports to Congress** -- at least the four most
> recent semiannual reports to Congress. (The semiannual reports will
> provide information regarding the nature and volume of investigative
> work being performed. The reports may also assist the review team in
> identifying closed case files to be reviewed.)
>
> c\. **A copy of the office's last PCIE/ECIE Quality Assessment Report
> and a summary of the corrective action taken in response to PCIE/ECIE
> QAR findings.**
>
> d\. **Closed Case Inventory** -- a listing of the cases closed during
> the past 12 months. (This listing should include information such as
> the case identifiers; dates the investigations were opened and closed;
> case types, e.g., employee integrity or procurement fraud; referral
> dates; disposition; types of action taken; hours charged; and grade
> levels of the investigators.)
>
> e\. **Self-Inspection Report** -- a copy (or appropriate portions) of
> self-inspection or internal evaluation reports conducted by the
> organization may be provided in advance or held until the onsite
> visit.
>
> Requests for information should be submitted to the OIG being reviewed
> approximately 60 to 90 calendar days before the on-site review begins.

**4. Working Environment.** Before beginning the on-site work, the
PCIE/ECIE QAR team leader should arrange with the reviewed agency to
have adequate workspace for the review team. The Assistant Inspector
General for Investigations (AIGI), or a designee, should facilitate the
coordination of logistics for the PCIE/ECIE QAR team and obtaining
requested materials.

**5. Review Schedule.** The PCIE/ECIE QAR will be scheduled by mutual
agreement between the review team and the agency to be reviewed. Once a
tentative schedule is established, the reviewing organization should
send the reviewed organization an engagement letter modeled on the
example in Appendix E. The size of the organization or level of detail
of the review may impact the time required to complete a review.

> The goal of the review team should be to complete a qualitative
> assessment in as little time as may be required. Therefore, the
> following timeframes are provided as general guidance:

  -------------------------------------- --------------------------------
  **Action Item**                        **Recommended Timeframe**
                                         (calendar days)

  Send engagement letter to reviewed     90 days before site visit
  organization                           

  Appointment of PCIE/ECIE QAR team      90 days before a site review
  leader and selection of review team.   

  Conduct pre-site review and request    60 to 90 days before the on-site
  necessary information from office      review begins
  being reviewed.                        

  Conduct on-site review and exit        5 to 10 days
  briefing.                              

  Complete the draft PCIE/ECIE QAR       30 days after completing the
  report and submit the draft report to  on-site review
  the reviewed office for comment.       

  Allow offices being reviewed to        15 days
  comment on the draft report.           

  Finalize PCIE/ECIE QAR report and      15 days after comment(s) by
  related documents and distribute       reviewed office

  Memorandum from reviewed agency on the 60 days after issuance of final
  status of corrective actions it        report
  committed to implement                 
  -------------------------------------- --------------------------------

**6. Entrance Briefing.** An entrance briefing will be conducted with
the IG or designee of the OIG being reviewed. The senior investigations
field office staff from each office reviewed should be invited to attend
the entrance briefing. This meeting provides an opportunity to outline
the objectives of the PCIE/ECIE QARs; review the methodology, and
express any areas of management concern.

1.  

**7. Sample Selection.** It may be prohibitive in terms of time and
resources for the review team to examine each field location and the
entire population of OIG records to answer specific items in the
appendices.

> The selection of field locations (satellite offices) included in the
> review involves the exercise of considerable professional judgment.
> The review team should strive to include offices that are
> representative of the OIG with greater weight given to locations with
> a lower level of centralized control. If prior internal inspections
> show a location had problems in the past, the team may want to review
> a sample of that location's work to ensure that corrective actions
> have been implemented and, if so, if they were effective.
>
> Factors to be considered in selecting the field location(s) to be
> reviewed include the following:

-   Number, size, and geographic dispersion of field offices

-   Changes in organizational structure, control, and leadership

-   Number, type, and importance of reports issued by location

-   Degree of centralized control over field locations

-   Results of prior internal inspection reports or other external
    reviews

-   The need to verify the results of internal inspection reports

> Due to the sensitive and dynamic nature of active investigations, it
> is recommended that the review team sample closed cases during the
> PCIE/ECIE QAR (see Appendix D1/D2). In determining the number of
> closed cases in the sample, it should be kept in mind that the
> objective of the PCIE/ECIE QAR is to obtain information regarding the
> performance of the OIG overall, not each individual office. Therefore,
> team leaders should not feel that they need to select a certain number
> of reports at each location. Rather, to the extent possible, sample
> selection should facilitate the review of a cross-section of
> investigation types performed by the OIG staff at the location (e.g.,
> procurement fraud, environmental crimes, technology crimes,
> traditional crimes, employee misconduct, etc.). Additionally, the
> review team may, at its discretion, review closed cases from prior
> years for further validation if the original sample is either too
> small or suggests potential significant deficiencies. However, the
> review team generally should not examine cases closed more than two
> years prior to the review.
>
> The following guidance is furnished to assist the review team in
> determining the number of closed cases selected in the sample:

+-------------------------------+--------------------------------------+
| **Number of Cases Closed**    | **Minimum Number of Closed Cases In  |
|                               | the Sample**                         |
| **In the 12 Months Preceding  |                                      |
| Field Work**                  |                                      |
+-------------------------------+--------------------------------------+
| 0-20                          | Review all files                     |
|                               |                                      |
| 20 -- 100 Cases               | 20 Closed Cases                      |
+-------------------------------+--------------------------------------+
| 101 -- 500 Cases              | 30 Closed Cases                      |
+-------------------------------+--------------------------------------+
| 500 (or more) Cases           | 50 Closed Cases                      |
+-------------------------------+--------------------------------------+

> The review team must apply a no-advance notice policy in advising the
> OIG of the closed case files selected for review during the on-site
> visit, if legally possible.
>
> Sampling may also be used to perform either of the following review
> steps.
>
> a\. A sample from the staff of investigators may be selected to ensure
> that they meet the basic qualifications for investigators.
>
> b\. A sample of training profiles or the equivalent may be selected to
> ensure that investigators maintain their investigative and law
> enforcement skills.

**8. Assessment Options.** The PCIE/ECIE QAR team has the following two
options for assessing an OIG's performance:

  ---------------------------- ----------------------------------------------------
  **Degree of                  **[Explanation]{.underline}**
  [Compliance]{.underline}**   

  **Compliant**                This is an Unqualified Opinion. No material
                               deficiencies (reportable findings) were identified
                               during the review.

  **Noncompliant**             This is an Adverse Opinion. One or more reportable
                               findings were noted during the review.
  ---------------------------- ----------------------------------------------------

> **a.** **Compliant**. A rating of "compliant" conveys that the
> reviewed organization has adequate internal safeguards and management
> procedures to ensure that PCIE/ECIE standards are followed and that
> law enforcement powers conferred by the 2002 amendments to the
> Inspector General Act are properly exercised (for applicable
> agencies).
>
> **b.** **Noncompliant.** A rating of non-compliance indicates a
> breakdown in practices, programs and/or policies that had an actual
> notable adverse impact on, or has a likelihood of materially
> affecting, the integrity of the investigative process (e.g., planning,
> conducting, reporting) or law enforcement operations (i.e., powers
> conferred by the IG Act). A reportable finding is defined as a
> material failure to conform to applicable standards---the Attorney
> General Guidelines for Statutory Law Enforcement Authority and/or the
> PCIE/ECIE Quality Standards for Investigation. A reportable finding
> can result from a material failure in one critical area as well as a
> series of weaknesses across multiple areas that have a cumulative
> adverse impact on the organization's ability to adequately comply with
> the PCIE/ECIE QSI and/or Attorney General Guidelines.
>
> The following circumstances generally do not constitute a reportable
> finding:

-   Deficiencies were found in a limited number of case files or at one
    of several sites reviewed;

-   A deficiency exists in an area outside the exclusive or substantial
    control of the OIG;

-   The reviewed OIG lacked stand-alone internal written policy but, in
    practice, complied with applicable standards; and,

-   The organization has violated its own internal policy but has
    complied with the PCIE/ECIE QSI and the Attorney General's
    Guidelines (e.g., internal policy documents require semi-annual
    refresher training in federal criminal and civil updates but it is
    actually provided annually).

> All reportable findings must be included in an attachment to the
> opinion letter. The decision to document a reportable finding must be
> supported by clear and convincing evidence of noncompliance as well as
> a specific listing of the standard(s) violated.

**9. Views of Responsible Officials.** PCIE/ECIE QAR findings must be
both complete and fair. Exaggeration of a finding's significance must be
avoided. One way to ensure the objectiveness, accuracy, and completeness
of the findings is to obtain the views of responsible officials prior to
finalizing the findings. When apparent deficiencies are found *during*
the review, the team must discuss the situation with the appropriate
responsible official(s) designated by the reviewed OIG. On-the-spot
corrections will be viewed favorably but must be completed prior to the
issuance of the final report. Depending on the gravity of the deficiency
corrected on the spot, the finding and correction may be discussed in
either the opinion letter or letter of observations. All preliminary
[reportable]{.underline} findings must be presented during the review to
the official(s) designated by the reviewed OIG. This action will help to
avoid any misunderstandings and help ensure that all facts are
considered before a formal draft report is prepared.

**10. Exit Conference.** The review team must prepare and present the
preliminary findings of the review to the IG and other members of the
senior management team at the conclusion of the on-site visitation.

**[REPORTING REVIEW RESULTS (See APPENDIX E)]{.underline}**

**General.** The findings of the PCIE/ECIE QAR are reported in two
separate documents: (1) an opinion letter, and (2) a letter of
observations.

**1. Opinion Letter.** This letter is prepared by the PCIE/ECIE QAR team
and furnished to the IG of the reviewed organization along with a letter
of observations, if issued (see below). If a rating of noncompliance is
issued, an attachment to the opinion letter must detail the reportable
finding(s). The body of the opinion letter contains information such as:

> a\. Scope of the review, including any limitations thereon, and any
> expansion of the review beyond the basic review guide, if applicable.
>
> b\. Description of the review methodology, including the field offices
> visited and a listing, by case number, of each investigative file
> reviewed.
>
> c\. The review team's opinion regarding the compliance or
> non-compliance with PCIE/ECIE QSI and applicable law enforcement
> standards.
>
> d\. An explanation of review team actions taken in response to the
> OIG's official comments to the draft report.

**2.** **Letter of Observations.** A supplemental letter of observations
may be furnished to the IG of the reviewed office. Observations may fall
into three categories:

a.  **"Best Practices"** **or similar notable positive attributes of the
    organization.** In keeping with the constructive nature of the
    PCIE/ECIE QAR program, the reviewing agency will highlight
    practices, policies, programs, accomplishments, etc., that are
    particularly worthy of praise or acknowledgement. Examples include
    but are not limited to a comprehensive management development
    program, an advanced management information system, and quality
    report writing and reviewing process.

> In coordination with the reviewed agency, the team should report
> particularly noteworthy accomplishments found during the review to the
> PCIE Investigations Committee for dissemination. Other OIGs may
> benefit from this information. This may be done in a separate letter
> from the team leader to the Committee.

b.  **Areas for Improvement.** These are non-material, non-systemic
    deficiencies identified during the review and are furnished to point
    out areas that need improvement. The reviewing team will identify a
    specific applicable PCIE/ECIE or Attorney General standard as a
    benchmark. Isolated instances of policy or procedural nonconformity
    are included here. Additionally, a review team should identify
    poorly constructed office policies or nonconformities or
    inconsistencies with professional standards.

c.  **Recommendations for Increased Efficiency/Effectiveness.** This
    category is used when the team identifies a policy, program or
    practice that could be examined by the organization for increased
    efficiency or effectiveness. Recommendations in this area are
    reserved for *notable* inefficiencies and ineffectiveness, not a
    violation of a specific standard. The team may suggest a benchmark
    or policy from similar organizations. Examples include a streamlined
    process for conducting case progress reviews, processing management
    information system entries, etc.

**3. Views of Responsible Officials.** The OIG being reviewed must be
afforded an opportunity to comment on the formal draft report prior to
the issuance of a final assessment report. All material facts provided
by the reviewed organization must be considered by the review team to
determine whether the initial comments included in the draft report
should be revised.

**4. Dispute Resolution.** The IG of the reviewed organization may refer
a significant dispute about a draft finding to the PCIE Investigations
Committee for review and resolution [if the IG cannot resolve the matter
with the PCIE/ECIE QAR team]{.underline}. The OIG of the reviewed
organization should provide the Investigations Committee: (a) a copy of
the *draft* PCIE/ECIE QAR report and attachments, (b) the reviewed
organization's response to the draft PCIE/ECIE QAR findings, and (c) a
written summary of the material facts regarding the disagreement.

> The Investigations Committee should work with the OIG being reviewed
> and the PCIE/ECIE QAR team leader to resolve the dispute. A range of
> options are available to the Investigations Committee. For example,
> the Investigations Committee may elect to: (a) accept the PCIE/ECIE
> QAR team's initial findings, (b) accept the reviewed organizations
> explanations, (c) request the PCIE/ECIE QAR review team conduct
> additional work to facilitate the resolution of the disagreement, (d)
> form a new PCIE/ECIE QAR team tasked with conducting further review of
> the disputed findings, or (e) other options not specifically
> anticipated here.
>
> As mentioned previously, the Investigations Committee should be
> furnished a copy of each final PCIE/ECIE QAR report conducted in
> PCIE/ECIE organizations. If the reviewed organization receives an
> overall opinion rating of "noncompliance," the organization must
> provide the Investigations Committee a detailed corrective action plan
> to bring the organization into compliance with professional standards.
> Where appropriate, this plan will be made available to the U.S.
> Department of Justice, upon request. An organization receiving an
> overall noncompliance rating will not be allowed to conduct PCIE/ECIE
> QAR reviews at other agencies until the corrective action plan has
> been developed and the Investigations Committee has approved its
> implementation.

**5. Letter Distribution.** The opinion letter and any letter of
observation will be issued to the IG of the reviewed office. A copy of
the opinion letter must be forwarded to the U.S. Attorney General in
accordance with Section 6(e)(7) of the IG Act for those agencies that
receive their law enforcement authority pursuant to Section 6(c) of the
IG Act. In order to track the completion of peer reviews, a copy of the
opinion letter will also be furnished to the Chairperson of the PCIE
Investigations Committee (for PCIE agencies) or the ECIE representative
on the Investigations Committee (for ECIE agencies).

> Additionally, consistent with the PCIE/ECIE Quality Standards for
> Federal Offices of Inspector General, a reviewed OIG may provide a
> copy of the final letters resulting from the PCIE/ECIE QAR to the head
> of the agency or department.
>
> If the reviewed agency commits to implementing corrective actions as a
> result of observations or recommendations, the reviewed agency will
> provide periodic written updates (not to exceed every 60 days) to the
> review team. The updates will continue until the action(s) is
> implemented.

**6. Files Maintenance.** All files, records, notes, and memoranda or
copies obtained from the office reviewed will be returned after issuing
the final report. The OIG conducting the PCIE/ECIE QAR should retain a
copy of the final report and supporting appendices. It is recommended
that these documents be retained by the reviewing OIG for at least two
review cycles.

> The OIG conducting the PCIE/ECIE QAR will institute a record retention
> policy in accordance with guidelines established by the National
> Archive and Records Administration. All requests for access to the
> PCIE/ECIE QAR files, to include Freedom of Information (FOIA) and
> Privacy Act (PA) requests, must be processed in consultation with the
> IG of the reviewed organization.

[^1]: Incorporation of Appendix C2 (a review of computer forensics
    activities) is not mandatory. It is an "opt-in" feature of a peer
    review. If the OIG organization being reviewed has computer forensic
    capability, it may, prior to commencement of the review, opt to have
    its computer forensics activities reviewed. If an organization does
    opt in, the results of the computer forensics review will be
    included in the overall assessment of the OIG organization. Note
    that regardless of whether or not an organization opts in, the
    *investigative aspects* of computer-related cases *(*planning,
    execution, and reporting) will be reviewed relative to the QSI if
    such cases are part of the sample case file selection. Appendix C2
    involves an additional review step---focusing on the *technical*
    aspects of computer forensics operations. If the OIG organization
    conducting the peer review does not have in-house personnel with
    computer forensic capability to conduct the review, it may seek
    assistance from other PCIE/ECIE OIG organizations.