```

[Federal Register Volume 65, Number 193 (Wednesday, October 4, 2000)]
[Notices]
[Pages 59200-59214]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 00-25397]

=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF JUSTICE

Confidentiality in Federal Alternative Dispute Resolution 
Programs; Evaluation of Federal Alternative Dispute Resolution Programs

AGENCY: Department of Justice/Federal Alternative Dispute Resolution 
Council.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: This notice solicits public comment on two documents designed 
to assist Federal agencies in developing alternative dispute resolution 
(ADR) programs: ``Confidentiality in Federal Alternative Dispute 
Resolution Programs'' and ``Evaluation of Federal Alternative Dispute 
Resolution

[[Page 59201]]

Programs.'' These documents were created by the Federal ADR Steering 
Committee, a group of subject matter experts from federal agencies with 
active ADR programs. They were approved for publication in draft form 
by the Federal ADR Council, a group of high level government officials 
chaired by the Attorney General. The first document contains detailed 
guidance on the nature and limits of confidentiality in Federal ADR 
programs and also includes a statement on these issues for Federal 
neutrals to use in ADR proceedings. The second document contains 
detailed recommendations for agencies to follow when evaluating their 
ADR programs.
    All interested individuals or organizations are invited to submit 
comments on these documents for the consideration of the Federal ADR 
Council before they are published in final form at the end of this 
year.

DATES: All comments must be postmarked by November 1, 2000, in order to 
receive consideration.

ADDRESSES: Address all comments to Jeffrey M. Senger, Deputy Senior 
Counsel for Dispute Resolution, United States Department of Justice, 
950 Pennsylvania Ave. NW, Room 4328, Washington, D.C., 20530

    Dated: September 27, 2000.
Jeffrey M. Senger,
Deputy Senior Counsel for Dispute Resolution, Department of Justice.

SUPPLEMENTARY INFORMATION: In recent years, the government and the 
private sector increasingly have been using techniques known as 
alternative dispute resolution (ADR). Our experience has shown that ADR 
can resolve disputes in a manner that is quicker, cheaper, and less 
adversarial than traditional processes such as litigation. In ADR, 
parties meet with each other directly, under the guidance of a neutral 
professional who is trained and experienced in handling disputes. They 
talk about the problems that led to the complaint and the resolution 
that will work best for them in the future. While litigation often 
silences the parties and severely restricts their control over the 
outcome of their own dispute, ADR allows them instead to work 
collaboratively to find creative, effective solutions that are 
agreeable to all sides.
    The Administrative Dispute Resolution Act of 1996 (ADRA), 5 U.S.C. 
571-584, requires each Federal agency to promote the use of ADR and 
calls for the establishment of an interagency committee to assist 
agencies in the use of ADR. Pursuant to this Act, a Presidential 
Memorandum dated May 1, 1998, created the Interagency ADR Working 
Group, chaired by the Attorney General, to ``facilitate, encourage, and 
provide coordination'' for Federal agencies. In the Memorandum, the 
President charged the Working Group with assisting agencies with 
training in ``how to use alternative means of dispute resolution'' and 
evaluation ``to ascertain the benefits of alternative means of dispute 
resolution.'' The following two documents are designed to serve these 
goals.
    The first document describes the nature and limits of 
confidentiality in Federal ADR proceedings. Confidentiality is vital 
for the success of ADR for several reasons. Parties must be free to 
engage in candid, informal discussions of their interests in order to 
reach the best possible settlement of their claims. Guarantees of 
confidentiality permit parties to speak openly, without fear their 
statements will be used against them later. Confidentiality also 
facilitates ADR by encouraging parties to avoid the posturing that 
often occurs when proceedings are on the record. Further, 
confidentiality gives parties the ability to trust the mediator because 
they are assured he or she will not later take sides and talk publicly 
in favor of one party or the other. At the same time, members of the 
public have a general right to know what happens in government 
proceedings and do not want ADR to be used to shield improper activity 
that involves public business. The ADRA is designed to strike the 
appropriate balance between the public interest in access to government 
decision-making and the necessity for certain guarantees of 
confidentiality in ADR in order for the process to be effective.
    Understandably, there has been a great deal of interest in 
understanding what statements made in the context of a Federal ADR 
proceeding are confidential and what statements are not. This document 
is designed to give a detailed explanation of the reasonable 
expectations of confidentiality for parties who participate in ADR 
involving the government. The first section of the report reprints the 
confidentiality provisions of the ADRA. Next, the report contains a 
section-by-section analysis of these confidentiality provisions. Then 
the report sets forth, in question-and-answer format, an expanded 
analysis of the issues likely to arise in practice. Finally, the report 
presents a model confidentiality statement suitable for use by neutrals 
in Federal ADR proceedings.
    The second document contains detailed guidance for agencies to use 
when conducting evaluations of their ADR programs. Proponents of ADR 
have described many benefits from its use, including savings of time 
and money, increased party satisfaction with the process and its 
outcome, increased settlement rates, and improved relationships. In 
order to ensure the growth of ADR programs, these benefits must be 
rigorously documented and communicated to the public. If evaluations 
determine problems with ADR programs, these must be remedied. 
Evaluation is a vital part of any ADR program, and it is consistent 
with the obligations of all Federal agencies under the Government 
Performance and Results Act (Pub. L. 103-62).
    The first part of this document is a two-page description of 
general evaluation recommendations for Federal ADR programs. It sets 
forth specific data that agencies should capture and gives a brief 
introduction to other important concepts, such as validity, 
reliability, and presentation of data. The remainder of the report is a 
twenty-page detailed description of evaluation, including planning and 
design, methodology, and communicating results. The report concludes 
with a bibliography of additional resources in this area.
    The Federal ADR Council encourages all interested parties to submit 
comments on these documents. The Council will consider all comments in 
connection with its review of the final versions of these documents at 
the end of 2000.
    Nothing in these guidance documents shall be construed to create 
any right or benefit, substantive or procedural, enforceable at law or 
in equity, by a party against the United States, its agencies, its 
officers or any other person.

The Federal ADR Council

Chair: Janet Reno, Attorney General, Department of Justice
Vice Chair: Erica Cooper, Deputy General Counsel, Federal Deposit 
Insurance Corporation
    Members: Leigh A. Bradley, General Counsel, Department of Veterans 
Affairs; Meyer Eisenberg, Deputy General Counsel, Securities and 
Exchange Commission; Mary Anne Gibbons, General Counsel, U.S. Postal 
Service; Gary S. Guzy, General Counsel, Environmental Protection 
Agency; Jeh C. Johnson, General Counsel, Department of the Air Force; 
Harold Kwalwasser, Deputy General Counsel, Department of Defense; 
Rosalind Knapp, Acting General Counsel, Department of Transportation; 
Anthony N. Palladino, Director, Office of Dispute Resolution, Federal 
Aviation Administration, Department of Transportation; Janet S.

[[Page 59202]]

Potts, Counsel to the Secretary, Department of Agriculture; Harriett S. 
Rabb, General Counsel, Department of Health and Human Services; Henry 
L. Solano, Solicitor, Department of Labor; John Sparks, Principal 
Deputy General Counsel, Department of the Navy; Peter R. Steenland, 
Jr., Senior Counsel for Dispute Resolution, U.S. Department of Justice; 
Mary Ann Sullivan, General Counsel, Department of Energy; Robert Ward, 
Dispute Resolution Specialist, Environmental Protection Agency.

Report on the Reasonable Expectations of Confidentiality Under the 
Administrative Dispute Resolution Act of 1996

Table of Contents

I. Introduction
II. Administrative Dispute Resolution Act
III. Section-By-Section Analysis of Confidentiality Provisions
IV. Questions & Answers on Confidentiality under the Administrative 
Dispute Resolution Act (ADR Act)
V. Model Confidentiality Statement for Use by Neutrals

I. Introduction

    The Administrative Dispute Resolution Act of 1996 (``ADR Act'') 
contains provisions that affect the confidentiality of administrative 
ADR proceedings. Neutrals and participants in federal dispute 
resolution proceedings need to have an accurate understanding of these 
provisions. The Federal ADR Council directed the Interagency ADR 
Working Group Steering Committee to review the ADR Act confidentiality 
provisions and provide the Council with a report outlining reasonable 
expectations of confidentiality for parties in federal dispute 
resolution. This report, the product of that effort, describes the ADR 
Act confidentiality provisions principally located at 5 U.S.C. Section 
574.
    The report has four sections: (1) A reprint of the confidentiality 
provisions of the ADR Act; (2) a section-by-section analysis of the 
confidentiality provisions; (3) a set of questions and answers designed 
to expand upon the analysis and address issues likely to arise in 
practice; and (4) a model confidentiality statement suitable for use by 
neutrals in federal ADR proceedings.
    During preparation of this report, several issues emerged regarding 
implementation of the ADR Act that are not fully addressed in this 
report. These issues are important to the practice of federal ADR and 
would benefit from further investigation and study. As federal sector 
experience with ADR evolves, some issues addressed in this report will 
be refined and new issues are likely to arise. It is also important to 
note that the ADR Act is not the only means of maintaining 
confidentiality and other laws, regulations, and agency policies may 
impact confidentiality. A complete analysis of all such authorities is 
beyond the scope of this report.

II. Administrative Dispute Resolution Act

Definitions (5 U.S.C. 571)

    For the purposes of this subchapter, the term--
    (1) ``agency'' has the same meaning as in section 551(1) of this 
title;
    (2) ``administrative program'' includes a Federal function which 
involves protection of the public interest and the determination of 
rights, privileges, and obligations of private persons through rule 
making, adjudication, licensing, or investigation, as those terms are 
used in subchapter II of this chapter;
    (3) ``alternative means of dispute resolution'' means any procedure 
that is used to resolve issues in controversy, including, but not 
limited to, conciliation, facilitation, mediation, factfinding, 
minitrials, arbitration, and use of ombuds, or any combination thereof;
    (4) ``award'' means any decision by an arbitrator resolving the 
issues in controversy;
    (5) ``dispute resolution communication'' means any oral or written 
communication prepared for the purposes of a dispute resolution 
proceeding, including any memoranda, notes or work product of the 
neutral, parties or nonparty participant; except that a written 
agreement to enter into a dispute resolution proceeding, or final 
written agreement or arbitral award reached as a result of a dispute 
resolution proceeding, is not a dispute resolution communication;
    (6) ``dispute resolution proceeding'' means any process in which an 
alternative means of dispute resolution is used to resolve an issue in 
controversy in which a neutral is appointed and specified parties 
participate;
    (7) ``in confidence'' means, with respect to information, that the 
information is provided--
    (A) with the expressed intent of the source that it not be 
disclosed; or
    (B) under circumstances that would create the reasonable 
expectation on behalf of the source that the information will not be 
disclosed;
    (8) ``issue in controversy'' means an issue which is material to a 
decision concerning an administrative program of an agency, and with 
which there is disagreement--
    (A) between an agency and persons who would be substantially 
affected by the decision; or
    (B) between persons who would be substantially affected by the 
decision;
    (9) ``neutral'' means an individual who, with respect to an issue 
in controversy, functions specifically to aid the parties in resolving 
the controversy;
    (10) ``party'' means--
    (A) for a proceeding with named parties, the same as in section 
551(3) of this title; and
    (B) for a proceeding without named parties, a person who will be 
significantly affected by the decision in the proceeding and who 
participates in the proceeding;
    (11) ``person'' has the same meaning as in section 551(2) of this 
title; and
    (12) ``roster'' means a list of persons qualified to provide 
services as neutrals.

Confidentiality (5 U.S.C.574)

    (a) Except as provided in subsections (d) and (e), a neutral in a 
dispute resolution proceeding shall not voluntarily disclose or through 
discovery or compulsory process be required to disclose any dispute 
resolution communication or any communication provided in confidence to 
the neutral, unless--
    (1) all parties to the dispute resolution proceeding and the 
neutral consent in writing, and, if the dispute resolution 
communication was provided by a nonparty participant, that participant 
also consents in writing;
    (2) the dispute resolution communication has already been made 
public;
    (3) the dispute resolution communication is required by statute to 
be made public, but a neutral should make such communication public 
only if no other person is reasonably available to disclose the 
communication; or
    (4) a court determines that such testimony or disclosure is 
necessary to--
    (A) prevent a manifest injustice;
    (B) help establish a violation of law; or
    (C) prevent harm to the public health or safety, of sufficient 
magnitude in the particular case to outweigh the integrity of dispute 
resolution proceedings in general by reducing the confidence of parties 
in future cases that their communications will remain confidential.
    (b) A party to a dispute resolution proceeding shall not 
voluntarily disclose or through discovery or

[[Page 59203]]

compulsory process be required to disclose any dispute resolution 
communication, unless--
    (1) the communication was prepared by the party seeking disclosure;
    (2) all parties to the dispute resolution proceeding consent in 
writing;
    (3) the dispute resolution communication has already been made 
public;
    (4) the dispute resolution communication is required by statute to 
be made public;
    (5) a court determines that such testimony or disclosure is 
necessary to--
    (A) prevent a manifest injustice;
    (B) help establish a violation of law; or
    (C) prevent harm to the public health and safety, of sufficient 
magnitude in the particular case to outweigh the integrity of dispute 
resolution proceedings in general by reducing the confidence of parties 
in future cases that their communications will remain confidential;
    (6) the dispute resolution communication is relevant to determining 
the existence or meaning of an agreement or award that resulted from 
the dispute resolution proceeding or to the enforcement of such an 
agreement or award; or
    (7) except for dispute resolution communications generated by the 
neutral, the dispute resolution communication was provided to or was 
available to all parties to the dispute resolution proceeding.
    (c) Any dispute resolution communication that is disclosed in 
violation of subsection (a) or (b), shall not be admissible in any 
proceeding relating to the issues in controversy with respect to which 
the communication was made.
    (d)(1) The parties may agree to alternative confidential procedures 
for disclosures by a neutral. Upon such agreement the parties shall 
inform the neutral before the commencement of the dispute resolution 
proceeding of any modifications to the provisions of subsection (a) 
that will govern the confidentiality of the dispute resolution 
proceeding. If the parties do not so inform the neutral, subsection (a) 
shall apply.
    (2) To qualify for the exemption established under subsection (j), 
an alternative confidential procedure under this subsection may not 
provide for less disclosure than the confidential procedures otherwise 
provided under this section.
    (e) If a demand for disclosure, by way of discovery request or 
other legal process, is made upon a neutral regarding a dispute 
resolution communication, the neutral shall make reasonable efforts to 
notify the parties and any affected nonparty participants of the 
demand. Any party or affected nonparty participant who receives such 
notice and within 15 calendar days does not offer to defend a refusal 
of the neutral to disclose the requested information shall have waived 
any objection to such disclosure.
    (f) Nothing in this section shall prevent the discovery or 
admissibility of any evidence that is otherwise discoverable, merely 
because the evidence was presented in the course of a dispute 
resolution proceeding.
    (g) Subsections (a) and (b) shall have no effect on the information 
and data that are necessary to document an agreement reached or order 
issued pursuant to a dispute resolution proceeding.
    (h) Subsections (a) and (b) shall not prevent the gathering of 
information for research or educational purposes, in cooperation with 
other agencies, governmental entities, or dispute resolution programs, 
so long as the parties and the specific issues in controversy are not 
identifiable.
    (i) Subsections (a) and (b) shall not prevent use of a dispute 
resolution communication to resolve a dispute between the neutral in a 
dispute resolution proceeding and a party to or participant in such 
proceeding, so long as such dispute resolution communication is 
disclosed only to the extent necessary to resolve such dispute.
    (j) A dispute resolution communication which is between a neutral 
and a party and which may not be disclosed under this section shall 
also be exempt from disclosure under section 552(b)(3).

III. Section-by-Section Analysis of Confidentiality Provisions (5 
U.S.C. 574)

Section 574(a)

    In general, a neutral in a dispute resolution proceeding is 
prohibited from disclosing any dispute resolution communication or any 
communication provided to him or her in confidence. Unless the 
communication falls within one of the exceptions listed below, the 
neutral cannot voluntarily disclose a communication and cannot be 
forced to disclose a communication through a discovery request or by 
any other compulsory process.
    The exceptions to this general rule are found in subsections 
574(a)(1)-(4), 574(d) and 574(e).

Section 574(a)(1)

    A neutral may disclose a communication if all parties and the 
neutral agree in writing to the disclosure. If a nonparty provided the 
communication, then the nonparty must also agree in writing to the 
disclosure.

Section 574(a)(2)

    A neutral may disclose a communication if the communication has 
already been made public.

Section 574(a)(3)

    A neutral may disclose a communication if there is a statute which 
requires it to be made public. However, the neutral should not disclose 
the communication unless there is no other person available to make the 
disclosure.

Section 574(a)(4)

    A neutral may disclose a communication if a court finds that the 
neutral's testimony, or the disclosure, is necessary to:
    A. prevent a manifest injustice;
    B. help establish a violation of law; or
    C. prevent harm to the public health and safety.
    In order to require disclosure, a court must determine that the 
need for disclosure is of sufficient magnitude to outweigh the 
detrimental impact on the integrity of dispute resolution proceedings 
in general. The need for the information must be so great that it 
outweighs a loss of confidence among other potential parties that their 
dispute resolution communications will remain confidential in future 
proceedings.

Section 574(b)

    Unless the communication falls within one of the exceptions listed 
below, the party cannot voluntarily disclose a communication and cannot 
be forced to disclose a communication through a discovery request or by 
any other compulsory process.

Section 574(b)(1)

    The party who makes a statement or communication is free to 
disclose it.

Section 574(b)(2)

    A party may disclose a communication if all the parties agree in 
writing to the disclosure.

Section 574(b)(3)

    A party may disclose a communication if the communication has 
already been made public.

Section 574(b)(4)

    A party may disclose a communication if there is a statute which 
requires it to be made public.

[[Page 59204]]

Section 574(b)(5)

    A party may disclose a communication if a court finds that the 
party's testimony, or the disclosure, is necessary to:
    A. prevent a manifest injustice;
    B. help establish a violation of law; or
    C. prevent harm to the public health and safety.
    In order to require disclosure, a court must determine that the 
need for disclosure is of sufficient magnitude to outweigh the 
detrimental impact on the integrity of dispute resolution proceedings 
in general. The need for the information must be so great that it 
outweighs a loss of confidence among other potential parties that their 
dispute resolution communications will remain confidential in future 
proceedings.

Section 574(b)(6)

    (1) Parties may use dispute resolution communications to show that 
a settlement agreement was in fact reached or to show what the terms of 
this agreement mean.
    (2) Parties may also use communications in connection with later 
issues regarding enforcing the agreement.
    (3) Communications may only be revealed to the extent that they 
meet the above purposes.

Section 574(b)(7)

    (1) There is no confidentiality protection for parties' dispute 
resolution communications that were available to everyone in the 
proceeding. For example, in a joint mediation session with all parties 
present, statements made and documents provided by parties are not 
confidential.
    (2) Communications coming from the neutral are confidential. For 
example, early neutral evaluations or settlement proposals from the 
neutral are protected.
    (3) A party may not use this provision to gain protection for a 
communication by providing it to the neutral who then provides it to 
the other party.

Section 574(c)

    No one may use any dispute resolution communication in a related 
proceeding, if that communication was disclosed in violation of Section 
574 (a) and (b).

Section 574(d)(1)

    (1) Parties may agree to alternative confidentiality procedures to 
limit disclosure by a neutral.
    (2) Parties must inform the neutral of the alternative procedures 
before the dispute resolution proceeding begins.
    (3) If parties do not inform the neutral of the alternative 
procedures, the procedures outlined in Section 574(a) will apply.

Section 574(d)(2)

    (1) Dispute resolution communications covered by alternative 
confidentiality procedures may be protected from disclosure under FOIA.
    (2) To qualify for this protection, the alternative procedures must 
provide for as much, or more, disclosure than the procedures provided 
in Section 574.
    (3) Dispute resolution communications covered by alternative 
confidentiality procedures do not qualify for protection from 
disclosure under FOIA if they provide for less disclosure than those 
outlined in Section 574.

Section 574(e)

    (1) A neutral who receives a demand for disclosure, in the form of 
a discovery request or other legal process, must make reasonable 
efforts to notify the parties and any affected non-party participants 
of the demand.
    (2) Parties and non-party participants who receive a notice of a 
demand for disclosure from a neutral:
    a. must respond within 15 days and offer to defend a refusal to 
disclose the information; or
    b. if they do not respond within 15 days, will have waived their 
objections to disclosure of the information.

Section 574(f)

    Evidence that is otherwise discoverable or admissible is not 
protected from disclosure under this Section merely because the 
evidence was presented during a dispute resolution proceeding.

Section 574(g)

    The provisions of Section 574 (a) and (b) do not affect information 
and data that are necessary to document agreements or orders resulting 
from dispute resolution proceedings.

Section 574(h)

    Information from and about dispute resolution proceedings may be 
used for educational and research purposes as long as the parties and 
specific issues in controversy are not identifiable.

Section 574(i)

    (1) Dispute resolution communications may be used to resolve 
disputes between the neutral in a dispute resolution proceeding and a 
party or non-party participant.
    (2) Dispute resolution communications may be disclosed only to the 
extent necessary to resolve a dispute between a neutral and party or 
non-party participant.

Section 574(j)

    A dispute resolution communication between a neutral and a party 
that is protected from disclosure under this section is also protected 
from disclosure under FOIA (Section 552(b)).

IV. Questions and Answers on Confidentiality under the 
Administrative Dispute Resolution Act of 1996 (ADR Act)

General Confidentiality Rules

1. What communications are confidential?
    Subject to certain exceptions, the following two types of 
communications are potentially confidential under the ADR Act:
    A. A dispute resolution communication. A dispute resolution 
communication is any oral or written statement made by a party or a 
neutral that occurs during a dispute resolution proceeding and any 
writing prepared specifically for the purposes of a dispute resolution 
proceeding. Written agreements to enter into a dispute resolution 
proceeding and any written final agreement reached as a result of the 
proceeding are not dispute resolution communications. Citation: 5 
U.S.C. 571(5).
    B. A ``communication provided in confidence to the neutral.'' A 
``communication provided in confidence to the neutral'' is any oral 
statement or document provided to a neutral during a dispute resolution 
proceeding. The communication must be made: (1) With the express intent 
that it not be disclosed, or (2) provided under circumstances that 
would create a reasonable expectation that it not be disclosed. 
Citation: 5 U.S.C. 571(7) and 574(a).
2. What confidentiality protection is provided for dispute resolution 
communications?
    Generally, neutrals and parties may not voluntarily disclose or be 
compelled to disclose dispute resolution communications. The ADR Act 
contains specific exceptions to the general rule. Citation: 5 U.S.C. 
574(a), (b).
3. What confidentiality protection applies to a ``communication 
provided in confidence'' by a party to a neutral?
    A neutral may not disclose any communication provided in 
confidence. Citation: 5 U.S.C. 574(a).

[[Page 59205]]

4. What is a dispute resolution proceeding?
    A dispute resolution proceeding is any process involving the 
services of a neutral that is used to resolve an issue in controversy 
arising from an agency's program, operations, or actions. A dispute 
resolution proceeding includes any stage of such a dispute resolution 
process. Citation: 5 U.S.C. 571(6) and (8). See also, Question 10.
5. Who is a Neutral?
    A neutral is anyone who functions specifically to aid the parties 
during a dispute resolution process. A neutral may be a private person 
or a federal government employee who is acceptable to the parties. 
There may be more than one neutral during the course of a dispute 
resolution process (e.g., an ``intake'' neutral, a ``convener'' 
neutral, as well as the neutral who facilitates a face-to-face 
proceeding). It is important that agencies clearly identify neutrals to 
avoid misunderstanding.
    The ADR Act supports a broad reading of the term ``neutral.'' An 
intake or convening neutral is included in this definition as ``an 
individual who * * * functions specifically to aid the parties in 
resolving the controversy'' because such neutrals take the necessary 
first steps toward a potential resolution of a dispute.
    In situations where an intake neutral is identified by an agency, a 
party's willingness to contact and/or work with the intake neutral to 
initiate an ADR process is an indication that the intake neutral is 
acceptable to the party. Citation: 5 U.S.C 571(9), 571(6), 571(3), 
573(a).

    Example: An employee contacts an agency ADR program and 
describes a dispute to an intake person. The conversation is 
confidential only if the intake person has been appropriately 
identified as a neutral by the agency to aid parties in resolving 
such disputes.
6. Who Is a Party?
    A party is any person or entity who participates in a dispute 
resolution proceeding and is named in a legal proceeding or will be 
affected significantly by the outcome of the proceeding. The 
obligations of parties extend to their representatives and agents. 
Citation: 5 U.S.C. 571(10).
7. What Constitutes Disclosure?
    Disclosure is not defined in the ADR Act. Disclosure occurs when a 
neutral, a party, or a non-party participant makes a communication 
available to some other person by any method.
8. May a Party or Neutral Disclose Dispute Resolution Communications in 
Response to Discovery or Compulsory Process?
    In general, neither a neutral nor a party can be required to 
disclose dispute resolution communications through discovery or 
compulsory process. Compulsory processes include any administrative, 
judicial or regulatory process that compels action by an individual. 
(See also Question 15) Citation: 5 U.S.C. 574(a) & 574(b).
9. What Confidentiality Protection Is Provided for Communications by a 
Nonparty Participant in a Dispute Resolution Proceeding?
    A nonparty participant in a dispute resolution proceeding is an 
individual other than a party, agent or representative of a party, or 
the neutral. This could be an individual who is asked by the neutral to 
present information for use of the neutral or parties. A nonparty 
participant has an independent right to protect his or her 
communications from disclosure by a neutral. A neutral needs to obtain 
the consent of all parties and the nonparty participant to disclose 
such a communication. Citation: 5 U.S.C. 574(a)(1).
10. When in an ADR Process do the Confidentiality Protections of ADR 
Act Apply?
    Confidentiality applies to communications when a person seeking ADR 
services contacts an appropriate neutral. A communication made by a 
party to a neutral is covered even if made prior to a face-to-face ADR 
proceeding. Confidentiality does not apply to communications made after 
a final written agreement is reached, or after resolution efforts aided 
by the neutral have otherwise ended. Citation: 5 U.S.C. 571(6), 574(a) 
and (b).

Exceptions to Confidentiality Protection

11. What Communications Are Not Protected by the ADR Act?
    A. A party's own communications made during a dispute resolution 
proceeding. A party may disclose any oral or written communication 
which the party makes or prepares for a dispute resolution proceeding. 
Citation: 5 U.S.C. 574(b)(1).
    B. A dispute resolution communication that has ``already been made 
public.'' The ADR Act's confidentiality protections do not apply to a 
communication that has already been made public. Examples of 
communications that have ``already been made public'' include:
    1. The communication has been discussed in a Congressional hearing;
    2. The communication has been posted on the Internet;
    3. The communications has been released to the media;
    4. The communication has been placed in a court filing or testified 
about in a court in a proceeding not under seal;
    5. The communication has been reported in the newspapers;
    6. The communication has been discussed in an open meeting;
    7. The communication has been released under FOIA.
    Citation: 5 U.S.C. 574(a)(2) & 574(b)(3).
    C. Communications required by statute to be made public. FOIA is an 
example of a federal statute which requires agency records to be made 
public under certain circumstances. NOTE: A protected dispute 
resolution communication which is between a neutral and a party is 
exempt from disclosure under FOIA. (See Question 23) Citation: 5 U.S.C. 
574(a)(3), 574(b)(4), & 574(j).
    D. When a court orders disclosure. A federal court may override the 
confidentiality protections of ADR Act in three limited situations. In 
order to override the confidentiality protections, a court must 
determine that testimony or disclosure of a communication is necessary 
to either (1) prevent a manifest injustice, (2) help establish a 
violation of law, or (3) prevent harm to the public health or safety. 
The court must also determine that the need for the information is of a 
sufficient magnitude in the particular case to outweigh the integrity 
of dispute resolution proceedings in general by reducing the confidence 
of parties in future cases that their communications will remain 
confidential. There are no cases as of August 2000 that have 
interpreted these provisions. Citation: 5 U.S.C. 574(a)(4) & (b)(5).
    E. In order to resolve a dispute over the existence or meaning of a 
settlement arrived at through a dispute resolution proceeding. The ADR 
Act creates an exception to the general rule of nondisclosure for the 
limited purpose of determining the existence or meaning of an agreement 
arrived at through a dispute resolution proceeding. Parties may also 
disclose communications as required to enforce an agreement arrived at 
through a dispute resolution proceeding. Citation: 5 U.S.C. 574(b)(6).

    Example: Parties may disclose dispute resolution communications 
as required to show that a settlement agreement was reached or to 
show what the terms of this agreement were.

[[Page 59206]]

    F. Parties' communications in joint session, with all parties 
present. A neutral may not disclose communications made in joint 
session. However, there is no prohibition against a party disclosing 
communications available to everyone in the proceeding. Citation: 5 
U.S.C. 574(b)(7).
    G. Information sought for specific purposes. The ADR Act allows for 
the disclosure of information for educational and research purposes, in 
cooperation with agencies, governmental entities, or dispute resolution 
programs. It is essential that the parties and specific issues in 
controversy not be identifiable, however. Citation: 5 U.S.C. 574(h).

    Example: An individual who has served as a neutral in a number 
of agency ADR proceedings may share collected experiences when 
participating in a training program provided that the parties and 
specific issues are not identifiable.
    Example: An ADR program administrator may collect statistics to 
monitor the results of the program.

    H. Communications required to resolve disputes that arise between 
the neutral and a party. If there is a dispute between a neutral and a 
party regarding the conduct of a dispute resolution proceeding, both 
may disclose information to the extent necessary to resolve the 
dispute. Citation: 5 U.S.C. 574(i)

    Example: If a party refuses to pay the neutral for services, the 
neutral can disclose communications to the extent necessary to 
establish that payment is due.
12. Are a neutral's communications to parties in joint session or 
provided to all parties confidential?
    Yes. ADR Act protects communications by a neutral.

    Example: Early neutral evaluations or settlement proposals 
provided to the parties by a neutral are protected.

    Note: A party, however, may not use this provision to gain 
protection for a communication by providing it to the neutral who 
then provides it to the other party. The statute says that the 
communication must be ``generated'' by the neutral, not just passed 
along by the neutral. Citation: 5 U.S.C. 574 (b)(7). (See H. Rept. 
104-841,142 Cong. Rec. H11108-11 (September 25, 1996).

13. Can confidentiality attach to communications that are provided to 
or available to fewer than all of the parties?
    Yes. The ADR Act does not prohibit disclosure of dispute resolution 
communications that are ``provided to or * * * available to all parties 
to the dispute resolution proceeding.'' Under a plain reading of the 
statute, communications are not protected when provided to, or 
available to, all parties; thus, they remain protected if they are 
provided to, or are available to, some (but not all) of the parties in 
a dispute.
    The legislative history states, ``A dispute resolution 
communication originating from a party to a party or parties is not 
protected from disclosure by the ADR Act.'' H.R. Rep. No. 104-841, 142 
Cong. Rec. H11, 110 (Sept. 25, 1996). The plain language of the statute 
is not inconsistent with this piece of legislative history, in that it 
can be interpreted to mean both parties in a two-party (``party to the 
other party'') or all parties in a multi-party dispute (``party to all 
other parties''). Citation: 5 U.S.C. 574(b)(7).
14. Does ADR Act provide confidentiality protection for all evidence 
used in the course of a dispute resolution proceeding?
    No. All evidence that is otherwise discoverable is not protected 
merely because it was presented at a dispute resolution proceeding. 
Citation: 5 U.S.C. 574(f).
15. Does the ADR Act protect against the disclosure of dispute 
resolution communications in response to requests by federal entities 
for such information?
    Section 574 of the ADR Act prohibits a neutral or a party from 
disclosing, voluntarily or in response to discovery or compulsory 
process, any protected communication. The ADR Act further states that 
neutrals and parties shall not ``be required'' to disclose such 
communications. However, a number of federal entities have statutory 
authority to request disclosure of documents from federal agencies and 
employees. Examples of such statutes include, but are not limited to, 
The Inspector General Act (5 U.S.C. App.); The Whistleblower Protection 
Act (5 U.S.C. Section 1212(b)(2)); and the Federal Service Labor-
Management Relations Act (5 U.S.C. Section 7114(4)). None of the 
exceptions to the ADR Act's confidentiality provisions directly applies 
to requests for disclosure of information from federal entities. For 
example, these statutes do not require information to be made public 
under ADR Act Section 574 (a)(3) & (b)(4). In addition, the judicial 
override procedure outlined in Section 574 (a)(4) & (b)(5) is not 
always available to federal entities with authority to access 
information. Some federal entities may lack jurisdiction to seek a 
court order to compel disclosure. Other federal entities may have such 
jurisdiction, but may seek disclosure under other statutory authority.
    In summary, a tension between these statutory authorities exists. 
The issues of statutory interpretation of these differing authorities 
have not yet been considered in an appropriate forum. We do not 
anticipate that there will be many occasions when such requests will be 
directed to neutrals or participants. However, it is important for 
agencies, neutrals and participants to be aware of the potential for 
requests.
    In order to prevent unnecessary disputes over requests for 
information pursuant to an access statute and to mitigate damage to ADR 
programs, we recommend:
     Agency ADR programs should enter into a dialogue with 
potential requesting entities so that each may be educated about their 
respective missions.
     Procedures should be established for access to information 
that recognize the importance of confidentiality in dispute resolution 
processes and protect the integrity of the agency's ADR program.
     ADR programs should identify classes of information that 
are not confidential.
     Requesting entities should use non-confidential 
information as a basis for information requests.
     Requesting entities should seek confidential information 
only after other potential sources have been exhausted.
     Requesting entities should seek information from a neutral 
only as a last resort.
     The ADR program and requesting entities should agree to 
procedures to resolve specific disagreements that arise with regard to 
the disclosure of information.
     If a federal employee party or neutral receives a request 
for disclosure, he or she should contact the agency's ADR program as 
soon as possible to discuss appropriate courses of action. Neutrals 
must also notify parties of any such request (See Question 19).

Alternative Procedures To Establish Confidentiality Protection

16. May parties agree to confidentiality procedures which are different 
from those contained in ADR Act?
    Yes. Parties may agree to more, or less, confidentiality protection 
for disclosure by the neutral or themselves than is provided for in the 
Act.
    Subsection 574(d)(1) provides that the parties can agree to 
alternative confidential procedures for disclosures by a neutral. While 
there is no parallel provision for parties, the exclusive wording of 
this subsection should not be construed as indicating Congressional 
intent to limit alternative

[[Page 59207]]

procedures by parties. Parties have a general right to sign 
confidentiality agreements, and there is no reason this should change 
in a mediation context.
    If the parties agree to alternative confidentiality procedures 
regarding disclosure by a neutral, they must so inform the neutral 
before the dispute resolution proceeding begins or the confidentiality 
procedures in the ADR Act will apply. An agreement providing for 
alternative confidentiality procedures is binding on anyone who signs 
the agreement. (See Questions 23 and 24 for potential FOIA 
implications.)

    Example: Parties to an ADR proceeding can agree to authorize the 
neutral to use his or her judgment about whether to voluntarily 
disclose a protected communication, as long as the neutral is 
informed of this agreement before the ADR proceeding commences.
    Example: Parties to an ADR proceeding can agree that they, and 
the neutral, will keep everything they say to each other in joint 
session confidential.

Issues Regarding the Disclosure of Protected Communications

17. What restrictions are put on the use of confidential communications 
disclosed in violation of the ADR Act?
    If the neutral or any participant discloses a confidential 
communication in violation of Sections 574(a) or (b), that 
communication may not be used in any proceeding that is related to the 
subject of the dispute resolution proceeding in which the protected 
communication was made. A dispute resolution communication that was 
improperly disclosed may not be protected from use in an unrelated 
proceeding. Citation: 5 U.S.C. 574(c).
18. What is the penalty for disclosing confidential communications in 
violation of the statute?
    The ADR Act does not specify any civil or criminal penalty for the 
disclosure of a protected communication in violation of the Act. 
However, such disclosure may violate other laws, regulations or 
agreements of the parties.
19. What must a neutral do when he or she receives a ``demand for 
disclosure'' of confidential communications?
    A demand for disclosure is a formal request for confidential 
information. The demand must be made by a discovery request or some 
other legal process. Upon receiving a demand for disclosure of a 
confidential communication, a neutral must make a reasonable effort to 
notify the parties and any affected non-party participants of the 
demand. Notice must be provided even if the neutral believes that there 
is no basis for refusing to disclose the communication.
    Notice should be delivered to the last address provided by a party. 
Parties have fifteen days, from the date they receive the notice, in 
which to offer to defend the neutral against disclosure. Therefore, 
notice should be sent by a process that provides certification of 
delivery. For example, delivery could be by registered mail, by any 
carrier that provides tracking and certification of delivery, or by 
courier. Use of telephone or email communications as notice could be 
problematic. Since the parties must respond within 15 days or waive 
their right to object to disclosure, there must be a written record of 
when the notice was sent and when it was received. Citation: 5 U.S.C. 
574(e).
    Example: A colleague asks a neutral what happened in a 
mediation. The neutral must simply refuse to discuss the matter. The 
neutral does not need to notify the parties of the request.
    Example: A neutral receives a formal discovery request for 
information on what happened in a mediation. The neutral must notify 
the parties of this demand for disclosure using the procedures 
described above.
20. What can/must parties do when they receive notice of a demand for 
disclosure from the neutral?
    If a party has no objection to the disclosure of confidential 
communications, it need not respond to the notice. On the other hand, 
if a party believes that the sought-after communications should not be 
disclosed, it should notify the neutral and make arrangements to defend 
the neutral. Where the party is a federal agency, it should develop 
departmental procedures for processing the notice.
21. What responsibilities do agencies have for ensuring that the 
notification requirement is met?
    In some federal ADR programs, the neutral may be a federal employee 
performing collateral duty. Imposing an obligation upon these neutrals 
to keep records of parties to dispute resolution proceedings may be 
unduly onerous and ineffective. Agencies should develop administrative 
procedures to assure that the notification functions are fulfilled.
22. May a neutral refuse to disclose communications even when the 
parties have failed to agree to defend the neutral?
    Yes. The ADR Act permits, but does not compel, a neutral to 
disclose if the parties have waived objections to disclosure under 
Section 574(e). While the statute is clear that a neutral ``shall not'' 
disclose where a party objects, the statute does not say that a neutral 
must disclose if a party does not object.
    The effectiveness and integrity of mediation and other ADR 
processes is largely dependent on the credibility and trustworthiness 
of neutrals. In order to safeguard the integrity of ADR programs and to 
eliminate the potential for eroding confidence in future ADR 
proceedings, neutrals should be allowed to rely on established codes of 
ethics and confidentiality standards to support a decision not to 
disclose. Citation: 5 U.S.C. 574(a) & (e).

Issues Related to the Freedom of Information Act (FOIA)

23. What dispute resolution communications are protected from 
disclosure under FOIA?
    Dispute resolution communications between a neutral and a party 
that are covered by the confidentiality protections of the ADR Act are 
specifically exempted from disclosure under the Freedom of Information 
Act. This includes communications that are generated by a neutral and 
provided to all parties, such as an Early Neutral Evaluation. In 
addition, other FOIA exemptions may apply.
    Since only federal records are subject to FOIA, dispute resolution 
communications that are not federal records are not subject to the 
disclosure requirements of FOIA. Therefore, this subsection would not 
apply to oral dispute resolution communications. Citation: 5 U.S.C. 
574(j).
24. If parties agree to alternative confidentiality procedures, are 
dispute resolution communications subject to FOIA?
    Parties may agree to confidentiality procedures that differ from 
those provided for in the ADR Act. Parties should be aware, however, 
that the FOIA exemption may not apply to all the communications 
protected under their agreement.
    If the agreement provides for the same or more disclosure than 
provided by the Act, dispute resolution communications are exempt from 
disclosure under FOIA. If the agreement provides for less disclosure, 
communications are not exempt from disclosure under FOIA. The ADR Act, 
in effect, establishes a ceiling on the extent to which confidential 
communications will be exempt. Parties cannot contract for more FOIA 
protection than the ADR Act provides.

V. Model Confidentiality Statement for Use by Neutrals

    The confidentiality provisions of the Administrative Dispute 
Resolution Act

[[Page 59208]]

(ADR Act) apply to this process. Generally, if you tell me something 
during this process, I will keep it confidential. The same is true for 
written documents you prepare for this process and give to me. 
[Similarly, you are generally required to keep information confidential 
that you receive during conversations with other parties or me and from 
writings prepared for this process. *
---------------------------------------------------------------------------

    * Include for multi-party disputes.
---------------------------------------------------------------------------

    Be advised, there are limits on our ability to keep information 
confidential. If you say something or provide documents to all the 
other parties it is not confidential. Under rare circumstances, a judge 
can order disclosure of confidential information. Even though not 
required by the ADR Act, information about a violation of criminal law, 
or an act of fraud, waste, or abuse, or an imminent threat of serious 
harm may have to be disclosed to appropriate authorities by a 
participant, but not necessarily by me.
    You can agree to more confidentiality if you want to. For example, 
you can agree to keep confidential things you share with all the 
parties. If you want to do any of that, it will require the agreement 
of all parties and should be memorialized in writing. You should be 
aware that if you agree to more confidentiality, written documents may 
still be available to others, for example, through the Freedom of 
Information Act. Confidentiality provisions other than those in the ADR 
Act may also apply to this process.

ADR Program Evaluation Recommendations

I. Introduction

    The alternative dispute resolution (ADR) field has long promoted 
the various benefits of using non-traditional methods to resolve 
disputes, such as savings of time and money, party satisfaction with 
the ADR process and outcomes, high settlement rates, and improved 
relationships. The ADR Council recognizes that ADR has the potential to 
produce these results, and notes the value of hard data to back up the 
assertion that ADR really delivers these benefits to agencies. The 
Council's Core Principles for Non-binding Workplace ADR Programs [and 
if approved, the ADR Pledge] identify evaluation as a key component of 
successful ADR program management. Up-front and thorough evaluation 
initiatives allow ADR program managers to ensure the quality of their 
programs, to identify programmatic successes and difficulties, and to 
make necessary improvements. Therefore, it is important that all 
federal ADR programs engage in a rigorous evaluation of ADR's use and 
benefits to ensure quality ADR programs and to provide the necessary 
information to sustain and increase support of ADR.
    As the use of ADR becomes institutionalized within federal 
agencies, the government has a heightened interest in evaluating the 
benefits and impact of these dispute resolution initiatives. This type 
of formal evaluation is consistent with the legal obligations of all 
federal programs, under the Government Performance and Results Act 
(Pub. L. 103-62) which requires that agencies create a performance 
plan, define goals, and track the extent to which they achieve their 
desired outcomes. ADR program management best practices emphasize the 
importance of an evaluation component in program design as well as 
practice, and some federal agencies have initiated evaluations of their 
ADR programs. However, the federal sector will benefit from agencies' 
coordinated and uniform efforts at ADR program evaluation.

II. Recommendations

    The Council acknowledges that throughout the government, ADR 
program goals and services differ dramatically among Federal agencies. 
Consequently, it is appropriate to tailor evaluation plans and methods 
to meet the needs of a particular program. Even with agency-specific 
tailoring, effective evaluations will include certain common elements. 
Therefore, to promote consistency and coordination among Federal ADR 
evaluation efforts, the Council makes the following recommendations to 
agencies:
    1. Importance of Evaluation. Each agency should engage in an up-
front and ongoing evaluation of its ADR programs.
    2. Data to be Captured. At a minimum, evaluators should attempt to 
capture and analyze in a timely manner the following information:
    a. Usage: the extent to which ADR is considered and used.
    b. Time Savings: the time it takes for a case to be resolved 
through ADR as compared to traditional dispute resolution processes.
    c. Cost Avoidance: the amount of financial savings (or costs) to 
the agency, including staff time, dollars, or other quantifiable 
factors, by resolving cases through ADR as compared to traditional 
dispute resolution processes.
    d. Customer Satisfaction: parties' satisfaction with the process 
and outcomes, including the quality of the neutral.
    e. Improved Relationships: where ongoing relationships are 
important, to what extent relationships are improved.
    f. Other Appropriate Indicators: in line with the agency's 
strategic goals and objectives.
    3. Validity and Reliability of Data. Methodologies should be valid 
and reliable. ADR program results should be compared to results from 
alternate or previously existing dispute resolution methods.
    4. Presentation of Data. ADR Program Managers should present a 
realistic, accurate and complete picture of the results of their 
program.
    5. Use of Data. ADR success stories should be summarized and 
publicized, to help foster a culture in which ADR is accepted as 
beneficial to Federal agencies and their customers. If areas for 
improvement are identified, that information should be used to enhance 
the ADR program.
    6. Reporting. Federal ADR Program Managers are encouraged to report 
the results of their evaluations to the Federal Interagency ADR Working 
Group.
    7. Potential Resources. In undertaking ADR activities, agencies 
should consult: (1) The Federal ADR Program Manager's Resource Manual, 
Chapter 8: Evaluating ADR Programs, and (2) The Electronic Guide to 
Federal Procurement ADR. Both of these resources, as well as other 
valuable information are available electronically at: 
www.financenet.gov/iadrwg

Evaluating ADR Programs

I. Introduction

    For the past ten years the practice of ADR, the creation of ADR 
programs, and the discipline of ADR evaluation have been developing in 
tandem. We have learned that organizations best design and develop ADR 
programs by knowing an organization's conflict resolution culture, we 
see that evaluation can and should be a reflective feedback mechanism 
for ADR program development, and that evaluation belongs at the 
beginning of ADR program design. While evaluation is ideally present at 
the beginning of ADR program development, we recognize that there are 
many ADR programs already up and running that do not have evaluation 
components. This chapter will address ADR programs at any stage along 
the way of program development.

II. Planning and Designing the Evaluation

    Traditional ADR program evaluation is a way to determine whether an 
ADR program is meeting its goals and

[[Page 59209]]

objectives. Evaluation data are useful in finding out what works and 
what does not work and may be a critical factor in decisions to modify 
or expand a program.
    When planning and designing a federal ADR program evaluation, it is 
important to understand what components of the program are essential to 
comply with federal statutes and initiatives. To the extent that an ADR 
program maintains compliance with federal ADR requirements, it fulfills 
a necessary and useful function for your organization or agency. A good 
design will build upon an existing program structure and will establish 
an evaluation methodology for each program ``core'' area, core areas 
being defined by statute or initiative. Overall program effectiveness 
can then be determined by combining data from all function areas, with 
consideration being given to intangible benefits and consumer 
satisfaction.
    Evaluation is an art as well as a science, even, perhaps, a state 
of mind. It is almost never a linear process. Decisions made early in 
the evaluation planning and design process will almost certainly need 
to be reconsidered and modified as your ADR program grows and develops. 
In addition, traditional cost/benefit analysis does not capture many of 
the benefits derived from ADR service programs because these benefits 
are often intangible and not easily quantifiable. With all of this in 
mind, evaluators need to strive for a workable balance between the need 
for defensible results and practical limitations.
    Key questions to ask when planning and designing an ADR program 
evaluation are:
     What are your goals and objectives for your ADR program 
evaluation?
     How will you pay for your ADR evaluation?
     Who will evaluate your ADR program?
     Who is your audience for this evaluation?
     What is your evaluation design strategy?
     What are your measures of success?

A. What Are Your Goals and Objectives for Your ADR Program Evaluation?

    The goals and objectives of an evaluation should link closely with 
the goals and objectives of the ADR program being evaluated, should 
reflect the needs and interests of those requesting the evaluation, and 
should be sensitive to the needs and interests of the expected 
audiences for the results. Ideally, the ADR program's goals and 
objectives will have been established early on. Sometimes, however, 
these goals may not have been clearly articulated, may not be 
measurable as stated, or may have changed. Evaluators may need to ask 
program managers and other stakeholders to provide input (and hopefully 
arrive at a consensus) on the program's goals, while addressing 
questions such as, how well is the program working, should changes be 
made, should the program be continued or expanded, and how well is the 
ADR program working in a particular federal context?

B. How Will You Pay For Your ADR Evaluation?

    The cost of conducting an ADR program evaluation depends upon a 
number of factors, such as the number and complexity of success 
measures, the type of ADR program selected, the level of statistical 
significance required of the results, the availability of acceptable 
data, and who is selected to carry out the evaluation. Costs can be 
controlled, however, by careful planning, appropriate adjustments in 
the design phase, and a creative use of outside evaluators, from 
universities, for example.

C. Who Will Evaluate Your ADR Program?

    When selecting an evaluator, or a team of evaluators, a number of 
qualifications should be considered. Objectivity (i.e. no stake in the 
outcome) is essential for your results to be seen as credible. An 
evaluator should have sufficient knowledge of the ADR process as well 
as program expertise to design the evaluation, perform the data 
collection process and data analysis as well as present your results to 
your audience if you chose to have the evaluator present your results. 
Such expertise may be found inside some agency policy and program 
evaluation offices, at the U. S. General Accounting Office, or at 
various outside evaluation consulting firms and university departments 
specializing in social science research. Some understanding of the 
organization or the context in which the program operates can be 
helpful to the evaluator, as are good interpersonal and management 
skills.
    Evaluations can be conducted by people outside the agency, within 
the agency but outside the program being evaluated, or by people 
involved with the ADR program. There are advantages and disadvantages 
to each option. An outside evaluator has the potential for the greatest 
impartiality, lending credibility and validity to your results. In 
addition, depending upon the expertise available in a particular 
agency, an outside evaluator may have more technical knowledge and 
experience. Outside evaluation may be relatively expensive, however, 
depending upon the affiliation of the evaluators (e.g. colleges or 
universities, other non-profit groups, or private sector entities such 
as management consulting or social science research firms). If the 
agency has evaluation capacity inside the organization where the ADR 
program is being implemented, the requisite neutrality may be available 
at a potentially lower cost. An inside evaluator involved in ADR 
program implementation or design may be the least expensive, and offer 
the best understanding of program context, but it also carries with it 
potential perceptions of a lack of impartiality. One way to avoid some 
of the disadvantages of each of these approaches is to use a team of 
people, representing internal and external groups.
    Regardless of who does the evaluation (outside or inside), it is 
useful to have someone in the ADR program who can serve as a liaison 
with the evaluator to ensure access to the necessary information. The 
liaison might be the person responsible for planning the evaluation.

D. Who Is Your Audience For This Evaluation?

    There are usually a variety of people who have an interest in the 
results of a program evaluation. These audiences may be interested in 
different issues and seek different types of information. Potential 
audiences should be identified as early as possible, and kept in mind 
while planning the evaluation, so that their questions will be 
addressed.
    Possible audiences for an ADR program evaluation include ADR 
program officials, other agency officials, program users, members of 
Congress, the general public, and others. Agency program officials may 
be interested in finding out how the ADR program is working, and how it 
might be improved. Their interests might focus, for example, on the 
program's impact on case inventory (backlogs), the effects of ADR use 
on long-term relationships among disputants, or how well information 
about the program is being disseminated. Program officials involved in 
the day-to-day operation may have different interests than those at 
higher levels.
    Other agency officials such as budget officers, staff within 
offices of General Counsel and Inspector General, or managers from 
other programs may also have an interest in evaluation results. Budget 
officials may be interested in

[[Page 59210]]

whether cost savings have been achieved through implementation of the 
program. The Inspector General may be interested in the nature of the 
settlements and whether ADR use promotes long-term compliance. General 
Counsels may care about how long it takes to resolve cases or the 
nature of outcomes; other managers may want to know how effectively the 
program was implemented.
    Members of Congress and their staffs may be interested in how ADR 
use affects budgets and how related laws, such as the Administrative 
Dispute Resolution Act, are being implemented. Members of the public 
may be interested in how efficiently the agency is resolving its 
disputes, and how satisfied participants are with ADR processes. 
Disputants may be interested in finding out how typical their 
experience was compared to other users. Officials in other federal 
agencies may find evaluation results helpful as they plan or modify 
their own ADR programs. There may be other audiences whose interests or 
desire for information should be considered.
    Although terminology differs, evaluations are commonly 
characterized as either: (1) Program effectiveness (also known as 
impact, outcome, or summative) evaluations, which focus on whether a 
program is meeting its goals and/or having the desired impact; or (2) 
program design and administration (also known as process or formative) 
evaluations, which examine how a program is operating. Program 
effectiveness evaluations may be useful in determining whether a 
program should be continued or expanded; program design/administration 
evaluations often focus on how a continuing program can be improved.
    Remember that decisions on the future of programs (or even how they 
could be improved) are usually not made solely on the basis of program 
evaluation results. Agency priorities, other institutional concerns, 
budget limitations, and other factors will also affect program 
decisions.
    While it is not possible to satisfy every audience by answering all 
potential questions, it is useful to figure out what the possible 
questions are and then focus the evaluation on the most important ones. 
Talking to members of the various potential audiences can help identify 
the issues they are interested in, and may help develop consensus about 
which issues to address. Such discussions also improve the likelihood 
that evaluation results will be a useful and meaningful part of future 
decision making processes.

E. What Is Your Evaluation Design Strategy?

    ADR program design is based on an understanding that certain 
components of a program are essential to comply with federal statutes 
and initiatives. Program effectiveness evaluations are conducted to 
answer fundamental questions about a program's utility, e.g., does the 
program provide a necessary or useful function, is the program 
accomplishing its goals, and is the program being administered 
effectively. A comprehensive evaluation system measures tangible and 
intangible benefits, including customer satisfaction, using both 
quantitative and qualitative data. To be a useful and effective 
management and planning tool, an evaluation system must do more than 
provide comparison data. It also must provide a flexible process for 
reevaluating the goals of the program, modifying the evaluation 
methodology, and implementing necessary changes.
    Development of an evaluation design might include the following 
steps:
1. Identification and Clarification of ADR Program Goals
    Clear goals and objectives mean that useful conclusions can be 
drawn from the data collected.
2. Development of an Appropriate Evaluation Methodology
    It is necessary to determine what is to be measured and how, what 
the sources of the data are, and how the data will be collected. To do 
this most effectively, core functional areas of ADR program practice 
need to be identified, as do quantitative and qualitative sources of 
data.
3. Development of an Analysis Plan and Research Methodologies
    Traditionally-based experimental designs (time-cost benefit 
analysis) provide statistically reliable results. Program analysis, 
while producing quantifiable results, must go beyond a bare assessment 
of program outcomes to explain the outcomes and to offer suggestions 
for program improvement.
4. Collection Data Mechanisms
    Status reports, case studies, time series collections, agency 
databases, logs, surveys, and evaluation forms are all sources of 
information, as are personal interviews.

F. What Are Your Measures of Success?

1. Program Effectiveness (Impact)
    Program effectiveness measures are aimed at assessing the impact of 
the program on users/participants, overall mission accomplishment, etc.
    The indicators of program effectiveness can be further divided into 
three categories: efficiency, effectiveness, and customer satisfaction.
     Efficiency
    {time}  Cost to the Government of using alternative dispute 
resolution vs. traditional dispute resolution processes:
    Is the use of ADR more or less costly than the use of traditional 
means of dispute resolution? (Cost may be measured in staff time, 
dollars, or other quantifiable factors.)
    {time}  Cost to disputants of using alternative dispute resolution 
vs. traditional dispute resolution processes:
    Is the use of ADR more or less costly than the use of traditional 
means of dispute resolution? (Cost may be measured in terms of staff 
time, dollars, or other quantifiable factors.)
    {time}  Time required to resolve disputes using alternative dispute 
resolution vs. traditional means of dispute resolution:
    Are disputes resolved more or less quickly using ADR, compared to 
traditional means of dispute resolution? Such factors as administrative 
case processing, participant preparation, dispute resolution activity 
timeframes, and/or days to resolution may be considered.
     Effectiveness
    {time}  Dispute Outcomes
    Number of settlements achieved through the use of mediation vs. 
traditional dispute resolution processes:
    Does the use of alternative dispute resolution result in a greater 
or a fewer number of settlements?
    Number of cases going beyond mediation steps:
    Does the use of alternative dispute resolution result in a greater/
fewer number of investigations, further litigation activities, etc.?
    Nature of outcomes:
    What impact does the use of alternative dispute resolution have on 
the nature of outcomes, e.g. do settlement agreements ``look 
different''? Do settlement agreements reflect more ``creative'' 
solutions? Do outcomes vary according to the type of alternative 
dispute resolution process used?
    Correlations for cases selected for alternative dispute resolution, 
between dispute outcomes and such factors as complexity or number of 
issues, or number of parties:
    Is there any correlation, where ADR is used, between the complexity 
and/or number of parties/issues in a case and the outcome of the case?
    {time}  Durability of Outcomes
    Rate of compliance with settlement agreements:

[[Page 59211]]

    Does the use of alternative dispute resolution result in greater or 
lesser levels of compliance with settlement agreements?
    Rate of dispute recurrence:
    Does the use of alternative dispute resolution result in greater or 
lesser levels of dispute recurrence, i.e. recurrence of disputes among 
the same parties?
    {time}  Impact on Dispute Environment
    Size of case inventory:
    Does the use of alternative dispute resolution result in an 
increase/decrease in case inventory?
    Types of disputes:
    Does the use of alternative dispute resolution have an impact on 
the types of disputes that arise?
    Negative impacts:
    Does the use of alternative dispute resolution have any negative 
consequences, e.g. an inability to diagnose and correct systemic 
problem/issues?
    Timing of dispute resolution:
    Does the use of alternative dispute resolution affect the stage at 
which disputes are resolved?
    Level at which disputes are resolved:
    Does the use of alternative dispute resolution have any impact on 
where and by whom disputes are resolved?
    Management perceptions:
    What are the quantitative and qualitative effects of using 
alternative dispute resolution on management, e.g. how does the use of 
ADR impact upon allocation and use of management time and resources? 
Does the use of ADR ease the job of managing?
    Public perceptions:
    Is the public satisfied with alternative dispute resolution 
outcomes? Is there any perceived impact of use of ADR on effectiveness 
of the underlying program? ``Public'' may be defined differently, 
depending on the particular program/setting involved.
     Customer Satisfaction
    {time}  Participants' Satisfaction with Process
    Participants' perceptions of fairness:
    What are participant perceptions of access to alternative dispute 
resolution, procedural fairness, fair treatment of parties by neutrals, 
etc.?
    Participants' perceptions of appropriateness:
    What are participant perceptions of appropriateness of matching 
decisions (i.e. matching of particular process to particular kinds of 
disputes or specific cases)?
    Participants' perceptions of usefulness:
    What are participant perceptions of the usefulness of alternative 
dispute resolution in the generation of settlement options, the 
quantity and reliability of information exchanged, etc.?
    Participants' perceptions of control over their own decisions:
    Do participants feel a greater or lesser degree of control over 
dispute resolution process and outcome through the use of alternative 
dispute resolution? Is greater control desirable?
    {time}  Impact on Relationships Between Parties
    Nature of relationships among the parties:
    Does the use of alternative dispute resolution improve or otherwise 
change the parties' perceptions of one another? Is there a decrease or 
increase in the level of conflict between the parties? Are the parties 
more or less likely to devise ways of dealing with future disputes? Are 
the parties able to communicate more directly or effectively at the 
conclusion of the ADR process and/or when new problems arise?
    {time}  Participants' Satisfaction with Outcomes
    Participants' satisfaction with outcomes:
    Are participants satisfied or unsatisfied with the outcomes of 
cases in which alternative dispute resolution has been used?
    Participants' willingness to use alternative dispute resolution in 
the future:
    Would participants elect to use alternative dispute resolution in 
future disputes?
2. Program Design and Administration (Structure and Process)
    How a program is implemented will have an impact on how effective a 
program is in meeting its overall goals. Program design and 
administration measures are used to examine this relationship and to 
determine how a program can be improved.
    The indicators of program design and administration are further 
divided into three categories: program organization, service delivery, 
and program quality.
     Program Organization
    {time}  Program structure and process:
    Are program structure and process consistent with underlying laws, 
regulations, executive orders, and/or agency guidance? Do program 
structure and process adequately reflect program design? Are program 
structure and process adequate to permit appropriate access to and use 
of the program?
    {time}  Directives, guides, and standards:
    Do program directives, guides, and standards provide staff/users 
with sufficient information to appropriately administer/use the 
program?
    {time}  Delineation of responsibilities:
    Does the delineation of staff/user responsibilities reflect program 
design? Is the delineation of responsibilities such that it fosters 
smooth and effective program operation?
    {time}  Sufficiency of staff (number/type):
    Is the number/type of program staff consistent with program design 
and operational needs?
    {time}  Coordination/working relationships:
    Is needed coordination with other relevant internal and external 
individuals and organizations taking place? Have effective working 
relationships been established to carry out program objectives?
     Service Delivery
    {time}  Access and Procedure
    Participant access to alternative dispute resolution:
    Are potential participants made aware of the program? Is the 
program made available to those interested in using ADR?
    Relationship between participant perceptions of access and usage of 
alternative dispute resolution:
    What impact do participants' perceptions about the availability of 
the program have on the levels of program usage?
    Participant understanding of procedural requirements:
    Do program users understand how the program works? Did they feel 
comfortable with the process in advance?
    Relationship between procedural understanding and rates of usage:
    Is there any relationship between the level of participant 
understanding and the degree of program use, e.g. is a lack of 
participant understanding serving as a disincentive to using the 
program?
    {time}  Case Selection Criteria
    Participants' perceptions of fairness, appropriateness:
    Do participants feel that appropriate types of cases are being 
handled in the program? Do participants or non-participants feel that 
the criteria for which cases are eligible for alternative dispute 
resolution are fair? Are cases being sent to the program at the 
appropriate dispute stages?
    Relationship between dispute outcomes and categories of cases:
    Is there a correlation between the nature (size, types of 
disputants, and/or stage of the dispute) of cases and the outcome of 
the dispute? Are certain types of cases more likely to be resolved 
through alternative dispute resolution than other types?
     Program Quality
    {time}  Training

[[Page 59212]]

    Participants' perceptions of the appropriateness of staff and user 
training:
    Do participants feel that they were provided with sufficient 
initial information and/or training on how to use the program? Do they 
feel that program staff had sufficient training and/or knowledge to 
appropriately conduct the program?
    Relationship between training variable and dispute outcomes:
    Is there a relationship between the type/amount of training (for 
participant and/or staff) and dispute outcomes?
    {time}  Neutrals
    Participants' views of the selection process:
    Are participants satisfied with the manner in which neutrals were 
selected and assigned to cases? Were they involved in the selection 
decision? If not, did they feel they should be?
    Relationship between participants' views of the selection process, 
perceptions of neutral competence and objectivity, and dispute 
outcomes:
    Is there any relationship between participant views about the 
neutrals selection process and dispute outcomes? How do these views 
affect participants' assessment of the competence and neutrality of 
neutrals?
    Participants' perceptions of competence (including appropriateness 
of skill levels/training):
    Do participants feel that neutrals were sufficiently competent or 
trained? Do participants feel that more or less training was needed?
    Participants' perceptions of neutrality/objectivity:
    Do participants feel that neutrals were sufficiently objective? Do 
participants feel that neutrals were fair in their handling of the 
dispute?

G. Other Specific Program Features

    Every dispute resolution program is unique. Those requesting and/or 
conducting an evaluation may want to consider examining other aspects 
of the program. These unique features may relate to the design of a 
program, who was and continues to be involved in program design and 
administration, etc. Each is likely to have at least some impact on 
service delivery and the quality of the program, and should be 
considered for inclusion in either a comprehensive or selected 
evaluation of the program, as appropriate.

II. Presentation, Dissemination, and Use of Results

    Results should be communicated in ways that will allow meaningful 
decisionmaking by program administrators and decisionmakers.
    It is easier to make decisions about the best way to present and 
disseminate results if the people who will use the results (the 
audience) have been consulted during the initial and subsequent 
evaluation processes. Such consultation can avoid costly or 
embarrassing errors; e.g., omission of a key area for analysis, and can 
ensure the report meets the needs of those who will be using it.

A. What Is the Best Method for Communicating Your Findings?

    There are a variety of ways that evaluators can communicate results 
to potential audiences. Evaluators or program staff may provide 
briefings, hold meetings with users, and/or prepare a written report.
    Briefings and presentations allow evaluators or program staff to 
convey important evaluation information quickly and selectively. In 
selecting material to be presented, care should be taken to avoid bias 
or presentation of material out of context. Some discussion of 
methodology is important, as are appropriate cautions about the limits 
and appropriate use of evaluation data. Providing for interaction with 
or feedback from the audience may allow issues and potential problems 
to be identified.
    Written reports typically take a great deal of time to prepare, but 
allow evaluators to provide considerably more detail on both 
methodology and results. Legislation or executive decisions often 
require a final, written report. If it is important to ensure that 
there is one ``official'' source of information on evaluation 
methodology and results, a formal, written report may be an important 
and/or required format in addition to briefings and presentations by 
evaluators or staff.

B. What Kind of Information Needs to Be Communicated?

    Although the potential audiences, program content, and evaluation 
objectives will vary for each ADR program evaluation, it is generally 
helpful to include the following kinds of information in a report or 
other type of presentation:
     Description of the ADR program and how it operates;
     Goals and objectives of the evaluation;
     Description of the evaluator's methodology;
     Presentation of evaluation findings;
     Discussion of program strengths and weaknesses;
     Implications for program administration (e.g., training, 
budget, staff.); and
     Recommendations as appropriate.
    Presentation style is entirely a matter of what works for whom. It 
is always important, however, to make sure that evaluation data are 
presented accurately and completely, to prevent charges of 
misrepresentation or overreaching, and to avoid misuse of results.

B. How Can You Enhance the Effectiveness of Your Presentation?

    Variations in presentation format and style aside, we offer the 
following suggestions for making the presentation of evaluation results 
as effective as possible.
     Involve potential users as early as possible in 
determining presentation format and style:
    Evaluation data should be organized and communicated in a way that 
is useful for potential audiences and users.
     Tailor presentation method, format, and style to audience 
needs:
    Select the method of presentation (e.g., oral briefing, written 
report), format, and style of presentation (e.g., formal vs. informal, 
briefing vs. discussion) based on who your audience is and what their 
needs are. There may be multiple audiences with multiple needs. Be 
flexible and willing to adapt material as appropriate.
     Be clear and accurate:
    Evaluation information must be presented clearly and accurately. 
Always keep the audience in mind as you prepare to describe your ADR 
program and present evaluation data. Avoid any gaps in describing the 
program or presenting the results. A clear and accurate portrayal of 
the program and evaluation results will allow the audience to draw 
appropriate conclusions about program effectiveness and any need for 
change.
     Be honest and direct:
    Sharing evaluation findings with potential users and involving them 
in key decisions concerning presentation format and style does not mean 
publishing only those findings that reflect well on the program or 
those affiliated with it. Evaluators must present the story 
objectively; too heavy an emphasis on the positive may cast doubt on 
the integrity of the results as well as the integrity of the 
evaluators. Data that suggest weaknesses in program design or 
administration or that reveal failure to accomplish program goals or 
objectives should be reported and can be used as a basis for suggesting 
appropriate changes. Honest analysis and thoughtful consideration of 
the information will enhance both the credibility and usefulness of the 
results.
     Keep the body of the report or the bulk of the 
presentation simple: Reduce

[[Page 59213]]

complex data to understandable form, use graphic illustrations where 
appropriate. Evaluation results must be presented so that the most 
essential data are available, understandable, and useful. Too complex a 
format or over-reliance on narrative may detract from evaluation 
results and analysis. Organize the presentation or report for multiple 
uses. Use headings and subheadings to help the audience identify useful 
information quickly.
    Limit the use of technical jargon. Prevent misinterpretation or 
misuse by considering how the data will look if lifted from the context 
of the presentation or report. Use simple graphics to illustrate 
results and call attention to key findings. Use footnotes and make 
technical data available in handouts or appendices so that the body of 
the presentation or report is as uncomplicated as possible.
     Provide an executive summary or abstract:
    Evaluators should provide an overview. The ``quick take'' should be 
supplemented by more detailed discussion later in the report.
     Make survey instruments and other data collection tools 
available: Materials can be made available as handouts, at an oral 
presentation or face-to-face meeting, or as appendices to a written 
report. The availability of such material enhances both understanding 
and credibility. It also allows other ADR program evaluators to learn 
from the experiences of their peers.
     Note limitations on the interpretation and use of 
evaluation data, where appropriate: Limitations on the interpretation 
of the data, such as those that might relate to the ability to study 
results, should be communicated to the audience. Evaluators need to 
exercise caution in expressing their own views and conclusions. Where 
conclusions are not an objective reflection of the data, they need to 
be labeled appropriately; i.e., as the views of the evaluators and not 
necessarily of officials responsible for the program.
     Expect the need for follow-up; be flexible and responsive:
    Have extra copies of reports and presentation handouts available. 
Keep materials accessible. Provide addresses and telephone numbers for 
follow-up discussion or questions. Be available for consultation. Stay 
abreast of how results are being used; provide clarification or added 
direction in the case of misinterpretation or misuse. Prepare 
additional materials as needed. Tailor subsequent releases to customer 
needs.

B. Who Is Responsible for Making Decisions Regarding the Dissemination 
of Evaluation Results?

    It is important to think about dissemination of the results at two 
points: early in the planning process, and again as results become 
available. Decisions about dissemination may be made solely by the 
evaluator, solely by program officials or other entity that has 
requested the evaluation, or, more typically, cooperatively. Such 
decisions may be circumscribed by contract or agreement, or may be 
discussed and resolved informally by evaluators and decisionmakers.

C. When Should Evaluation Results Be Made Available?

    Decisionmakers need to consider the implications of releasing 
evaluation results at different times. For example, if you want 
publicity for the results, select slower news days. The timing of data 
release may be defined by contract or agreement, or may otherwise be 
discussed and resolved by evaluators and decisionmakers. Releasing 
preliminary data before all data are collected or analyzed may be 
risky.

D. How Widely Will Evaluation Results Be Disseminated?

    Evaluation results may be disseminated widely or narrowly. Cost, 
convenience, and level of interest are likely to play a role. It is 
rare that either the evaluator or program officials will have complete 
control over dissemination of the results.

E. How Will Evaluation Results Be Disclosed Initially?

    Evaluation results can be initially disclosed in different ways, 
with more or less fanfare. They may be made available to the selected 
audiences by memorandum, by press release, by press conference, etc. 
Typically, such decisions will be made at the executive level, by those 
who have the authority to make the disclosure.

Evaluation Checklist

     Is your ADR program ongoing or in the formative stage?
     What are your goals and objectives for your ADR program 
evaluation?
     How will you pay for your ADR program evaluation?
     Who will do the evaluation?
     Who is your audience?
     What is your evaluation design strategy?
     What are your measures of success?
     What do you need to know about your program effectiveness 
(impact)?
     What do you need to know about your program structure and 
administration?
     How and when will you disseminate your evaluation results?

Resources

Administrative Conference of the United States. (1995). Dispute 
Systems Design Working Group. Evaluating ADR Programs: A Handbook 
for Federal Agencies. Washington, D.C.: Administrative Conference of 
the United States.
Brett, J. M., Barsness, Z. I., & Goldberg, S. B. (1996). The 
Effectiveness of Mediation: An Independent Analysis of Cases Handled 
by Four Major Service Providers. Negotiation Journal, 12(3), 259-
269.
Costantino, Cathy and Sickles-Merchant, Christine. (1996). Designing 
Conflict Management Systems: A Guide to Creating Productive and 
Healthy Organizations. Jossey-Bass.
Empowerment Evaluation: http://www.stanford.edu/davidf/
empowermentevaluation.html
Federal Deposit Insurance Corporation. (1999). Checklist for 
Evaluation of Federal Agency ADR Programs: Short and Long Term. 
Attorney General's ADR Working Group, Workplace Session Notes, 5/18/
99.
Federal Deposit Insurance Corporation. (1997). ADR Program 
Evaluation Project, Annual Report.
Galanter, M. (1989). Compared to What? Assessing the Quality of 
Dispute Processing. Denver University Law Review, 66(3), xi-xiv.
Honeyman, C. (1990). On Evaluating Mediators. Negotiation Journal, 
23-36.
Honeyman, C. (1995). Financing Dispute Resolution. Madison, WI: 
Wisconsin Employment Relations Commission.
McEwen, C. A. (1991). Evaluating ADR Programs. In F. E. A. Sander, 
Emerging ADR Issues in State and Federal Courts. Washington, D.C.
Patton, Michael. (1990). Qualitative Evaluation and Research 
Methods. Sage: Beverly Hills, CA.
Posovac, Emil J. and Raymond B. Carey. (1997). Program Evaluation: 
Methods and Case Studies, 5th Edition. Prentice Hall Humanities/
Social Sciences.
Rossi, Peter and Howard Freeman. (1993). Evaluation: A Systematic 
Approach. Sage: Beverly Hills, CA.
Scher, E. (1996). Evaluations: What for, by Whom, Who Pays? 
Consensus, October 5, 7-8.
Susskind, L. E. (1986). Evaluating Dispute Resolution Experiments. 
Negotiation Journal, April, 135-139.
Tyler, T. (1989). The Quality of Dispute Resolution Procedures and 
Outcomes. Denver University Law Review, 66, 419-436.
Wholey, Joseph S., Harry P. Hatry, and Kathryn E. Newcomer, Eds. 
(1994). Handbook of Practical Program Evaluation. Jossey-Bass.
Worthen, B.R., J.R. Sanders, and J. Fitzpatrick. (1997). Program 
Evaluation: Alternative Approaches and Practical Guidelines. 
Addison, Wesley, Longman.

[[Page 59214]]

    This document was written by Lee Scharf, ADR Specialist at the 
Environmental Protection Agency, and draws from the work of Cathy 
Costantino and Christine Sickles-Merchant as well as that of the 
Administrative Conference of the United States. See the Resources 
section for cites.

[FR Doc. 00-25397 Filed 10-3-00; 8:45 am]
BILLING CODE 4410-AR-U
```