Administrative Conference of the United States
PUBLIC ENGAGEMENT WITH AGENCY
RULEMAKING

Final Report: November 19, 2018

Michael Sant’Ambrogio
Michigan State University
College of Law

Glen Staszewski
Michigan State University
College of Law

This report was prepared for the consideration of the Administrative Conference of the United States. The
opinions, views and recommendation expressed are those of the authors and do not necessarily reflect
those of the members of the Conference or its committees, except where formal recommendations of the
Conference are cited.

PUBLIC ENGAGEMENT WITH AGENCY RULEMAKING
Final Report for the Administrative Conference of the United States

Michael Sant’Ambrogio & Glen Staszewski

TABLE OF CONTENTS
Table of Contents .......................................................................................................... i
Introduction.................................................................................................................. 1
I. Study Methodology .................................................................................................. 8
II. Reasons For Public Engagement in Rulemaking .................................................... 9
A. More Effective Regulations ............................................................................ 10
B. Democratic Accountability and Legitimacy................................................... 12
C. Public Support for Regulations ....................................................................... 16
III. Enhancing Public Participation in Rulemaking ................................................... 17
A. The Notice-and-Comment Process ................................................................. 17
B. Obstacles to Broader Public Participation in Rulemaking ............................. 19
C. The Challenges of Mass Comments ............................................................... 21
D. Lowering the Barriers to Participation by Absent Stakeholders .................... 26
1. Improving the Effectiveness of Regulation .................................................. 26
2. Enhancing the Democratic Accountability, Legitimacy, and
Public Support For Regulation ..................................................................... 28
E. Beyond Notice and Comment ......................................................................... 29
IV. Levels of Public Engagement .............................................................................. 30
V. The Chronology of Public Engagement in Rulemaking ....................................... 33
VI. Agenda Setting .................................................................................................... 35
A. Rulemaking Petitions...................................................................................... 37
B. Federal Advisory Committees ........................................................................ 44
C. Focus Groups .................................................................................................. 48
D. Requests for Information (“RFIs”) ................................................................. 50
E. Listening Sessions .......................................................................................... 52
F. Reverse Industry Days .................................................................................... 54
G. Hotlines or Suggestion Boxes ........................................................................ 57
H. Public Complaints .......................................................................................... 60
I. Public Notice and Comment ............................................................................ 61
VII. Early Rule Development .................................................................................... 62
A. Federal Advisory Committees ........................................................................ 62
B. Focus Groups .................................................................................................. 64
i

C. Requests for Information (“RFIs”) ................................................................. 65
D. Listening Sessions .......................................................................................... 65
E. Internet and Web-Based Outreach .................................................................. 68
1. Prior ACUS Recommendations Concerning the Internet and Social Media 69
2. Successful Uses of the Internet and Social Media ........................................ 71
3. Challenges of Using the Internet and Social Media ..................................... 72
4. Best Practices for Using the Internet and Social Media
to Engage Absent Stakeholders .................................................................... 73
F. Using Public Input and Providing Feedback to Participants .......................... 75
VIII. Advanced Rule Development ........................................................................... 77
A. Advance Notices of Proposed Rulemaking .................................................... 78
B. Federal Advisory Committees ........................................................................ 80
C. Focus Groups .................................................................................................. 80
D. Public Meetings .............................................................................................. 80
E. “Shuttle Diplomacy” & Technical Workshops ............................................... 81
F. Internet and Web-based Outreach ................................................................... 82
G. Negotiated Rulemaking .................................................................................. 84
IX. The Notice-and-Comment Process ...................................................................... 89
A. Plain Language NPRMs ................................................................................. 91
B. Using Social and Visual On-line Media to Publicize Rulemakings ............... 94
C. User-Friendly Rulemaking Dockets ............................................................... 98
D. Effective Commenting Tutorials .................................................................. 105
E. Public Meetings ............................................................................................ 107
F. Supplemental Deliberative Exercises ........................................................... 108
G. Reply Comment Periods ............................................................................... 111
H. Status Reports and Notifications .................................................................. 114
I. Acknowledgment and Impact Reports .......................................................... 114
X. Retrospective Review ......................................................................................... 114
A. “Open” or “Living” Rulemaking Dockets .................................................... 118
B. Petitions for Rulemaking .............................................................................. 121
C. Requests for Information .............................................................................. 122
D. Public Meetings ............................................................................................ 124
E. Federal Advisory Committees ...................................................................... 125
F. Focus Groups ................................................................................................ 128
G. Public Notice and Comment ......................................................................... 128
XI. Enhanced Forms of Deliberation ....................................................................... 128
A. Deliberative Public Engagement in Government Decision-making ............ 130
B. In-person Versus Online Deliberative Exercises .......................................... 133
C. Best Practices for Enhanced Deliberative Exercises .................................... 134
XII. The Importance of Planning and Outreach ...................................................... 138
A. General Public Engagement Policies............................................................ 138
B. Specific Plans for Public Engagement for Each Rulemaking Initiative ....... 143
C. Outreach and Communication ...................................................................... 149
ii

XIII. Recommendations ........................................................................................... 157
A. Public Engagement Planning and Infrastructure .......................................... 157
1. Public Engagement Policies ....................................................................... 157
2. Public Engagement Plans ........................................................................... 157
3. Enhancing Agency Capacity for Public Engagement................................. 158
B. Agenda Setting ............................................................................................. 159
1. Petitions for Rulemaking ............................................................................ 159
2. Hotlines and Suggestion Boxes .................................................................. 160
3. Requests for Information (RFIs)................................................................. 160
4. Listening Sessions ...................................................................................... 161
5. Advisory Committees ................................................................................. 161
6. Focus Groups .............................................................................................. 161
7. Public Complaint Databases ....................................................................... 162
8. Public Notice and Comment ....................................................................... 162
C. Early and Advanced Rule Development ....................................................... 162
1. Traditional or Negotiated Rulemaking ....................................................... 162
2. Requests for Information (RFIs)................................................................. 162
3. Internet and Web-Based Outreach .............................................................. 163
4. Advance Notices of Proposed Rulemaking (ANPRMs)............................. 164
5. Advisory Committees ................................................................................. 164
6. Listening Sessions and Public Meetings .................................................... 164
7. Focus Groups .............................................................................................. 165
D. The Notice-and-Comment Process............................................................... 165
1. Plain Language NPRMs ............................................................................. 165
2. Regulations.gov .......................................................................................... 165
3. User-Friendly Rulemaking Portals ............................................................. 166
4. Supplemental Deliberative Exercises ......................................................... 166
5. Reply Comment Periods ............................................................................. 167
E. Retrospective Review ................................................................................... 168
1. Preambles to Final Rules ............................................................................ 168
2. Open or Living e-Rulemaking Dockets ...................................................... 168
3. Requests for Information (RFIs)................................................................. 168
4. Advisory Committees ................................................................................. 169
5. Focus Groups .............................................................................................. 169
6. Listening Sessions and Public Hearings ..................................................... 169
Appendix A ………………………………………………………………………..171
Appendix B………………………………………………………………………...174
Appendix C………………………………………………………………………...175

iii

PUBLIC ENGAGEMENT WITH AGENCY RULEMAKING
Final Report for the Administrative Conference of the United States
Michael Sant’Ambrogio & Glen Staszewski 1
INTRODUCTION
Notice-and-comment rulemaking is frequently praised for its highly democratic
character. 2 Before promulgating a final rule, federal agencies generally must apprise
the public of their plans and afford any interested person the opportunity to comment
on the proposal. 3 Moreover, agencies review the comments and must respond to any
that raise significant regulatory issues, regardless of their source. 4 The public

1

The authors are both professors of law at Michigan State University College of Law.
We are indebted to the many participants in this study who gave generously of their time.
We also gratefully acknowledge the research assistance of Elliott Borchardt and the
enormously helpful comments from the staff of the Administrative Conference of the United
States, especially Cheryl Blake, Reeve Bull and Francis Massaro, attorney advisor for this
project, and Cary Coglianese, Chair of the Committee on Rulemaking. Finally, we benefited
enormously from discussions concerning many of the issues addressed in this report with Deb
Dalton, Neil Eisner, Cynthia Farina, John Kamensky, Yogin Kothari, Carolyn Lukensmeyer,
Nina Mendelson, Sabeel Rahman, Genna Reed, Peter Shane, and Bill West, as well as
feedback based on early presentations of the project at the 2018 Annual Meeting of the Law
& Society Association and the Third Annual Administrative Law New Scholarship
Roundtable.
This report was prepared for the consideration of the Administrative Conference of the
United States. The opinions, views and recommendations expressed are those of the authors
and do not necessarily reflect those of the members of the Conference or its committees,
except where formal recommendations of the Conference are cited.
2
See KENNETH CULP DAVIS, DISCRETIONARY JUSTICE: A PRELIMINARY INQUIRY 65-66
(1969) [hereinafter DAVIS, DISCRETIONARY JUSTICE] (“Rule-making procedure which allows
all interested parties to participate is democratic procedure.”); CORNELIUS M. KERWIN &
SCOTT R. FURLONG, RULEMAKING: HOW GOVERNMENT AGENCIES WRITE LAW AND MAKE
POLICY 31 (4th ed. 2011) (“Rulemaking adds opportunities for and dimensions to public
participation that are rarely present in the deliberations of Congress or other legislatures.”);
Donald Kochan, The Commenting Power: Agency Accountability Through Public
Participation, 70 OKLA. L. REV. 601, 601 (2018) (“The commenting power given to ordinary
individuals is rather extraordinary.”).
3
See Administrative Procedure Act, 5 U.S.C. §§ 553(b)-(c) (2012). The notice and
comment requirements of Section 553 do not apply to “interpretative rules, general
statements of policy, or rules of agency organization, procedure, or practice; or when the
agency for good cause finds . . . that notice and public procedure thereon are impracticable,
unnecessary, or contrary to the public interest.” Id. §§ 553(b)(A)-(B).
4
See, e.g., United States v. N.S. Food Prods. Corp., 568 F.2d 240, 252-53 (2d Cir. 1977)
(quoting Auto. Parts & Accessories Ass’n v. Boyd, 407 F.2d 330, 338 (2d Cir. 1968) and

1

comments become part of the record in the event of judicial review of the final rule,
encouraging agencies to give public comments careful consideration and respond to
them in a reasoned fashion. Thus, it is hard to imagine a government decisionmaking process more open and accessible to the public, at least formally.
Yet there is a widespread perception that in practice certain sophisticated
stakeholders dominate the notice-and-comment process. 5 These stakeholders
typically include the regulated entities, industry groups, and professional associations
that have the motivation, resources, and capacity to participate regularly and
effectively in agency rulemaking. 6 National public interest organizations that focus
on the benefits of regulation to a broader public do also sometimes participate, but
these groups are generally not as active as parties concerned with compliance costs. 7
Typically less present, however, are most regulatory beneficiaries; smaller regulated
entities; state, local, and tribal governments; unaffiliated experts; citizens with
practical knowledge of the regulatory issues; and members of the general public. 8
The absence of many stakeholders from the regulatory process can undermine the
goals of public participation in rulemaking. First, it can undermine the effectiveness
of regulations. Absent stakeholders may have important information “about impacts,
ambiguities and gaps, enforceability, contributory causes, [and] unintended
consequences” based on “their lived experience in the complex reality into which the
Associated Indus. of N.Y. State, Inc. v. U.S. Dep’t of Labor, 487 F.3d 342, 352 (2d Cir.
1973) (requiring agencies to address “major issues” discussed in the public comments)).
5
See Cynthia R. Farina et al., Knowledge in the People: Rethinking “Value” in Public
Rulemaking Participation, 47 WAKE FOREST L. REV. 1185, 1191, 1194 (2012) [hereinafter
Farina et al., Knowledge in the People] (citing Mariano-Florentino Cuéllar, Rethinking
Regulatory Democracy, 57 ADMIN. L. REV. 411, 414 (2005) (defining “sophisticated
stakeholders” and discussing how “sophisticated stakeholders” make significantly more
comments than others)).
6
See Stephen M. Johnson, Beyond the Usual Suspects: ACUS, Rulemaking 2.0, and a
Vision for Broader, More Informed, and More Transparent Rulemaking, 65 ADMIN. L. REV.
77, 78 (2013) (collecting views that rulemaking is “dominated by regulated entities and
industry groups, rather than public interest groups”). But cf. Daniel E. Walters, Capturing
Regulatory Agendas?: An Empirical Study of Industry Use of Rulemaking Petitions, 43
HARV. ENT’L L. REV. (forthcoming 2019) (canvassing political science literature on agency
capture and finding that changes triggered by rulemaking petitions “inure mostly to the
benefit of regulated entities, but it is difficult to square with theories of excessive influence or
capture of the regulatory process by business interests”).
7
See Johnson, supra note 6, at 78 n.2.
8
See Cary Coglianese, Citizen Participation in Rulemaking: Past, Present, and Future,
55 DUKE L.J. 943, 964, 967-68 (2006) [hereinafter Coglianese, Citizen Participation]
(explaining that citizens are not infrequently involved in rulemaking and noting the collective
actions problems of public engagement); Cynthia R. Farina et al., Rulemaking in 140
Characters or Less: Social Networking and Public Participation in Rulemaking, 31 PACE L.
REV. 382, 385-86, 386 n.6 (2011) [hereinafter Farina et al., Rulemaking in 140 Characters or
Less] (citing evidence that only a “limited range of stakeholders” participate in notice and
comment rulemaking).

2

proposed regulation would be introduced.” 9 Harnessing such “situated knowledge”
may improve the quality of agency regulations. 10 Although agencies frequently rely
on interest group organizations to represent the views of individuals who do not
participate in rulemaking, these groups do not always provide an adequate
substitute. 11 Representative organizations may choose not to participate in a
rulemaking for strategic institutional reasons or because of limited resources; they
may not represent the full range of interests and views among their constituencies;
they may lack relevant experiential knowledge; or in some cases, there may be no
such organizations.
Second, barriers to broad and meaningful public participation in agency
rulemaking can weaken democratic accountability and legitimacy. In our democratic
system agencies have an obligation to consider the public’s views when making
discretionary decisions about how to implement their statutory mandates. 12 This
certainly does not mean rulemaking is a plebiscite in which agencies should merely
follow public opinion. 13 But we do expect federal agencies to render an account of
what they are doing based on the “republican idea [that] the business of government
is public business.” 14 Indeed, the absence of electoral controls calls for heightened

9

Farina et al., Knowledge in the People, supra note 5, at 1197.
See Memorandum from Barack Obama, President of the U.S., to the Heads of
Executive Departments and Agencies (Jan. 21, 2009) (“Public engagement enhances the
Government's effectiveness and improves the quality of its decisions. Knowledge is widely
dispersed in society, and public officials benefit from having access to that dispersed
knowledge.”).
11
See, e.g., Richard B. Stewart, The Reformation of American Administrative Law, 88
HARV. L. REV. 1669, 1767 (1975) (“There are no accepted means of determining whether the
views of the organization are congruent with the interests of the broader class.”).
12
See Batterton v. Marshall, 648 F.2d 694, 703 (D.C. Cir. 1980) (“The essential purpose
of according § 553 notice and comment opportunities is to reintroduce public participation
and fairness to affected parties after governmental authority has been delegated to
unrepresentative agencies.”).
13
See Reeve T. Bull, Making the Administrative State "Safe for Democracy": A
Theoretical and Practical Analysis of Citizen Participation in Agency Decisionmaking, 65
ADMIN. L. REV. 611, 633 (2013) [hereinafter Bull, Making the Administrative State “Safe”]
(citing universal rejection of the idea that rulemaking should be a plebiscite); Cynthia R.
Farina et al., Rulemaking vs. Democracy: Judging and Nudging Public Participation that
Counts, 2 MICH. J. ENVTL. & ADMIN. L. 123, 131 (2012) [hereinafter Farina et al.,
Rulemaking vs. Democracy]; Farina et al., Rulemaking in 140 Characters or Less, supra note
8, at 429-30. But see Nina A. Mendelson, Rulemaking, Democracy, and Torrents of E-Mail,
79 GEO. WASH. L. REV. 1343, 1372, 1374-75 (2011) [hereinafter Mendelson, Torrents of EMail] (suggesting that agencies give greater consideration to public policy and value
preferences in certain circumstances).
14
Jeremy Waldron, Accountability: Fundamental to Democracy 19 (N.Y. Univ. Sch. of
Law, Working Paper No. 14-13, 2014), https://lsr.nellco.org/nyu_plltwp/462 (emphasis
omitted). For a comprehensive extension of deliberative democratic theory to administrative
10

3

accountability on the part of agencies to both individuals and groups, by considering
their interests and perspectives, responding to them in a deliberative fashion, and by
giving justifications for regulatory decisions that could reasonably be accepted by
citizens with fundamentally competing views. 15 Thus, agencies have an obligation
not only to render an account of their thinking in a form that can be understood and
accepted, but also in a way that gives the public a meaningful opportunity to
participate in the decision-making process by sharing their experiences and views.
The Administrative Conference commissioned this Report to survey the tools and
practices utilized by federal agencies to enhance public understanding of agency
rulemaking and to foster meaningful participation in the regulatory process by
stakeholders who have traditionally been absent. In addition, the Administrative
Conference seeks to provide agencies with guidance on best practices to help them
invest resources in a way that maximizes the probability that rulewriters obtain high
quality public information throughout the course of the rulemaking process.
We are not writing on a blank slate. The Administrative Conference has already
produced important studies resulting in recommendations on rulemaking comments,16
legal considerations and agency innovations in e-Rulemaking, 17 the use of social
media in rulemaking, 18 plain language in regulatory drafting, 19 negotiated
rulemaking, 20 and other topics discussed in our Report, all of which seek to enhance
public engagement with the regulatory process. Therefore, we seek to build on this
body of work in two main ways.
First, we focus on when and how agencies can encourage greater participation by
traditionally absent stakeholders in the regulatory process. We are less concerned
law, see generally HENRY S. RICHARDSON, DEMOCRATIC AUTONOMY: PUBLIC REASONING
ABOUT THE ENDS OF POLICY (2002).
15
See Glen Staszewski, Political Reasons, Deliberative Democracy, and Administrative
Law, 97 IOWA L. REV. 849, 857 (2012) [hereinafter Staszewski, Political Reasons]; Glen
Staszewski, Reason-Giving and Accountability, 93 MINN. L. REV. 1253, 1255 (2009)
[hereinafter Staszewski, Reason-Giving] (explaining that agencies are held “accountable by a
requirement or expectation that they give reasoned explanations for their decisions”).
16
See generally Admin. Conf. of the U.S., Recommendation 2011-2, Rulemaking
Comments, 76 Fed. Reg. 48,789, 48,791 (Aug. 9, 2011).
17
See generally id.; Admin. Conf. of the U.S., Recommendation 2011-1, Legal
Considerations in e-Rulemaking, 76 Fed. Reg. 48,789, 48,789 (Aug. 9, 2011).
18
See generally Admin. Conf. of the U.S., Recommendation 2013-5, Social Media
in Rulemaking, 78 Fed. Reg. 76,269, 76,269 (Dec. 17, 2013).
19
See generally Admin. Conf. of the U.S., Recommendation 2017-3, Plain
Language in Regulatory Drafting, 82 Fed. Reg. 61,728, 61,728 (Dec. 29, 2017); CHERYL
BLAKE & BLAKE EMERSON, ADMIN. CONFERENCE OF THE U.S., PLAIN LANGUAGE IN
REGULATORY DRAFTING (2017).
20
See generally Admin. Conf. of the U.S., Recommendation 2017-2, Negotiated
Rulemaking and Other Options for Public Engagement, 82 Fed. Reg. 31,039, 31,040 (July 5,
2017).

4

with enhancing participation by the insiders who routinely participate, although our
Report will touch upon ways in which the quality of public participation can be
improved more generally. We focus on the absent public because of the significant
barriers to participation in rulemaking faced by rulemaking novices, and the need for
careful planning and outreach by agencies to overcome these barriers. 21 Public
participation in rulemaking is not like a field of dreams—if you build it, they will not
necessarily come. 22 Absent stakeholders are often unaware that potential rules are
being considered and that they have opportunities to participate. 23 Even when they
are aware, many members of the public lack the incentive to become involved. The
interests at stake for individual beneficiaries of regulation may simply be too small to
justify the time and attention required. 24 Or they may assume someone else will
represent their views or that the agency has already made up its mind. 25
Moreover, even when members of the public are sufficiently motivated to
participate, rulemaking novices may not have the capacity to participate effectively.
Although submitting a comment requires only a few clicks on Regulations.gov,
submitting an effective comment requires much more. In most cases, understanding
the proposed rule requires reading a lengthy, complex, dense, and (for most people)
quite boring NPRM, written at an advanced level of education. 26 And even if
rulemaking novices make it this far, they generally do not know how to submit
effective comments—i.e., comments containing the kinds of information agencies

21

See Farina et al., Rulemaking in 140 Characters, supra note 8, at 389–90 (emphasis
omitted) (citing “[i]gnorance about the rulemaking process[, u]nawareness that rulemakings
of interest are going on[,] and [i]nformation [o]verload from the length, and linguistic and
cognitive density, of rulemaking materials” as three barriers to participation).
22
See, e.g., CYNTHIA R. FARINA & MARY J. NEWHART, IBM CTR. FOR THE BUS. OF
GOV’T, RULEMAKING 2.0: UNDERSTANDING AND GETTING BETTER PUBLIC PARTICIPATION
21 (2013) [hereinafter FARINA & NEWHART, IBM CENTER].
23
See id. at 11; Cynthia R. Farina et al., Democratic Deliberation in the Wild: The
McGill Online Design Studio and the RegulationRoom Project, 41 FORDHAM URB. L.J. 1527,
1550 (2014) (discussing barriers to participation).
24
Regulations often impose concentrated costs on regulated entities and diffuse benefits
on a broader public, creating collective action problems exacerbated by cognitive biases. See
MANCUR OLSON, JR., THE LOGIC OF COLLECTIVE ACTION: PUBLIC GOODS AND THE
THEORY OF GROUPS 53-65 (8th prtg. 1980) (explaining how small, organized groups are
usually more effective than larger, diffuse groups in shaping policy); James Q. Wilson, The
Politics of Regulation, in THE POLITICAL ECONOMY: READINGS IN THE POLITICS AND
ECONOMICS OF AMERICAN PUBLIC POLICY 82, 87-88 (Thomas Ferguson & Joel Rogers eds.,
1984).
25
Wilson, supra note 24, at 85.
26
See FARINA & NEWHART, IBM CENTER, supra note 22, at 12 (describing a
Department of Transportation NPRM as written at a “late-college/early-graduate school
reading level”).

5

seek and the types of arguments agencies are likely to find persuasive. 27 Rulemaking
novices therefore frequently need additional instruction and support to provide
agencies with beneficial information.
Although many believed that the advent of e-Rulemaking and the proliferation of
social media would lower many of the barriers to public participation, this has proven
challenging for various reasons, and technological innovations have not yet
revolutionized public engagement in rulemaking or fully democratized the regulatory
process. 28
Thus, our Report focuses on how agencies have attempted to overcome these
barriers and reach beyond the usual suspects in their rulemakings. Some of these
efforts could be widely adopted in a cost-effective manner, whereas others would
likely prove beneficial only in a limited range of circumstances. 29 Therefore,
agencies must think carefully about precisely when additional efforts are most likely
to prove worthwhile, and we hope to provide some useful guidance on this score.
Second, we seek to broaden the discussion of public participation in rulemaking
beyond the notice-and-comment process and include regulatory agenda setting, early
and advanced rule development, and retrospective review or regulatory “lookbacks.”
The literature on agency rulemaking and efforts to more fully democratize it has
focused overwhelmingly on the notice-and-comment stage. 30 The agenda-setting and
rule-development stages, in contrast, have received much less focused attention. 31
The existing literature generally treats what happens before publication of the notice
of proposed rulemaking (NPRM) as “a black box,” and suggests that rule

27

See Farina et al., Knowledge in the People, supra note 5, at 1187 (explaining that
while agency decision-makers value “objective,” empirical evidence and quantitative data,
presented in analytical, premise-argument-conclusion reasoning, rulemaking novices tend to
offer “highly contextualized, experiential information, often communicated in the form of
personal stories”).
28
See MICHAEL HERZ, ADMIN. CONFERENCE OF THE U.S., USING SOCIAL MEDIA IN
RULEMAKING: POSSIBILITIES AND BARRIERS 2 (2013) [hereinafter HERZ, USING SOCIAL
MEDIA] (“[T]he move online has not produced a fundamental shift in the nature of noticeand-comment rulemaking.”); see also Coglianese, Citizen Participation, supra note 8, at 958
(“[N]either agencies’ acceptance of comments by e-mail nor the development of the
Regulations.gov portal have led to any dramatic changes in the general level or quality of
public participation in the rulemaking process.”).
29
See Farina et al., Rulemaking in 140 Characters or Less, supra note 8, at 423 (“[T]he
public in general likely has little useful knowledge to add to federal rulemaking in general.
This does not mean that segments of the public have nothing useful to add to specific
rulemakings.”) (emphasis added).
30
See William F. West, Inside the Black Box: The Development of Proposed Rules and
the Limits of Procedural Controls, 41 ADMIN. & SOC’Y 576, 577 (2009).
31
See id. at 583-84; CORNELIUS M. KERWIN, IBM CTR. FOR THE BUS. OF GOV’T, THE
MANAGEMENT OF REGULATION DEVELOPMENT: OUT OF THE SHADOWS (2008) [hereinafter
KERWIN, OUT OF THE SHADOWS].

6

development is primarily influenced by political considerations and pressure from
well-organized interest groups. 32 Yet in conducting our research we have spoken to
numerous agency officials who described significant efforts to engage the public with
their agenda setting and rule development activities. In addition, some agencies have
used some of these same tools in the context of retrospective review.
Thus, evaluating public engagement in rulemaking requires a broader and more
holistic view of the regulatory process than taken by much of the literature focused on
the notice-and-comment process. In most cases, agency engagement with the public
does, and should, begin long before the publication of an NPRM. 33 Agency agenda
setting is of the utmost importance to regulatory governance because it determines
which issues or problems agencies will address and which issues or problems will go
unresolved. In addition, the best prospects for more fully democratizing the
rulemaking process may be meaningful and consistent efforts to solicit informed
public engagement during rule development, before the agency has made up its mind
about which course of action to pursue. And regulatory lookbacks could be
handicapped without the participation of stakeholders who live under the regulatory
regime under review.
The public—and, indeed, different publics 34—may have different contributions to
make at each stage of the regulatory process. For example, agencies may seek to
understand the general public’s values, priorities, and preferences when setting their
regulatory agendas or choosing among alternative approaches to a regulatory
problem. Agencies may seek the situated knowledge of stakeholders based on their
practical experiences early in the process of developing a rule or conducting a
regulatory lookback. And agencies may be most interested in technical data about
compliance costs and potential benefits, as well as unanticipated consequences that
stakeholders might reveal, when drafting an NPRM, revising a Final Rule as part of
the notice-and-comment process, or conducting retrospective review. One of the
themes of this Report is that agencies must think carefully at each stage of the
rulemaking process about what information and stakeholders may be missing and use
the available tools that are most likely to generate this information.
To engage the public successfully, agencies need to plan for public participation
early in the regulatory process. This includes developing policies for public
engagement in rulemaking and establishing mechanisms to ensure that those policies

32

See, e.g., Wendy Wagner et al., Rulemaking in the Shade: An Empirical Study of
EPA’s Air Toxic Emission Standards, 63 ADMIN. L. REV. 99, 153 (2011) [hereinafter Wagner
et al., Rulemaking in the Shade]; West, supra note 30, at 583-84.
33
See West, supra note 30, at 584 (noting such efforts occur but they tend to be
unstructured and ad hoc).
34
See Archon Fung, Recipes for Public Spheres: Eight Institutional Design Choices and
Their Consequences, 11 J. POL. PHIL. 338, 338-39 (2003) [hereinafter Fung, Recipes for
Public Spheres] (describing efforts to convene groups of citizens “in self-consciously
organized public deliberations” as “minipublics”).

7

are consistently followed. 35 In addition, agencies should develop specific plans for
public engagement for each regulatory initiative they undertake or seriously consider.
These plans should include internal and external situation assessments, and consider:
(1) why the agency wants to engage the public, (2) who the agency is trying to reach,
(3) what types of information the agency seeks, (4) how this information might be
obtained, (5) when these efforts should occur, and (6) what the agency will do with
the information.
Thus, our Report recommends first and foremost that agencies take a holistic
approach to enhancing public engagement in rulemaking. In addition, our Report
highlights the kinds of information most useful to agencies at each stage of the
regulatory process, the tools and practices that are available to generate such
information from otherwise missing stakeholders, and the best practices for soliciting
meaningful public input or comment.
Finally, we suggest how agencies can
approach public engagement as part of a comprehensive plan that utilizes different
modes of public engagement at different stages of rulemaking in a synergistic
fashion. In this way, we hope to contribute to efforts already underway to more fully
democratize the regulatory process.
I. STUDY METHODOLOGY
We began this project by reviewing the substantial literature on public
participation in notice-and-comment rulemaking and in regulatory governance
generally. This included the literature on enhanced deliberative exercises on public
policy questions. In addition, we reviewed the relevant statutory, executive,
administrative, and judicial authorities bearing on public engagement in the
rulemaking process. Finally, we reviewed previous reports produced for ACUS that
touch on public participation in rulemaking and the recommendations adopted by
ACUS based on those reports.
Then, with the assistance of ACUS and its federal agency contacts, we sent the
written questionnaire included in Appendix A to forty-three federal agencies to
identify the different institutional structures, procedures, and practices used to engage
the public with their regulatory agendas, their rulemaking proceedings, and
retrospective review. We received partial or complete responses from fourteen
agencies. During our follow-up contacts, we also gave agencies the option of
responding to the questionnaire by telephone interview. Between the survey
responses and telephone interviews, we obtained information concerning the efforts
to engage the public of twenty-one federal agencies. 36 They included a mix of large

35

See e.g., ENVTL. PROT. AGENCY, PUBLIC INVOLVEMENT POLICY OF THE U.S.
ENVIRONMENTAL PROTECTION AGENCY (2003) [hereinafter EPA PUBLIC INVOLVEMENT
POLICY]; NAT’L PARK SERV., DIRECTOR’S ORDER #75A: CIVIC ENGAGEMENT AND PUBLIC
INVOLVEMENT (2007) [hereinafter DIRECTOR’S ORDER #75A].
36
The agencies are listed in Appendix B. We use the definition of “agency” set forth in
section 551(1) of the Administrative Procedure Act: “each authority of the Government of the

8

and small agencies, executive branch and independent agencies, and agencies that
engage in rulemaking both frequently and infrequently. Based on the survey
responses, other follow-up with our agency contacts, and our literature review, we
conducted twenty in-depth telephone interviews of agency officials, leading scholars
in the field, and others who have been involved in projects designed to enhance
meaningful public engagement in government policymaking. During our interviews,
we frequently identified other potential sources of information to explore. Where
necessary, we reviewed the rulemaking dockets and public comments associated with
rulemakings that we studied and discuss in this Report. We also reviewed numerous
agency websites and other on-line material relevant to public engagement in
government decision-making.
Finally, we presented preliminary findings and solicited feedback at several
public forums. In April 2018, we met with the Rulemaking Committee of ACUS to
discuss the project and obtain additional input on the reasons for enhancing public
engagement, the tools agencies have found most useful, those which have proved less
successful, and to consider the potential benefits and challenges of enhancing public
engagement. In addition, we discussed various issues addressed in this report in June
2018 at the Annual Meeting of the Law & Society Association held in Toronto,
Canada, and the Third Annual Administrative Law New Scholarship Roundtable held
at the University of Michigan Law School. We are tremendously grateful to everyone
for the time they spent talking with us, and the insights they shared during this
process. A complete list of agencies and non-governmental organization that we
spoke with during the study is included in Appendix B.
II. REASONS FOR PUBLIC ENGAGEMENT IN RULEMAKING
Before turning to the modes of public engagement in rulemaking it is important
to understand the reasons why agencies might want to enhance public participation in
this process, including participation by traditionally absent stakeholders. Scholars,
judges, and agency officials have offered three sets of justifications. 37 First, and
likely of most interest to agencies themselves, the public is an important source of
information for agencies designing regulatory programs. Thus, public engagement
can improve the effectiveness of regulations by providing agencies with better and
more comprehensive information. Second, in a system founded on the principle that
government officials are the agents or trustees of “the people,” public engagement
enhances the democratic legitimacy and accountability of agency regulations. Third,
it is often suggested that the opportunity to participate in an agency’s decisionmaking process will enhance public support for the final rule, even if the agency does

United States, whether or not it is within or subject to review by another agency[.]”
Administrative Procedure Act, 5 U.S.C. § 551(1) (2012).
37
See, e.g., Cary Coglianese et. al., Transparency and Public Participation in the
Federal Rulemaking Process: Recommendations for the New Administration, 77 GEO. WASH.
L. REV. 924, 927 (2009) (describing how transparency and public participation can “enhance
regulators’ ability to achieve society's goal of high-quality and legitimate rules”).

9

not adopt all of the participants’ views. We discuss each of these justifications in
turn.
A.

More Effective Regulations

One of the primary goals of public participation in rulemaking is to provide
agencies with the information they need to promulgate effective rules and regulations.
Rules and regulations are effective when they achieve roughly their intended benefits
at roughly their expected costs in the way anticipated by the agency decision-maker.
This largely turns on the quality of information available to the agency when it is
developing the rule. Congress delegates decisions to agencies, in part, because of
their subject-matter expertise in their regulatory areas. 38 Agencies develop this
expertise through the personnel they hire, the research they conduct, and the
experience they develop administering federal programs. But even agencies with
deep in-house knowledge depend upon outside parties for a great deal of information.
In particular, agencies need information from the industries they regulate, other
experts, and citizens with situated knowledge of the field in order to understand the
problems they seek to address, the potential regulatory solutions, their attendant costs,
and the likelihood of achieving satisfactory compliance. 39
The notice-and-comment process “ensure[s] that agency regulations are tested
via exposure to diverse public comment.” 40 The Administrative Procedure Act
(APA) generally requires agencies to publish proposed rules in the Federal Register
and accept comments on their proposals from any interested member of the public.41
Moreover, agencies must read the public comments and respond to those that raise

38

See David B. Spence & Frank Cross, A Public Choice Case for the Administrative
State, 89 GEO. L.J. 97, 135 (2000) (“[A] commonly cited and crucial reason for the delegation
to agencies is the desire to have decisions made by public officials with expertise and
extensive information-gathering capabilities.”).
39
See STEPHEN BREYER, REGULATION AND ITS REFORM 109 (1982) (“The central
problem of the standard-setting process and the most pressing task facing many agencies is
gathering the information needed to write a sensible standard.”); see also Cary Coglianese, et
al., Seeking Truth for Power: Informational Strategy and Regulatory Policymaking, 89 MINN.
L. REV. 277, 277 (2004) (“Information is the lifeblood of regulatory policy.”); Teresa Moran
Schwartz, The Role of Federal Safety Regulations in Products Liability Actions, 41 VAND. L.
REV. 1121, 1147-48 (1988). But see Wendy E. Wagner, Administrative Law, Filter Failure,
and Information Capture, 59 DUKE L.J. 1321, 1351–52 (2010) [hereinafter Wagner,
Administrative Law] (“[I]nformation gluts . . . can estrange marginally financed interest
groups, undermine the hope of pluralistic engagement that could help the agency sift through
at least some of the incoming information, and ultimately put the agency at the mercy of the
party in control of most of the relevant information.”).
40
United States v. Reynolds, 710 F.3d 498, 517 (3d Cir. 2013) (quoting Prometheus
Radio Project v. Fed. Commc’ns Comm’n, 652 F.3d 431, 449 (2011)).
41
See Administrative Procedure Act, 5 U.S.C. §§ 553(b)-(c).

10

salient issues, explaining why and how they decided to proceed in light of or in spite
of particular comments. 42
These procedures, and public participation in rulemaking more generally, are
designed to provide agencies with more and better information upon which to base
their regulatory choices. 43 Public participation in rulemaking “broadens an agency’s
perspective, which otherwise might not extend beyond the views of the staff or the
client groups with whom the staff regularly consults.” 44 The public may raise
problems the agency has not seen, illuminate direct and collateral effects, propose
solutions the agency has not considered, and identify unintended consequences of
certain actions. Potential regulatory beneficiaries and their advocates can provide
agencies with information about the problems agencies seek to address and the impact
of policies on individuals. 45 Regulated parties can provide agencies with information
about the workability and costs of different proposals, collateral consequences, and
the difficulty of achieving compliance. 46 At the most basic level, public participation
may help to clarify ambiguities in an agency proposal that would undermine the
agency’s goals merely due to confusion on the part of the public regarding what a rule
requires.
The information justification for public participation does not necessarily call for
engaging all members of the public in all rulemakings. Rather, it requires agencies to
engage those members of the public with information the agency needs based on the
particular regulatory decision the agency must make. This will usually be a smaller
“public” than the public as a whole. It will generally include those who are likely to
benefit or be burdened by a regulatory proposal, and those with situated knowledge of
the subject of regulation. But members of these groups may not all have useful
information in equal measure. For example, the agency may possess more
information about the public health consequences of a pollutant than about the

42

See, e.g., United States v. N.S. Food Prods. Corp., 568 F.2d 240, 252 (2d Cir. 1977)
(“It is not in keeping with the rational process to leave vital questions, raised by comments
which are of cogent materiality, completely unanswered.”).
43
See, e.g., TOM. C. CLARK, U.S. DEP’T OF JUSTICE, ATTORNEY GENERAL’S MANUAL
ON THE ADMINISTRATIVE PROCEDURE ACT 31 (1947) (“The objective [of notice and
comment] should be to assure informed administrative action and adequate protection to
private interests.”); cf. FINAL REPORT OF ATTORNEY GENERAL’S COMMITTEE ON
ADMINISTRATIVE PROCEDURE 6 (Senate Document No. 8, 77th Congress, First Session,
1941).
44
Michael Asimow, Nonlegislative Rulemaking and Regulatory Reform, 1985 DUKE L.J.
381, 402–03 (1985).
45
See Richard B. Stewart & Cass R. Sunstein, Public Programs and Private Rights, 95
HARV. L. REV. 1193, 1202-03 (1982) (describing regulatory beneficiaries); Nina A.
Mendelson, Regulatory Beneficiaries and Informal Agency Policymaking, 92 CORNELL L.
REV. 397, 429 (2007) (same).
46
See Wagner, Administrative Law, supra note 39, at 1346; Stewart, supra note 11, at
1713-14 (“[T]he information upon which the agency must ultimately base its decision must
come to a large degree from the groups being regulated.”).

11

feasibility of installing different pollution control devices. Thus, the slice of the
public likely to have the most useful information for the agency will depend upon the
nature of the specific rulemaking. 47
Therefore, the information justification for public participation requires agencies
to design public engagement in a way that is most likely to obtain the information
they need in each particular rulemaking. There is no one-size-fits-all approach. To
be sure, agencies are not always aware of all the information they need. It is one
thing to plan for soliciting public comments on “known unknowns”; another to plan
for “unknown unknowns,” such as unintended consequences. Thus, while designing
public engagement around specific rules, agencies also need more general strategies
to ensure they obtain information they might not anticipate but nevertheless would be
quite valuable for crafting effective regulation. The notice-and-comment process is
one such tool, notwithstanding the shortcomings discussed in this Report. But in
most cases the agency will be able to identify specific information that it needs for a
particular rule and the members of the public most likely to have it. Accordingly, the
need for better information has generally been the primary justification for agency
efforts to enhance public engagement beyond the general provisions of the noticeand-comment process.
B.

Democratic Accountability and Legitimacy

A second justification for public participation in rulemaking is to enhance the
democratic legitimacy and accountability of federal agencies and the regulations they
promulgate. 48 The administrative state has long suffered from questions regarding its
constitutional status and concerns that agencies exercise immense policy-making
authority without electoral checks. 49 Of course, agencies are delegated authority by
Congress and supervised by the political branches. But some question the ability of
Congress and the President to provide sufficient democratic legitimacy to agency
action on their own. Political scientists have long observed that Congress faces acute

47

Farina et al., Rulemaking in 140 Characters or Less, supra note 8, at 423 (“[T]he
public in general likely has little useful knowledge to add to federal rulemaking in general.
This does not mean that segments of the public have nothing useful to add to specific
rulemakings.”) (emphasis added).
48
See, e.g., Beth Simone Noveck, The Electronic Revolution in Rulemaking, 53 EMORY
L.J. 433, 436 (2004) [hereinafter Noveck, The Electronic Revolution] (“Participation . . .
makes regulatory rulemaking more legitimate and accountable.”).
49
JAMES O. FREEDMAN, CRISIS AND LEGITIMACY 10 (1978) (“[C]riticism of the
administrative agencies has been animated by a strong and persisting challenge to the basic
legitimacy of the administrative process itself.”); see also Jody Freeman, Collaborative
Governance in the Administrative State, 45 UCLA L. REV. 1, 4 n.5 (1997) (“The crisis of
legitimacy in administrative law stems from the lack of constitutional status accorded to
administrative agencies and the need for oversight from the three branches of government to
ensure that agency decision making is accountable to the public.”); David Fontana,
Reforming the Administrative Procedure Act: Democracy Index Rulemaking, 74 FORDHAM L.
REV. 81, 100, 100 n.118 (2005).

12

challenges controlling agency discretion after it has been delegated through
legislation. 50 The President is in a better position to shape the actions of his
appointees, and over the past four decades presidents of both parties have built an
institutional framework for centralized review of agency rulemaking. 51 Accordingly,
some legal scholars (and to a large extent the Supreme Court) have turned to the
President to bring greater democratic accountability to agency decisions. 52 But the
President is unlikely to be able to supervise the vast majority of rules promulgated by
federal agencies in any meaningful way. 53 In addition, even if he could, some
question whether any single political representative can provide regulatory actions
with much democratic accountability given the diverse interests and perspectives of
the American public. 54
Moreover, in a democratic republic founded on principles of popular sovereignty,
Congress and the President are not an agency’s only “principals,” and public

50

See, e.g., Matthew D. McCubbins et al., Administrative Procedures as Instruments of
Political Control, 3 J.L. ECON. & ORG. 243, 250-51 (1987) (explaining challenges to
congressional control of the administrative state); Matthew D. McCubbins et al., Structure
and Process, Politics and Policy: Administrative Arrangements and the Political Control of
Agencies, 75 VA. L. REV. 431, 434-47 (1989) (same); Stewart, supra note 11, at 1695 n.128
(same).
51
See Nicholas Bagley & Richard L. Revesz, Centralized Oversight of the Regulatory
State, 106 COLUM. L. REV. 1260, 1263-67 (2006).
52
See, e.g., Chevron U.S.A. Inc. v. Nat. Res. Def. Council, Inc., 467 U.S. 837, 865-66
(1984) (“While agencies are not directly accountable to the people, the Chief Executive is,
and it is entirely appropriate for this political branch of the Government to make such policy
choices—resolving the competing interests which Congress itself either inadvertently did not
resolve, or intentionally left to be resolved by the agency charged with the administration of
the statute in light of everyday realities.”); see also Elena Kagan, Presidential Administration,
114 HARV. L. REV. 2245, 2332 (2001) (“Presidential administration promotes accountability
in two principal and related ways. First, presidential leadership enhances transparency,
enabling the public to comprehend more accurately the sources and nature of bureaucratic
power. Second, presidential leadership establishes an electoral link between the public and
the bureaucracy, increasing the latter's responsiveness to the former.”).
53
See OFFICE OF MGMT. & BUDGET, 2016 DRAFT REPORT TO CONGRESS ON THE
BENEFITS AND COSTS OF FEDERAL REGULATIONS AND AGENCY COMPLIANCE WITH THE
UNFUNDED MANDATES REFORM ACT 7 (2016) (“From FY 2006 through FY 2015, [f]ederal
agencies published 36,289 final rules in the Federal Register.”).
54
Cynthia R. Farina, The Consent of the Governed: Against Simple Rules for a Complex
World, 72 CHI.-KENT L. REV. 987, 988 (1997) (“Strong presidentialism . . . is premised upon
a fundamentally untenable conception of the consent of the governed. The ‘will of the
people,’ as invoked in that effort, is artificially bounded in time, homogenized, shorn of
ambiguities—in short, fabricated.”); Staszewski, Political Reasons, supra note 15, at 867-72
(criticizing the presidential control model as “deeply problematic”).

13

engagement in democratic governance does not end with elections. 55 Consequently,
agencies have a continuing obligation to consider the public’s views when making
discretionary decisions about how to implement their statutory mandates. 56 This does
not mean that rulemaking is a plebiscite in which the agency should merely follow
public opinion. 57 Agencies are not designed to be electorally accountable, and direct
democracy is completely alien to our federal representative system. 58 But agencies’
lack of electoral accountability may be an advantage rather than a defect if we
conceive of democratic accountability more broadly than merely standing for periodic
elections. Political theorists note that democratic accountability also requires
government officials to render a justifiable account of what they are doing on behalf
of the public based on the “republican idea [that] the business of government is public
business.” 59 Furthermore, the relevant public is not only the electorate as a whole,
much less the constituents of a prevailing party suggested by majoritarian politics.
The government does not merely owe a duty of account to “We the People” as a
disembodied, collective whole or the majorities that elected it. Rather, the duty of
account by government “is owed to persons individually, to persons arrayed in ragged
and sometimes ad hoc sub-sets of ‘the people,’ as well as to ‘the people’ itself as a
notionally and occasionally unified entity.” 60 In other words, public officials in a
democracy should be held accountable to everyone who is interested in or affected by
their decisions.
Thus, democratic accountability imposes on agencies an obligation to render an
account to the public of what they are doing and to “[do] so in a form that can be
understood by the [public].” 61 While elected officials are ultimately accountable (in
some sense) to their constituents through elections, the absence of electoral controls
for agencies calls for heightened accountability on their part to individuals and groups
by considering their interests and perspectives, responding to them in a deliberative

55

See Michael Sant’Ambrogio, Standing in the Shadow of Popular Sovereignty, 95 B.U.
L. REV. 1869, 1883-84 (2015) (discussing the relationship between the people and the federal
government established by the Constitution).
56
Batterton v. Marshall, 648 F.2d 694, 703 (D.C. Cir. 1980) (“The essential purpose of
according § 553 notice and comment opportunities is to reintroduce public participation and
fairness to affected parties after governmental authority has been delegated to
unrepresentative agencies.”).
57
See Bull, Making the Administrative State “Safe”, supra note 13, at 633 (citing
universal rejection of the idea that rulemaking should be a plebiscite); Farina et al.,
Rulemaking vs. Democracy, supra note 13, at 131; Farina et al., Rulemaking in 140
Characters or Less, supra note 8, at 436-37. But see Mendelson, Torrents of E-Mail, supra
note 13, at 1375-76 (suggesting that agencies give greater consideration to public policy and
value preferences in certain circumstances).
58
Sant’Ambrogio, supra note 55, at 1888-90 (discussing the Framers’ rejection of direct
democracy while embracing popular sovereignty).
59
Waldron, supra note 14, at 19.
60
Id. at 16.
61
Id. at 7.

14

fashion, and by giving justifications for regulatory decisions that could reasonably be
accepted by citizens with fundamentally competing views. 62 We might take this a
step further and say that agencies have an obligation not only to render an account of
their thinking in a form that could reasonably be understood and accepted, but also in
a way that gives the public a meaningful opportunity to participate in the decisionmaking process. 63
The democratic justification for public engagement in rulemaking conceives of a
broader public than the information justification. To be sure, we might still be
particularly concerned that the most significantly affected interests, whether benefited
or burdened, are heard in the rulemaking process. 64 But democratic theory supports
opportunities for engagement by a broader, and more diverse, public. Democratic
theory suggests that agencies should be accountable in some form to each and every
citizen. At the same time, unlike the information justification for public participation,
democratic theory does not require an agency to design public engagement around the
specific information it needs for each particular rule. To date, agencies have relied

62

Staszewski, Political Reasons, supra note 15, at 857; Staszewski, Reason-Giving,
supra note 15, at 1255 (agencies are held “accountable by a requirement or expectation that
they give reasoned explanations for their decisions”); see also AMY GUTMANN & DENNIS
THOMPSON, WHY DELIBERATIVE DEMOCRACY? 144 (2004) [hereinafter GUTMANN &
THOMPSON, WHY DELIBERATIVE DEMOCRACY?] (explaining that the justifications given by
decision-makers must be understandable to those governed by them); Edward Rubin, The
Myth of Accountability and the Anti-Administrative Impulse, 103 MICH. L. REV. 2073, 2073
(2005) (defining accountability as “the ability of one actor to demand an explanation or
justification of another actor for its actions and to reward or punish that second actor on the
basis of its performance or its explanation”); Mark Seidenfeld, A Civic Republican
Justification for the Bureaucratic State, 105 HARV. L. REV. 1511, 1515 (1992) (“Having
administrative agencies set government policy provides the best hope of implementing civic
republicanism’s call for deliberative decisionmaking informed by the values of the entire
polity.”); Peter L. Strauss, Legislation that Isn’t—Attending to Rulemaking’s “Democracy
Deficit”, 98 CAL. L. REV. 1351, 1354 (2010) (“political will alone cannot suffice to validate
regulations[—] acts of reasoned judgment along legislatively prescribed lines are required”).
63
Cf. RICHARDSON, supra note 14, at 17 (claiming that “our ideal of democracy
commits us to reasoning together, within the institutions of a liberal republic, about what we
ought to do in such a way that it is plausible to say that we, the people, rule ourselves”);
Robert B. Reich, Public Administration and Public Deliberation: An Interpretive Essay, 94
YALE L.J. 1617, 1637 (1985) (“The job of the public administrator is not merely to make
decisions on the public’s behalf, but to help the public deliberate over the decisions that need
to be made.”).
64
See Stewart, supra note 11, at 1712. (“Faced with the seemingly intractable problem
of agency discretion, courts have changed the focus of judicial review . . . so that its
dominant purpose is . . . the assurance of fair representation for all affected interests in the
exercise of the legislative power delegated to agencies.”); see also Fontana, supra note 49, at
82 (“The more public participation in the promulgation of an agency rule, the more deference
that rule should receive when it is challenged in court.”).

15

largely on the notice-and-comment procedures and Regulations.gov to provide broad
democratic accountability and legitimacy. 65
C.

Public Support for Regulations

Finally, public engagement in agency rulemaking is sometimes justified on
instrumental grounds as enhancing public support for agency regulations. 66 The
theory is that stakeholders will be more supportive of agency rulemaking when their
voices are heard by the agency, even when they do not get everything they want. 67
Thus, public engagement fosters support for the administrative state, which is
currently viewed with suspicion by many Americans. 68 Indeed, this is one of the
leading justifications for negotiated rulemaking, in which a group of stakeholders
affected by regulation negotiate in the hope of reaching a consensus rule that the
agency will then propose using the normal notice-and-comment process. 69
There is some evidence to support the idea that individuals view government
decisions that affect them as more legitimate if they have the opportunity to
participate and be heard in the decision-making process. 70 Yet it is questionable
whether the interest groups that typically challenge agency action in court are any less
likely to do so when they participate actively in the rulemaking process. Interest
groups may be more likely to get what they can from participation in the regulatory

65

See Cynthia R. Farina et al. Rulemaking 2.0, 65 U. MIAMI L. REV. 395, 407 (2011)
[hereinafter Farina et al., Rulemaking 2.0] (discussing the idea that e-rulemaking “should
enable citizens to monitor what unelected agency decisionmakers are doing, and to participate
actively in the rulemaking process in ways that, until now, have been available only to wellresourced interests”).
66
This is sometimes described as “sociological legitimacy.” See Richard H. Fallon, Jr.,
Legitimacy and the Constitution, 118 HARV. L. REV. 1787, 1795-96 (2005).
67
See Guardian Fed. Savs. & Loan Ass’n v. Fed. Savs. & Loan Ins. Corp., 589 F.2d 658,
662 (D.C. Cir. 1978) (“[T]he procedure for public participation tends to promote
acquiescence in the result even when objections remain as to substance.”); see also SIDNEY
A. SHAPIRO, AGENCY, ADMIN. CONFERENCE OF THE U.S., REVIEW OF EXISTING
REGULATIONS 434 (1995) (“Unless an agency speaks with all stakeholders, the consultation
process is not likely to be considered legitimate.”).
68
See Bull, Making the Administrative State “Safe”, supra note 13, at 615
(“[W]idespread antipathy towards administrative agencies may also reflect a sense that the
public has been foreclosed from making decisions regarding the proper allocation of
resources, decisions that instead are made by relatively insulated bureaucrats.”).
69
Cary Coglianese, Assessing Consensus: The Promise and Performance of Negotiated
Rulemaking, 46 DUKE L.J. 1255, 1256–57 (1997).
70
CYNTHIA R. FARINA & CERI, IBM CTR. FOR THE BUS. OF GOV’T, RULEMAKING 2.0:
UNDERSTANDING WHAT BETTER PUBLIC PARTICIPATION MEANS, AND DOING WHAT IT
TAKES TO GET IT 12 (2013) [hereinafter FARINA & CERI, IBM CENTER] (citations omitted)
(“Political psychology research confirms that individuals who are able to provide meaningful
input into government decisions that affect them are more likely to view the process as
legitimate—and to accept the outcome, even if it is not what they had hoped for.”).

16

process and then turn to judicial review when the outcomes are less than ideal and
adversely affect them or their constituencies. Indeed, the politically charged context
of some rulemakings may further mobilize opposition to agency proposals. 71 At least
one study of negotiated rulemaking suggests that there is no relationship between this
robust form of stakeholder participation in regulatory decision-making and
acceptance of the outcome, at least as measured by whether the participants seek to
challenge the rule in court. 72 Moreover, members of the general public unfamiliar
with the rulemaking process may be unhappy with the outcome if their views are
rejected and they do not receive an explanation they regard as persuasive. 73
Finally, even under the best of circumstances, only a slice of the public will
participate in agency rulemaking. The vast majority of people who do not participate
in the process are unlikely to view rules more favorably merely because other
members of the public do participate. 74 Nonetheless, robust and widely publicized
efforts by agencies to engage with the public during the rulemaking process could
enhance the sociological legitimacy of regulatory decision-making. 75
III. ENHANCING PUBLIC PARTICIPATION IN RULEMAKING
A.

The Notice-and-Comment Process

The most well-known tool for engaging the public in rulemaking is the noticeand-comment process required by the APA. Pursuant to Section 553, agencies must
71

See, e.g., Gail L. Achterman & Sally K. Fairfax, The Public Participation
Requirements of the Federal Land Policy and Management Act, 21 ARIZ. L. REV. 501, 508
(1979) (“[T]here is no reason to assume that the opportunity to participate leads to more
readily accepted decisions. Public involvement programs . . . may easily mobilize dissent.”);
Charlotte Garden, Toward Politically Stable NLRB Lawmaking: Rulemaking vs.
Adjudication, 64 EMORY L.J. 1469, 1488 (2015) (“[T]he potential for NLRB rulemaking to
result in a final product that enjoys public legitimacy seems doubtful in the highly politically
charged moment in which Board actions of all stripes are likely to be targeted with
allegations of illegitimacy and union favoritism.”).
72
Coglianese, supra note 69, at 1309 (“As a means of reducing litigation, negotiated
rulemaking has yet to show any demonstrable success.”).
73
Bill Funk, The Public Needs a Voice in Policy. But is Involving the Public in
Rulemaking a Workable Idea? CPRBLOG (Apr. 13, 2010) (“How will people feel about their
experience participating in government when they discover that their comments in fact have
no impact whatsoever on the rulemaking?”).
74
The vast majority of people unhappy with a regulation are also unlikely to avail
themselves of judicial review.
75
See E. Allan Lind & Christiane Arndt, Perceived Fairness and Regulatory Policy: A
Behavioural Science Perspective on Government-Citizen Interactions 9 (Org. for Econ. Cooperation and Dev. Regulatory Policy, Working Paper No. 6, 2016) (discussing research that
supports this proposition and noting that citizens’ perceptions of fair treatment turn heavily
on whether they have had a chance to express their views, been treated with respect by the
agency, and been given a reasoned explanation for the agency’s decision).

17

publish a “notice of proposed rulemaking” (NPRM) in the Federal Register including
the “terms or substance of the proposed rule or a description of the subjects and issues
involved.” 76 Although the language of the APA does not seem to require much in the
way of notice, the practice has become much more elaborate over time. Today,
agencies generally publish the full text of their proposed rules along with detailed
preambles explaining the “basis and purpose” of the proposed rule. 77 Agencies must
then provide interested members of the public with the opportunity to “participate in
the rule making through submission of written data, views, or arguments”: 78 the
“comment” piece of the process. Moreover, as a result of judicial glosses on the APA
over many years, the agency must generally address all significant public comments
when issuing the final rule. 79 And if the Final Rule is not the “logical outgrowth” of
the proposed rule, a court may require an additional round of public comment. 80
Notice-and-comment rulemaking is considered an open and participatory
decision-making process because any member of the public with the wherewithal can
learn what the agency is planning, submit their views on the proposal, and provide the
agency with information relevant to the rulemaking decision. Moreover, with the
advent of regultions.gov, agency proposals, public comments, and various supporting
documents are now available online to anyone with access to the Internet. In the
foregoing ways, the notice-and-comment process augments the information resources
of agencies and enhances their democratic legitimacy and accountability. 81 Indeed, it
is hard to imagine a more open, transparent, and democratic process, at least formally.
Nevertheless, there is a widespread perception that representatives of industry,
large trade groups, professional associations, and national advocacy organizations
typically dominate notice-and-comment rulemaking. 82
These “sophisticated

76

Administrative Procedure Act, 5 U.S.C. § 553(b)(3) (2012). The notice and comment
requirements of Section 553 do not apply to “interpretative rules, general statements of
policy, or rules of agency organization, procedure, or practice; or when the agency for good
cause finds . . . that notice and public procedure thereon are impracticable, unnecessary, or
contrary to the public interest.” Id. §§ 553(b)(A)-(B).
77
KERWIN & FURLONG, supra note 2, at 64.
78
Administrative Procedure Act § 553(c).
79
See, e.g., United States v. N.S. Food Prods. Corp., 568 F.2d 240, 252-53 (2d Cir.
1977) (quoting Auto. Parts & Accessories Ass’n v. Boyd, 407 F.2d 330, 338 (2d Cir. 1968)
and Associated Indus. of N.Y. State, Inc. v. U.S. Dep’t of Labor, 487 F.3d 342, 352 (2d Cir.
1973)) (requiring agencies to address “major issues” discussed in the public comments); see
also Lisa Schultz Bressman & Glen Staszewski, Judicial Review of Agency Discretion, in A
GUIDE TO JUDICIAL AND POLITICAL REVIEW OF FEDERAL AGENCIES 195, 225-29 (Michael
E. Herz et al. eds, 2d ed. 2015).
80
See, e.g., Chocolate Mfrs. Ass’n v. Block, 755 F.2d 1098, 1105 (4th Cir. 1985).
81
There is little evidence that it increases public support for agency regulations.
82
See Johnson, supra note 6, at 78 (collecting views that rulemaking is “dominated by
regulated entities and industry groups, rather than public interest groups”); see also MarianoFlorentino Cuéllar, Rethinking Regulatory Democracy, 57 ADMIN. L. REV. 411, 414 (2005)

18

stakeholders” 83 have the motivation, resources, and capacity to submit comments that
provide agencies with the types of information they need and that advance the kinds
of arguments agencies are most likely to find persuasive. By contrast, regulatory
beneficiaries and many smaller regulated parties, as well as state, local, and tribal
governments, and members of the general public rarely participate. 84 Although
advocacy groups may participate on behalf of many absent stakeholders, public
interest groups do not participate as frequently as regulated industries and are
frequently out-resourced. In addition, advocacy groups sometimes make strategic
decisions not to participate even when their members may be rich sources of
information, and representative groups may not represent the diversity of views or
experiences of their members.
B.

Obstacles to Broader Public Participation in Rulemaking

Scholars suggest that many members of the public who might participate in
rulemaking, including those with a direct stake in the outcome either as potential
beneficiaries or as regulated entities, lack awareness, motivation, and capacity to
comment effectively. 85 First, they may not be aware that the rulemaking process is
taking place or that it is open to public participation. News coverage of agency
rulemaking rarely mentions the opportunities for public comment. 86 Particularly in
the era of “Presidential Administration,” the media often depicts agency decisionmaking as an opaque, politically driven process, in which political appointees choose
regulatory initiatives based on the agenda of the President and his supporters.
Second, many members of the public may lack the motivation or incentive to
become involved in rulemakings, even when they are aware of them and have a stake
in their outcomes. The interests at stake for many potential beneficiaries of
regulatory action may simply be insufficient to justify the expenditure of time and
attention on participating. Regulations often impose concentrated costs on regulated
entities and diffuse benefits on the general public, creating collective action
problems. 87 Absent stakeholders may assume that someone else will represent their
(noting that “the sophistication with which a comment is written seems to affect the
probability that the agency will accept suggestions in that comment”).
83
See Farina et al., Knowledge in the People, supra note 5, at 1191 (defining
“sophisticated stakeholders”).
84
See Farina et al., Rulemaking in 140 Characters or Less, supra note 8, at 385-86
(citing evidence that only a “limited range of stakeholders” participate in notice and comment
rulemaking).
85
See Coglianese, Citizen Participation, supra 8, at 967 (noting collective actions
problems of public engagement); Farina et al., Rulemaking in 140 Characters or Less, supra
note 8, at 389–90 (citing “[i]gnorance about the rulemaking process[,] [u]nawareness that
rulemakings of interest are going on[,] and [i]nformation [o]verload from the length, and
linguistic and cognitive density, of rulemaking materials” as three barriers to participation).
86
See FARINA & NEWHART, IBM CENTER, supra note 22, at 11.
87
See OLSON, supra note 24, at 53-65 (explaining how small, organized groups are
usually more effective than larger, diffuse groups in shaping policy).

19

views. Indeed, not unreasonably, many members of the public may assume that
regulatory agencies charged with pursuing the “public interest” or ensuring clean air,
clean water, safe air transportation, and other statutory mandates will advocate on
their behalf. Others may believe that interest groups will represent their views. Or
they may believe the agency has already made up its mind about the course it will
pursue and their participation will have no impact. 88
In addition, due to loss aversion, individuals are more motivated to protect
themselves from losing something than to obtain a new benefit. Because regulations
often involve the imposition of new costs on industry and new benefits for a broader
public, the potential beneficiaries will not be as motivated to fight for the benefit they
have lived without as regulated parties will be to avoid a new cost. 89
Third, even if parties are aware of the opportunity to participate in the
rulemaking and are motivated to become involved, they may not have the capacity to
participate effectively. Although agencies accept comments on their NPRMs through
Regulations.gov anytime day or night, proposed rules are often quite lengthy and
complex, address many different issues, are written at an advanced level of
education, 90 and can be downright boring to many readers.
Perhaps most importantly, many members of the public do not know how to
submit effective comments. Several scholars have suggested that there is “a
fundamental incongruence between the ways that ‘insiders’ think and talk in
rulemaking and the ways that novice commenters do.” 91 They suggest that the
“insiders” who regularly participate in rulemaking—i.e., executive branch officials,
regulated industries, trade associations and major advocacy organizations—form a
“community of practice,” with a “shared rhetoric, competencies, experiences, and
expertise[, developed] over sustained interactions.” 92 While agency decision-makers
value “objective,” empirical evidence and quantitative data, presented in analytical,
premise-argument-conclusion reasoning, unsophisticated stakeholders tend to offer
“highly contextualized, experiential information, often communicated in the form of
personal stories.” 93

88

Ironically, the substantial efforts that agencies undertake before publishing a NPRM,
including the many different forms of public outreach and consultation described in this
Report, may contribute to this perception of the notice-and-comment process because the
agency has already thought about most of the salient issues. In other words, if the agency
does a good job engaging the public when developing a proposed rule, it may not learn
anything new during notice and comment that causes it to revise the final rule.
89
Wilson, The Politics of Regulation, supra note 24, at 85.
90
See FARINA & NEWHART, IBM CENTER, supra note 22, at 12 (describing a
Department of Transportation NPRM as written at a “late-college/early-graduate school
reading level”).
91
Farina et al., Knowledge in the People, supra note 5, at 1187.
92
Id.
93
Id.

20

The absence of certain important stakeholders is not the only perceived
shortcoming of public engagement in notice-and-comment rulemaking. There is also
criticism of the quality of participation by those who do regularly participate.
Sophisticated stakeholders may be less interested in engaging in a dialogic process
with the agency and other stakeholders or improving the substance of a proposed rule
than in preparing a favorable record for a subsequent judicial challenge. 94 This may
be due in part, however, to the participation of these groups in the rule development
that precedes the publication of the NPRM. If an interested party has already
articulated its position to the agency and the agency has rejected it in the NPRM, the
party may correctly decide that it is more worthwhile to prepare for judicial review
than to try, try again.
Although many have hoped that e-Rulemaking and “Rulemaking 2.0” would
lower the barriers to public engagement beyond the usual suspects and lead to a more
dialogic regulatory process, to date the impact of the move online has been modest.95
While it is certainly easier for interested parties to obtain access to the relevant
information, the dawn of e-Rulemaking has not dramatically increased the level or
quality of participation. Moreover, even in the few high-profile rulemakings that
have attracted mass comments from individual members of the public, “rulemaking
novices” rarely participate effectively. In most cases, their comments are little more
than a “thumbs up” or “thumbs down,” adding little information agencies can use in
deciding how to proceed, other than providing perhaps a broad sense of the general
sentiment of those who have submitted mass comments. 96
C.

The Challenges of Mass Comments

The “mass comments” occasionally submitted in great volume in highly salient
rulemakings are one of the more vexing challenges facing agencies in recent years.
These comments are typically the result of orchestrated campaigns by advocacy
groups to persuade members or other like-minded individuals to express support for
or opposition to an agency’s proposed rule. Advocacy groups that organize these
campaigns frequently provide text or recommended language for individuals,
although they increasingly urge commenters to customize their input by adding
language or thoughts or their own, and they often provide portals through which
commenters can easily submit their views for inclusion in the rulemaking record. A

94

KERWIN & FURLONG, supra note 2, at 115.
See Coglianese, Citizen Participation, supra note 8, at 958 (“[N]either agencies’
acceptance of comments by email nor the development of the Regulations.gov portal have led
to any dramatic changes in the general level or quality of public participation in the
rulemaking process.”); see also Farina et al., Rulemaking in 140 Characters or Less, supra
note 8, at 387; HERZ, USING SOCIAL MEDIA, supra note 28, at 2 (“[T]he move online has not
produced a fundamental shift in the nature of notice-and-comment rulemaking.”).
96
See Coglianese, Citizen Participation, supra note 8, at 958; HERZ, USING SOCIAL
MEDIA, supra note 28, at 40-42 (noting many instances in which engagement by the “lay
public” with agency regulation has failed to produce novel or helpful ideas).
95

21

significant number of mass comments may therefore be identical or vary from each
other in only relatively marginal ways.
Because the public has a right “to participate in the rule making through
submission of written data, views, or arguments,” 97 and agencies are legally required
to consider those comments in a reasoned fashion, mass comments pose serious
practical challenges. First, simply reviewing the “torrents of email” 98 that agencies
receive in mass comment campaigns is resource intensive. While natural language
processing technology can be used to identify the identical comments and otherwise
streamline and improve the review process, 99 the costs of reviewing this material may
still potentially exceed its benefits. There may also be concerns about whether the
mass comments are genuine or truly representative of the views of the general
public. 100 The latter concern applies to some degree to all public comments, but they
are especially pressing in the context of mass comments that are specifically designed
to show that a significant segment of the public strongly supports or opposes a
proposed rule. If those comments are not genuine or truly representative, then they
cannot reliably perform this function.
There is also a more fundamental debate in the literature regarding whether mass
comments have any real value in rulemaking—and, relatedly, how agencies ideally
should respond to them. 101 Agencies currently tend to devote most of their attention
to relatively sophisticated comments that include technical data or analytical
arguments, and they are generally reluctant to engage with comments that merely
provide “a simple statement of viewpoint, value, or preference.” 102 Cynthia Farina
and her colleagues at the Cornell e-Rulemaking Initiative support this approach
because mass comments that simply reflect preferences about public policy outcomes
tend to be the product of relatively uninformed “spontaneous preferences” or “group
framed preferences,” rather than the type of reasoned deliberation that is supposed to

97

Administrative Procedure Act, 5 U.S.C. § 553(c) (2012).
See Mendelson, Torrents of E-Mail, supra note 13.
99
See Michael A. Livermore et al., Computationally Assisted Regulatory Participation,
93 NOTRE DAME L. REV. 977, 993 (2018).
100
We focus here on mass comments rather than fake comments, which are largely
beyond the scope of this report.
101
See Michael Herz, “Data, Views, or Arguments”: A Rumination, 22 WM. & MARY
BILL RTS. J. 351, 368-69 (2013) [hereinafter Herz, Data, Views, or Arguments].
102
Nina A. Mendelson, Should Mass Comments Count?, 2 MICH. J. ENVTL. & ADMIN. L.
173, 173 (2012) [hereinafter Mendelson, Should Mass Comments Count?] (noting that such
content tends to be characteristic of mass comments). For a leading empirical study that finds
a disparity between agencies’ treatment of comments based on their level of sophistication,
see generally Cuéllar, supra note 82. See also Mendelson, Torrents of E-Mail, supra note 13,
at 1363-64 (“In general, rulemaking documents only occasionally acknowledge the number
of lay comments and the sentiments they express; they very rarely appear to give them any
significant weight.”).
98

22

characterize rulemaking. 103 While mass comments might legitimately serve as a fire
alarm that could alert politicians to a potential need to intervene, they should not
influence the work of agencies’ rulemaking teams. 104 “Rulemaking is not supposed
to be a plebiscite,” 105 and thus “the types of preferences expressed in these comments
may be good enough for electoral democracy, but they are not good enough for
rulemaking, even when rulemaking is heavily laden with value choices.” 106
While agreeing wholeheartedly that rulemaking is not a “plebiscite,” Nina
Mendelson notes the propensity of most agencies to avoid responding to the positions
expressed in mass comments, even if they merely express support for or opposition to
the outcome of a proposed rule or strengthening or relaxing a regulatory standard. 107
Most agencies have broad discretionary authority, and rulemaking therefore
necessarily involves making value-laden policy judgments of precisely this nature.108
Mendelson claims that the democratic responsiveness of rulemaking is undermined if
agencies ignore the preferences or values expressed by the people during the noticeand-comment process. 109 She responds to Farina’s concern about the lack of
deliberation and poor information environment surrounding mass comments by
pointing out that “[a]gencies, not public commenters, are the decision makers, and so
the critical locus of deliberation is in the agency.” 110 Mendelson therefore contends
that agencies “should commit to acknowledging mass comments in the final rule
document and to offering a brief answer.” 111 More specifically, she suggests that
agencies should
recognize the possibility that public views, as expressed in comments, point
in an unexpected direction or a direction contrary to the agency’s initially
preferred policy. This should prompt the agency to use additional

103

See Cynthia R. Farina et al., Rulemaking vs. Democracy, supra note 13, at 132-45.
Farina and her colleagues also believe that situated knowledge presented through narrative
stories does have potential value, and they suggest that rule making teams should undertake
efforts to be more receptive to this kind of information. See Cynthia R. Farina et al.,
Knowledge in the People, supra note 5, at 1188.
104
See Farina et al., Rulemaking vs. Democracy, supra note 13, at 13, 141.
105
Id. at 131.
106
Id. at 137. See also Stuart W. Shulman, The Case Against Mass E-mails: Perverse
Incentives and Low Quality Public Participation in U.S. Federal Rulemaking, 1 POL’Y &
INTERNET 23 (2009) (criticizing this trend).
107
See Mendelson, Torrents of E-Mail, supra note 13, at 1361-62.
108
See id. at 1347-48.
109
See id. at 1359 (“An agency’s dismissal or pro forma treatment of significant
numbers of public comments would be very hard to square with a vision of rulemaking as a
democratic process.”).
110
Mendelson, Should Mass Comments Count?, supra note 102 , at 180.
111
Id. at 182.

23

procedures to engage public views or at a minimum to engage in more
extended deliberation, possibly including Congress or the White House. 112
Mendelson recognizes, however, that courts should be very deferential to agency
decisions in this context as long as agencies have given “some acknowledgment of
significant views expressed through lay comments.” 113
While commentators disagree about the extent to which mass comments have
value and thus merit a response from agencies, there is widespread consensus that
comments that include data or well-reasoned analytical arguments are more valuable
to agencies. 114 Yet comments in general, and mass comments in particular, may
blend simple statements of preference, the provision of relevant technical data,
situated knowledge, and analytical arguments regarding the strengths or weaknesses
of an agency’s proposed rule. Accordingly, even if everyone agreed that simple
statements of preference or value are not worthwhile contributions, agencies would
still need to review the comments to determine whether they contain useful data,
situated knowledge, or analytical arguments and respond appropriately.
We strongly agree that rulemaking is not a plebiscite, and that agencies should
not treat comments as “votes” or base their decisions on the unfiltered preferences of
a majority. We are therefore sympathetic to Professor Farina’s position that
comments reflecting simple preferences about outcomes are “not good enough for
rulemaking.” On the other hand, majoritarian preferences are not necessarily
irrelevant, even from a deliberative perspective, 115 and thus we tend to share
Professor Mendelson’s intuition that it would be troubling for agencies to ignore mass
comments on questions of value.
What tends to get glossed over in this debate, however, is the fact that
rulemaking is a multi-stage process, and that public notice and comment only occurs
after agencies have already devoted substantial attention to the basic issues at stake.
This means that the simple statements “of viewpoint, value, or preference” that tend
to be reflected in mass comments will generally already have been considered and
resolved by the agency when it set its agenda and developed the proposed rule.
Agencies should not be required to redo all of this work in response to mass
comments when they issue their final rules. While agencies are often criticized for
being inflexible and treating notice-and-comment as “Kabuki Theater,” 116 a certain

112

Mendelson, Torrents of E-Mail, supra note 13, at 1377.
Id. at 1379 (emphasis added).
114
Moreover, situated knowledge conveyed in the form of personal stories by
individuals with lived experience in the area being regulated can have substantial value as
well. See Farina et al., Knowledge in the People, supra note 5, at 1217-38.
115
See Staszewski, Political Reasons, supra note 15, at 894-912 (discussing potential
roles for majoritarian preferences in agency decision making from a deliberative perspective).
116
See E. Donald Elliott, Re-Inventing Rulemaking, 41 DUKE L.J. 1490, 1492 (1992)
(“Notice-and-comment rulemaking is to public participation as Japanese Kabuki theater is to
113

24

amount of path dependence is a natural and not entirely undesirable element of the
rulemaking process. At the same time, however, this pragmatic position puts a
premium on the importance of agencies conducting meaningful public engagement
during the agenda setting and rule development stages, when their highest priorities
are established and these basic questions of value tend to be resolved, and when
agencies are most open-minded about the proper treatment of policy questions as a
general matter. Consistent with this report’s emphasis on treating rulemaking in a
holistic fashion, we think that if agencies provide a reasoned explanation for pursuing
a proposed rule and address relevant questions of value in the course of developing a
rule proposal, then they should generally be able to rely on this information as an
adequate response to mass comments even if the explanation or justification was
initially provided during the agenda setting or rule development stages. If, however,
mass comments reflect a genuine groundswell of grassroots opposition to the
agency’s proposal that was not anticipated or addressed at earlier stages of the
rulemaking process, then the agency should either (1) address these comments on
their merits when it promulgates the final rule, or (2) treat the comments as a “yellow
light” that warrants further deliberation. In other words, the agency should consider
treating genuine and unanticipated grassroots opposition to a proposed rule as a
trigger for the kind of enhanced deliberation discussed in Part XI. 117
Finally, although we cannot fully resolve the debate over mass comments, we
will offer suggestions on how agencies might address some of the practical challenges

human passions—a highly stylized process for displaying in a formal way the essence of
something which in real life takes place in other venues.”).
117
See Herz, Data, Views, or Arguments, supra note 101, at 377 (“[Mass] comments are
a prompt; they flag an issue that the agency should focus on, but the agency should then
determine whether the more accepted bases of agency decisionmaking justify a particular
outcome”) (emphasis added); see also Mendelson, Should Mass Comments Count?, supra
note 102, at 181 (“[W]ithout dictating the outcome, mass comments might give agency
officials a reason to pause and to engage a wider range of viewpoints. Public comments
might also prompt the agency to take steps to more systematically consider public views,
whether that is though focused polling, focus groups, public deliberation efforts, so-called
citizen juries, or other devices.”).
NHTSA followed this approach in response to a disproportionate number of comments
from the general public that the agency believed reflected an unfounded fear of air bags. See
generally Air Bag On-Off Switches, 62 Fed. Reg. 62,406 (Nov. 21, 1997) (to be codified at
49 C.F.R. pts. 571 and 595); Mendelson, Torrents of E-Mail, supra note 13, at 1366
(discussing this example). Rather than changing its proposed rule, the agency conducted
focus groups on whether it could correct the public’s misperceptions. Based on the feedback
that it received from the focus groups, NHTSA ultimately promulgated a rule that adhered to
its substantive position “but added a ‘public education information campaign to put air bag
risks and benefits into their proper perspective.’” Mendelson, Torrents of E-Mail, supra note
13, at 1366 (quoting Air Bag On-Off Switches, 62 Fed. Reg. at 62,423). This is an excellent
illustration of an agency using what we would regard as the best practices for responding to
mass comments during notice-and-comment rulemaking.

25

of reviewing mass comments and extracting the information of relevance to their
rulemakings.
D.

Lowering the Barriers to Participation by Absent Stakeholders

Enhancing public engagement in rulemaking by traditionally absent stakeholders
can improve the effectiveness, democratic legitimacy, and public acceptance of
certain regulations.
1.

Improving the Effectiveness of Regulation

Absent stakeholders may have relevant information “about impacts, ambiguities
and gaps, enforceability, contributory causes, unintended consequences,” and similar
information based on “their lived experience in the complex reality into which the
proposed regulation would be introduced.” 118 Scholars call this “situated
knowledge.” Proponents of greater participation by absent stakeholders believe that
they can provide additional information based on their experiences that will improve
the effectiveness of regulation. Absent stakeholders may also have information that
is relevant to agenda-setting and retrospective review.
Cynthia Farina and her colleagues have documented many examples of situated
knowledge helpful to agency rulemaking from the Regulation Room project discussed
more fully below. 119 For example, when the Consumer Financial Protection Bureau
(CFPB) 120 conducted a rulemaking aimed at ensuring borrowers were offered
opportunities to restructure their mortgage payments before foreclosure on their
homes, individual borrowers shared their experiences with foreclosures that could
have been avoided but for erroneous calculations of their home’s net present value.
Based on these comments, the CFPB modified the final rule to require lenders to
disclose these calculations prior to foreclosure so that the borrowers might correct
them. Similarly, during a rulemaking by the Department of Transportation (DOT) to
require the installation of electronic on-board recording equipment (EOBR) in
commercial motor vehicles, independent commercial vehicle drivers noted that strict
enforcement of hours-of-service rules based on automatic monitoring could have
absurd and unintended consequences, such as requiring drivers to pull off the
highway just short of their destination or in an unsafe location. 121 There are many
other examples despite limited systematic efforts to date by agencies to reach these
types of individuals and groups.
In many cases, agencies rely upon interest group organizations to represent the
views of individuals who rarely participate in rulemakings. Yet these groups may not

118

Farina et al., Knowledge in the People, supra note 5, at 1187, 1197.
See infra Part IX.C & F.
120
Since the public engagement activities described in this Report, the CFPB has
undergone several changes, including adopting a different name, the Bureau of Consumer
Financial Protection.
121
See FARINA & CERI, IBM CENTER, supra note 70, at 17.
119

26

always provide an adequate substitute. 122 For example, in a DOT rulemaking
involving tarmac delays, many of the professional associations and unions
representing employees in the air travel industry, such as pilots, flight attendants, air
traffic controllers, gate personnel, ground crews, and travel agents, made a strategic
decision not to participate in the rulemaking lest they upset either the airlines (their
employers) or air travelers (their customers). 123
Even when organized interest groups do participate, they may not represent the
range of interests and views among those they purport to represent. For example,
some workers do not participate in unions because they do not think their unions
represent their views. 124 Some gun owners oppose the positions of the NRA. 125
Furthermore, national advocacy organizations generally take a position on issues
rather than convey a diversity of constituent views. For example, during a DOT
rulemaking concerning the accessibility of airport check-in kiosks and travel websites
to individuals with physical and cognitive disabilities, organizations representing
persons with disabilities uniformly sought the increased independence offered by
accessible technologies. But some individuals with disabilities were concerned that
requiring accessible technologies would result in fewer airline agents who could
assist travelers and adapt to their particular needs. 126 In the DOT rulemaking
proposing the installation of EOBR equipment, large trucking companies, most of
whom already used the technology, generally supported the proposal, while small
trucking companies and independent commercial motor vehicle drivers, most of
whom did not use the technology, generally opposed the proposal. 127 Among other
things, the commercial drivers and small companies noted that it would be more
difficult for them to absorb the costs of installing the equipment and suggested that
the agency’s cost-benefit analysis was “skewed toward a big business model.” 128

122

See Stewart, supra note 11, at 1767 (“There are no accepted means of determining
whether the views of the organization are congruent with the interests of the broader class.”).
123
See Farina et al., Rulemaking in 140 Characters or Less, supra note 8, at 406 n.59,
413.
124
See, e.g., Janus v. AFSCME, 138 S.Ct. 2448, 2461 (2018) (involving an employee
who refused to join his union because he opposed many of its polices).
125
See generally Christopher Ingraham, Most Gun Owners Don’t Belong to the NRA—
and They Don’t Agree with it Either, WASH. POST (Oct. 15, 2015),
https://www.washingtonpost.com/news/wonk/wp/2015/10/15/most-gun-owners-dont-belongto-the-nra-and-they-dont-agree-with-it-either/?noredirect=on&utm_term=.33ce8b43b5a9.
126
See Farina et al., Knowledge in the People, supra note 5, at 1204; FARINA & CERI,
IBM CENTER, supra note 70, at 17.
127
Farina et al., Knowledge in the People, supra note 5, at 1200, 1214 n.106.
128
Id. at 1200.

27

In some cases there may be no organization that represents stakeholders affected
by a rulemaking, or the organizations that do exist may not be strong enough to
participate regularly or effectively. 129
Finally, there may be rulemakings in which the agency needs individual rather
than collective responses. For example, the agency may seek to understand consumer
reaction to the content and form of disclosure requirements or trade-offs between
functionality and efficiency. 130 Or the representative organization (and especially the
lawyers who typically write comments on their behalf) may not have the experiential
knowledge the agency needs. During the DOT EOBR rulemaking, independent
commercial drivers and small trucking companies conveyed “rich and nuanced detail
of individual experiences and operations[, illuminating] a variety of business
practices around completing driving logs and … concerns about inflexibility.” 131
To be clear, the absence of many stakeholders will not always undermine the
effectiveness of regulations. In many rulemakings absent stakeholders will not
possess situated knowledge that is relevant to the decision or that is not available
from other sources. For example, when EPA is setting the permissible exposure level
for a pollutant, many people adversely affected by the pollutant may have relevant
health experiences to share. But the agency can likely obtain more valuable
information about the pollutant’s health effects in the aggregate from other sources. 132
Part of what this report emphasizes is that agencies should make an assessment as
early as possible when developing rules concerning whether there are individuals and
groups who are likely to be absent from the rulemaking without robust, targeted
outreach and whether these individuals or groups have experiences, views, or other
information relevant to the rulemaking that would not otherwise be available to the
agency without their participation.
2. Enhancing the Democratic Accountability, Legitimacy, and Public
Support For Regulation
The level of participation required to lend democratic accountability and
legitimacy to rulemaking, or to enhance public support for regulations, is much less
clear and remains theoretically contested. Some would argue that the notice-andcomment process and the ability of any interested person to use Regulations.gov to
review proposed rules and submit comments is all that democracy requires—perhaps

129

See, e.g., Richard L. Hasen, Clipping Coupons for Democracy: An Egalitarian/Public
Choice Defense of Campaign Finance Vouchers, 84 CAL. L. REV. 1, 16 (1996) (“The current
political market is stacked against the poor, who lack the political capital needed to influence
politics, as well as against other large, diffuse groups that have difficulty overcoming
collective action problems, like some racial minorities.”).
130
See Farina et al., Rulemaking in 140 Characters or Less, supra note 8, at 429
(discussing DOT design of tire labeling to provide information related to fuel efficiency).
131
FARINA & CERI, IBM CENTER, supra note 70, at 17.
132
See id. at 18.

28

more. 133 Whatever the validity of that view, we think that the legitimacy of the
process can still be enhanced through further engagement by the public in
rulemaking. As discussed above in Part II.B, government agencies have a democratic
obligation not only to render an account to the public of what they are doing, but to
do so in a way that can be understood and accepted by the public. 134 In addition,
agencies should ensure that the public has a meaningful opportunity to participate in
the decision-making process and that the entire range of relevant interests and
perspectives is considered. Thus, democratic theory does not necessarily require the
public to participate in every rulemaking. But it does require that citizens be able to
participate in a meaningful way when they have the motivation and incentive to do
so, particularly when they can present information or views that would otherwise be
missing. This means that agencies should explain what they are doing in plain
language form, publicize their plans early enough in the rulemaking process so that
the public can participate when it still might make a difference, and educate the
public on how to participate effectively in rulemaking if they are so inclined.
E.

Beyond Notice and Comment

Section 553 sets forth only the minimum requirements for informal rulemaking.
But agencies are free to and do provide additional opportunities for public
engagement in their regulatory work. This Report will address many of the different
tools for public engagement beyond the notice-and-comment process, with the goal of
helping agencies to think systematically about when and how to engage members of
the general public or unorganized stakeholders in their rulemakings. In doing so, we
describe various successes in which agencies, sometimes with the assistance of nongovernmental parties, have thoughtfully supplemented the APA procedural
requirements to overcome the barriers to participation by rulemaking novices. 135 In
addition, we offer suggestions on how to improve the use of social media and
eRulemaking as a way of obtaining broader participation or additional useful
information.
Of course, most agency rulemakings are not controversial, and substantially
enhancing public engagement beyond the basic APA requirements may not be a wise
use of resources. In addition, in many rulemakings absent stakeholders may be
adequately represented by organized interests who do participate; or rulemaking
novices may not have much useful information to add. Identifying and engaging
absent stakeholders in most cases will be more difficult than relying on sophisticated
stakeholders who are ubiquitous in agency rulemaking. Nevertheless, many of the
agency officials we spoke to who have managed to increase participation in
rulemaking by the public and traditionally absent stakeholders acknowledged its

133

See DAVIS, DISCRETIONARY JUSTICE, supra note 2, at 65-66.
See Waldron, supra note 14, at 7.
135
Farina et al. report that “agency rulemakers . . . reacted positively to the usefulness of
comments received from Regulation Room, even when they were initially skeptical about the
value of the project.” Farina et al., Knowledge in the People, supra note 5, at 1198.
134

29

value in furthering both the information and democratic functions of public
participation in the regulatory process.
We do not suggest that it is necessary for agencies to enhance public
participation in every rulemaking. Nevertheless, we recommend that agencies think
purposefully about the value of broader public engagement in every rulemaking they
undertake and consider whether it is worthwhile to supplement the traditional
opportunities for public participation. This includes thinking about whether there are
stakeholders affected by the potential regulations that do not traditionally participate
in the agency’s work or that are not likely to be adequately represented by other
groups. In addition, we recommend that agencies consciously consider the
appropriate level of public engagement in each and every rulemaking, develop a
public engagement plan when appropriate, and periodically revisit their plan as the
rulemaking progresses from early rule development to the notice-and-comment
process and beyond.
Finally, agencies should seek to promote the democratic goal that the public
understands what they are doing in their rulemakings and why. As discussed herein
and in other ACUS reports, 136 this requires explaining complex regulatory matters in
a way that is understandable to unsophisticated stakeholders. It may also include
broadening the types of comments that count in rulemaking 137 and developing a
greater appreciation for the non-technical values that inform discretionary
policymaking. 138
IV. LEVELS OF PUBLIC ENGAGEMENT
The existing literature on public engagement in rulemaking suffers from a couple
of limitations. There is no standard definition of what counts as “public
engagement,” and there is no definitive catalogue of the countless available
mechanisms for engaging the public and their particular characteristics. 139 Nor is
there general agreement about the best mechanisms for engaging with the public in
any particular situation. There is, however, widespread agreement that the best
mechanisms for engaging with the public will vary depending upon the situation.
For purposes of this report, we define public engagement to include (1) efforts to
enhance public understanding of agency rulemaking and (2) efforts to foster
meaningful participation in the rulemaking process by members of the public.
Consistent with our charge, we are especially interested in identifying ways of
enhancing public engagement with “absent stakeholders,” meaning individuals or
groups that a regulation may benefit or burden, or otherwise have a direct stake in the
136

See generally BLAKE & EMERSON, supra note 19.
See Farina et al., Knowledge in the People, supra note 5, at 1188.
138
Cuéllar, supra note 82, at 468.
139
See Gene Rowe & Lynn J. Frewer, A Typology of Public Engagement Mechanisms,
30 SCI., TECH., & HUM. VALUES 251, 252-53 (2005).
137

30

outcome of the agency’s rulemaking, but who do not traditionally participate in the
agency’s rulemaking process. We are also interested in identifying ways of
enhancing public engagement with “unaffiliated experts,” meaning scientific,
technical, or other professionals with expertise relevant to the agency’s rulemaking
who are neither direct stakeholders nor employed or retained by a stakeholder.
It is vital to recognize that there are different levels of public engagement in
rulemaking, and scholars have previously identified typologies of participation that
we find useful. 140 For example, the International Association for Public Participation
(“IAP2”) has developed a “Spectrum of Public Participation” that identifies five types
of public participation ranging from least to greatest citizen influence, and articulates
the goal, the promise to the public, and examples of techniques at each point along
the spectrum. 141 The main categories of the spectrum and their accompanying goals
are as follows:
Inform

To provide the public with balanced and objective information
to assist them in understanding the problem, alternatives,
opportunities, and/or solutions.

Consult

To obtain public feedback on analysis, alternatives, and/or
decisions.

Involve

To work directly with the public throughout the process to
ensure that public concerns and aspirations are consistently
understood and considered.

Collaborate

To partner with the public in each aspect of the decision
including the development of alternatives and the identification
of the preferred solution.

Empower

To place final decision-making in the hands of the public. 142

The notice-and-comment rulemaking process outlined in section 553 of the APA
is widely viewed as a prominent form of consultation, although it also involves
informing the public of an agency’s plans. Forms of participation that “involve” the

140

For the classic example, see generally Sherry R. Arnstein, A Ladder of Civic
Participation, 35 J. AM. INST. PLANNERS 216 (1969); see also CAROLYN J. LUKENSMEYER
ET AL., IBM CTR. FOR THE BUS. OF GOV’T, ASSESSING PUBLIC PARTICIPATION IN AN OPEN
GOVERNMENT ERA: A REVIEW OF FEDERAL AGENCY PLANS App. III at 59-61 (2011)
(discussing these typologies).
141
See CAROLYN J. LUKENSMEYER & LARS HASSELBLAD TORRES, IBM CTR. FOR THE
BUS. OF GOV’T, PUBLIC DELIBERATION: A MANAGER’S GUIDE TO CITIZEN ENGAGEMENT 7
(2006) (presenting the IAP2 spectrum).
142
Id.

31

public or result in “collaboration” tend to be more ongoing and deliberative in nature,
and they will also figure prominently in our report. In contrast, efforts to “empower”
the public by giving them final decision-making authority are outside the scope of our
report, since agencies are typically the authoritative decision-makers in rulemaking
under federal regulatory statutes.
Archon Fung has developed another influential typology of public participation:
“the democracy cube.” 143 He explains that forms of public participation vary along
three key dimensions: (1) who participates (or how participants are selected), (2) how
participants communicate and make decisions, and (3) the impact of public
participation. Each of these dimensions can, in turn, be understood along a spectrum
ranging from least to most inclusive, deliberative, or potent. Thus, for example, the
spectrum of “participant selection” ranges from relatively exclusive methods that are
limited to public officials (expert administrators or elected representatives), to
relatively inclusive methods that do not involve the state at all. 144
We are concerned primarily with methods of public participation that fall within
the middle-range of the spectrum and involve the creation of “mini-publics.” 145 From
most to least exclusive (or least to most inclusive), these methods can involve
participation by professional stakeholders, lay stakeholders, a random selection of
citizens, or mini-publics that are open to all (the latter of which can involve targeted
recruitment or outreach or rely solely on self-selection). Although notice-andcomment rulemaking is ostensibly open to anyone who wishes to participate, in
practice the process tends to be dominated by sophisticated stakeholders. 146 Our
report therefore focuses on methods of participation that would likely include lay
stakeholders, or that would involve targeted outreach efforts to recruit otherwise
missing stakeholders or unaffiliated experts. We will also discuss some methods of
participation by random selections of ordinary citizens.
Fung’s democracy cube also includes six “modes of communication . . . and
decision making” in democratic governance ranging from “least intense” to “most
intense.” 147 At the least intensive end of the spectrum participants merely listen as
spectators. As communication becomes more intensive, participants may express
their preferences or even develop their preferences through deliberation and
discussion. These are the most common modes of citizen participation in democratic
governance. Moving further along the spectrum participants become decision-makers
in governance by aggregating and bargaining over their preferences; deliberating and
negotiating over their preferences; and finally, at the most intensive end of the
spectrum, deploying technical expertise to make a decision. The last mode of

143

See Archon Fung, Varieties of Participation in Complex Governance, 66 PUB.
ADMIN. REV. 66, 70-71 (2006) [hereinafter Fung, Varieties of Participation].
144
See id. at 67-68.
145
See Fung, Recipes for Public Spheres, supra note 34, at 338-39.
146
See Johnson, supra note 6, at 78.
147
Fung, Varieties of Participation, supra note 143, at 69.

32

decision-making usually involves agency officials and professional experts rather
than citizens. 148
Finally, the third dimension of the democracy cube involves the “extent of
authority and power” exercised by the participants, ranging from the least to the
greatest authority. The least authoritative methods of public participation provide
participants with personal benefits (such as enhanced knowledge or a sense of civic
virtue), while the most authoritative methods provide participants with direct
lawmaking authority. In the middle, ranging from least to greatest authority,
participants exert communicative influence on decision makers; advise and consult
with decision makers; or engage in collaborative, co-governing partnerships with
governmental policymakers. Notice-and-comment rulemaking is, of course, a
prominent form of advice and consultation because “officials preserve their authority
and power but commit themselves to receiving input from participants.” 149
An important feature of public engagement that is omitted from the preceding
typologies, however, involves the timing of an agency’s efforts to engage the public.
We think that different methods of public engagement will often make sense at
different stages of the rulemaking process. Accordingly, we incorporate a fourth
dimension into our analysis: the timing or chronology of public engagement.
V. THE CHRONOLOGY OF PUBLIC ENGAGEMENT IN RULEMAKING
We believe that any effort to develop a set of best practices for public
engagement in rulemaking must take a broad and holistic view of the rulemaking
process. Rulemaking begins with agenda setting, when the agency decides which
matters will be the subject of rulemaking, continues with early and advanced
development of a rule for publication in the Federal Register, followed by the noticeand-comment process itself, and also includes retrospective review, when the agency
evaluates the effectiveness of previously promulgated rules. Taking a broad approach
to rulemaking and considering each stage is important for several reasons.
Chronology of Public Engagement Efforts
Agenda
Setting



Rule

Development

Notice &
Comment



Retrospective
Review

First, a broad and holistic view of rulemaking is valuable because to date the
literature on agency rulemaking, and efforts to more fully democratize it, have
focused overwhelmingly on the notice-and-comment stage. 150 The agenda setting

148

See id.
Id.
150
See West, supra note 30, at 577; see also ALBERTO ALEMANNO, STAKEHOLDER
ENGAGEMENT IN REGULATORY POLICY 44 (2015) (“While there seems to be mounting
awareness about the need to integrate engagement practices in the entire policy cycle, this
149

33

and rule development stages, in contrast, have received much less focused
attention. 151 The existing literature generally treats what happens before publication
of the notice of proposed rulemaking as “a black box,” and suggests that rule
development is primarily influenced by political considerations and pressure from
well-organized interest groups. 152 Meanwhile, the democratic value of notice-andcomment rulemaking, which was once praised as “one of the greatest inventions of
modern government,” 153 has been called into question by recent concerns regarding
the veracity of public comments and reasons to believe that agencies are reluctant to
make changes to their proposed rules after they have been published in the Federal
Register. Yet in conducting research for this project, we have spoken to numerous
agency officials who have described significant efforts to engage the public with their
agenda setting and rule development activities. We believe it is helpful to shed more
light on those efforts, and that the most productive way to improve the democratic
legitimacy and effectiveness of the rulemaking process may be to build on this
foundation and construct the best possible infrastructure for meaningful public
engagement before publication of a proposed rule in the Federal Register for public
comment. Nonetheless, we should not ignore potential ways to improve notice-andcomment rulemaking itself. Accordingly, we will also present examples of and
suggestions for enhancing public engagement during the notice-and-comment process
and retrospective review.
Second, agencies need to plan for public engagement as early as possible in the
rulemaking process and consider which modes of public engagement are likely to be
most useful at each stage of the process. As agencies develop a public engagement
plan, they should think about: (1) why they want to engage with the public; (2) who
they are trying to reach with their public engagement efforts; (3) what type of
information they are seeking to obtain from those individuals or groups; (4) how this
information could most likely be secured – i.e., through which participation methods;
(5) what to do with the input they receive and precisely how to respond; and (6) when
during the rulemaking process agencies should carry out these efforts. The answers
to the first five questions may vary at different stages of the process and according to
the particular decision the agency must make. For example, an agency may want
information about the general public’s values and priorities when setting its
regulatory agenda. The agency may seek the situated knowledge of absent
stakeholders based on their practical experiences with a regulatory program when
designing a proposed rule or engaging in retrospective review. The agency may need
technical data about compliance costs and anticipated benefits from regulated entities,
regulatory beneficiaries, or unaffiliated experts during advanced rule development or
the notice-and-comment process. A well-designed rulemaking process should

requires embracing a significant cultural shift in both the policymakers’ mind-set and that of
stakeholders.”).
151
See West, supra note 30, at 583-84; KERWIN, OUT OF THE SHADOWS, supra note 31.
152
See, e.g., Wagner et al., Rulemaking in the Shade, supra note 32, at 109-13; West,
supra note 30, at 583-84.
153
DAVIS, DISCRETIONARY JUSTICE, supra note 2, at 65.

34

provide avenues for various segments of the public to provide agencies with all of this
information and more.
Therefore, the bulk of this report—Parts VI through X—is organized around the
relevant stages of the rulemaking process. For each stage, we describe the most
common modes of public engagement in rulemaking, provide examples of how
agencies have used them effectively, and discuss the primary limitations or challenges
associated with each mode of engagement. Part XI then discusses the potential value
of incorporating more ambitious forms of deliberative democracy into the rulemaking
process in appropriate circumstances. In addition, Appendix C summarizes the
function, strengths, weaknesses or challenges, and other pertinent information about
each mode of public engagement, including the stage in the regulatory process when
each can be used.
Finally, the last section of the report turns to issues that cross the different stages
of rulemaking and require a broader lens. Part XII addresses the vital importance of
planning, outreach, and communication, and suggests institutional reforms that might
facilitate these efforts.
VI. AGENDA SETTING
Agency agenda setting is of the utmost importance to regulatory governance
because it determines which issues or problems agencies will address and which
problems will go unresolved. While the agenda-setting stage of the policymaking
process has received substantial attention from social scientists in other contexts,154
this vital topic has thus far received minimal attention from scholars of administrative
agencies and the regulatory process. 155 This Part builds on the exploratory work in
this area by identifying innovative practices currently used by agencies to
communicate with interested members of the public when setting their rulemaking
agendas and discussing strategies for enhancing those efforts.
An agency’s agenda includes the plans or activities identified in the semi-annual
Unified Agenda of Regulatory and De-Regulatory Actions (“Unified Agenda”) and
the agency’s annual Regulatory Plan. Agencies must prepare these documents under
the supervision of the Office of Information and Regulatory Affairs (“OIRA”) to
comply with the requirements of the Regulatory Flexibility Act and Executive Order
12,866. 156 While not much is known about how agencies prepare and update these
documents (and agencies presumably use a variety of different approaches), it is

154

See generally JOHN W. KINGDON, AGENDAS, ALTERNATIVES, AND PUBLIC POLICIES

(1984).
155

See Cary Coglianese & Daniel E. Walters, Agenda-Setting in the Regulatory State:
Theory and Evidence, 68 ADMIN. L. REV. 865, 866 (2016).
156
See generally Regulatory Flexibility Act, 5 U.S.C. § 602 (2012); see also Exec. Order
No. 12,866 § 4(b), 58 Fed. Reg. 51,735 (Sept. 30, 1993); CURTIS W. COPELAND, ADMIN.
CONFERENCE OF THE U.S., THE UNIFIED AGENDA: PROPOSALS FOR REFORM 6 (2015).

35

widely understood that they are frequently incomplete and contain some
inaccuracies. 157 Accordingly, the Unified Agenda and Regulatory Plan do not
reliably capture everything agencies are working on, or considering working on, at
any given time. Therefore it is useful to adopt a broader definition of agenda setting,
which includes “all the choices and opportunities that both agency officials and other
participants in the regulatory process have about what problems agencies emphasize
and what alternatives they consider.” 158 We are interested in potential ways of
enhancing public engagement both in the process of producing the Unified Agenda
and Regulatory Plan and in the process of otherwise determining which rules an
agency will (or will not) pursue.
Agencies do not always have complete control over their rulemaking agendas.
On the contrary, the White House sometimes directs agencies to promulgate rules (or,
at least, to consider promulgating rules) to address a specified problem. Moreover,
Congress frequently requires agencies to promulgate rules on designated subjects in
its enabling legislation. Mandatory rulemaking of this nature sometimes requires
agencies to meet statutory deadlines, and Congress sometimes provides “hammers”
that punish agencies that fail to comply with the requisite timeline. Federal courts
periodically enforce mandatory rulemaking obligations and associated statutory
deadlines in successful litigation brought pursuant to the APA to “compel agency
action unlawfully withheld or unreasonably delayed.” 159 Congress sometimes also
requires agencies to engage in periodic “lookbacks” or retrospective reviews of
previously enacted rules. For these reasons, the few scholars who have examined
agency agenda setting empirically have concluded that Congress plays a major—and
perhaps the dominant—role in this process. 160 Consistent with these findings, several
officials told us that much of their agency’s rulemaking activity is mandated by
statute, and that their agency’s rulemaking agendas are largely filled by those
statutory requirements.
Of course, even when an agency engages primarily in mandatory rulemaking, it
must still decide which rules to tackle first and which alternatives to consider, so the
agency still has some agenda-setting discretion.
Moreover, many agencies
promulgate a combination of mandatory and discretionary rules, or have nearly

157

See COPELAND, supra note 156, at 11; Jennifer Nou & Edward H. Stiglitz, Strategic
Rulemaking Disclosure, 89 S. CAL. L. REV. 733, 736 (2016).
158
Coglianese & Walters, supra note 155, at 869 (adopting this broader definition).
159
5 U.S.C. § 706(1) (2012); see also Norton v. S. Utah Wilderness All., 542 U.S. 55, 64
(2004) (holding that such actions “can proceed only where a plaintiff asserts that an agency
failed to take a discrete agency action that it is required to take”) (emphasis in original).
160
See MARISSA MARTINO GOLDEN, WHO CONTROLS THE BUREAUCRACY? THE CASE
OF AGENDA SETTING 19, 21 (2003); William F. West & Connor Raso, Who Shapes the
Rulemaking Agenda? Implications for Bureaucratic Responsiveness and Bureaucratic
Control, 23 J. PUB. ADMIN. RES. & THEORY 495, 502, 504-05 (2013); see also Coglianese &
Walters, supra note 155, at 875-77 (reviewing the literature and discussing Congress’s
“highly influential” role in agency agenda setting).

36

unfettered discretion to decide which rulemaking projects to undertake. Accordingly,
while the distinction between mandatory and discretionary rules is important to
agenda setting, agencies nearly always have at least some (and often substantial)
policy-making discretion in this area that could fruitfully be informed by public
engagement efforts.
There are a variety of different ways for agencies to engage with the public when
establishing or modifying their rulemaking agendas. The following sections discuss
these various modes and provide examples.
A.

Rulemaking Petitions

Consistent with a longstanding constitutional tradition, 161 the APA provides that
“[e]ach agency shall give an interested person the right to petition for the issuance,
amendment or repeal of a rule.” 162 The APA also requires agencies to respond to
rulemaking petitions within a reasonable time, 163 and suggests that agencies must
give a reasoned explanation for denying such requests. 164 Federal courts review the
denial of rulemaking petitions under an especially deferential version of the arbitrary
and capricious test. 165 Accordingly, unlike most of the other available mechanisms
for agencies to engage with the public when they set their rulemaking agendas,
citizens have a legal right to file rulemaking petitions and effectively to compel
agencies to respond to their requests in a reasoned fashion.
Congress has periodically established more elaborate petition processes for
particular statutory schemes, including major environmental laws such as the
Endangered Species Act. 166 These customized petition processes typically impose
statutory deadlines for agencies to accept or reject petitions and may provide more

161

For discussions of the constitutional roots of the right to petition the government for a
redress of grievances, see Maggie McKinley, Lobbying and the Petition Clause, 68 STAN. L.
REV. 1131, 1142-56 (2016) and JASON A. SCHWARTZ & RICHARD L. REVESZ, ADMIN.
CONFERENCE OF THE U.S., PETITIONS FOR RULEMAKING 7-8 (2014).
162
Administrative Procedure Act, 5 U.S.C. § 553(e) (2012).
163
See id. § 555(b) (providing that “within a reasonable time, each agency shall proceed
to conclude a matter presented to it”).
164
See id. § 555(e) (“Prompt notice shall be given of the denial in whole or in part of a
written application, petition, or other request of an interested person made in connection with
any agency proceeding. Except in affirming a prior denial or when the denial is selfexplanatory, the notice shall be accompanied by a brief statement of the grounds for denial.”).
165
See Massachusetts v. EPA, 549 U.S. 497, 527-28 (2007); Am. Horse Prot. Ass’n, Inc.
v. Lyng, 812 F.2d 1, 4-5 (D.C. Cir. 1987). ACUS has previously endorsed a “limited” scope
of judicial review in this context. See Admin. Conf. of the U.S., Recommendation 95-3,
Review of Agency Regulations, 60 Fed. Reg. 43,108, 43,109 (Aug. 18, 1995).
166
See Endangered Species Act, 16 U.S.C. §§ 1533(b)(3)(A)-(C) (2012); see also Eric
Biber & Berry Brosi, Officious Intermeddlers or Citizen Experts? Petitions and Public
Production of Information in Environmental Law, 58 UCLA L. REV. 321, 327-38 (2010)
(providing examples); SCHWARTZ & REVESZ, supra note 161, at 31-35 (same).

37

detailed procedural requirements for the processing of petitions. 167 While rulemaking
petitions—like agency agenda setting more generally—are understudied in the
scholarly literature, 168 ACUS has produced two sets of reports and recommendations
on this important subject, 169 and several empirical studies have recently been
produced on various aspects of the topic. 170
Rulemaking petitions can be a valuable form of public engagement for several
reasons. First, they promote democratic responsiveness because rulemaking petitions
can be filed by anyone and they engender a legal right to a meaningful response.171
Scholars have also pointed out that the petition process has been a fruitful avenue for
networking and coalition building by nascent social movements. 172 From an agency’s
perspective, rulemaking petitions can “be a source of some valuable ideas for
regulatory change, though this may vary from agency to agency and over time.”173
Rulemaking petitions may be particularly valuable when they include detailed
proposals and are accompanied by supporting data, and when interested members of
the public are able to provide agencies with “dispersed or diffuse” information that
would otherwise be unavailable or difficult to collect. 174
To be sure, sophisticated stakeholders may be in the best position to make
effective use of rulemaking petitions for many of the same reasons discussed above in
Part III.B. Yet a study of petitions to the U.S. Fish and Wildlife Service seeking
listings of endangered or threatened species under the Endangered Species Act
(ESA) 175 found that “environmental or scientific organizations that have a primary
focus on species protection” made the greatest use of them. 176 Indeed, it is reasonable
to assume that public interest groups make greater use of pro-regulatory petitions for

167

See William V. Luneburg, Petitioning Federal Agencies for Rulemaking: An
Overview of Administrative and Judicial Practice and Some Recommendations for
Improvement, 1988 WIS. L. REV. 1, 14-20 (1988) (describing petition processes under statutes
other than the APA).
168
See Biber & Brosi, supra note 166, at 328-29 (“[D]espite the prevalence of agency
petition provisions at the federal level, they have been almost completely ignored in the legal
and political science literature.”).
169
See generally WILLIAM V. LUNEBURG, ADMIN. CONFERENCE OF THE U.S., PETITIONS
FOR RULEMAKING: FEDERAL AGENCY PRACTICES AND RECOMMENDATIONS FOR
IMPROVEMENT (1986); SCHWARTZ & REVESZ, supra note 161.
170
See generally Biber & Brosi, supra note 166; ROBIN COOPER FELDMAN ET AL.,
EMPIRICAL EVIDENCE OF DRUG COMPANIES USING CITIZEN PETITIONS TO HOLD OFF
COMPETITION (2018); Walters, supra note 6.
171
See Walters, supra note 6 (manuscript at 14-16).
172
See, e.g., McKinley, supra note 161, at 1138.
173
Luneburg, supra note 167, at 62.
174
Biber & Brosi, supra note 166, at 364. See also SCHWARTZ & REVESZ, supra note
161, at 62.
175
See Endangered Species Act, 16 U.S.C. §§ 1533(b)(3)(A)–(C) (2012).
176
Biber & Brosi, supra note 166, at 363-64.

38

rulemaking while regulated industries make greater use of de-regulatory petitions to
amend or repeal a rule, discussed more fully below in Part X (Retrospective Review).
Thus, petitions for rulemaking may provide a vital tool for shaping an agency’s
rulemaking agenda in the public interest. 177 Nevertheless, agencies may find it
valuable to undertake greater efforts to involve missing stakeholders and unaffiliated
experts in the petition process in appropriate circumstances.
At the same time, rulemaking petitions raise some potential concerns. First,
agencies must devote limited resources to reviewing and responding to rulemaking
petitions, even when they are unsupported by data or otherwise unmeritorious. This
obligation “can impose a strain on already tight agency budgets and can be perceived
as an undesirable disruption of internally-established regulatory priorities.” 178 At
times, excessive devotion to rulemaking petitions pursuant to unusually demanding
statutory mandates has overwhelmed an agency’s agenda and interfered with its
ability to establish or pursue what would otherwise be the most sensible priorities.179
There are related concerns that because many rulemaking petitions are filed by
regulated entities that are seeking regulatory relief, rulemaking petitions can provide a
relatively low visibility mechanism for facilitating regulatory capture. Rulemaking
petitions can also be filed by business interests in an effort to gain an advantage over
their competitors. 180
Despite these concerns, there is some recent empirical evidence that agencies
tend to decide rulemaking petitions in a reasonably independent and even-handed
way that tends to favor relatively narrow and incremental changes, 181 and that
rulemaking petitions can provide agencies with valuable information that improves

177

See supra notes 171-176 and accompanying text.
Luneburg, supra note 167, at 62.
179
See Biber & Brosi, supra note 166, at 323 (“[T]he little scholarly commentary on
petitions has been strongly negative, focused on the idea that petitions divert agencies from
pursuing the most serious problems that deserve regulatory attention.”). The classic
cautionary tale involves the Consumer Product Safety Commission during the 1970s and
early 1980s. See Richard A. Merrill, CPSC Regulation of Cancer Risks in Consumer
Products: 1972-1981, 67 VA. L. REV. 1261, 1309 (1981); Teresa M. Schwartz, The
Consumer Product Safety Commission: A Flawed Product of the Consumer Decade, 51 GEO.
WASH. L. REV. 32, 48 (1982).
180
See Walters, supra note 6 (manuscript at 6-7) (“Rulemaking petitions . . . give
business interests the means to further ‘corrosive capture’ via deregulatory petitions, as well
as to further ‘anti-competitive capture’ via proposals for new regulatory requirements that
disproportionately affect competitors or establish a monopoly by creating barriers to firm
entry.”); see also FELDMAN ET AL., supra note 170, at 5 (concluding that big pharmaceutical
companies file rulemaking petitions with the Food and Drug Administration strategically to
delay entry of generic competitors into the market).
181
See Walters, supra note 6 (manuscript at 31-36) (finding little evidence of “business
influence,” and reporting that “agencies favor rulemaking petitions that request narrow,
technical changes in a deregulatory direction” regardless of the petitioner’s identity).
178

39

the quality of their agenda-setting processes in some situations.182 The success of any
rulemaking petition process, particularly as a tool for public engagement, will
necessarily depend to a large extent on the quality of its procedures. Yet the APA
does not provide any procedural requirements for the processing of rulemaking
petitions. 183 Accordingly, ACUS set forth a set of recommendations in 1986 that
agencies could adopt to improve the quality of their rulemaking petition processes,
some of which would also facilitate public engagement. These include:
•

Establishing by rule basic procedures for the receipt, consideration, and
prompt disposition of petitions for rulemaking;

•

Maintaining a publicly available petition file;

•

Providing guidance on the type of data, argumentation, or other
information the agency needs to consider petitions; and

•

Developing effective methods for providing notice to interested persons
that a petition has been filed and identifying the agency office or official
to whom inquiries and comments should be made. 184

A more recent ACUS study in 2014 found that most agencies still did not have
official procedures for handling rulemaking petitions. 185 ACUS explicitly recognized
that “[a]lthough the petitioning process can be a tool for enhancing public
engagement in rulemaking, in practice most petitions for rulemaking are filed by
sophisticated stakeholders and not by other interested members of the public.” 186
ACUS suggested that the petition process could be improved by increased
transparency, improved communication between agencies and petitioners, and more
prompt and thorough responses. Moreover, ACUS recognized that petitioners and
agencies would both benefit “from greater clarity as to how petitions can be filed,
what information should be included to make a petition more useful and easier for the
agency to evaluate, whether or when public comment will be invited, and how long it
may take to resolve a petition.” 187 Accordingly, ACUS recommended a set of best

182

See generally Biber & Brosi, supra note 166 (examining data from the listing of
species protected by the Endangered Species Act and finding that rulemaking petitions and
citizen suits did not result in suboptimal agenda-setting by the U.S. Fish and Wildlife Service,
and suggesting that public participation can affirmatively improve the performance of
environmental agencies by providing access to diffuse information about environment
conditions).
183
As noted above, some statutes do impose relatively detailed procedural requirements
on the petition processes they establish. See supra note 166-167 and accompanying text.
184
See Admin. Conf. of the U.S., Recommendation 1986-6, Petitions for Rulemaking,
51 Fed. Reg. 46,985, 46, 988, 46,988 (Dec. 30, 1986).
185
See Admin. Conf. of the U.S., Recommendation 2014-6, 79 Fed. Reg. 75,114,
75,117 (Dec. 17, 2014).
186
Id. at 75,117-18.
187
Id. at 75,118.

40

practices that could improve the rulemaking petition process, which included the
following suggestions:
•

Establish procedures that, among other things, explain what type of data,
argumentation, and other information make a petition more useful and
easier for the agency to evaluate, and that identify any information that is
statutorily required for the agency to act;

•

Accept the electronic submission of petitions via email, Regulations.gov,
or an existing online docketing system;

•

Designate a particular person or office to receive and distribute all
petitions for rulemaking to ensure that each petition is expeditiously
directed to the appropriate agency personnel for consideration and
disposition;

•

Encourage and facilitate communication between agency personnel and
petitioners, both prior to submission and while petitions are pending for
disposition;

•

Use online dockets to allow the public to monitor the status of petitions;

•

Consider inviting public comment on petitions for rulemaking by either (a)
soliciting public comment on all petitions for rulemaking; or (b) deciding,
on a case-by-case basis, whether to solicit public comment on petitions for
rulemaking;

•

Use available online platforms, including the agency’s website and
Regulations.gov, to implement this recommendation, including by
informing the public about the petitioning process, facilitating the
submission of petitions, inviting public comment, providing status
updates, improving the accessibility of agency decisions on petitions, and
annually providing information on petitions for rulemaking that have been
resolved or are still pending. 188

In connection with this final recommendation, ACUS also suggested that OIRA
should ask agencies to “include in their annual regulatory plan information on
petitions for rulemaking that have been resolved during that year or are still
pending.” 189
The report accompanying ACUS’s most recent set of recommendations on
rulemaking petitions identified several examples of agencies that follow some of
these best practices. For instance, the Nuclear Regulatory Commission has developed
some of the “best practices for educating the public about the right to petition,
transparently reporting the status of petitions, and regularly communicating with

188
189

See id. at 75,118-19.
Id. at 75,119.

41

petitioners.” 190 These best practices include the NRC’s provision of a “plain
language” description of the process for filing a rulemaking petition on its website,191
the announcement of filed petitions in the Federal Register, the public docketing of
all related communications on Regulations.gov, and the provision of reasoned
explanations for its decisions that are responsive to public comments. 192 The NRC
also communicates effectively with interested members of the public by providing
contact information on its website and by both permitting and encouraging potential
petitioners to consult with agency staff prior to filing their petitions. 193
Scholars have also identified the rulemaking petition process that is used by the
Fish and Wildlife Service to make listing decisions under the Endangered Species Act
as a notable success story. Biber and Brossi found in this context that public petitions
“result in the identification of species that are at least as deserving of protection under
the Act as species identified by the agency on its own.” 194 Not only do these findings
undermine the claim that rulemaking petitions will necessarily distract an agency
from establishing the most sensible priorities, but Biber and Brossi contend that
rulemaking petitions can improve the rationality of an agency’s agenda setting by
providing the agency with dispersed information that will help the agency to achieve
its goals. 195
Biber and Brossi recognize, however, that the ESA petition process is unusual in
various ways that may limit the transferability of the FWS’s experience to other
contexts. 196 In particular, ESA petitions require a discrete decision (whether or not to
list a species as threatened or endangered) that turns on a relatively specific technical
determination upon which relevant scientific knowledge is highly dispersed and
indeed often not otherwise publicly available. Moreover, the FWS has a designated
budget for listing decisions, which means that such efforts do not affect the agency’s
budget for other activities. 197 Biber and Brossi therefore suggest that public
participation, and rulemaking petitions in particular, can help to promote regulatory
rationality “where there is dispersed information that could lead to better
decisionmaking,” and “the agency is (a) making a relatively simple set of decisions

190

SCHWARTZ & REVESZ, supra note 161, at 49.
See id.; The Rulemaking Petition Process, U.S. NUCLEAR REG. COMMISSION,
http://www.nrc.gov/about-nrc/regulatory/rulemaking/petition-rule.html (last updated Nov. 7,
2017).
192
See SCHWARTZ & REVESZ, supra note 161, at 50.
193
See id.
194
Biber & Brossi, supra note 166, at 321. See also SCHWARTZ & REVESZ, supra note
161, at 60-62 (discussing this example).
195
Biber & Brossi, supra note 166, at 325.
196
See id. at 378-82.
197
See id. at 379.
191

42

(b) using technical factors that do not require complicated trade-offs with a range of
other activities.” 198
The foregoing studies, ACUS recommendations, and examples of best practices
with rulemaking petitions suggest several lessons regarding how the petition process
can be used to enhance public engagement in agency agenda setting. From a
substantive perspective, Biber and Brossi suggest that agencies interested in making
greater use of petition processes as an agenda-setting tool should:
(a) focus on tasks that are relatively simple and cheap to accomplish;
(b) result in the imposition of a default regulatory standard to
minimize the administrative burden on the agency; (c) allow for some
sort of safety valve if the default standard is inappropriate; (d) commit
the agency to take action on petitions for which the necessary standard
has been met; and (e) impose a cap on the maximum number of
petitions that can be granted in a year, in order to keep the petition
process from swamping the rest of the agency’s regulatory agenda. 199
Biber and Brossi suggest, for example, that a variety of other areas of
environmental law may fit these criteria, as well as the regulation of airline and traffic
safety and the securities industry. 200
From a procedural standpoint, the best ways to improve the capacity of
rulemaking petitions to facilitate public engagement in agency agenda setting involve
increased education and enhanced communication. First, this would include, on the
front end, plain language explanations of the opportunity to submit rulemaking
petitions in a prominent location on the agency’s website, along with clear guidance
regarding the appropriate format and the kinds of supporting information that are
most helpful for evaluating a petitioner’s request. Second, agencies should consider
providing examples of “model” rulemaking petitions that potential petitioners could
use for additional guidance. Third, agencies should provide contact information on
their websites and encourage potential petitioners to consult with agency staff prior to
filing their petitions. Fourth, the filing of rulemaking petitions should be announced
in the Federal Register and on Regulations.gov and the agency’s website. Fifth,
agencies should maintain publicly available dockets of all related communications on
their websites and Regulations.gov, and use these online platforms to otherwise keep
the public informed about the status of petitions that have been resolved or are still
pending. Sixth, as ACUS has previously recommended, agencies should consider
inviting public comment on rulemaking petitions either as a matter of course or on a
case-by-case basis, depending on whether a petition addresses a question of policy or
of general interest and whether members of the public could potentially provide the

198

Id. at 378-79.
Id. at 382.
200
See id. at 379-80, 383.
199

43

agency with relevant information that is not already in the agency’s possession.201
When agencies provide opportunities for public comment on rulemaking petitions,
they can effectively provide two layers of mechanisms for public engagement. 202
While the rulemaking petition process provides the potential for relatively robust
public engagement, it may also be worthwhile for agencies to consider additional
ways to facilitate greater balance in the interests that are represented. On the front
end, agencies should consider affirmatively soliciting rulemaking petitions from
absent stakeholders and unaffiliated experts when they are likely to have dispersed
knowledge that is otherwise unavailable to agencies, particularly when the other
circumstances identified by Biber and Brossi are present. Agencies could, in turn,
affirmatively solicit comments from absent stakeholders and unaffiliated experts on
rulemaking petitions filed by regulated entities to ensure that agencies are able to
make fully informed decisions. 203 Both of these strategies would require enhanced
education and communication of the kinds recommended above, as well as the use of
listserves and other outreach techniques that could potentially encourage more absent
stakeholders and unaffiliated experts to participate in the rulemaking petition
process. 204
B.

Federal Advisory Committees

A second mode of public engagement that agencies can use in agenda setting is
to solicit and obtain advice from federal advisory committees. The use of federal
advisory committees is governed by the Federal Advisory Committee Act
(“FACA”), 205 which formalizes the process for agencies that seek advice from groups
that include non-federal employees and imposes a variety of procedural obligations
that are designed to promote transparency and ensure that the composition of
committees reflects an appropriate balance “of the points of view represented and the
functions to be performed.” 206 While chartering a new advisory committee can take a
substantial amount of time, and complying with FACA’s procedural obligations
entails certain burdens, studies conducted on behalf of ACUS have concluded that
FACA generally strikes an appropriate balance between the goals of permitting

201

See Admin. Conf. of the U.S., Recommendation 2014-6, Petitions for Rulemaking,
79 Fed. Reg. 75,114, 75,117, 75,118 (Dec. 17, 2014).
202
We discuss best practices related to the notice-and-comment process below in Part
XIII.D.
203
We discuss best practices for petitions for repeal or amendment of a rule in the
context of retrospective review below in Part XIII.E.
204
We discuss the importance of outreach and planning in more detail below in Part XII.
205
Federal Advisory Committee Act, 5 U.S.C. App. § 2 (2012).
206
Id. §§ 5(b)(2)-(3), (c). For excellent overviews of FACA’s requirements, see
generally REEVE T. BULL, ADMIN. CONFERENCE OF THE U.S., THE FEDERAL ADVISORY
COMMITTEE ACT: ISSUES AND PROPOSED REFORMS (2011) [hereinafter BULL, THE FEDERAL
ADVISORY COMMITTEE ACT]; Steven P. Croley & William F. Funk, The Federal Advisory
Committee Act and Good Government, 14 YALE J. ON REG. 451 (1997).

44

agencies to obtain valuable information in a relatively efficient and cost-effective
manner and promoting transparency and avoiding capture by narrow special
interests. 207 Steven Croley and Bill Funk concluded that considering the breadth and
amount of quality advice that agencies can obtain at a marginal cost from federal
advisory committees, “the main virtue of the FACA is that it enables the federal
government to solicit what is tantamount to free advice.” 208 ACUS has, nonetheless,
made recommendations to improve FACA and promote agencies’ ability to obtain
quality advice from federal advisory committees in an efficient manner while also
promoting transparency. 209 Several of those recommendations would facilitate public
engagement in rulemaking, and therefore merit brief reiteration here:
•

“Upon creating a new advisory committee, agencies should announce the
committee’s mission in the Federal Register and/or on the agencies’ Web
site and invite nominations for potential committee members, from the
public, from expert communities with experience in the subject matter of
the committee’s assignment, and/or from groups especially likely to be
affected by the committee’s work.” 210

•

“Agencies should identify and prioritize those factors for achieving
balance among committee members that are directly relevant to the subject
matter and purpose of the committee’s work.” 211

•

Agencies should recognize that they may hold committee meetings via
online forums, and that such “virtual meetings . . . can occur electronically
in writing over the course of days, weeks or months on a moderated,
publicly accessible web forum.” 212

•

“Agencies should provide live webcasts of open committee meetings
and/or post recordings following such meetings unless the costs are
prohibitive.” 213

•

Agencies should post documents related to an advisory committee’s work
on a publicly-accessible committee website as soon as possible. 214

207

See BULL, THE FEDERAL ADVISORY COMMITTEE ACT, supra note 206, at 32-33;
Croley & Funk, supra note 206, at 526-27.
208
Croley & Funk, supra note 206, at 527 (emphasis added).
209
See generally Admin. Conf. of the U.S., Recommendation 2011-7, The Federal
Advisory Committee Act—Issues and Proposed Reforms, 77 Fed. Reg. 2,257, 2,261 (Jan. 17,
2012).
210
Id. at 2,264.
211
Id. at 2,263.
212
Id. See also BULL, THE FEDERAL ADVISORY COMMITTEE ACT, supra note 206, at 4547 (discussing “asynchronous virtual meetings of advisory committees”).
213
The Federal Advisory Committee Act—Issues and Proposed Reforms, 77 Fed. Reg. at
2,264.

45

While advisory committees can be used as a mode of public engagement at any
stage of the rulemaking process (and we have several examples of agencies that
routinely use them for purposes of rule development), 215 there is a significant and
perhaps relatively untapped potential for using federal advisory committees to inform
an agency’s rulemaking agenda. The goal of this form of public engagement would
be to obtain advice from a balanced group of well-informed stakeholders and
unaffiliated experts in setting the agency’s rulemaking priorities. Federal advisory
committees could be used generally to engage in strategic planning or otherwise to
obtain quality advice on a variety of more specific questions related to agenda setting:
•

Which rulemaking projects should an agency consider?

•

What should be in the agency’s Regulatory Plan or submission for the
Unified Agenda?

•

Should the agency initiate a particular rulemaking project?

•

How should an agency prioritize among the rulemaking projects it may or
must pursue?

In using advisory committees to obtain advice on such questions, there are
several distinctive issues that an agency would need to consider. First, the agency
would need to consider which advisory committee or what type of advisory
committee is best suited for its purposes. Many agencies could use existing advisory
committees to obtain advice on the foregoing questions, especially when those
committees are composed of a balanced group of interested stakeholders or
unaffiliated experts and their charters are broad enough to include these functions.
This approach would have the obvious advantage of limiting the costs, delays, and
other potential obstacles associated with chartering a new advisory committee. 216 It
would also allow agencies to take advantage of the knowledge and experience that
members of existing groups have already developed about the regulatory program at
issue.
On the other hand, existing advisory committees may not have the proper charge
or composition to provide the agency with sufficiently balanced advice on its
prospective regulatory agenda. The agency may therefore want to consider chartering
a new advisory committee (or committees) for these purposes. In this regard, the
agency should consider whether it is necessary or appropriate to charter a new
advisory committee to provide advice on whether to undertake a particular
rulemaking initiative. It seems likely that this level of effort would only be
sufficiently worthwhile in the agenda-setting context for particularly important or
controversial rulemaking initiatives, or perhaps where the agency is considering

214

Id.
See infra Part VII.A.
216
Cf. Exec. Order No. 12,838, 58 Fed. Reg. 8,207 (Feb. 10, 1993) (limiting the creation
of new advisory committees).
215

46

entering a new field of regulation that would have significant long-term implications
for its future direction. Consider, for example, the delegation to the Consumer
Financial Protection Bureau (CFPB) of new powers to regulate certain nonbank
financial institutions, such as mortgage companies, payday lenders, and private
education lenders, under the Dodd-Frank Wall Street Reform and Consumer
Protection Act. It may be appropriate in such situations to charter an advisory
committee to procure advice on how to proceed, and to ensure that representatives of
all relevant stakeholders—including both traditionally absent borrowers and small
lenders—are involved in the deliberations.
For more routine agenda-setting decisions, agencies should consider establishing
standing advisory committees that are explicitly charged with providing advice on the
agency’s rulemaking agenda. Such “agenda-setting advisory committees” could
regularly provide specific advice to agency officials on the kinds of questions
identified above. While advisory committees of this nature could be composed of a
balanced and representative group of the agency’s primary stakeholders—including
traditionally absent stakeholders and perhaps members of the general public—it
would be worthwhile for certain agencies (and especially those agencies whose
constituencies closely mirror the general public, such as the Department of Education
or the Consumer Financial Protection Bureau) to consider establishing agenda setting
advisory committees that are composed entirely of a representative sample of
ordinary citizens. Citizens advisory committees of this nature seem especially
appropriate in the agenda setting context, where agencies are establishing priorities
and determining how they will use their limited resources. 217 If an agency was
interested in comparing the priorities of its more traditional clientele with those of
reasonably informed members of the general public, the agency could seek advice on
its rulemaking agenda from both a traditional advisory committee and a citizens
advisory group.
Of course, the use of advisory committees as a means of public engagement in
agenda setting is not without challenges or limitations. First, as noted above,
establishing a new advisory committee in particular requires significant time and
effort. Second, in comparison to some other modes of public engagement, the use of
federal advisory committees is heavily regulated by law. Third, the membership of
an advisory committee may not truly be representative, either because some important
stakeholders are absent or because a representative’s supposed constituency has
competing interests and perspectives. 218 It is also possible for advisory committees to

217

For a proposal that advocates the use of citizen advisory committees in appropriate
circumstances, see Bull, Making the Administrative State “Safe”, supra note 13, at 640-647.
218
See JENNIFER NASH & DANIEL E. WALTERS, PENN PROGRAM ON REGULATION,
PUBLIC ENGAGEMENT AND TRANSPARENCY IN REGULATION: A FIELD GUIDE TO
REGULATORY EXCELLENCE 22 (2015), https://www.law.upenn.edu/live/files/4709nashwalters-ppr-researchpaper062015.pdf (discussing research finding that balanced interests
and viewpoints enhance both the effectiveness and credibility of advisory committees).

47

be stacked in a particular partisan direction. 219 Fourth, advisory committees cannot
function effectively or provide sound advice if they are not properly informed of the
relevant issues. And, achieving this task is significantly more difficult when advisory
committees are composed of unsophisticated stakeholders or ordinary citizens who
lack background knowledge of relevant technical or legal issues.
Accordingly, it is important for agencies that use advisory committees as a means
of public engagement in agenda setting to develop briefing materials that clearly
explain the relevant issues in a manner that non-experts can readily understand.
While this task can undoubtedly be difficult and resource-intensive, it is probably
easier at the agenda-setting stage than it would be at later stages of the rulemaking
process when the details of alternative approaches to a problem are evaluated.
Moreover, the background materials that are prepared for federal advisory
committees can be used by the agency as the basis for other public engagement
efforts. For example, such materials could be presented as the basis for discussion
with focus groups or at listening sessions or other public meetings. Similarly, those
materials could be placed on the agency’s website as the basis for soliciting public
comment on the agency’s potential priorities.
C.

Focus Groups

A third tool that agencies can use to facilitate public engagement in agenda
setting is focus groups. Focus groups involve facilitated small group discussions of a
set of questions by participants who are randomly selected or who have other specific
demographic characteristics, backgrounds, or qualifications. 220 They can be used to
gauge participants’ reactions to information, ideas, messages, or proposals, and to
begin to identify preferred alternatives or potential concerns. Focus groups are a
good way “to find out what issues are of most concern for a community or group
when little or no information is available.” 221 They are often used as a sounding
board or as a way to “trouble spot” by identifying preliminary issues or concerns that
will require further research and deliberation, and as a mechanism for obtaining
relatively detailed knowledge of the primary concerns of the participants. 222 Focus
groups will often produce ideas that would not emerge from surveys because they
provide greater opportunities for interactive discussion and dialogue. 223 Their use
should not trigger FACA so long as the agency is not seeking a report or set of
recommendations from the group, as a group, concerning how the agency should

219

See, e.g., Brian D. Feinstein & Daniel Jacob Hemel, Federal Advisory Committees &
the Internal Separation of Powers 23 (Jan. 23, 2018) (unpublished manuscript) (on file with
authors).
220
See JAMES L. CREIGHTON, THE PUBLIC PARTICIPATION HANDBOOK: MAKING
BETTER DECISIONS THROUGH CITIZEN INVOLVEMENT 113 (2005).
221
ENVTL. PROT. AGENCY, INTRODUCTION TO THE PUBLIC PARTICIPATION TOOLKIT 58
[hereinafter EPA, PUBLIC PARTICIPATION TOOLKIT].
222
See id.
223
See id.

48

exercise its discretion, but rather is engaged in fact finding and seeking to understand
the views of individual members of the group. 224
The basic steps for conducting a focus group include (1) identifying the target
population, (2) recruiting participants, (3) choosing and hiring a facilitator or
moderator, (4) preparing the questions, (5) conducting (and recording) the session,
and (6) preparing a report that summarizes the results. 225 The convener should also
provide relevant background materials for the participants when necessary. The
primary challenges or limitations of focus groups are that they require skilled
facilitation and careful planning, the participants may face a steep learning curve, and
their views may not be representative (even in an informal sense). Numerous federal
agencies have used focus groups, primarily to ask questions about different
approaches to consumer disclosure or product labeling, or to gauge public knowledge
or attitudes regarding potential subjects of regulation, particularly when there may be
widespread misinformation or confusion on the topic. For example, NHTSA has
conducted focus groups to address public fears about airbags, 226 potential labels on
tire fuel efficiency, 227 and whether drivers understand advanced crash avoidance
technologies. 228 Similarly, FDA has conducted focus groups on food nutrition
labeling, 229 the usefulness of prescription drug labeling under existing regulations,230
and the public’s views on genetically modified foods. 231

224

See BULL, THE FEDERAL ADVISORY COMMITTEE ACT, supra note 206, at 13-14
(describing the “group” requirement of the statute, and explaining that “the case law has also
created an amorphous exception to FACA that arises when an agency seeks advice from an
assemblage of persons acting not as a formal group but as a collection of individuals”);
Steven P. Croley, Practical Guidance on the Applicability of the Federal Advisory Committee
Act, 10 ADMIN. L.J. AM. U. 111, 129 (1996); W. Herbert McHarg, The Federal Advisory
Committee Act: Keeping Interjurisdictional Ecosystem Management Groups Open and Legal,
15 J. ENERGY NAT. RESOURCES & ENVTL. L. 437, 465 (1995).
225
See CREIGHTON, supra note 220, at 114-16.
226
See Nina A. Mendelson, Torrents of E-Mail, supra note 13, at 1366 (discussing this
example).
227
See Douglas A. Kysar, Politics by Other Meanings: A Comment on “Retaking
Ratonality Two Years Later”, 48 HOUS. L. REV. 43, 52 (2011).
228
See Stephen P. Wood et al., The Potential Regulatory Challenges of Increasingly
Autonomous Motor Vehicles, 52 SANTA CLARA L. REV. 1423, 1497 (2012).
229
See Food Labeling: Format for Nutrition Label, 57 Fed. Reg. 32,058, 32,060 (July 20,
1992).
230
See Erika Fisher Lietzan & Sarah E. Pitlyk, Thoughts on Preemption in the Wake of
the Levine Decision, 13 J. HEALTH CARE L. & POL’Y 225, 250 n.141 (2010).
231
See FDA Report on Consumer Focus Groups on Biotechnology, MERIDIAN
INSTITUTE
(Feb.
13,
2001),
http://www.merid.org/en/Content/News_Services/Food_Security_and_AgBiotech_News/Arti
cles/2001/02/13/FDA_Report_On_Consumer_Focus_Groups_On_Biotechnology.aspx.

49

Focus groups could also be a potentially useful tool for facilitating public
engagement in regulatory agenda setting. The goal would be to obtain feedback from
missing stakeholders, unaffiliated experts, or the general public on potential
regulatory priorities and issues of greatest concern. Because agenda setting is
primarily about an agency’s need to establish sensible priorities in the face of limited
resources, this may be an ideal setting to use focus groups “to find out what issues are
of most concern for a community or group when little or no information is
available.” 232 Moreover, because the agency can choose its target audience and the
precise questions for discussion, it could potentially use focus groups to get a better
sense of the priorities of different sets of stakeholders and identify areas of consensus
and disagreement. This use of focus groups would also produce information about
the primary concerns of the various participants.
Therefore, we anticipate that focus groups could be a relatively easy and
inexpensive way for agencies to obtain meaningful feedback on their regulatory
agendas from missing stakeholders or unaffiliated experts who are otherwise difficult
to reach and tend not to participate in public engagement efforts where participation
is self-selecting. 233 It may also be useful for agencies to use focus groups composed
of regulatory beneficiaries for purposes of agenda setting because this exercise would
force participants to reckon with the agency’s resource limitations and identify their
highest priorities. In addition to the preceding recommendations, the best practices
for using focus groups in agency agenda setting would include carefully selecting the
facilitator and the participants, developing an agenda with just a handful of major
questions or topics for discussion, providing participants with appropriate briefing
materials to familiarize them with the relevant issues, and perhaps most importantly,
compiling a report that summarizes the feedback and identifies topics for further
study.
D.

Requests for Information (“RFIs”)

A fourth tool of public engagement that agencies frequently use early in the
rulemaking process, including at the agenda-setting stage, is what is generally known
as requests for information or “RFIs.” 234 A request for information is typically

232

EPA, PUBLIC PARTICIPATION TOOLKIT, supra note 221, at 58.
See id. at 74 (recognizing that focus groups can be a useful way to obtain the views of
people who do not generally participate in larger public meetings).
234
Agencies sometimes refer to this method of public engagement by other names. For
example, the FCC refers to this tool as a “notice of inquiry.” RFIs are also very similar to,
and in some cases, indistinguishable from Advance Notices of Proposed Rulemaking
(“ANPRMs”), which are published in the Federal Register to obtain comments on a potential
rule that is under consideration by an agency. For purposes of this report, we use the term
“RFI” to refer to requests for comments early in the rule development process or
retrospective review when the agency is still considering whether to engage in a rulemaking
project and is just beginning to consider potential approaches to a problem. We use the term
“ANPRM” to refer to requests for comments later in the rule development process when the
agency has prepared a rough draft of a proposed rule or has significantly narrowed the
233

50

published in the Federal Register and on Regulations.gov, and generally describes a
problem or an issue that is of interest to an agency. Agencies use RFIs to request
data, comments, or other information from the public on a designated problem or
issue. Responses to RFIs are intended “to assist the [a]gency in determining an
appropriate course of action—which may or may not involve rulemaking—to address
the problem.” 235
For some agencies, such as the Consumer Financial Protection Bureau (CFPB)
and the Department of Energy (DOE), the issuance of an RFI is nearly standard
practice in early rule development, discussed more fully below in Part VII.C.
Interviewees told us that RFIs are valuable in part because commentators are more
likely to communicate their situated knowledge in this setting rather than advocate a
position for or against a proposed rule. Moreover, RFIs also provide agencies with
valuable information about how to communicate their work to the affected
communities. Interviewees told us they consistently found RFIs most useful when
the agency is unsure of the best course of action or lacks important information
regarding the nature of a problem, the appropriate regulatory strategies, or
compliance-related questions. They recommended that RFIs remain neutral regarding
the agency’s future direction and ask lots of questions aimed at soliciting the relevant
data.
For many of the same reasons, RFIs can also be a useful tool for facilitating
public engagement in agency agenda setting. Indeed, some agencies have already
been using RFIs for precisely this purpose. For example, the IRS has for some time
issued an annual notice in the Internal Revenue Bulletin requesting public input on
IRS priorities. 236 Similarly, many agencies recently used RFIs to seek input on the
regulatory and deregulatory actions they should consider under Executive Order
13,771 (“Reducing Regulation and Controlling Regulatory Costs”). In response to
such an RFI, the Pension Benefit Guaranty Corporation (“PBGC”), received 38
comments from individuals, which was both a quantitatively and qualitatively
superior response than this agency ordinarily receives when it solicits public
comments.
We endorse these practices and recommend that other agencies consider using
RFIs to solicit public feedback on their agendas. In addition, we reiterate the advice
of our interviewees that RFIs should (1) be used whenever the agency is open-minded
about appropriate courses of action, (2) be neutral about whether or how the identified
issues or problems will or should be addressed, and (3) pose detailed questions that
are aimed at soliciting the situated knowledge and data necessary for the agency to
make informed decisions. We would also encourage agencies to engage in robust
options that are seriously under consideration. In any event, we recognize that RFIs and
ANPRMs are neither legally nor categorically distinct.
235
Requests
for
Information
(RFI),
U.S.
DEP’T
LAB.,
https://arlweb.msha.gov/REGSRFI.asp (last visited Oct. 22, 2018) (defining “RFIs” and
providing numerous examples).
236
See Coglianese & Walters, supra note 155, at 887 (discussing the IRS’s practice).

51

outreach efforts to bring their RFIs to the attention of missing stakeholders and
unaffiliated experts to increase the likelihood that they will receive balanced
comments that reflect all of the relevant interests and perspectives, and to maximize
the likelihood that they will receive the situated knowledge and data they are seeking.
For example, the CFPB’s External Affairs and Consumer Education and Engagement
Offices undertake efforts to ensure that different communities are aware of RFIs and
able to participate. CFPB focuses its outreach efforts on specific communities of
absent stakeholders, such as the elderly, students, and “the unbanked.” 237
E.

Listening Sessions

A fifth tool of public engagement that agencies often use early in the rulemaking
process, including at the agenda setting stage, is “listening sessions” or other forms of
public meetings designed to gather information, comments, and data from the public
on a designated problem or issue. 238 They also provide the agency with an
opportunity to educate the public about a problem and the agency’s current treatment
or potential plans regarding the matter. Unlike focus groups, in which the
participants are selected by the agency or outside consultants to represent particular
demographic groups, listening sessions are typically open to the general public.
Unlike RFIs, listening sessions are typically done in person, rather than in writing,
although current technology allows such meetings to be conducted online or through
video-conferencing. Even when the meetings are held in person, they can be livestreamed and recorded so interested persons can observe the proceedings remotely.
Those observers could participate in the meeting electronically or offer written
comments on the topic at a later date. Moreover, listening sessions can be done at
regional sites around the country on important issues or problems that vary
geographically. Listening sessions can be designed to allow for more “back-andforth” than is generally possible solely through written communications. Even when
the agency attends primarily to “just listen,” agency officials can pose questions and
encourage deliberation among the participants. 239 Listening sessions are another
potentially valuable method of helping the agency to determine an appropriate course
of action—which may or may not involve rulemaking—to address the problem.
Several agencies have used listening sessions as an effective method of gathering
information. For example, the Nuclear Regulatory Commission (“NRC”) frequently

237

Telephone Interview with Consumer Financial Protection Bureau (Dec. 5, 2017).
For a useful discussion of the potential value, challenges, and best practices
associated with conducting public meetings in general, see EPA, PUBLIC PARTICIPATION
TOOLKIT, supra note 221, at 74-77. Once again we have adopted a nomenclature based on
when these types of meetings are held. Therefore, we refer to them as listening sessions
during agenda setting and early rule development when the agency’s regulatory direction is
more open and it primarily wishes to listen to stakeholders. By contrast, we refer to such
events as public meetings during advanced rule development and the notice-and-comment
process, when the agency is heading in a particular direction and seeks to explain its proposal
in addition to gaining public input.
239
Telephone Interview with Forest Service (Nov. 17, 2017).
238

52

holds public meetings around the country where the public can raise issues and
express concerns about a topic. The NRC’s meetings tend to focus on a particular
topic rather than what should appear on the agency’s rulemaking agenda, but the
discussions sometimes influence the agency’s agenda by suggesting that a new rule is
necessary. While the NRC will travel for meetings if necessary, the agency prefers to
host live webinars for budgetary reasons, and the public can nearly always participate
in its meetings regarding rulemaking online. The agency has a policy of making the
agenda and any related materials publicly available several days in advance of its
meetings.
Similarly, the Forest Service conducted numerous public meetings and listening
sessions as part of the process of developing its 2012 Planning Rule. Although these
meetings were open to the public, the agency also conducted targeted outreach to
important stakeholder communities, including (1) users of the forests for recreational
and economic purposes, (2) Native American tribal communities, (3) state and local
government officials, and (4) the scientific community. The Forest Service hired the
U.S. Institute for Environmental Conflict Resolution (IECR) to design and facilitate
the public engagement in connection with developing the Planning Rule. The Forest
Service personnel were “there to listen.” The public engagement efforts related to the
Planning Rule also included a science forum, four national roundtables and thirtythree regional roundtables, national and regional public forums, national and regional
tribal roundtables, tribal consultation meetings, Forest Service employee feedback,
and comments posted to the Planning Rule blog. The agency considered all feedback
received through these efforts, and used public input, science, and agency expertise to
develop the 2012 Planning Rule. 240
While these examples do not involve the use of listening sessions as a means of
agenda setting per se, listening sessions could easily be an effective tool for this
purpose. Their primary challenges or limitations include securing attendance from a
broad range of interested stakeholders, obtaining balanced participation at the
meetings, and obtaining sufficiently detailed or focused advice on potential regulatory
actions. Nonetheless, similar to focus groups, listening sessions could be an ideal
setting “to find out what issues are of most concern for a community or group when
little or no information is available,” 241 and thus to help the agency establish its
rulemaking priorities. Moreover, much like RFIs, listening sessions could routinely
be used whenever agencies are open-minded about the appropriate course of action
and seek situated knowledge from absent stakeholders or information about the
values and priorities of interested members of the public. In conducting listening
sessions regarding agenda setting, agencies should remain neutral about whether or
how the identified issues or problems will or should be addressed and pose detailed
questions aimed at soliciting the targeted information or data. Agencies should
consider conducting listening sessions in multiple locations and use available

240

Id.
EPA, PUBLIC PARTICIPATION TOOLKIT, supra note 221, at 58 (discussing focus
groups).
241

53

technology to facilitate remote attendance. Agencies should also engage in robust
outreach to bring listening sessions to the attention of missing stakeholders,
unaffiliated experts, and other interested parties. Agencies should also strongly
consider using a facilitator to plan its listening sessions and preside over the meetings
to ensure that targeted stakeholders are included and the discussion is balanced and
productive. Perhaps most important, agencies should recognize that the primary
purpose of listening sessions is to learn from the participants rather than to explain or
justify their decisions. Accordingly, agencies should use listening sessions to “listen”
rather than as another opportunity to speak.
One particularly ambitious listening session to inform a regulatory agenda was
Global Pulse 2010, which was “organized by the U.S. Agency for International
Development (USAID) with the assistance of the Commerce, Education, Health and
Human Services, and State departments.” USAID described Global Pulse as “a threeday virtual event aimed at bringing together thousands of people from around the
globe to discuss the world’s most pressing challenges and envision solutions.” 242
USAID organized this event in partnership with a broad range of stakeholders “to
enable listening, learning and sharing of ideas” regarding “10 pressing global
challenges within the fields of science and technology, economic opportunity, and
human development.” 243 Global Pulse was specifically intended to reach “individuals
who are not normally seated at the table with key decision makers,” and to take
advantage of technological advances that allow agency officials “to engage in
dialogue with individuals and communities from around the globe.” 244 The event
featured “international leaders from government, private industry, and civil society
organizations, along with influential individuals,” who helped “guide the live
conversations, encourage participation, provoke deeper thinking, and offer insight
into the topics at hand.” 245 The Jams technology used to conduct this event was able
to support thousands of participants from around the world, and directed them to their
preferred topics of conversation. 246
F.

Reverse Industry Days

An innovative way for federal agencies to engage with the public that has been
used in the procurement process is known as “reverse industry days” (“RIDs”).
Industry days have long been used in procurement to provide government contractors

242

Press Release, U.S. Agency for International Development, Global Pulse 2010:
Innovative Ideas Wanted: U.S. Government to Host an Online Conversation: ‘Global Pulse
2010’ (Mar. 15, 2010) (on file with authors).
243
Id.
244
Id.
245
Id.
246
See id. This technology can also be used to conduct textual analysis and data mining
to identify emerging trends and otherwise analyze the dialogue. See IBM & USAID,
GLOBAL PULSE 2010, MARCH 29-31, 2010 4 (2010).

54

with information from federal agencies about an impending acquisition. 247 The focus
is generally on the details of a particular acquisition, and on providing industry with
an improved understanding of the government’s needs and potentially allowing the
government to improve or refine the requirement and solicitation. 248 “Reverse
industry days,” in contrast, invite industry representatives “to tell the government”
what they need “to be active, responsive, and effective participants in the
government’s acquisition process.” 249 Specified industry members are carefully
selected to organize and run a “training session” for agency employees that focuses
on the state of the industry and its business practices, how the government’s
acquisition practices influence industry’s participation and performance in federal
acquisitions, and unintended or unappreciated barriers or obstacles that hinder or limit
industry participation. In reverse industry days, “the government is there to listen and
learn while industry presents its knowledge and perspective.” 250 Thus, unlike
listening sessions or focus groups, regulatory stakeholders set the agenda during
RIDs.
Several federal agencies have recently held RIDs to inform their acquisitions
practices. For example, the Department of Homeland Security’s “Acquisition
Innovations in Motion” project seeks to facilitate efforts for government and industry
to learn from one another. 251 This project has featured several RIDS, which included
large group presentations from industry representatives and contractors, smaller
breakout sessions, and roundtable discussions. These RIDs have provided DHS
acquisitions personnel with an opportunity to obtain a better understanding of the
industry’s perspective on a variety of issues relevant to the procurement process.
DHS subsequently made the materials from these RIDs available on the agency’s
website for the benefit of other acquisitions professionals. RIDs also provide agency
officials with industry contacts that could be tapped during future efforts to evaluate
or improve the acquisitions process or other aspects of the agency’s regulatory
activities. Other agencies that have recently held RIDs in the procurement context
include the IRS and the General Services Administration (GSA). 252 GSA’s RIDs
have focused in part on the bidding process and on how industry interprets relevant
legal requirements. During these events, GSA officials told us they try “to look at
issues from the perspective of industry,” and identify information they “didn’t realize
was a concern.” One interviewee told us that RIDs are capable of generating “aha
moments” for agency officials. More than 400 people attended GSA’s RIDs, and the
feedback provided by attendees in subsequent surveys was reportedly “tremendous.”

247

See VIRTUAL ACQUISITION OFFICE RESEARCH INSTITUTE, REVERSE INDUSTRY DAY:
BETTER ACQUISITION OUTCOMES FROM A CHANGE IN PERSPECTIVE 1 (2017).
248
See id.
249
Id. at 1-2.
250
Id. at 2.
251
See id. at 3.
252
See id. at 4.

55

While RIDs are perhaps best suited for the procurement context (where agencies
and contractors are in a business relationship), this format could also potentially be
used as a tool to facilitate public engagement in agency agenda setting (and, as
discussed later, as a tool for conducting retrospective review). The primary benefits
of RIDs are to give agency officials an opportunity to hear about the agency’s
practices from the perspective of the regulated community and to build stronger
relationships with the participants in these events. RIDs can also bring unintended or
unappreciated problems to an agency’s attention, and suggest potential reforms that
could improve the agency’s processes and outcomes. These benefits suggest that
RIDs may be a useful tool for facilitating public engagement about how agencies
could improve their public engagement efforts. Moreover, these are the kinds of
benefits that could potentially improve and usefully inform agency agenda setting in
appropriate circumstances.
The best practices for hosting RIDs, particularly for purposes of improving an
agency’s public engagement efforts or informing its agenda setting, would include
(1) securing a broad stakeholder perspective in organizing the event; (2) broadly
publicizing RIDs to secure adequate and balanced participation; (3) prioritizing
agenda items; (4) carefully considering the best methods for obtaining the targeted
information; (5) coordinating and cooperating with relevant stakeholders in
organizing and hosting the event; (6) devoting adequate resources to planning and
hosting the event; and (7) establishing a “charter” or blueprint for planning and
executing the event, which includes “personnel designations for leads, points of
contact, reporting, various required approvals, responsibility for related
documentation before and after the RID, and ‘rules of the road’ for interacting with
participants,” as well as any necessary legal advice. 253
As these best practices suggest, the most significant challenge that would be
posed by using RIDs to inform an agency’s public engagement efforts or agendasetting priorities is to ensure adequate and balanced participation from the entire
range of the agency’s relevant stakeholders. The “industry” targeted by RIDs in the
procurement context is a relatively narrow group of government contractors and
perhaps associated representatives from the regulated industry. The extension of
RIDs to the public engagement or agenda setting contexts would significantly
broaden the range of potentially interested parties in a way that would almost
certainly pose challenges for the organization and execution of such events. In
addition, to avoid regulatory capture, it is essential for agencies to include not only
representatives of regulated industries in these events, but also representatives of
regulatory beneficiaries, absent stakeholders, and unaffiliated experts or ordinary
citizens in appropriate circumstances. Like in the contexts of focus groups and
listening sessions, agencies should consider organizing and hosting RIDs that
specifically target each of the foregoing groups, or hosting fewer events that are
organized and attended by a sufficiently balanced group of participants.

253

Id. at 7-9 (proposing a number of best practices for RIDs).

56

G.

Hotlines or Suggestion Boxes

While the public could influence an agency’s agenda directly by filing a petition
for rulemaking, hotlines or suggestion boxes provide a less formal mechanism for
agencies to solicit information, ideas, or experiences from the public relevant to their
rulemaking agendas. Therefore, these tools may be more open and accessible to
traditionally absent stakeholders than petitions for rulemaking. Hotlines or
suggestion boxes include widely advertised telephone numbers or web-based forms of
communication that are established to allow interested persons to ask questions or
comment on an issue. They can be set up to allow the caller or user to communicate
in “real time” with an agency employee, or to allow the caller or user to leave a
message for a subsequent agency response. They can be used specifically to solicit
suggestions for problems that should be added to an agency’s agenda, or they can be
established to obtain more general questions or comments regarding the agency’s
statutory mission or operations. The key to an effective hotline is providing callers or
users with an appropriate response to their questions or comments. 254 This means
that agencies must provide well-trained staff to answer calls or respond to messages
in a reasonably prompt fashion, and that callers or users should feel that agency
officials are interested in their concerns and that their replies are well-informed and
responsive. Agency staff will, of course, sometimes need to conduct research to
provide an adequate response.
Hotlines or suggestion boxes have the potential to bring previously unknown or
underappreciated problems or concerns to an agency’s attention. They can also
provide agencies with some indication of whether a particular problem or concern is
widespread or at least warrants greater scrutiny. Hotlines or suggestion boxes also
have the advantage of providing agencies with a relatively simple mechanism for
being (and appearing to be) accessible and responsive to the general public. Yet
hotlines of this nature can backfire if they are not well run. If users do not receive a
response from the agency or are unsatisfied with an agency’s responses, they may
undermine the agency’s perceived legitimacy and discourage subsequent possibilities
for engagement. Running a hotline well therefore requires adequate staffing and
resources. Moreover, some questions or comments will necessarily be difficult to
answer in an entirely satisfactory fashion. And, some callers may use hotlines or
suggestion boxes in a problematic fashion or merely as an opportunity to vent.
Accordingly, agencies should carefully consider whether the potential benefits of
establishing and operating hotlines or suggestions boxes exceed the likely costs.
One example of a web-based hotline operated by a federal agency is the “Tell
Your Story” feature managed by CFPB’s Office of Consumer Education and
Engagement. The CFPB’s website invites consumers to tell the agency about their

254

See CREIGHTON, supra note 220, at 119; EPA, PUBLIC PARTICIPATION TOOLKIT,
supra note 221, at 63-64; IAP2, PUBLIC PARTICIPATION TOOLBOX 2 (2006).

57

“experiences with money and financial services, good and bad.” 255 In addition, the
agency promises that “[t]he CFBP is listening.” 256 The website provides basic
information about the agency, explains how “telling your story works,” gives
examples of stories shared by other consumers, and notes that CFPB uses this
information to identify “trends and work to head off problems,” partly through its
enforcement actions. 257 CFPB’s website also includes a feature called “Ask CFPB,”
which provides consumers with the opportunity to obtain “answers to
frequently-asked financial questions about student loans, credit cards, mortgages,
credit scores and reporting, getting out of debt and more.” 258
A popular “suggestion box” operated by the federal government was the “We the
People” petition process provided by the White House during the Obama
Administration. 259 The White House established this website in 2012 to solicit online
petitions from members of the public for recommended governmental action, and to
allow visitors to sign those petitions indicating their support. While the White House
initially promised to respond to any petition that received 5,000 signatures, the
popularity of the website prompted the Administration to raise the signature threshold
for a response to 100,000 signatures in a 30-day period. Beth Simone Noveck points
out that petitions of this nature “can get a topic on the public agenda by opening a
channel of communication other than lobbying or appealing to congressional
officials.” 260 For example, a petition that was designed to force telephone companies
to unlock a consumer’s cell phone for use on a competitor’s network when their
service contract expired generated 114,000 signatures and prompted substantial
attention among media and political elites. 261 Noveck also points out, however, that
most petitions fell short of the signature threshold that triggered a response, interest in
the website precipitously declined in a relatively short period of time, and there is no

255

Your Money. Your Story., CONSUMER FIN. PROTECTION BUREAU,
https://www.consumerfinance.gov/your-story/ (last visited Nov. 8, 2018).
256
Id.
257
Id. ACUS has, however, recommended that agencies that make consumer complaints
publicly available through online databases or downloadable data sets inform the public of
the source and limitations of the information and permit entities publicly identified to respond
or request corrections or retractions. See Admin. Conf. of the U.S., Recommendation 2016-1,
Consumer Complaint Databases, 81 Fed. Reg. 40,259, 40,259-60 (June 21, 2016).
258
You Have the Right to Have Your Financial Complaint Heard, CONSUMER FIN.
PROTECTION
BUREAU,
https://www.consumerfinance.gov/you-have-the-right/submit-acomplaint/ (last visited Aug. 14, 2018).
259
For discussions of “We the People,” see BETH SIMONE NOVECK, SMART CITIZENS,
SMARTER STATE: THE TECHNOLOGIES OF EXPERTISE AND THE FUTURE OF GOVERNING 7576 (2015) [hereinafter NOVECK, SMART CITIZENS]; Jonathan Weinberg, The Right to be
Taken Seriously, 67 U. MIAMI L. REV. 149, 212-14 (2012).
260
NOVECK, SMART CITIZENS, supra note 259, at 76.
261
See id. at 75-76.

58

evidence that “We the People” had any meaningful impact on policymaking. 262 Part
of the problem was with petitions that could not be taken seriously (e.g., deport Justin
Bieber back to Canada, and build a “Death Star”) or petitions that were beyond the
scope of the President’s authority. Part of the problem, however, may also have been
the unsatisfying nature of some White House responses. 263 In this regard, one
submission that received 37,167 signatures petitioned the Obama Administration to
“[a]ctually take these petitions seriously instead of just using them as an excuse to
pretend you are listening.” 264
The preceding examples illustrate both the advantages and limitations of hotlines
or suggestion boxes and suggest some best practices related to their use. First,
agencies should have realistic expectations about the utility of these methods of
public engagement for agenda setting purposes. Hotlines and suggestion boxes can
bring new ideas or problems to an agency’s attention and perhaps highlight the
significance of problems with which the agency is already familiar. These tools also
give agencies a concrete opportunity to demonstrate their responsiveness to the
interests and concerns of the public. Nonetheless, hotlines and suggestions boxes will
not typically provide ready solutions to the problems users identify, and they should
be viewed simply as a potentially useful supplement to the other modes of public
engagement in agenda setting. Moreover, agencies that provide hotlines or
suggestion boxes should widely advertise their availability to their targeted audiences,
including missing stakeholders and unaffiliated experts, and devote sufficient
resources toward their operation to ensure that the agency’s responses are timely,
accurate, and otherwise appropriate.
Agencies should also consider publicly recognizing the best ideas they receive
through hotlines and suggestion boxes, allowing commentators to “endorse”
suggestions that are submitted by other users, and providing more detailed substantive
responses to serious or major proposals when possible. Agencies should report when
they have acted on suggestions they have received via their hotlines or suggestion
boxes. If hotlines or suggestion boxes are used for general purposes beyond agenda
setting or other matters related to rulemaking, agencies should establish mechanisms
to ensure that their rulemakers have access to the relevant database or reports that
summarize the nature of the comments received. Agencies should also consider
providing users of these tools with information about other agency activities that may
be of interest to them, including upcoming opportunities to engage with the agency’s
rulemaking efforts.

262

See id. at 76. See also Paul Hiltin, ‘We the People’: Five Years of Online Petitions,
PEW RES. CTR. (Dec. 28, 2016), http://www.pewinternet.org/2016/12/28/we-the-people-fiveyears-of-online-petitions/ (calling the impact of the petitions on policy “modest and varied”).
263
See Weinberg, supra note 259, at 213 (criticizing the nature of the White House’s
responses, suggesting that this outcome is unsurprising, and claiming that “it makes the whole
enterprise seem pointless”).
264
Id. at 213.

59

H.

Public Complaints

Public complaints are another form of civic engagement that can be used to
inform an agency’s rulemaking agenda. Public complaints are in a sense a more
specific version of hotlines and suggestion boxes. While the public can submit a
range of proposals using suggestion boxes, including informal requests for
rulemaking, complaints are typically focused on regulatory violations and agency
enforcement. Complaints sometimes seek a specific enforcement action, but they can
also provide a broader mechanism for the public to express concerns and for the
agency to assess the frequency or magnitude of various problems, which could
potentially inform the agency’s agenda setting and provide an impetus for rule
development. Much like hotlines or suggestion boxes, agencies can solicit or accept
complaints via telephone or by establishing mechanisms for filing complaints via the
Internet. Public complaints have many of the same advantages and limitations as
hotlines as well—they can bring previously unknown or underappreciated problems
to the agency’s attention and provide a service that demonstrates the agency’s
responsiveness to public concerns. But it is essential that agencies provide the
resources necessary to respond to complaints in a timely and reasoned fashion.
Although agencies will rarely establish complaint processes primarily to inform their
rulemaking agendas, the value of public complaints for agenda setting is a potentially
significant byproduct of those efforts that agencies should use to their advantage. 265
The CFPB’s Office of Consumer Response is responsible for handling the
Bureau’s consumer complaints. This Office has received approximately 1.2 million
consumer complaints to date. Eighty percent of the complaints are submitted via the
Bureau’s website; the Bureau also operates a call center to handle complaints. 266 The
CFPB began collecting consumer complaints about credit card products in 2012 and
gradually expanded the system over the next two years. The Bureau now has a
universal complaint form that asks consumers specific questions about the nature of
their complaints, which are then routed to the appropriate part of the agency for
processing. The Bureau’s data scientists periodically mine the complaints database
using natural language processes. While this database has reportedly been used more
heavily to date for purposes of rule development, it could also be used as a tool to
inform the Bureau’s rulemaking agenda. Other agencies that receive and respond to
consumer complaints could likewise use their complaint databases as a resource to
inform their rulemaking agendas.
The CFPB’s experiences with public complaints offer some best practices for
their use. Agencies that accept public complaints should develop a “complaints

265

As noted above, ACUS has previously recommended that if this information is made
publicly available, agencies “should adopt and publish online written policies” that “[i]nform
the public of the source(s) and limitations of the information, . . . permit entities publicly
identified . . . to respond . . . or request corrections or retractions, . . . and give appropriate
consideration to privacy interests.” Admin. Conf. of the U.S., Recommendation 2016-1,
Consumer Complaint Databases, 81 Fed. Reg. 40,259, 40,259-60 (June 21, 2016).
266
Telephone Interview with Consumer Financial Protection Bureau (Dec. 5, 2017).

60

database” that could be used to identify recurring problems, begin to assess their
magnitude and frequency, and develop processes for using this information to inform
the agency’s rulemaking agenda. For example, data scientists who study or examine
the complaints database should provide periodic reports that summarize their findings
to officials who are responsible for developing the agency’s rulemaking agenda.
I.

Public Notice and Comment

The most widely known and frequently used form of public engagement in
rulemaking is almost certainly notice and the opportunity for public comment.
Although this process is generally required for proposed rules, notice and comment
can also enhance public engagement in agency agenda setting, although with some
limits discussed below. For example, agencies can provide public notice and
opportunities to comment on what should appear on their rulemaking agendas. As
discussed above, the IRS and the PBGC already use RFIs to do this. Such requests
for public comments could be open, in the nature of RFIs, or agencies could provide
notice and opportunities to comment in connection with the Regulatory Plans they
submit for the Unified Regulatory Agenda under E.O. 12,866. Agencies could
publish their Regulatory Plans for public comment in advance of submitting them to
the Office of Management and Budget (OMB) or merely provide a transparent means
for the public to comment on the Plans after they are published in the Unified
Regulatory Agenda. Finally, agencies can provide public notice and opportunities to
comment on some or all of their rulemaking petitions. As indicated above, this last
form of public notice and comment provides two layers of public engagement in
agency agenda setting. Consistent with prior ACUS recommendations, 267 we believe
that agencies should consider providing opportunities for public notice and comment
in each of these three contexts.
To be sure, we are not suggesting that agencies comply with all the requirements
of notice and comment under 4 U.S.C. § 553(c). Because section 553(c) notice and
comment is not required for agenda setting, the agency need not have a fixed window
for receiving comments, respond to all public comments on its agenda, nor respond
with the level of detail and support typical in section 553 rulemaking. 268 Moreover,
judicial review of agency agenda setting is extremely deferential. 269 Rather, agencies
should merely commit to reviewing public comments on their agenda and, to the
extent appropriate and realistic in light of agency resources, providing a concise and
reasoned justification for their decisions based on the petitions and comments

267

See Admin. Conf. of the U.S., Recommendation 2014-6, Petitions for
Rulemaking, 79 Fed. Reg. 75,114, 75,117, 75,118 (“Agencies should consider inviting
public comment on petitions for rulemaking by either: (a) Soliciting public comment on
all petitions for rulemaking; or (b) Deciding, on a case-by-case basis, whether to solicit
public comment on petitions for rulemaking.”); see also SCHWARTZ & REVESZ, supra note
161, at 76-78.
268
See SCHWARTZ & REVESZ, supra note 161, at 77.
269
Id.

61

received. We recognize that agencies would generally not be able to respond to
individual comments in this context unless they are unusually important or
decisive. 270
Following this advice could enhance public engagement in agenda setting, which
is frequently the stage of rulemaking when agencies are most open minded and where
the general public’s values and priorities can and should most readily influence
agency decision-making.
VII. EARLY RULE DEVELOPMENT
There are many ways for agencies to engage with the public when they set their
rulemaking agendas. Of course, once an issue or problem is placed on the agency’s
agenda as a potential subject of rulemaking, the agency’s opportunity to engage with
interested members of the public on the best course of action are only beginning.
While the scholarly literature on public engagement with rulemaking has focused
primarily on ways to improve public notice and comment, the best prospect for more
fully democratizing the rulemaking process may be to provide meaningful and
consistent efforts for informed public engagement during rule development. Many of
the modes of public engagement that are available at the agenda-setting stage can also
be used—sometimes even more effectively—during early rule development, when
agencies study the problem at issue, begin to gather relevant data, identify and
evaluate potential approaches to the regulatory problem, and begin the process of
drafting a proposed rule. This section identifies available modes of public
engagement during early rule development, provides examples of their effective use
by federal agencies, discusses the challenges and limitations of each of those
methods, and recommends best practices for facilitating meaningful, balanced, and
informed public engagement during this vital and formational stage of the rulemaking
process.
A.

Federal Advisory Committees

Federal advisory committees can be an effective tool for public engagement early
in the process of developing most rules subject to notice and comment. As in the
context of agenda setting, agencies would need to consider whether to charter a new
advisory committee or to use existing advisory committees for this purpose. This
would depend primarily on whether existing advisory committees are structured
adequately to represent all of the relevant stakeholders with an interest in the matter
and whether they have the necessary forms of expertise. If not, agencies should
consider chartering new advisory committees to provide advice on particular
rulemaking initiatives, especially if a potential rule would be especially significant or

270

For a discussion of analogous ways in which voluntarily conducting public notice and
comment on guidance documents is less burdensome than for legislative rules, see NICHOLAS
R. PARRILLO, ADMIN. CONFERENCE OF THE U.S., FEDERAL AGENCY GUIDANCE: AN
INSTITUTIONAL PERSPECTIVE 143-50 (2017).

62

controversial. Agencies should also consider whether seeking advice from a Citizens
Advisory Committee would make sense under the circumstances. This is most likely
to be the case, once again, for rulemaking initiatives that are especially important or
controversial.
The primary goal of using federal advisory committees at this stage is to obtain
advice from a balanced group of well-informed stakeholders and unaffiliated experts
on potential approaches to a proposed rule. During the earliest stages of the process,
advisory committees could help to identify the relevant issues that need to be
considered, provide advice on how to obtain the requisite data, make suggestions or
provide preliminary evaluations of potential approaches to the regulatory problem,
and recommend potential approaches to begin drafting a proposed rule. As the
agency’s thinking progresses, advisory committees could be used to evaluate the
agency’s preliminary work, identify additional problems that still need to be resolved,
and provide feedback on the agency’s tentatively preferred solutions (including early
drafts of proposed rules). In short, federal advisory committees can be used as a
relatively simple and cost-effective way to “involve” representatives of the public in
rule development, and even as a potential mechanism for “collaboration,” whereby
federal agencies “partner with the public in each aspect of the decision including the
development of alternatives and the identification of the preferred solution.” 271
Moreover, the briefing materials and other documents prepared for advisory
committees, and the advice that is provided from advisory committees to federal
agencies, can be shared with the public and provide a basis for relatively informed
public comments from other interested parties early in a rule’s development.
We have found several examples of agencies that use federal advisory
committees for advice relatively early in rule development. The Department of
Energy has a “process rule” that sets forth its procedures for promulgating rules.272
The agency routinely consults with its advisory committees as part of this process
before issuing an NPRM. The FCC also has advisory committees through which the
public “is afforded an opportunity to provide input[]into a process that may form the
basis for government decisions.” 273 The FCC currently receives advice from eight
committees subject to FACA and three committees not subject to FACA. 274 The
NRC reportedly uses advisory committees to inform the development of its proposed
rules, and the agency has found that the briefing materials that it prepares for those
committees are also useful for gathering feedback from the general public. Similarly,
the CFPB regularly uses Small Business Enforcement Fairness Act (“SBREFA”)
panels to review proposed rules before they are noticed. The panels help the agency
identify unintended consequences and provide information about how industry does

271

See LUKENSMEYER & TORRES, supra note 141, at 7 (describing IAP2’s “spectrum of
public participation”).
272
See infra Part XII.A (providing more detailed discussion of DOE’s process rule).
273
Advisory Committees of the FCC, FED. COMMS. COMMISSION,
https://www.fcc.gov/about-fcc/advisory-committees-fcc (last visited May 2, 2018).
274
See id.

63

business, compliance matters, appropriate regulatory thresholds, how best to
communicate the Bureau’s regulatory requirements, and trade-offs between potential
costs and benefits. The CFPB also publishes SBREFA briefing materials on its
website to solicit broader public input prior to the issuance of a notice of proposed
rulemaking.
As discussed above, the primary challenges or limitations associated with
advisory committees are that using them requires significant time and effort (as
compared with not using them), their use is relatively heavily regulated by law, their
composition may not be sufficiently balanced or representative, and they must be
properly briefed on the relevant legal and technical issues and the agency’s tentative
thinking about the problem. Nonetheless, when federal advisory committees are
properly composed and sufficiently informed, seeking their feedback and advice early
in rule development will routinely be worth the time and effort. The best practices
associated with using advisory committees at this stage therefore include
(1) identifying all of the relevant stakeholders and making sure that their interests and
perspectives are adequately represented; (2) carefully considering whether to use
existing advisory committees or to charter a new advisory committee to obtain advice
or recommendations on the development of a potential ruleor perhaps whether it
would be best to seek advice on the matter from a combination of different advisory
committees; (3) regularizing the solicitation of advice from advisory committees early
in the process of developing potential rules, as opposed to using advisory committees
for these purposes in an inconsistent or ad hoc manner; (4) preparing briefing
materials for advisory committees that clearly explain the relevant issues and
potential alternatives and the agency’s tentative plans (if any); and (5) publishing
those materials and any subsequent advice or recommendations from federal advisory
committees on the agency’s website and in other appropriate venues as a basis for
seeking additional public comments.
B.

Focus Groups

Focus groups can also be used as a form of public engagement during the early
stages of rule development. Because they are a useful mechanism for identifying the
primary concerns of participants regarding a problem or their initial reactions to a
proposal, focus groups are generally likely to be more useful during the early stages
of rulemaking than when a proposal is more fully developed. Similar to the agendasetting context, focus groups could be a relatively easy and inexpensive way for
agencies to obtain useful feedback on potential regulatory alternatives from missing
stakeholders, unaffiliated experts, and ordinary citizens who tend not to participate in
notice-and-comment proceedings or open public meetings. 275 Agencies should give
careful consideration to whom to invite to participate in focus groups based on the

275

See EPA, PUBLIC PARTICIPATION TOOLKIT, supra note 221, at 74 (recognizing that
focus groups can be used to obtain feedback from “people with common concerns who may
not feel confident speaking up in a larger public gathering,” and explaining that “findings
from smaller meetings can be presented at larger public meetings or in summary reports,
giving ‘voice’ to those in the community who are unable to speak up in a larger setting”).

64

nature of the rule at issue and the type of feedback they are seeking. In any event, it
is crucial that agencies provide skilled facilitation and conduct careful planning to
maximize the likelihood of getting the most productive input possible. Agencies
should also provide participants with briefing materials that clearly explain the
relevant issues and the primary policy alternatives, and they should prepare a report
after the session that summarizes the feedback and identifies potential concerns for
further consideration.
C.

Requests for Information (“RFIs”)

RFIs are another tool of public engagement that can be used effectively during
the early stages of developing many rules. As explained above, RFIs are published
requests for data, comments, or other information from the public on designated
problems or issues. The goal of using RFIs at this stage is to obtain data and
feedback on potential approaches to a regulatory problem and primary areas of
concern early in the rulemaking process before any firm decisions have been made.
Responses to RFIs can help agencies to determine “an appropriate course of action—
which may or may not involve rulemaking—to address the problem.” 276 Some
agencies, including CFPB and the Department of Energy, issue RFIs during the early
stages of rule development on a regular basis.
The best practices for using RFIs during agenda setting would also apply to their
use early in rule development. In particular, agencies should use RFIs as a matter of
course when they are open to a variety of potential courses of action or need
additional information or data before they can begin crafting a regulatory approach to
an issue. When publishing an RFI, agencies should (1) remain neutral regarding
potential regulatory alternatives; (2) pose specific questions or requests to obtain the
information they need; (3) identify their intended audiences; (4) conduct focused
outreach to bring RFIs to the attention of missing stakeholders, unaffiliated experts,
and other interested parties, when appropriate, and consider whether there are
additional ways to incentivize robust participation; and (5) acknowledge responses to
RFIs, seriously consider the resulting feedback, and explain how the submitted
information, data, or comments were useful to the agency.
D.

Listening Sessions

Listening sessions are another potentially valuable tool that can be used to
facilitate public engagement during the early stages of rule development. Listening
sessions provide “live” opportunities for agencies to obtain information, comments,
or data from the public on a designated problem or issue. They can be conducted in
person or online (or both), and their primary goal is to provide information about the
agency’s tentative goals or plans and give the public an opportunity to ask questions
and provide comments. Listening sessions provide a potentially useful way for
agencies to obtain situated knowledge from absent stakeholders and concrete data or
other information from unaffiliated experts. They also provide agencies with an

276

See Requests for Information (RFI ), supra note 235.

65

opportunity to hear about participants’ primary concerns and to receive feedback on
the perceived advantages or disadvantages of various policy alternatives. Similar to
RFIs, this information and data can help an agency determine an appropriate course
of action—which may or may not involve rulemaking—to address the problem.
As discussed above, the Forest Service made extensive use of listening sessions
and other public meetings during the early stages of developing its 2012 Planning
Rule. 277 The NRC is another agency that has made relatively extensive use of
listening sessions. Indeed, the NRC has a group of employees who are trained as
facilitators who can be called upon to help organize and run these meetings. The
meetings normally occur early in the process of creating a rule, and they focus on a
designated issue upon which the agency “wants to get input about what people think.”
An interviewee told us that “these meetings are fairly open, and the NRC will
occasionally have panel discussions.” The NRC’s listening sessions usually last half
a day or a full day. To identify specific topics to discuss at such meetings, the NRC
staff will “brainstorm” or the agency will reach out to the public and interest groups
to suggest topics. In addition, the NRC will include language in its notice of the
meeting asking what issues and topics participants would like to discuss, and if
enough people suggest a topic, the NRC will place it on the agenda for the meeting.
The public can generally attend the meeting in person or they can watch it online.
Even if they are participating online, the public is still invited to ask questions.
The NRC tries to “educate the public” about the relevant topic during these
listening sessions. This may be an advantage of listening sessions over RFIs when
the agency wants to get input from an informed public or engage in a more
deliberative exchange. We were told that “the public is appreciative of these efforts.”
After an informative introduction, the agency will typically open up the meeting to a
question and answer session. The meeting is generally led by someone from the
relevant program office, and the facilitator is there primarily to “help out” and to
serve as “an internal consultant.” Before the meeting, the facilitator will also help
with logistics. During the meeting, the facilitator is there to advise, ask questions,
and make sure that everyone has a chance to participate. The facilitator also helps
explain “technical issues and complex language” in a manner that the public can
understand. This can prove challenging if the facilitator is not sufficiently familiar
with the topic or with what the agency’s experts are trying to say. The NRC’s
facilitators have also helped to conduct listening sessions online on several occasions.
These meetings are similar to a “radio show” in that “there is a phone bridge set up
and an operator controls the phone line.” For online meetings, the facilitator will help
with a presentation and then work with the operator regarding questions. One
advantage of online listening sessions is that everyone is on relatively equal footing,
unlike live meetings where people who are physically present often have an easier
time participating than people who are attending remotely.

277

See supra Part VI.E.

66

EPA is another agency that has used listening sessions effectively early in rule
development. For example, the agency conducted five listening sessions in large
cities around the country “as a first step in developing a new rule that would require
states, territories, and authorized tribes to develop Total Maximum Daily Loads
(TMDL) of pollutants that a body of water can receive while still meeting water
quality standards.” 278 These listening sessions were designed “to obtain stakeholder
perspectives on key issues associated with the TMDL program and related issues in
the National Pollutant Discharge Elimination System (DPDES) program.” 279 EPA
hosted the listening sessions, which included participation from other federal, state,
and local agencies, and representatives of environmental, agriculture, forestry, and
industry groups. The listening sessions were facilitated by a hired contractor, and
included presentations from EPA management, as well as designated “listening
panels” composed of EPA officials, officials from USDA and state agencies, “and
industry and environmental stakeholders who listened to the attendees’ perspectives
and shared their own perspectives as well.” 280 The listening sessions included a large
group presentation and background discussion of the general topic, numerous
facilitated small group discussions, and a plenary session with presentations from
spokespersons for each of the smaller groups. At the end of the meetings, the
listening panels “responded to what they had heard” and “both presented their
viewpoints and listened and reacted to the discussions of the participants.” 281
Comments from all of the participants were included in meeting summaries, which
were posted on EPA’s website and considered by the agency’s rule-writing team as it
determined how to proceed. While EPA ultimately withdrew its proposed rule on the
grounds that significant changes were needed, the agency issued related guidance
“based in part on information gathered from the listening sessions.” 282 EPA
subsequently reported that “[t]his case study illustrates that listening sessions can
provide an effective forum for agencies to encourage proactive and constructive
engagement early in the policy development process.” 283 Moreover, “input obtained
from listening sessions can be used to influence EPA guidance, even if the
rulemaking that occasioned the information exchange is cancelled or the proposed
rule withdrawn.” 284

278

DEBORAH DALTON & PHILLIP J. HARTER, CONFLICT PREVENTION & RESOLUTION
CTR., BETTER DECISIONS THROUGH CONSULTATION AND COLLABORATION app. V at 115
[hereinafter BETTER DECISIONS MANUAL].
279
Id.
280
Id. at app. V at 116.
281
Id.
282
Id. at app. V at 115.
283
Id.
284
Id.

67

Listening sessions tend to be most useful if public education and input are both
desired and possible. 285 The primary challenges associated with listening sessions
include the potential difficulty of securing balanced attendance and participation.
Moreover, the discussions could become “heated” if the issues are controversial, and
listening sessions are not necessarily the best format for facilitating “interactive
dialogue”—unless, as in the EPA example, moderated small group discussions are
incorporated into the event. Accordingly, the effective use of listening sessions in
early rule development will necessarily require careful planning and extensive
outreach efforts. Agencies should generally use trained facilitators to plan and help
conduct these events. They should also consider holding listening sessions in
multiple locations and use available technology to facilitate remote participation. As
discussed in the agenda-setting context, agencies should remain neutral about the best
approach to a regulatory problem when they hold such meetings, and they should
pose detailed questions that are designed to produce the targeted information or data.
Agencies should also consider establishing “listening panels” composed of agency
officials to respond to the views and perspectives that are conveyed at the meetings.
And, they should provide summaries of the results of their listening sessions, and
explain how this feedback subsequently influenced the agency’s decision making.
E.

Internet and Web-Based Outreach

The Internet is constantly evolving to allow greater possibilities for usergenerated content, and thus, to provide increased opportunities for public
participation and dialogue in agency rulemaking. Social media, which includes “any
online tool that facilitates two-way communication, collaboration, interaction, or
sharing between agencies and the public,” 286 can be carefully used in some situations
to enhance public engagement in connection with notice-and-comment rulemaking. 287
As ACUS has previously recognized, however, the potential for using social media to
facilitate public engagement in rulemaking is significantly greater during the early
stages of rule development because “the APA and other legal restrictions do not
apply, and agencies are often seeking dispersed knowledge or answers to more openended questions that lend themselves to productive discussion through social
media.” 288 The Conference has therefore recently issued recommendations to
facilitate the use of social media for purposes of public outreach and education, and
also as a potentially useful mechanism for soliciting situated knowledge and other
helpful information during the early stages of rule development from missing
stakeholders, unaffiliated experts, and other citizens who do not traditionally

285

For a more thorough discussion of the challenges associated with public meetings,
and a detailed list of their potential advantages and the best practices for conducting them, see
EPA, PUBLIC PARTICIPATION TOOLKIT, supra note 221, at 74-75.
286
Admin. Conf. of the U.S., Recommendation 2013-5, Social Media in Rulemaking, 78
Fed. Reg. 76,269, 76269, 76,269 (Dec. 17, 2013).
287
See infra Part IX.B.
288
Social Media in Rulemaking, 78 Fed. Reg. at 76,270.

68

participate in notice-and-comment proceedings. 289 This section briefly highlights the
most pertinent ACUS recommendations, provides a few examples of agencies using
social media effectively to facilitate public engagement in early rule development,
and identifies some of the primary challenges and best practices in this area. 290
1.

Prior ACUS Recommendations Concerning the Internet and Social Media

The primary goals of using the Internet and social media for outreach in early
rule development are: (1) to inform the public of an agency’s interest in a potential
rule and opportunities to participate in the process of developing the rule and (2) to
encourage public discussion and input concerning potential regulatory initiatives.
Social media can be particularly effective for purposes of public outreach, “helping to
increase public awareness of agency activities, including opportunities to contribute
to policy setting, rule development, or the evaluation of existing regulatory
regimes.” 291 Accordingly, ACUS has recommended that “[a]gencies should use
social media to inform and educate the public about agency activities, their
rulemaking process in general, and specific rulemakings.” 292 Social media can be a
particularly effective (and likely essential) means of reaching absent stakeholders,
and agencies should therefore “take an expansive approach to alerting potential
participants to upcoming rulemakings by posting to the agency Web site and sending
notifications through multiple social media channels.” 293 ACUS’s more specific
advice concerning the use of social media for public outreach includes the following
suggestions:
•

Develop “communications plans specifically tailored to the rule and to all
types of missing stakeholders or other potential new participants the
agency is trying to engage.”

•

Clearly explain “the mechanisms through which members of the public
can participate in the rulemaking, what the role of public comments is, and
how the agency will take comments into account.”

•

Provide clear and specific information “about how the proposed rule
would affect the targeted participants and what input will be most useful to
the agency.”

•

Ask organized groups “to spread the participation message to members or
followers,” explaining that “individual participation can be beneficial,”

289

See id. at 76,271-72.
Of course, some of the same tools and practices could also be used in appropriate
circumstances to enhance public engagement in agency agenda setting, advanced rule
development, and retrospective review.
291
Id. at 76,270.
292
Id.
293
Id. at 76,270-71.
290

69

and encouraging these groups “to solicit substantive, individualized
comments from their members.” 294
ACUS has also recommended that agencies consider “ways to publicize, and
allow members of the public to receive, regular, automated updates on developments”
related to their significant rulemaking activities. 295 Finally, ACUS has recommended
that agencies should consider using social media during rule development “where the
goal is to understand the current state of affairs, collect dispersed knowledge, or
identify problems.” 296 When using social media for these purposes, agencies should
clearly identify the kinds of information they are seeking and how the agency plans to
use this feedback, and they should acknowledge receipt of submissions, pose followup questions, and provide substantive responses to reward participation and promote
further dialogue with commentators. 297
ACUS has also recommended other ways to use social media effectively in early
rule development. For example, the Conference has suggested that “agencies should
consider maintaining a blog or other appropriate social media site” for each
rulemaking that can provide interested persons with “information, updates, and
clarifications regarding the scope and progress of the rulemaking.” 298 ACUS has also
recommended that agencies consider “using such a site to generate a dialogue.”299
When agencies undertake such efforts, ACUS has recommended that agencies
consider “retaining facilitator services to manage rulemaking discussions conducted
through social media.” 300 Moreover, agencies should provide information and
otherwise communicate in a manner that would be understandable to the general
public. 301 The Conference has emphasized that “[a]gencies have maximum flexibility
under the APA to use social media before an NPRM is issued or after a final rule has
been promulgated,” 302 and it has even recommended that they “consider
experimenting with collaborative drafting platforms, both internally and, potentially,
externally, for purposes of producing regulatory documents.” 303
Such

294

Id. at 72,271.
Id.
296
Id.
297
See id.
298
Id.
299
Id.
300
Id.
301
See Admin Conf. of the U.S., Recommendation 2017-3, Plain Language in
Regulatory Drafting, 82 Fed. Reg. 61,728, 61,730 (Dec. 29, 2017) (“The preambles to
proposed rules should include a summary of the rule that non-specialists and the general
public can understand.”).
302
Social Media in Rulemaking, 78 Fed. Reg. at 76,272.
303
Id. at 76,271.
295

70

experimentation would generally make the most sense early in the rule development
process. 304
2.

Successful Uses of the Internet and Social Media

The Department of Transportation has been a leader in using social media and
the Internet to involve the public in some of the foregoing ways. In addition to its
work with Regulation Room, which is discussed below, 305 DOT has used “National
Online Dialogues” to solicit public input and encourage public discussion concerning
various regulatory initiatives. These dialogues typically describe a general topic or
problem and invite members of the public “to participate and lend [their] ideas.”306
In addition to providing background information, the online dialogues invite
interested persons “to post an idea; review, comment, and vote on others’ input; and
provide specific feedback to [the agency’s] questions.” 307 The public is provided
with a specific time frame in which to submit comments and told, “your voice can
make a difference!” 308 A number of agencies have experimented with the use of the
“Idea Scale” platform for similar purposes. 309
DOT has also been a pioneer in the use of “status reports” and “effects reports,”
which allow interested persons to keep track of the status of rulemaking that could
affect them. Members of the public can sign up to receive “status reports” on
particular rules or on broader topics of interest. Status reports are issued by the
agency each month, and they keep subscribers up to date on the progress of
significant rulemakings. Status reports also allow recipients to sign up for “alerts”
that notify them when proposed rules are open for public comment. Subscribers to
this service do not need to monitor the Federal Register; they can just click on a link
near the bottom of the notice that takes them directly to the NPRM.
“Effects reports” are issued as another way to keep the public informed of DOT’s
rulemaking activities. 310 The agency regularly produces 21 separate reports for each
“effect” of its proposed or potential rules. Members of the public can sign up to
receive a specific effects report when a proposed or potential rule has an “effect” that

304

For informative discussions of “wikis” and other collaborative drafting processes, see
HERZ, USING SOCIAL MEDIA, supra note 28, at 59-62.
305
See infra Part IX.C & F.
306
See, e.g., FTA 5-Year Research Strategic Plan National Online Dialogue, FED.
TRANSIT
ADMIN.,
https://www.transit.dot.gov/research-innovation/fta-5-year-researchstrategic-plan-national-online-dialogue (last updated Aug. 24, 2016).
307
Id.
308
Id.
309
For a discussion and evaluation of these efforts, see HERZ, USING SOCIAL MEDIA,
supra note 28, at 42-46.
310
See Reports on the Effects of Rulemakings, DEP’T TRANSP.,
https://www.transportation.gov/regulations/reports-on-effects-DOT-rulemakings
(last
updated Mar. 16, 2017).

71

is of interest to them. Examples of relevant effects include “Regulatory Flexibility
Analysis,” which may be of interest to small businesses; state, local, or tribal
government effects, which may be of interest to representatives of those sovereign
entities; and “major” or “other significant effects,” which could potentially be of
interest to anyone. Effects reports are provided so that interested members of the
public can stay abreast of the progress of proposed or potential rules that would affect
their particular interests. DOT creates and uses “listserves” to keep subscribers of
these services notified by email of significant actions or events regarding specific
rules or more general topics. The creation and use of listserves is a relatively easy
way for agencies to conduct web-based outreach that has the potential to enhance
public engagement in rulemaking significantly.
The FCC has also made relatively extensive use of web-based outreach and
public engagement during early rule development. For example, in developing its
National Broadband Plan, the FCC established a blog called “Blogband” for the
public to discuss broadband policy, and conducted workshops and field hearings that
were “lively, interactive and valuable for the staff tasked with collecting data and
forming recommendations.” 311 The public engagement effort related to the National
Broadband Plan was unusually extensive, and included the issuance of a “notice of
inquiry” (sometimes called a “request for information”), and 36 public workshops or
“listening sessions” that were held at FCC headquarters and streamed online—which
drew more than 10,000 in-person or online attendees and reportedly provided the
framework for the ideas contained in the Plan. Significantly for present purposes, the
FCC also used the Internet to engage the public in this effort. The agency posted
more than 130 blog entries on a website created specifically for this project and
received nearly 1,500 comments in return. 312 Based partly on efforts related to this
project, the FCC’s Twitter feed reportedly has more than 330,000 followers, “making
it the third most popular governmental Twitter feed after the White House and the
Centers for Disease Control.” 313
3.

Challenges of Using the Internet and Social Media

While social media and other technological innovations have the greatest
potential to facilitate public engagement during the early stages of rule development,
such efforts undoubtedly pose substantial challenges. Michael Herz identifies and
discusses those challenges at length in his ACUS report on the use of social media in

311

Jennifer A. Manner & Ronnie S. Cho, Broadband in America: Introduction to a New
Federal Priority, 19 MEDIA L. & POL’Y 5, 9-10 (2010).
312
In discussing this effort, however, Michael Herz concluded that “[d]espite some
fanfare, comments were neither especially numerous nor especially substantive.” HERZ,
USING SOCIAL MEDIA, supra note 28, at 41.
313
FED. COMMC’N COMM’N, CONNECTING AMERICA: THE NATIONAL BROADBAND
PLAN IX (2010).

72

rulemaking. 314 The bottom line is that the general public is unaware of most agency
rulemaking activity and opportunities to participate in the process, they generally
have other things they would rather do with their time, they do not know how to
comment effectively, and, even if all of the foregoing barriers could be overcome,
they may not have anything useful to add. 315 These challenges are potentially
compounded by the fact that organized groups and trade associations are sometimes
reluctant to encourage their members to participate as individuals with independent
interests or perspectives. 316
Though these are undoubtedly daunting challenges, we are confident that social
media and other web-based outreach can be thoughtfully designed “to increase
participation by certain small, targeted groups or to improve the quality of the
participation that is already occurring,” 317 especially during early rule development.
4. Best Practices for Using the Internet and Social Media to Engage Absent
Stakeholders
Social media and other technological innovations can be used for several distinct
purposes, which can and should be disaggregated. The first purpose is simply to
inform targeted stakeholders about an agency’s activities and to notify them of related
opportunities to participate. The related participatory opportunities could involve any
of the “modes” of public engagement discussed in this report. Social media and
listserves (or other subscription services) can be used relatively easily and effectively
for these informational purposes. But agencies must identify their targeted audiences
and engage in special recruitment efforts to ensure that they are “signed up” to
receive the relevant information.
The second purpose that can be served by social media and other technological
innovations is to help educate the public regarding how to participate effectively.
This means providing the agency’s targeted audiences with the procedural and
substantive information they need to participate effectively. Specifically, agencies
need to provide relevant members of the public with information about (1) how a
particular mode of public engagement works, (2) their potential regulatory plans, and
(3) precisely what information or data they seek. And, of course, all of this
educational material needs to be presented in a manner that the targeted audience can
clearly understand. While likely more challenging, social media and other
technological innovations could also be used effectively for these educational
purposes. As discussed further below, agencies could produce (either individually or

314

See HERZ, USING SOCIAL MEDIA, supra note 28, at 21-28. See also FARINA &
NEWHART, IBM CENTER, supra note 22, at 11-12 (identifying the primary barriers to
meaningful public engagement in rulemaking).
315
See HERZ, USING SOCIAL MEDIA, supra note 28, at 8 (summarizing the challenges).
316
See supra Part III.D.
317
HERZ, USING SOCIAL MEDIA, supra note 28, at 29 (recognizing these may be realistic
goals even for those who are “dubious about the value added by public participation”).

73

collectivelyor through some intermediary such as ACUS) materials that clearly
describe how each mode of participation works and how citizens can contribute most
effectively to those public engagement efforts. Agencies could also provide citizens
with a set of best practices for making contributions through each of the respective
modes of public engagement, and samples of particularly useful or effective
contributions. Finally, agencies could use social media and other technological
innovations to provide their targeted audiences with information that clearly describes
each of their specific rulemaking initiatives and clearly identifies the information or
data they are seeking from relevant members of the public. Armed with this
procedural and substantive knowledge, the agency’s targeted audiences would be in
the best possible position to participate effectively in early rule development.
Social media and other technological innovations can also be used as a means to
gather information from the public or to facilitate a dialogue about the agency’s
potential regulatory plans. These are by far the most challenging uses of social media
for purposes of public engagement, and they require careful planning and facilitation.
The e-rulemaking literature frequently and correctly recognizes that the use of social
media in rulemaking is not like a field of dreams—if you build it, they will not
necessarily come. 318 One could add that even if they come, they will not necessarily
give you the information that you want or need. Nonetheless, if social media is used
effectively for the information and educational purposes described above, agencies
should also be able to use social media as a productive means for gathering useful
information or comments from their targeted audiences.
Agencies should carefully consider what type of information they seek to obtain
through public engagement. For example, are they seeking situated knowledge about
a designated issue or problem, technical information or data about a potential subject
of regulation, public preferences regarding potential forms of consumer disclosure, or
general information about the public’s values or priorities? Answering these
questions will help agencies identify who they need to reach to generate the desired
information. Once they know who they need to reach, agencies can consider how this
information is most likely to be obtained, when those efforts should occur, and what
the agency will do with the input.
Accordingly, agencies must (1) build the appropriate opportunities for public
engagement, (2) inform targeted stakeholders about those opportunities, (3) persuade
them to participate, and (4) provide the targeted audience with the education and
guidance they need to participate effectively. The first of these requirements
essentially involves identifying the best available modes of public participation for
the task at hand. While this entire report is focused on helping with this task, social
media and the Internet can be used to supplement most of the conventional modes of
public engagementand they can also be used as independent tools for engaging
with the public when carefully done in appropriate circumstances.

318

See, e.g., FARINA & NEWHART, IBM CENTER, supra note 22, at 21.

74

The second of these requirements involves conducting the necessary public
outreach. As ACUS has previously recognized, social media can be especially
helpful for this purpose. In addition to following ACUS’s prior recommendations on
this score, the best practices for effective outreach include (1) providing targeted
information to targeted stakeholders, (2) constructing and using listserves to identify
targeted stakeholders and communicate with them regarding relevant rulemaking
initiatives and related opportunities to participate, and (3) creating websites and
operating blogs that are specifically designed for each significant rulemaking
initiative and engaging in appropriate efforts to bring these resources to the attention
of agencies’ targeted audiences.
The final prerequisite to the effective use of social media in public engagement
efforts involves providing citizens with the information they need to participate
effectively. We will further develop the best practices in this area in our discussion of
potential ways to improve notice-and-comment rulemaking, but it is especially
important for agencies to make special efforts to reach out to missing stakeholders
during the early stages of rule development, and to provide them with clear
information about the available ways to participate in each rulemaking initiative,
basic instructions and examples regarding how to participate effectively, and simple
overviews of the relevant issues with detailed descriptions of the information the
agency is seeking and the questions it is trying to answer.
F.

Using Public Input and Providing Feedback to Participants

While the issues that are briefly discussed in this section apply to public
engagement efforts during every stage of rulemaking, and we discuss the importance
of evaluating public engagements efforts as part of a broader discussion of
planning, 319 it is worthwhile to emphasize here that agencies should use the
information they receive from the public in early rule development and they should
also provide participants with appropriate feedback regarding how their input was
used. The EPA’s Public Involvement Policy, for example, identifies reviewing and
using public input and providing feedback as one of seven steps for effective public
participation. 320 This step is considered essential for building public trust and
establishing the credibility of the agency’s public engagement efforts. 321 The
agency’s more detailed guidance for implementing this step of its Policy emphasizes
that EPA “should demonstrate, in its decisions and actions, that it has understood and
fully considered public concerns,” and “the Agency should communicate the decision
to the public and discuss how the public’s input influenced the decision.” 322
Agencies can provide feedback to the public through formal channels, such as
preambles or “responsiveness summaries,” or through relatively informal

319

See infra Part XII.
See BETTER DECISIONS MANUAL, supra note 278, at 3 (discussing this Policy).
321
See EPA PUBLIC INVOLVEMENT POLICY, supra note 35, at 19.
322
Id. at 19.
320

75

mechanisms such as public meetings, press briefings or news releases, or even “thank
you letters.” 323 While preambles and responsiveness summaries often accompany an
agency’s final decisions, agencies should consider preparing a responsiveness
summary for each significant public engagement effort. EPA, for example, is
required by its own regulations to prepare a responsiveness summary for certain
rulemaking activities by its advisory committees. 324 These documents express
respect for the participants and effectively reward “stakeholders who participated by
discussing how their contributions affected the decision.” 325 According to EPA’s
regulations, responsiveness summaries “shall identify the public participation activity
conducted; describe the matters on which the public was consulted; summarize the
public’s views, significant comments, criticisms and suggestions; and set forth the
agency’s specific responses in terms of modifications of the proposed action or an
explanation for rejection of proposals made by the public.” 326 Agencies should also
consider telling “participants in stakeholder involvement events what will happen to
the summaries or discussions—who will get them, what other information the
[a]gency will produce and consider, and where and when the responsiveness
summary will be posted.” 327 Finally, in planning how to use the input they receive
from public engagement efforts and how to notify the public about the impact of its
feedback, agencies should consider the following specific questions: (1) How will
the agency incorporate the results of its public engagement efforts into its decisions?
(2) What specific measures will the agency undertake to ensure this happens? and
(3) How will agencies inform stakeholders of the impact of their comments on the
agency’s decision making? 328 Agencies should also consider how often and through
what methods their officials will “debrief” the public engagement process, and how
they will evaluate the success of their efforts. 329

323

See BETTER DECISIONS MANUAL, supra note 278, at 78.
See 40 C.F.R. § 25.8 (2001).
325
EPA, BETTER DECISIONS MANUAL, supra note 278, at 79.
326
40 C.F.R. § 25.8.
327
EPA, BETTER DECISIONS MANUAL, supra note 278, at 79.
328
See id. at 83. With respect to the second question, EPA’s Conflict and Resolution
Center provides a series of potential examples, including (a) maintaining two-way
communication with workgroup and management; (b) keeping an updated web-site or listserve; (c) encouraging workgroup members to attend public involvement events; (d)
providing opportunities for upper management to be on the agenda at meetings or to listen to
stakeholder concerns; (e) conducting regular meetings or conference calls to update the
agency’s management and workgroup; (f) inviting the facilitator to make presentations to the
workgroup or management; (h) inviting the chair or executive committee to brief
management; and (i) prepare fact sheets or discussion papers. Id.
329
See id. at 83-84.
324

76

VIII. ADVANCED RULE DEVELOPMENT
At some point during rule development the shape of an agency’s likely course of
action comes into focus. There is no hard and fast line between early and advanced
rule development. Nevertheless, it is useful to distinguish between the stage in the
regulatory process when an agency remains open to a variety of different actions,
including no action, and the stage in the process when the agency has committed to
developing a regulation along certain lines, even if the agency is still considering one
or more alternatives or needs to flesh out the details of a proposed rule. First,
agencies seek different types of information at each stage. During early rule
development, the agency needs information about the nature of a problem and what
types of regulatory solutions may or may not be appropriate or feasible. During
advanced rule development, however, the agency has a good idea of the direction it is
headed. Nevertheless, the agency may need more specific information to choose
among competing alternatives, flesh out a proposed rule in more detail, and prepare
for a productive notice-and-comment process, including drafting a proposed rule that
is clear, effective, and avoids unanticipated problems. Second, our interviewees
reported that the public often engages with the agency in different ways depending on
whether the agency is open-minded about how to address a regulatory issue, if at all,
or the agency has made clear the direction in which it is headed. To wit, stakeholders
may be more open and forthcoming with situated knowledge during early rule
development, while focusing on promoting their positions during advanced rule
development. 330
It is critical for agencies to engage the public as they refine their regulatory
proposals and prepare for notice and comment. Executive Order 12,866 directs that,
“before issuing a notice of proposed rulemaking, each [executive] agency should,
where appropriate, seek the involvement of those who are intended to benefit from
and those expected to be burdened by any regulation (including, specifically, State,
local, and tribal officials).” 331 Moreover, it may be essential to the success of the
notice-and-comment process for the agency to receive feedback from a broad array of
stakeholders as it develops its proposed rule. Otherwise, the agency may discover
relevant information or unintended consequences only after publishing an NPRM.
This may require the agency to conduct a second round of notice and comment, thus
delaying the publication of a final rule and wasting agency resources. 332
Many of the tools of public engagement described above in connection with early
rule development will naturally also be relevant to advanced rule development.
330

To be sure, there may be cases in which the process of early rule development is
inapplicable or less involved because the agency knows the policy direction it will take based
on a congressional mandate, the agency’s prior knowledge, or the existing regulatory regime.
331
Exec. Order No. 12,866, 58 Fed. Reg. 51,735 (Sep. 30, 1993).
332
To protect the rule from being vacated on judicial review, the agency may need to
undertake an additional round of notice and comment when it makes significant changes
between the NPRM and Final Rule. See Chocolate Mfrs. Ass’n v. Block, 755 F.2d 1098,
1104-05 (4th Cir. 1985) (describing the logical outgrowth test).

77

Rather than repeat what we have said about these modes of public engagement, in this
Part we highlight additional or different considerations relevant to their use with
advanced rule development. In addition, we discuss several tools that may be unique
to advanced rule development.
A.

Advance Notices of Proposed Rulemaking

Advance Notices of Proposed Rulemaking (ANPRMs) are important tools for
public engagement during advanced rule development. ANPRMs are similar to RFIs
and there is no formal legal distinction between them. 333 In both cases, the agency is
seeking comments, data, and other information from the public in the course of
developing a regulatory proposal. We merely use different terminology to signal the
stage in the regulatory process when the agency is seeking public comment prior to an
NPRM. The agency may issue an RFI to request comments when it is genuinely open
regarding whether to address a matter with new rulemaking and, if so, how. By
contrast, the agency may use an ANPRM to request comments on a tentative proposal
or different potential alternatives and obtain the information it needs to develop and
refine a rule in preparation for issuing an NPRM. Indeed, based on these distinctions,
the Department of Energy’s process rule includes both RFIs and ANPRMs as part of
rule development before publishing an NPRM. 334
Agencies should routinely consider using ANPRMs in connection with their
rulemakings unless they have good cause to do otherwise. Good cause might include
situations in which the agency has full information and full authority to implement
the statutory mandate, the proposed rule is non-controversial, the agency has solicited
meaningful feedback from the relevant stakeholders using other means, or the public
has no information or views relevant to the proposed rule. 335 In many cases,
333

Neither is defined in the APA and many agencies use both. For a useful discussion of
the purposes of ANPRMs and an argument that they should be used more regularly, see
Andrew Emery & Fred Emery, Maybe the Experts Were Wrong about the ANPRM, 34
ADMIN. & REG. L. NEWS 10, 25 (2009).
334
See infra Part XII.A.
335
Cf. SUSAN L. MOFFITT, MAKING POLICY PUBLIC: PARTICIPATORY BUREAUCRACY IN
AMERICAN DEMOCRACY 32 (2014) (“From a bureaucratic perspective, public participation
for independent tasks in which bureaucrats enjoy sufficient information comes fraught with
risk and little potential value in terms of advancing an agency’s image of unique expertise.”).
On the other hand, Moffitt contends that public participation can have substantial value for
administrators when they lack full information about a regulatory problem and when the
successful implementation of a regulatory regime depends in part on the independent actions
of third parties. See id. at 48 (“Tasks with emergent knowledge and interdependent
implementations create opportunities for public participation to offer value to bureaucratic
administration in terms of the information it brings into agencies and in terms of the
information it disseminates from agencies.”) (emphasis in original). Moffitt’s model suggests
that public participation will routinely provide added value for bureaucrats when agencies
promulgate rules on complex subjects in the modern regulatory state. However, public
engagement with rulemaking may be unnecessary or counterproductive when agencies
already have full information and complete jurisdictional control over a regulatory problem.

78

however, ANPRMs can help agencies craft better NPRMs and avoid surprises during
notice and comment.
ANPRMs are most useful when the agency has some doubts or reservations
about a proposed regulatory action, lacks full information about the costs and benefits
of its proposal, or is trying to choose from more than one alternative. In these cases,
ANPRMs can provide critical information about the likelihood of compliance, the
costs of achieving the agency’s goals, the range of potential benefits, and any
unintended consequences. The public response, along with questions raised during
public meetings held in connection with an ANPRM, may also help agencies
understand how best to communicate their proposals and specific legal requirements
to the affected communities in the NPRM. The information garnered from an
ANPRM may cause the agency to alter or revise its proposal before publishing an
NPRM, thus saving the agency time down the road.
Of course, merely publishing an ANPRM in the Federal Register and
Regulations.gov is not enough. The agency must also engage in outreach to obtain
participation from a broad range of stakeholders, including regulated parties, potential
beneficiaries, citizens with situated knowledge, and unaffiliated experts who may
have useful information to share. In addition, ANPRMs are likely to be most
productive when they ask specific questions. 336 Thus, ANPRMs seeking comments
from traditionally absent stakeholders should address them directly and ask them to
share their relevant experiences. For example, if an agency is considering extending
a regulatory requirement to new groups or new contexts, it might ask the stakeholders
currently subject to the requirement about their experiences or ask the new group
what it would cost to comply with the new rule. Or if the agency is proposing a
regulation to address a problem causing harm to consumers, it might ask consumers
already harmed whether the proposed change would have helped them. Agencies
should also ask stakeholders to identify ambiguities or unanticipated problems that
could result from the agency’s tentative proposal, and to suggest potential ways to
improve a proposed rule on the subject.
Finally, whenever an agency publishes an NPRM after using an ANPRM, the
preamble to the NPRM should acknowledge the public input received during the
ANPRM and describe how it influenced the agency. This could include describing
changes the agency made to its proposal based on the public comments or explaining
how the public comments confirmed the appropriateness of the agency’s preferred
course of action. Although the APA does not impose legal constraints on ANPRMs,
we recommend that agencies strive to treat comments received in response to
ANPRMs in the same way they treat comments received in response to NPRMs. At a
minimum, of course, they should review the comments. In addition, agencies should

In these latter situations, using the kinds of public engagement tools that we believe would
routinely be useful, including RFIs, listening sessions, advisory committees, and ANPRMs,
would generally be unnecessary.
336
Cf. Neil R. Eisner & Judith S. Kaleta, Federal Agency Reviews of Existing
Regulations, 48 ADMIN. L. REV. 139, 149, 156 (1996).

79

endeavor to respond in a reasoned fashion to the significant comments in the
preamble to a subsequent NPRM. Moreover, if the agency decides to abandon the
regulatory initiative, the agency should inform the public of its plan and explain its
decision, although it need not provide the same level of detail as a preamble to an
NPRM or final rule.
B.

Federal Advisory Committees

Federal advisory committees can also be used for engaging the public during
advanced rule development. At this stage, the agency may be less interested in
hearing whether it should regulate or about alternative approaches to a rule than in the
committee’s views regarding the costs and benefits of a particular rule under
development and any unintended consequences the committee foresees with the
agency’s proposal. As discussed above in Part VI.B, a well-designed advisory
committee can provide agencies with input from a broad group of informed
stakeholders and unaffiliated experts.
We recommend that agencies consider using their advisory committees in
conjunction with RFIs and ANPRMs during rule development. When an agency
publishes an ANPRM, it should also ask its relevant advisory committees for
feedback or advice regarding the proposal. For example, the Department of Energy
routinely consults an advisory committee concerning the information gathered from
its RFI/ANPRM before deciding whether to proceed with rulemaking. 337 At
whatever stage of rule development an agency consults with its advisory committees,
the agency should post the FACA materials online, preferably in the kind of erulemaking docket described below in Part IX.C. This includes any materials the
agency provided to the committee and the committee’s recommendations or report.
C.

Focus Groups

Agencies may also want to use focus groups during advanced rule development
to gauge the reaction of a particular group to an agency’s proposal or potential
alternatives. Nevertheless, as discussed in Part VII.B, we believe focus groups are
generally more useful during the early stages of rule development. However, focus
groups of unaffiliated experts, in particular, could also provide useful feedback during
the advanced rule development stage.
D.

Public Meetings

Agencies hold public meetings at various stages of rule development. As
discussed above in Part VII.D., public meetings can be conducted in-person,
telephonically, online, or using some combination of formats to allow for both “live”
in-person exchanges and remote access and participation. Agencies typically hold
public hearings when the rulemaking is expected to have a significant impact on the

337

Telephone Interview with Department of Energy (Jan. 19, 2018).

80

public or may prove politically controversial. 338 At the advanced stage of rule
development, public meetings provide an informal setting for the agency to explain
what it is proposing and continue to grapple with the views and concerns of
stakeholders. If properly designed and facilitated, public meetings provide a more
involved form of public participation than an RFI or ANPRM because the agency and
stakeholders with different interests or perspectives can engage in a deliberative
exchange. Agencies should always remind participants, however, that they need to
submit their comments in the rulemaking process to ensure they are part of the
administrative record.
Public meetings also provide agencies with an opportunity to describe how they
responded to public input during rule development, publicize a current or upcoming
ANPRM or NPRM, and encourage public participation in the written comment
process. As we emphasize in Part XII, agencies need to conduct robust outreach to
ensure broad public participation in the notice-and-comment process. A public
meeting during advanced rule development can play an important role in outreach
related to an ANPRM or NPRM. The agency can educate the public on the logistics
of the comment process and highlight the types of information the agency needs.
Finally, questions or comments during public meetings may uncover ambiguities
in an agency’s proposal or misunderstandings on the part of certain stakeholders. The
agency can clarify ambiguities or confusion that emerges during the meeting, which
will help the public submit better comments in response to an ANPRM or NPRM.
The agency may also decide to revise its proposed rule or the preamble to an NPRM
to prevent such misunderstandings from undermining the usefulness of the noticeand-comment process. The public meetings discussed in this section are primarily
designed to inform the public of the agency’s plans and to consult with interested
parties. If the agency is interested in involving or collaborating with a diverse group
of stakeholders, it should use one or more of the enhanced deliberative exercises
discussed in Part XI.
E.

“Shuttle Diplomacy” & Technical Workshops

Agencies also hold meetings with discrete groups of stakeholders during
advanced rule development. For example, after preparing a draft NPRM, EPA often
seeks feedback from stakeholders in one-on-one meetings and listening sessions.339
Although meeting with a specific stakeholder does not foster deliberation among
parties with diverse interests, it does allow agency officials to gain a deeper
understanding of the group’s perspective. In such a setting, some stakeholders may
be more forthcoming with information they do not want to share publically and
willing to engage in a more informal back-and-forth with the agency to identify
possible solutions to problems or challenges identified in the regulatory process. In
addition, agencies sometimes hold technical workshops with stakeholders or other
338

See KERWIN & FURLONG, supra note 2, at 83.
See CHERYL BLAKE & REEVE T. BULL, ADMIN. CONFERENCE OF THE U.S.,
NEGOTIATED RULEMAKING 15 (2017).
339

81

experts to obtain feedback on the data and analysis used by the agency in developing
a proposed rule. 340 These meetings will usually involve sophisticated stakeholders
who routinely participate in rulemaking. Thus, when agencies need to hold one-onone meetings with these groups during the development of their rules, they should
consider whether they are also creating opportunities for participation in the process
by regulatory beneficiaries and other stakeholders who might have useful information
to contribute and a different perspective. They should also consider soliciting
feedback from unaffiliated experts as part of this process.
F.

Internet and Web-based Outreach

Internet and web-based outreach continue to be important tools as agencies move
into advanced rule development, and the strategies, examples, and challenges
discussed in Part VII.E are also relevant at this stage of the rulemaking process.
Agencies can use their websites, social media, and status reports to inform the public
regarding the progress of the rulemaking as the agency refines its proposal and
prepares to publish an NPRM. This includes keeping the public apprised of the
impact of public engagement to date on the agency’s decision-making. Agencies can
also use these tools to educate the public on the upcoming notice-and-comment
process and how to participate effectively. Finally, agencies can use e-tools to gather
information and facilitate dialogue concerning the agency’s plans.
Some of the web-based enhanced deliberation efforts discussed below in Parts IX
(Notice-and-Comment Process) and XI (Enhanced Forms of Deliberation) may be
most fruitful during advanced rule development. One can easily re-imagine the
Regulation Room projects as deliberative exercises conducted during advanced rule
development, when the agency has a rough outline of what it plans to propose, but
before it has published an NPRM. Indeed, Regulation Room conducted online
deliberations regarding an ANPRM that was issued by the Department of
Transportation. 341 Using these modes of public engagement during late rule
development enables the agency to obtain situated knowledge from traditionally
absent stakeholders before the notice-and-comment process begins. As we emphasize
throughout this Report, such enhanced public engagement will not be necessary or
productive in all rulemakings. But when agencies decide that they need to target
rulemaking novices for participation in the rulemaking process, it would be better for
them to play a role before the agency has committed itself to a proposal in an
NPRM. 342

340

Id. at 15-18.
See Enhancing Airline Passenger Protections, 75 Fed. Reg. 32,318, 32,319 (Proposed
June 8, 2010) (to be codified at 14 C.F.R. pt. 259.5); see also Farina et al., Rulemaking 2.0,
supra note 65, at 428 n.116.
342
“[S]ome proposals might be too ‘hard-baked’ to easily switch to other, quite different
options after the formal comment period close[s].” Patricia A. McCoy, Public Engagement in
Rulemaking: The Consumer Financial Protection Bureau’s New Approach, 7 BROOK. J.
CORP. FIN. & COM. L. 1, 2 (2012).
341

82

The CFPB has been at the forefront of using the Internet and social media to
engage the public during advanced rule development. 343 When the agency was
developing new regulations governing disclosure requirements for home mortgages, it
posted prototypes of the disclosure forms on its website and invited both consumers
and industry to comment on the alternatives and which they preferred. 344 The agency
asked consumers if the forms were missing important information and encouraged
them to submit comments online. The agency asked industry participants about the
“usability and ease of implementation.” 345 The agency received more than 27,000
text box comments and emails in response to the forms. 346 In addition, the public
could click on parts of the forms they “liked” or “disliked.” The agency used this
information to create heat maps showing where readers focused their attention. These
heat maps were then posted online. The comments and heat maps helped the agency
identify problems with the disclosure forms and develop solutions. In this way, the
public’s participation in advanced rule development helped the CFPB to further refine
its proposed disclosure form prior to publishing an NPRM.
A few things about the success of this exercise are worth noting. First, merely
posting material on an agency’s website is not sufficient to ensure public
participation. As the CFPB official who oversaw all mortgage policy initiatives for
the Bureau described it:
In the weeks leading up to roll out, the … rulemaking team posted
announcements on the CFPB’s website, on Facebook, and on Twitter
explaining the purpose of mortgage disclosures, talking about the need to
improve those disclosures, and announcing our plans to unveil the prototype
forms in the upcoming weeks for people’s reaction. For readers who wanted
to get involved, the Bureau suggested that they sign up for e-mail updates
and tell their friends and family about the project on Twitter and Facebook
and through e-mail. 347
These extensive outreach efforts were likely critical to obtaining robust public
participation in the rule development process.
Second, these online exercises were part of an iterative, multi-modal approach to
public engagement in the development of CFPB’s mortgage disclosure rules. Early in
the process, the agency:

343

The example described below may have been the first time “any federal banking
regulator … elicited mass public input on prototype disclosure forms before a proposed rule
was published.” Id. at 7.
344
See id. at 8; Elizabeth G. Porter & Kathryn A. Watts, Visual Rulemaking, 91 N.Y.U.
L. REV. 1183, 1213 (2016).
345
McCoy, supra note 342, at 8.
346
See id.
347
Id. at 7. Patricia McCoy founded the Mortgage Markets section of the CFPB.

83

[held] brainstorming sessions with affected stakeholders—such as consumer
representatives, housing counselors, lenders, other agencies, settlement
services providers, and vendors—to identify issues and possible solutions;
review[ed] the existing research and comment letters on past disclosure
proposals . . . ; hir[ed] experienced consultants to assist . . . with the design
and testing; and [held] an academic research symposium to explore
consumer decision-making processes and better ways to design
disclosures. 348
Then, once the agency had developed two prototype forms, it conducted five rounds
of qualitative testing of each form using one-on-one interviews conducted in different
parts the country. 349 Ninety-two consumers and twenty-two lenders participated. 350
After each round of testing, the agency evaluated the results, revised the forms, and
tested the new forms during the next round. 351 The agency tested the prototypes in
both English and Spanish. 352 At the same time as the agency conducted the
qualitative testing using small groups, the agency used the web-based exercises
involving the broader public. 353
The CFPB conducted similarly robust outreach and public engagement before
publishing an NPRM during its development of regulations and disclosures
concerning college costs, overdraft fees, pre-paid cards, overdraft protection, payday
lending, and private educational loans. 354 These types of Internet and web-based
exercises lend themselves well to disclosure and labeling requirements because
consumer preferences and reactions are directly relevant to their effectiveness.
Finally, agencies should include the results of these exercises in some form,
comparable to responsiveness summaries, 355 in the e-rulemaking dockets we discuss
more fully below in Part IX.C. The information, comments, and views provided by
the public should be part of the decision-making process if members of the public
take the time and effort to participate. 356
G.

Negotiated Rulemaking

Agencies sometimes use negotiated rulemaking, or “Reg Neg,” during advanced
rule development. In negotiated rulemaking, the agency convenes an advisory
committee comprising representatives of stakeholders affected by the rule to

348

Id. at 5.
See id. at 5-6.
350
See id. at 6.
351
See id.
352
See id. at 6-7.
353
See id. at 7.
354
See id. at 3.
355
See supra notes 323-325 and accompanying text.
356
See FARINA & NEWHART, IBM CENTER, supra note 22, at 8.
349

84

collaborate on the formulation of a rule for notice and comment. 357 The negotiating
committee is led by a convener or facilitator trained in ADR who “establishes
mutually agreed upon ground rules for the negotiation, seeks to flesh out the
stakeholders’ positions, identifies and gathers information on relevant questions of
fact, and guides participants towards producing a draft rule text based on group
consensus.” 358 If the committee reaches a consensus on a proposed rule, the agency
then conducts the notice-and-comment process required by the APA. If the
committee does not reach consensus, the agency must decide whether or not to
proceed with a proposal.
Negotiated rulemaking arose in response to concerns first voiced in the 1960s
that notice-and-comment rulemaking “had become increasingly adversarial and
formalized.” 359 Stakeholders with different interests did not meet or communicate
with each other, which led them to take “conflicting and antagonistic positions and to
engage in expensive and time-consuming litigation over agency rules.” 360
In 1982, the Administrative Conference adopted a series of recommendations
concerning regulatory negotiation with affected stakeholders based on a report written
by Philip J. Harter. 361 Harter argued that
Negotiations among directly affected groups conducted within both the
existing policies of the statute authorizing the regulation and the
existing policies of the agency, would enable the parties to participate
directly in the establishment of the rule. The significant concerns of
each could be considered frontally. Direct participation in rulemaking
through negotiations is preferable to entrusting the decision to the
wisdom and judgment of the agency, which is essential under the basic
provisions of the APA, or to relying on the more formal, structured
method of hybrid rulemaking in which it is difficult for anyone to
make the careful trade offs necessary for an enlightened regulation. A
regulation that is developed by and has the support of the respective

357

5 U.S.C. § 565(a)(1) (2012) (the committee is composed of representatives who “can
adequately represent the interests that will be significantly affected by a proposed rule”); see
also BLAKE & BULL, supra note 339, at 1. Negotiated rulemaking committees must comply
with the Federal Advisory Committee Act (FACA) unless they have been exempted by
statute. § 565(a)(1).
358
BLAKE & BULL, supra note 339, at 1.
359
Admin. Conf. of the U.S., Recommendation 85-5, Procedures for Negotiating
Proposed Regulations, 50 Fed. Reg. 52,893, 52,895, 52,895 (Dec. 27, 1985).
360
Negotiated Rulemaking Act of 1990, Pub. L. No. 101–648, § 2, 104 Stat. 4969, 4969.
361
See generally Admin. Conf. of the U.S., Recommendation 82-4, Procedures for
Negotiating Regulations, 47 Fed. Reg. 30,701 30,708 (July 15, 1982); Philip J. Harter,
Negotiating Regulations: A Cure for Malaise, 71 GEO. L.J. 1 (1982).

85

interests would have a political legitimacy that regulations developed
under any other process arguably lack. 362
The Administrative Conference issued a second set of recommendations in 1985
based on the experience of four agencies that had used negotiated rulemaking since
1982. 363 The two sets of recommendations provided a framework and procedures for
planning and conducting a rule negotiation. 364
Based on the work of the Administrative Conference and advocates who had had
success with regulatory negotiation, Congress formally endorsed the practice in the
Negotiated Rulemaking Act of 1990. 365 The Act clarified the legal authority of
federal agencies to conduct Reg Neg and established certain procedural
requirements. 366 Among other things, the agency head must find that the use of
negotiated rulemaking is “in the public interest” based on criteria set forth in the
Act. 367 Then, if the agency decides to establish a negotiating committee, the agency
must publish a notice in the Federal Register announcing the committee; describing
the issues to be negotiated, the interests “likely to be significantly affected by the
rule,” and the “persons proposed to represent such interests”; soliciting comments on
both the proposal and the proposed membership of the committee; and explaining
“how a person may apply or nominate another person for membership on the
committee.” 368 In addition, the Act authorized the Administrative Conference to
provide agencies with training on Reg Neg, maintain a roster of trained facilitators,
cover agencies’ Reg Neg expenses, and report to Congress on the process. 369
Negotiated rulemaking generated a great deal of enthusiasm among some after
the passage of the Act. The Clinton Administration repeatedly endorsed its use, and
between 1991 and 1999 federal agencies convened 63 negotiated rulemaking
committees. 370 But there was a significant decline in its use during the George W.

362

Harter, supra note 361, at 7.
See generally Procedures for Negotiating Proposed Regulations, 50 Fed. Reg. at
52,985; see also Henry H. Perritt, Jr., Negotiated Rulemaking Before Federal Agencies:
Evaluation of Recommendations by the Administrative Conference of the United States, 74
GEO. L.J. 1625, 1627 (1986).
364
Perritt, supra note 363, at 1628.
365
See generally Negotiated Rulemaking Act of 1990, Pub. L. No. 101-648, 104 Stat.
4969 (codified as amended by Pub. L. No. 104-320, 110 Stat. 3870 (1996), at 5 U.S.C. §§
561-70).
366
See BLAKE & BULL, supra note 339, at 6-7.
367
5 U.S.C. § 563(a) (2012).
368
Id. § 564(a).
369
See BLAKE & BULL, supra note 339, at 7.
370
Exec. Order No. 12,866 § 6(a)(1), 58 Fed. Reg. 51,735 (Sept. 30, 1993) (directing
agencies to consider using negotiated rulemaking); Memorandum from William J. Clinton,
President of the U.S. to Executive Departments & Selected Agencies Administrator of the
Office
of
Information
and
Regulatory
Affairs
(Sept.
30,
1993),
363

86

Bush Administration. 371 Agencies convened only 22 negotiated rulemaking
committees between 2000 and 2007. 372 The trend continued during the Obama
Administration, with only 13 negotiated rulemaking committees noticed between
2007 and 2013. 373 Eighty-five percent of these committees were statutorily
required. 374 Even during the Clinton Administration, however, negotiated rulemaking
was used in only a tiny percentage of the rulemakings conducted each year.
The Administrative Conference published its most recent recommendations on
negotiated rulemaking in June 2017, in the wake of this decline. 375 The
Administrative Conference’s 2017 Recommendations place negotiated rulemaking in
the context of a variety of ways, including many discussed in this Report, that
agencies can obtain public input on their regulatory work. 376 The Administrative
Conference recommended that federal agencies deciding whether to use negotiated
rulemaking consider whether:
• “there are a limited number of identifiable interests that will be
significantly affected by the rule;”
• “there is a reasonable likelihood that a committee can be convened with a
balanced representation of persons who (a) can adequately represent the
[identifiable and significantly affected] interests and (b) are willing to
negotiate in good faith to reach a consensus on the proposed rule;”
• there is adequate time to complete negotiated rulemaking and the agency
possesses the necessary resources to support the process; and
• “the agency, to the maximum extent possible consistent with the legal
obligations of the agency, will use the consensus of the committee with

http://govinfo.library.unt.edu/npr/library/direct/memos/2682.html (directing federal agencies
“to (i) identify to OIRA at least one rulemaking which the agency will, within the upcoming
year, develop through the use of negotiated rulemaking or (ii) explain to OIRA why the use
of negotiated rulemaking will not be feasible in the upcoming year”); Jeffrey S. Lubbers,
Achieving Policymaking Consensus: The (Unfortunate) Waning of Negotiated Rulemaking,
49 S. TEX. L. REV. 987, 1001 (2008) (counting negotiated rulemaking committees noticed in
the Federal Register).
371
See Lubbers, supra note 370, at 996.
372
See id.
373
See Peter H. Schuck & Steven Kochevar, Reg Neg Redux: The Career of A
Procedural Reform, 15 THEORETICAL INQUIRIES L. 417, 439 (2014).
374
See id.
375
See generally Admin. Conference of the U.S., Recommendation 2017-2, Negotiated
Rulemaking and Other Options for Public Engagement, 82 Fed. Reg. 31,039, 31,040 (July 5,
2017); BLAKE & BULL, supra note 339.
376
See Negotiated Rulemaking and Other Options for Public Engagement, 82 Fed. Reg.
at 31,042.

87

respect to the proposed rule as the basis for the rule proposed by the
agency for notice and comment.” 377
The first two considerations are most salient for purposes of this Report. Even in
negotiated rulemaking’s heyday, Professor William Funk raised concerns that parts of
the public were left out of the process:
The rulemaking has “parties” who make the agreement. They make the
agreement among and for themselves. They bargain and deal to achieve
their own interests. There is no mention of the “public.” The wisdom and
fairness of the rule is equated with the satisfaction of the parties. Public law
has been subtly transformed into private law relationships. 378
Under such circumstances, it is essential that the parties to the negotiation
adequately represent all affected interests. Moreover, “[g]iven the inherent
challenges of group dynamics,” 379 achieving consensus becomes increasingly
difficult as the number of affected interests multiply. Therefore, the Negotiated
Rulemaking Act “limit[s] membership on a negotiated rulemaking committee to 25
members, unless the agency head determines that a greater number of members is
necessary for the functioning of the committee or to achieve balanced
membership.” 380 This number includes at least one person who represents the
agency. 381 Given the limited size of negotiating committees and the requirement that
committees represent all the affected interests, Professor Susan Rose-Ackerman
argues that regulatory negotiation is “ill-advised” in areas such as environmental
policymaking “because the notion of interest representation on which the method is
based does not apply to most environmental issues.” 382 There are simply too many
diverse interests at stake to be “represented effectively by standard environmental
groups.” 383 Indeed, even the most ardent advocates of negotiated rulemaking
recognize that it is not appropriate for “a regulation that would affect many interests
in such diverse ways that representation by a few individuals or teams of individuals
would be impossible.” 384
Thus, negotiated rulemaking is likely inappropriate when agencies identify
absent stakeholders who are not adequately represented by other parties for the

377

Id. (alteration in original) (quoting Negotiated Rulemaking Act, 5 U.S.C. §§
563(a)(2)-(7) (2012)).
378
William Funk, Bargaining Toward the New Millennium: Regulatory Negotiation and
the Subversion of the Public Interest, 46 DUKE L.J. 1351, 1386 (1997).
379
BLAKE & BULL, supra note 339, at 35.
380
5 U.S.C. § 565(b) (2012).
381
Id.
382
Susan Rose-Ackerman, Consensus Versus Incentives: A Skeptical Look at Regulatory
Negotiation, 43 DUKE L.J. 1206, 1210 (1994).
383
Id.
384
Harter, supra note 361, at 46.

88

reasons discussed above in Part III.D: “All the interest groups participating must be
well organized and similar in knowledge and bargaining skill.” 385 Nor is negotiated
rulemaking appropriate where there is a large group of diverse interests affected in
different ways or who hold a variety of different preferences. In these circumstances,
the enhanced deliberative exercises discussed in Part XI may be substantially more
fruitful.
Nevertheless, negotiated rulemaking remains an important tool for agencies
seeking a more deliberative, consensus-driven process for developing a subset of
rules that lend themselves to the criteria set forth in the 1990 Act 386 and the
Administrative Conference’s 2017 Report and Recommendations. 387 When those
criteria are not satisfied—because there are interested stakeholders who are not
adequately represented by organized groups—that should be a strong signal that the
agency should consider using the other tools of public engagement discussed in this
report.
IX. THE NOTICE-AND-COMMENT PROCESS
The notice-and-comment process set forth in section 553 of the APA is the most
prominent and well-studied aspect of rulemaking. Far more has been written about
notice-and-comment rulemaking than agenda setting, rule development, or
retrospective review. 388 Indeed, “notice-and-comment rulemaking” is often used as
short-hand for the administrative process of creating legislative rules, even though, as
this Report emphasizes, the process of rulemaking begins much earlier.
Notice-and-comment rulemaking is generally considered to be the most open part
of the regulatory process, and more open to participation by members of the general
public than the legislative process. 389 The agency must afford any interested person
the opportunity to submit comments on proposed rules, there is no particular form
that comments must take, the agency is obligated to consider the public comments,
and the agency must generally respond in a reasoned fashion to significant issues
raised in the comments, regardless of their source. Although all members of the
public can write letters to their political representatives, politicians have no obligation
385

Rose-Ackerman, supra note 382, at 1210.
5 U.S.C. § 563(a) (2012).
387
See generally BLAKE & BULL, supra note 339.
388
See, e.g., Coglianese & Walters, supra note 155, at 866 (noting how little is
known about agenda-setting by agencies).
389
See STEVEN P. CROLEY, REGULATION AND PUBLIC INTERESTS: THE POSSIBILITY OF
GOOD REGULATORY GOVERNMENT 74 (2008) (“[T]he legal-procedural ways in which
agencies regulate attenuate the advantages that more powerful interest groups enjoy over less
powerful interest groups in the legislative arena.”); KERWIN & FURLONG, supra note 2, at 31
(“Rulemaking adds opportunities for and dimensions to public participation that are rarely
present in the deliberations of Congress or other legislatures.”); Kochan, supra note 2, at 601
(“The commenting power given to ordinary individuals is rather extraordinary.”).
386

89

to respond in a reasoned fashion to constituent mail, and those letters do not become
part of the record that Congress must consider when taking legislative action. The
procedural requirements of notice-and-comment rulemaking, by contrast, allow
members of the general public to share their views directly with the responsible
decision-maker and compel the decision-maker to address their concerns, lest the
agency face reversal on judicial review.
Nevertheless, as discussed above in Part III.A, there is a widespread perception
that certain “sophisticated stakeholders,” 390 including representatives of industry,
large trade groups, professional associations, and national advocacy organizations,
typically dominate notice-and-comment rulemaking.
By contrast, regulatory
beneficiaries; small regulated businesses; state, local, and tribal governments;
ordinary citizens with situated knowledge; and members of the general public rarely
participate. In addition, even in high-profile rulemakings that do attract mass
comments, members of the public merely register their general preferences and rarely
participate effectively. 391
This raises at least two concerns: First, there may be stakeholders who possess
information that will help the agency to write a more effective rule but do not
participate, or participate effectively, in the rulemaking. Although this depends on
the specific rulemaking, there are very likely some rulemakings in which rulemaking
novices possess valuable information.
Second, the minimal procedural requirements of notice and comment as currently
practiced may not meet the democratic obligation of agencies to render an account of
what they are doing and why in a way that can be understood and accepted by the
public, in all of its various forms, and to provide the public, in all of its various forms,
with a meaningful opportunity to participate in the decision-making process by
sharing their experiences and views. 392
Agencies have used a variety of tools to address these concerns in the notice-andcomment process. First, agencies have begun to rethink how they draft their NPRMs
to make them more understandable to the general public and highlight key issues in
the rulemaking and the types of helpful information that absent stakeholders might be
able to provide. 393 Second, some agencies have recently made creative use of social
and other visual media to raise awareness about their rulemakings, to encourage
public participation, and to direct interested persons to submit comments through
Regulations.gov. Third, some agencies have created more user-friendly e-rulemaking
390

See Farina et al., Knowledge in the People, supra note 5, at 1191 (defining
“sophisticated stakeholders”).
391
See supra Part III.C (discussing the challenges of mass comments).
392
See supra Part II.B.
393
See generally Admin. Conf. of the U.S., Recommendation 2017-3, Plain
Language in Regulatory Drafting, 82 Fed. Reg. 61,728, 61,728 (Dec. 29, 2017); see also
BLAKE & EMERSON, supra note 19, at 12-13 (discussing the Department of Education’s
attempts to make regulatory documents more understandable).

90

dockets, which summarize different parts of the NPRM, divide it into more digestible
sections, and allow persons to comment on specific issues and reply to the comments
of others. Fourth, both government and non-governmental parties have developed
tutorials for rulemaking novices that explain the process and describe how to submit
effective comments that could have an impact on the agency’s decision. Fifth,
agencies have long used public meetings to supplement notice-and-comment
rulemaking and provide for a more informal forum in which to clarify their regulatory
proposals, obtain feedback from interested stakeholders, and engage in a more
dialogic process than typically is possible through written notice and comment.
Sixth, agencies sometimes use reply comment periods to flesh out issues raised
during the initial comment period and allow for a more dialogic process. Finally, a
few agencies have engaged in supplemental deliberative exercises in rulemakings
where they suspect that missing stakeholders who face barriers to participating may
have important information to contribute. We review each of these tools, their
successes and their limits, in turn.
A.

Plain Language NPRMs

Meaningful public participation in notice-and-comment rulemaking begins with
the NPRM. To achieve its information-gathering function, the NPRM must be
written so that it is understandable to the stakeholders who will be impacted by the
regulation and are likely to have information the agency needs. 394 This includes
sophisticated stakeholders who routinely participate in rulemaking as well as
rulemaking novices who might nevertheless be valuable sources of information
otherwise unavailable. To achieve the democratic function of notice and comment
the NPRM should also be understandable to members of the general public.
Achieving these twin goals may in some cases, particularly where complex technical
issues are involved, require the NPRM to address different audiences in different
places.
In December 2017, ACUS adopted a series of recommendations regarding “Plain
Language in Regulatory Drafting.” 395 The Recommendations identify “tools and
techniques agencies have successfully used to facilitate plain language drafting in
rulemaking[.]” 396 Rather than repeat the many sensible recommendations ACUS has
already made regarding plain language regulations, we (1) highlight the
recommendations particularly useful for engaging absent stakeholders and members

394

The Plain Writing Act (PWA) exempts rulemaking from its list of covered
documents, but does cover other documents related to regulations that are directed to the
public. Pub. L. No. 111-274, § 3, 124 Stat. 2861, 2861 (2010). OMB interpreted the Act as
applying to “rulemaking preambles.” But it is not enforceable in court. Memorandum from
Cass R. Sunstein, Adm’r, Office of Info. & Regulatory Affairs, to the Heads of Executive
Departments
and
Agencies
(Apr.
13,
2011),
https://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2011/m11-15.pdf.
395
Plain Language in Regulatory Drafting, 82 Fed. Reg. at 61,728.
396
Id. at 61,730.

91

of the general public and (2) offer some additional recommendations to improve the
accessibility of NPRMs.
First, agencies should pitch the language of their NPRMs, particularly the
preambles, to the relevant audiences for the rulemaking. These audiences include not
just sophisticated stakeholders, but also the potential beneficiaries, smaller regulated
entities, and members of the broader public. 397 To apprise members of the general
public and unsophisticated stakeholders of the agency’s proposals, the preambles to
NPRMs should plainly explain the goals of a proposal, the agency’s means-ends
analysis, and how to comment on the rule. In addition, the preambles should
highlight issues the agency believes would particularly benefit from public ventilation
based on outreach conducted during rule development. To the extent the NPRM must
also address sophisticated commenters with specialized expertise likely unfamiliar to
the general public, the preamble should address these issues after the discussions
aimed at rulemaking novices. Stakeholders who routinely participate in rulemaking,
such as industry and national advocacy organizations and public interest groups, do
not face the same barriers to participation as rulemaking novices. Unlike members of
the general public, sophisticated stakeholders are unlikely to throw up their hands and
merely submit a thumbs-up or thumbs-down comment if the preamble does not speak
to them until page 100. Most rulemaking novices will.
Several agencies have begun employing visual elements such as bullet points,
Q&A formats, and more colloquial language to emphasize key issues for public
comment. For example, the Department of Education used a bulleted list of “Specific
Issues Open for Comment” to solicit information relevant to a 2015 proposal that all
Department grantees awarded direct competitive grant funds be required to openly
license intellectual property created with Department funds. 398 The Department
explained that “[i]n addition to your general comments, we are particularly interested
in your feedback on the” bulleted questions. Some of the questions also addressed
specific members of the public directly and asked them to share their experiences.
For example, one question asked, “What experiences do you have implementing
requirements of open licensing policy with other Federal agencies? Please share your
experiences with these different approaches, including lessons learned and
recommendations that might be related to this document.” 399 Thus, the Department
highlighted a “known unknown” and sought information directly from the
stakeholders the Department believed were most likely to have the relevant
information.
Many agencies have adopted internal manuals, practices, and procedures to
ensure their NPRMs are understandable to unsophisticated stakeholders and members
of the general public. These may include express plain language directions in internal

397

See id.
See Open Licensing Requirement for Direct Grant Programs, 80 Fed. Reg. 67,672,
67,672 (U.S. Dep’t of Educ. Nov. 3, 2015) (to be codified at 2 C.F.R. pt. 3574).
399
Id.
398

92

agency guidelines for rulemaking. 400 In addition, multiple offices within an agency
usually play a role in drafting regulatory documents, including policy offices with
different areas of subject-matter expertise, the Office of the General Counsel,
economists, and others. The Office of Information and Regulatory Affairs (OIRA) in
the Office of Management and Budget (OMB) also reviews certain draft NPRMs
under E.O. 12,866. A collaborative process involving different offices can help to
catch ambiguities and confusion in the NPRM. A designated office may also be
given specific responsibility for reviewing the NPRM for clarity and accessibility. 401
The Office of General Counsel (OGC) may be well suited to play this role because it
works outside of a specific subject-matter area.
Nevertheless, OGCs are staffed with lawyers, who have a high level of education
and draft language using the conventions of their own legal and regulatory
communities of practice. Thus, OGCs may face their own obstacles in determining
whether an NPRM is clear and understandable to a non-expert, non-lawyer audience.
Therefore, agencies may also want to consider sharing draft NPRMs with reviewers
without expertise in the subject area of the rulemaking or who do not draft and read
regulatory documents on a regular basis.
For example, the Department of Transportation (DOT) shared draft NPRMs with
the Regulation Room researchers at Cornell University in connection with several
rulemakings discussed more fully below. 402 Although the Regulation Room team
included at least one expert in and many students familiar with administrative law,
they were not experts in commercial trucking, accessible air travel, or the other
regulatory topics of the rulemakings. In addition, although many members of the
team were trained as lawyers, they did not routinely draft regulatory documents. An
interviewee reported that having someone familiar with the rulemaking process but
not the substantive area of the rulemaking read the draft NPRM can help make the
NPRM more comprehensible to rulemaking novices and those unfamiliar with the
technical aspects of the regulations. 403 The Forest Service likewise reported
benefitting from having unaffiliated experts review and comment on the drafting of
proposed rules.
Accordingly, this Report reiterates ACUS Recommendation 2017-3, No. 2, that
agencies “consider directing one or more offices involved in drafting rules and
guidance to review them for plain language.” In addition, the Report recommends

400

See, e.g., BLAKE & EMERSON, supra note 19, at 13-14 (discussing the Federal
Highway Administration and the Federal Aviation Administration).
401
See, e.g., id. at 26-27.
402
See infra Part IX.C & F.
403
Telephone Interview with Neil Eisner (Jan. 12, 2018). There may be obstacles to
sharing some NPRMs with parties outside the agency prior to publication, however. For
example, the Office of Federal Procurement Policy is an obstacle to early engagement with
the public by GSA regarding rule development because its guidelines make rules confidential
until they reach a certain stage.

93

that agencies consider including non-subject-area experts and non-lawyers in
reviewing draft NPRMs to make sure that designated parts of the NPRM speak in
plain language to rulemaking novices and members of the general public. Agencies
might consider tasking their public communications office, for example, with this
responsibility. Moreover, NPRM reviewers should consider the specific stakeholders
the agency seeks to reach and whether they are likely to be able to find and
understand the specific questions that the agency wants them to answer. When these
stakeholders may include rulemaking novices, reviewers should ask whether such
stakeholders would know that they are being solicited and understand the agency’s
questions.
B.

Using Social and Visual On-line Media to Publicize Rulemakings

Agencies have long promoted their regulatory activities to the general public in
various ways. But these efforts have not usually focused on rulemakings. 404
Agencies are required to publish their NPRMs in the Federal Register and post them
using Regulations.gov. Some agencies also post a description of their NPRMs on
their website, with a link to the Federal Register or Regulations.gov. But some
proposed rules are buried deep within an agency’s portal. Moreover, even prominent
placement on an agency’s website does little to ensure broad public awareness of the
rulemaking. Beyond the sophisticated stakeholders who routinely participate in
rulemaking and monitor agency activities, most persons with something at stake in
rulemakings do not monitor either the agency’s website or Regulations.gov. The
media covers some particularly salient or politically controversial rulemakings. But it
too rarely discusses the public comment process, and even more rarely, if ever, tells
the public how to participate in notice and comment.
Professors Elizabeth Porter and Kathryn Watts have called for greater use of
visual media by agencies to leverage information “inflows”—i.e., encouraging public
participation in rulemakings. 405 Although agencies have long used visual media to
communicate with the public, it has not historically played an important role in
rulemaking. 406 Porter and Watts suggest that various regulatory stakeholders,
including agencies themselves, the President, members of Congress, interest groups,
and members of the general public, have begun making greater use of visual media—
including still images, videos, infographics, and GIFs—to promote their regulatory
goals. 407 To date, however, agencies that have made the most use of visual media in
rulemaking have focused on information “outflows”—i.e., efforts to “sell[]their
rulemaking stories to the American people.” 408 Agencies have made less use of
visuals to encourage public engagement with rulemaking. 409

404

See Porter & Watts, supra note 344, at 1193-94.
See id. at 1278.
406
See id. at 1186, 1200.
407
See id. at 1185-86, 1198.
408
Id. at 1200.
409
See id. at 1210.
405

94

The Department of Labor (DOL) and the Environmental Protection Agency
(EPA) have been “at the forefront” of using visuals to publicize their rulemakings. 410
For example, after DOL published an NPRM to expand overtime pay under the Fair
Labor Standards Act (FLSA), the agency posted a 4-minute whiteboard video to its
YouTube channel explaining in a simple and visually engaging manner the nature of
overtime pay and the benefits of updating overtime protections. 411 In addition, DOL
posted a GIF on its overtime webpage that used humor to communicate the
advantages of its regulatory initiative to members of the general public. 412 Similarly,
after publishing an NPRM to require financial advisers to avoid conflicts of interest
when providing investment advice, DOL posted a white-board video on its YouTube
channel to explain the proposal. In addition, DOL posted an emotional video on its
website in which a woman recounts how she was taken advantage of by an
investment adviser with conflicts when her husband was diagnosed with Alzheimer’s
disease. 413
EPA has also used social and visual media to publicize its rulemakings. For
example, EPA marketed the benefits of its Clean Power and Clean Water Plans to the
public using whiteboard videos, infographics, photographs, and other social media
that portrayed a variety of negative consequences of pollution for everyday
Americans in simple, and often emotional, terms. 414
These uses of the Internet and social media helped the agencies communicate
clearly, simply, and in a manner likely to be understood by the general public, the
problems that the agencies’ NPRMs sought to address and the benefits they hoped to
achieve. But none of these social media campaigns included links to the rulemaking
docket or advice on how to participate in the rulemaking. For example, although the
end of an EPA whiteboard video on the Clean Water Rule told viewers that, “We are
starting a national conversation on this, and we encourage you to tell us what you
think of our proposal and make your voices heard,” the video said nothing more about
how, where, when, or why to comment on the NPRM. 415
The CFPB is one of the few agencies that has made innovative use of social
media to encourage public engagement with, not just publicize, its regulatory work.
For example, when proposing requirements for pre-paid cards, 416 the agency tweeted
a picture of two fee-disclosure forms for pre-paid cards, and asked the public to “Let

410

See id. at 1200.
See id. at 1201.
412
See id. at 1204.
413
See id. at 1204-05.
414
See id. at 1206-10.
415
Id. at 1211 (quoting U.S. Envtl. Prot. Agency, EPA White Board: Clean Water Act
Rule
Proposal
Explained,
YOUTUBE
(Mar.
25,
2014),
https://
www.youtube.com/watch?v=fOUESH_JmA0, at 02:28–02:35).
416
Pre-paid cards are financial instruments that consumers can load money onto.
411

95

us know what you think.” 417 The tweet included a link to a blog post elaborating on
how to comment at Regulations.gov, provided access to key regulatory documents,
and posed specific questions the agency wanted the public to answer. 418
Social media offers a powerful tool to reach unsophisticated stakeholders and the
broader public concerning the goals and rationales of regulatory initiatives and in
some cases has generated significant public engagement in rulemaking. Welldesigned visual media can break down a rulemaking into understandable terms and
highlight what is at stake for regulatory beneficiaries and smaller regulated parties
who do not regularly participate in the rulemaking process.
But generating awareness of and interest in rulemaking is only a first step
towards meaningful public engagement. Agencies that engage in public outreach
regarding a rulemaking should, at a minimum, also invite the public to participate in
the rulemaking by submitting comments. This requires providing the audiences for
the agency’s media with links to basic information about how and when to submit
public comments, the public’s role in the rulemaking process, and the kinds of
information that would be most valuable for the agency to receive. This should be
relatively simple for agencies that have made the decision to invest in social and other
visual online media. Such steps are not enough to ensure meaningful public
engagement—prospective commenters also need accessible information about the
substance of the proposed rule and sufficient incentives to read it.419 But they are still
a good first step to overcoming the “awareness” barrier to broader public participation
in rulemaking.
The downside to such efforts is that they may generate more public comments
but not necessarily better public comments. In other words, web-based outreach may
result in “torrents of email” 420 merely endorsing or opposing a proposed regulatory
action. 421 The agency must review all these comments even when they provide little
useful information. 422 To be sure, this danger already exists without social media
outreach by agencies encouraging public participation in rulemaking. Advocacy

417

See Porter & Watts, supra note 344, at 1212 (citing Consumerfinance.gov (@CFPB),
TWITTER (Nov. 19, 2014, 9:33 AM), https://twitter.com/CFPB/status/535123637582708736
[https://perma.cc/LC65-QP7G].).
418
Id. at 1212-13. See also Eric Goldberg, Prepaid Products: New Disclosures to Help
You Compare Options, CONSUMER FIN. PROTECTION BUREAU (Nov. 13, 2014),
https://www.consumerfinance.gov/about-us/blog/prepaid-products-new-disclosures-to-helpyou-compare-options/.
419
See HERZ, USING SOCIAL MEDIA, supra note 28, at 32-33.
420
See Mendelson, Torrents of E-Mail, supra note 13, at 1361.
421
See FARINA & NEWHART, IBM CENTER, supra note 22, at 12 n.2 (“Mass e-mail
comment campaigns mounted by advocacy groups illustrate low participation literacy. These
comments are typically short, generalized statements urging the agency to do (or not do)
something about the primary topic the rule addresses.”).
422
See supra Part III.C (discussing the challenges of mass comments).

96

groups are quite capable of undertaking mass comment campaigns using their own
social media and the current form of Regulations.gov, or even just an old-fashioned
mailing address. 423 Mass comments can also be generated by the efforts of a single
television host. 424 Still, agencies may be reluctant to contribute to the inflow of
unhelpful information.
In most cases, however, merely advertising a rulemaking and creating links to the
rulemaking page of the agency’s website or Regulations.gov will not be enough to
generate substantial comments. 425 Such social media outreach may increase public
awareness of rulemaking beyond sophisticated stakeholders but it will not do much to
overcome the “incentive” and “capacity” barriers that prevent most people from
submitting public comments, let alone meaningful public comments. Therefore, such
efforts must be paired with other tools to overcome the disincentives to commenting
before they are likely to produce a substantial improvement in public engagement.
Moreover, if agencies pair social media outreach with tools to enhance the capacity of
the public to submit meaningful comments, it may have the collateral benefit of
reducing the number of unhelpful comments received through mass commenting
campaigns. In addition, as discussed in the next section, agencies may be able to use
natural language processing technologies to help organize and analyze a large number
of public comments.
Finally, agencies that publicize their rulemaking using the Internet and social
media must be careful to avoid running afoul of federal anti-lobbying laws. In 2015,
the Government Accountability Office (GAO) concluded that EPA violated
prohibitions on the use of appropriated funds for “covert propaganda” and “grassroots lobbying.” 426 First, the GAO cited EPA’s use of Thunderclap, “a
‘crowdspeaking’ platform that allows a single message to be shared across multiple
Facebook, Twitter and Tumbler accounts at the same time.” GAO found that EPA
violated a statutory prohibition on “covert propaganda” 427 because EPA’s role in
creating messages such as “clean water is important to me” and “I support EPA’s

423

See HERZ, USING SOCIAL MEDIA, supra note 28, at 7 (“Printing out and mailing a
document is not that hard either.”); ARGIVE, IMPROVING REGULATIONS.GOV: A PERSPECTIVE
FROM SILICON VALLEY 2 (2017) (noting the high volume of fake, duplicative, or
unproductive comments submitted in certain rulemakings through Regulations.gov).
424
See Harper Neidig, John Oliver Urges Net Neutrality Supporters to Tone Down FCC
Comments, HILL (May 15, 2017), https://thehill.com/policy/technology/333440-john-oliverurges-net-neutrality-supporters-to-tone-down-fcc-comments (“When the [FCC] was first
deliberating over [net neutrality rules] in 2014, it was flooded with a record of nearly 4
million comments—in part thanks to Oliver’s high-profile activism on the subject.”).
425
See, e.g., HERZ, USING SOCIAL MEDIA, supra note 28, at 34-35.
426
See generally letter from U.S. Government Accountability Office to Hon. James M.
Inhofe, Chairman, Comm. On Env’t & Pub. Works, U.S. Senate (Dec. 14, 2015) [hereinafter
Letter from GAO]; Eric Lipton & Michael D. Shear, EPA Broke Law with Social Media Push
for Water Rule, Auditor Finds, N.Y. TIMES, Dec. 15, 2015, at A1.
427
Letter from GAO, supra note 426, at 3, 11.

97

efforts” was not clear to the networks of friends and followers who ultimately viewed
the messages in their newsfeeds and dashboards. 428 Second, GAO concluded that
EPA’s creation of hyperlinks to external webpages that contained links to contact
Members of Congress concerning proposed legislation violated the prohibition on
“grass roots lobbying.” 429 A full analysis of the anti-lobbying questions surrounding
use of the Internet and social media is beyond the scope of this study. But agencies
should be cognizant of these concerns as they use the Internet and social media to
publicize rulemakings, educate the public on how to participate, and solicit
information from absent stakeholders.
C.

User-Friendly Rulemaking Dockets

Upon the launch of Regulations.gov in 2003, OMB Director Mitchell E. Daniels,
Jr. predicted the website would “democratize an often closed process and enable
every interested citizen to participate in shaping the rules which affect us all.” 430
Regulations.gov replaced scores of dispersed electronic and paper-based rulemaking
docket systems maintained by each agency with a single, centralized web-based
source for all federal rulemaking dockets. Before the advent of e-rulemaking dockets,
most members of the public interested in learning about an agency’s regulatory
proposal had to go to a library that carried a hard copy of the Federal Register.431
But the library might not receive the Federal Register in a timely manner given the
relatively brief window for public comments, and typically other public comments
could only by accessed in the docket rooms maintained by the agency in Washington,
D.C. 432
The arrival of e-rulemaking dockets, albeit not Regulations.gov per se, has made
it substantially easier for interested members of the public to see what an agency is
doing and to comment on regulatory proposals. But it does not appear to have led to
any meaningful role by citizens in “shaping the rules which affect us all.” 433 As one
scholar has put it, the website “continues to reflect an ‘insider’ perspective—i.e., the
viewpoint of someone familiar with rulemaking and the agencies that conduct it.”434
If you know what you are looking for and have experience commenting in
rulemakings, the website has most of what you need. Otherwise, it does little to
overcome the barriers to participating in rulemaking for novices who manage to

428

See id. at 12-13.
See id. at 17-18.
430
Press Release, Office of Mgmt. & Budget, Regulations.gov to Transform U.S.
Rulemaking Process and Save Nearly $100 Million—New E-Democracy Web Site First Step
in Improving the Quality of Rulemaking Decisions (Jan. 23, 2003) (on file with authors).
431
See Cary Coglianese, E-Rulemaking: Information Technology and the Regulatory
Process, 56 ADMIN. L. REV. 353, 362 (2004).
432
See Coglianese, Citizen Participation, supra note 8, at 949.
433
See HERZ, USING SOCIAL MEDIA, supra note 28, at 8-11.
434
CYNTHIA R. FARINA, COMM. ON THE STATUS & FUTURE OF FED. E-RULEMAKING: A
REPORT TO CONGRESS AND THE PRESIDENT 4 (2008).
429

98

become aware of the rulemaking and are sufficiently interested to make it this far.
This is due in part to the challenges of the underlying rulemaking process and in part
to the technical shortcomings of the website.
When citizens arrive at Regulations.gov, they confront an antiquated user
interface that for the most part has merely moved documents formatted for paper to
an online website. 435 The first thing a visitor sees when selecting a proposed rule on
Regulations.gov is the NPRM. The text of the beginning of the NPRM occupies the
bulk of the web browser page and visitors can scroll down to read the rest. The top of
the web page also includes a link to “Open [the] Docket Folder” and a button to
“Comment Now!,” while the right-hand side of the page includes identifying
information for the regulatory document and docket along with snippets of a few
public comments if such comments have been received and docketed. But the most
prominent part of the page is the text of the NPRM. Therefore, the NPRM can have a
significant impact on whether the user continues to participate in the rulemaking or
directs his or her attention elsewhere.
Unfortunately, NPRMs tend to be long, complex, and quite boring for general
readers. 436 They are usually written at a college or graduate-level of education 437 and
include an abundance of both “legalese,” such as references to statutory authorities,
executive orders, and steps in the administrative process, and detailed descriptions
and analyses of substantive regulatory issues that may be quite technical and
complex. Thus, the text of most NPRMs is littered with terms of art unknown to
individuals inexperienced with either the legal or technical aspects of rulemaking.
NPRMs are often challenging even for those with some experience in rulemaking.
Moreover, in many cases the most relevant information for members of the
general public is buried deep inside the NPRM. For example, the Agricultural
Marketing Service recently sought public comments on its proposal for mandatory
disclosures concerning bioengineered foods. The most valuable information that
consumers can provide the agency likely concerns their reaction to the text and
symbols the agency proposes be required to disclose a bioengineered food, including
their design, color, size, and placement on the packaging. Yet the reader must scroll
through more than 10,000 words of text, roughly 26 pages, before the NPRM
describes the specific appearance and placement of the proposed disclosures.
Moreover, the reader has no idea of the length of the NRPM until she has scrolled
down to the end of it. 438 Finally, while the actual symbols proposed by the agency

435

See id. at 3; (the federal government’s eRulemaking initiative “has been largely
limited to putting existing notice-and-comment processes online”); Farina et al., Rulemaking
2.0, supra note 65, at 418.
436
FARINA & NEWHART, IBM CENTER, supra note 22, at 12; Farina et al. Rulemaking
2.0, supra note 65, at 402, 434.
437
FARINA & NEWHART, IBM CENTER, supra note 22, at 12.
438
We had to copy and paste the NPRM into a Word document to determine its length.

99

include color graphics, the NPRM on Regulations.gov is in black and white. 439 Thus,
merely posting NPRMs on-line does little to overcome the barriers to public
engagement for most rulemaking novices.
If the agency has received and docketed public comments on the proposed rule,
they are accessible on Regulations.gov. This is a huge improvement over the
previous system, which required interested members of the public to visit the agency
to obtain the comments. It is hard to overstate how much this has improved the
accessibility of these documents for those who wish to read them. Nevertheless,
viewing the comments is still a tedious process for most users due to the antiquated
architecture of the website.
As noted above, a few “teaser” comments, or at least the first few lines of them,
are portrayed on the right-hand side of the homepage for the NPRM. When
comments are submitted as attachments, however, there may be nothing to see on the
rulemaking home page other than “See attached file(s).” If the visitor wants to see
any of the comments on the homepage in full, the user must click “View Comment”
below the “teaser” lines to open up a new page with the full comment. If the
comment was submitted as an attachment, however, the visitor must click again to
open the attachment, which may also involve choosing whether to open the file or
save it to a hard drive. Then, if the visitor wants to look at another comment, the
visitor must hit the back button on their browser, return to the rulemaking homepage,
and begin the process again. There is no way to move easily from comment to
comment or search the comments for particular information. Moreover, the only way
to see all the public comments is to select “Open Docket Folder,” which brings the
visitor to a page with links to the primary and supporting documents for the
rulemaking and the public comments. Once again, however, not all the comments are
visible and the visitor must select “View All” to see all the comments, or at least the
first few lines of them. And once again there is no way for a visitor to scroll through
or search the comments. Rather, the visitor must click on the name of the commenter
and download any attachments to read the comment in full. Then to view the next
comment, the visitor must use the back key on the browser, return to the full list of
comments, and begin the process again.
Given the difficulty of using the website, a visitor is likely to become
discouraged, disengaged, or just throw up their hands and select “Comment Now!”
without sufficiently understanding the issues involved or the kinds of information
they possess that would be helpful to the agency. Moreover, once visitors open the
commenting window, they must remember what they have already read on the

439

Color versions of the symbols can be found on the agency’s website at
https://www.ams.usda.gov/sites/default/files/media/ProposedBioengineeredLabels.pdf
and
the agency did ask the public for suggestions and advice regarding the appearance and
placement of required disclosures during the rule development process. See Proposed Rule
Questions Under Consideration, U.S. DEP’T AGRIC., https://www.ams.usda.gov/rulesregulations/gmo-questions#Q2 (last visited Oct. 12, 2018).

100

different pages of the website. 440 It is a wonder that anyone not paid to comment on a
proposed rule or with a strong professional or personal stake in the outcome would
wade through the typical NPRM and decide to participate in the rulemaking. This is
not a website designed for usability by the public. Moreover, it should not be
surprising that many of the comments that are submitted by the general public
through Regulations.gov are not particularly helpful and merely voice the
commenter’s approval or disapproval of the agency’s general proposal or
performance, address themselves to issues outside the agency’s jurisdiction or beyond
the scope of the NPRM, or otherwise do not provide the agency with any actionable
information. Indeed, it would be surprising if the public provided any other types of
comments when confronted with lengthy and complex NPRMs without any focused
questions.
Agencies could improve the experience of using Regulations.gov by revising the
preambles to the NPRMs in the ways proposed above in Part IX.A. Highlighting
specific questions or information the agency seeks at the top of the NPRM and
directing the requests to specific audiences would likely facilitate better comments
from rulemaking novices and members of the general public. But if an agency wants
to enhance participation by certain absent stakeholders, more ambitious efforts are
needed.
A number of agencies and non-governmental organizations have created a more
user-friendly on-line experience for rulemakings in which they sought enhanced
public participation by absent stakeholders. 441 For example, the CFPB created a blog
in connection with its proposed disclosures for prepaid cards that explained the
reasoning behind the NPRM, summarized the proposal, included visual images of the
proposed disclosures, and invited the public to answer specific questions:
Tell us what you think
Now, we want to hear from you! Take a look, and tell us if you think this model form
does a better job of disclosing fee information compared to other forms you’ve seen
on prepaid card packaging. We’re eager to get feedback from consumers, industry,
advocacy organizations, and anyone else who is interested in making prepaid

440

See FARINA & NEWHART, IBM CENTER, supra note 22, at 28.
For example, Syracuse University established a Website early in 2018 entitled
“Climate Comments,” which was designed to provide the public with accessible information
about proposed environmental regulations and to encourage them to submit comments to
EPA. The Website provided plain language summaries of pertinent information about
policies related to climate change that were under consideration, examples of comments both
favoring and opposing relevant proposals, and the opportunity to submit comments via the
website to Regulations.gov for inclusion in the administrative record. See Martin Walls,
‘Climate Comments’ Website Translates Complex Climate Change Policy into Plain
Language, SYRACUSE U. NEWS, MEDIA, L. & POL’Y (Jan. 17, 2018),
https://news.syr.edu/blog/2018/01/17/climate-comments-website-translates-complex-climatechange-policy-into-plain-language/.
The
Website,
which
was
accessible
at
www.climatecomments.org, was no longer in operation when this report was completed.
441

101

account disclosures better.
While you’re looking at the form, some questions to consider might be:


Does the short form disclosure above make it clear how much the account
would cost you to use?



What would you like to see added or changed? Is there some way to make
the information clearer?



Is there anything you find confusing?

We want to get your feedback so that we can consider it as we develop a final rule.
If you want to influence the design of a new prepaid card fee disclosure, let us know
what you think. Submit a comment at Regulations.gov.
To learn more, check out the preamble, the proposed rule, and the official
interpretations. 442

The summary and bullet points direct the reader to the specific issues that would
be useful for members of the public who purchase prepaid cards to address in their
comments. Someone who has bought prepaid cards is more likely to feel like they
have something to add to the rulemaking than if the agency simply summarized the
proposal and invited the public to comment generally. Nevertheless, the consumer
who would like to submit a comment must still brave the Regulations.com website.
The Regulation Room project went a few steps further. The Regulation Room
was an experimental e-rulemaking platform created by the Cornell eRulemaking
Initiative (CeRI) in collaboration with DOT and the CFPB during the Obama
Administration. 443 A cross-disciplinary group of faculty and students summarized in
plain language the significant issues raised in several NPRMs, highlighted the
questions the agency wanted commenters to address, and created a more user-friendly
interface for commenting on the Regulation Room website. 444 Rather than
commenting on the proposal as a whole, the website allowed participants to comment
on the specific issues summarized, and their comments remained attached to those
issues. 445 When a visitor clicked on a particular issue, the website displayed a
column of explanatory text on the left and a column for public comments on the
right. 446 Once the user clicked on a section of the explanatory text, the comments
connected with that section would open in the column on the right. Thus, users could

442

Goldberg, supra note 418.
Farina et al., Rulemaking 2.0, supra note 65, at 397. Professor Cynthia R. Farina of
Cornell Law School served as the Principle Investigator for CeRI, and Mary J. Newhart, also
of Cornell Law School, served as its Executive Director. The CeRI team included professors
of Computing and Information Science and Information Science and Communication, as well
as experts in design, communications, conflict resolution, technology, and e-Government.
444
Farina et al., Rulemaking 2.0, supra note 65, at 412; Farina et al., Rulemaking in 140
Characters or Less, supra note 8, at 390-91.
445
See Farina et al., Rulemaking 2.0, supra note 65, at 412.
446
See FARINA & NEWHART, IBM CENTER, supra note 22, at 30.
443

102

learn about both the NPRM and the comments already filed at the same time. Finally,
in addition to submitting their own comments, participants could “Endorse” or
“Reply” to other comments submitted on the same section of the rule.
FedThread, a now defunct project of the Center for Information Technology
Policy at Princeton University, used a somewhat similar system for commenting. The
FedThread website allowed users to comment on and create a conversation connected
with any paragraph of an NPRM in the Federal Register. 447
In general, it would be useful for agencies to draft summaries of regulatory
proposals and the types of information the agency is most eager to receive from the
public. Agencies have more flexibility with their websites than they do with
Regulations.gov and can use them to create webpages for their rulemakings that are
more accessible to rulemaking novices. Although it is useful for agencies to highlight
the information they want from all stakeholders, it is particularly important when they
seek information from rulemaking novices. Agencies should use their websites to
summarize their proposals and highlight key information they want from different
stakeholders.
At least one agency experimented with similar interfaces before the
centralization of e-rulemaking on Regulations.gov. An IT entrepreneur created a
web-based commenting forum for the National Marine Fisheries Service that broke
up the NPRM into issue areas and allowed the public to submit comments in boxes
that accompanied each issue. The agency also asked questions of commenters. There
was an analytical algorithm that kept track of key words in the comments and allowed
the agency to analyze and categorize the comments. 448
Such agency innovations beyond plain-language summaries appear to be
disfavored in the wake of the centralization and uniformity of Regulations.gov. 449
This is unfortunate. It would be useful for federal agencies to experiment with
different forms of commenting technology to see which ones produce the most
valuable information and meaningful public participation. We suspect, for example,
that commenting on plain language summaries of NPRMs organized by issue would
be more fruitful than the paragraph-by-paragraph comment structure utilized by
FedThread. The latter is unlikely to overcome the challenges of reading NPRMs.
Moreover, issues are likely to span more than one paragraph and some paragraphs
might contain multiple issues.
When agencies suspect that rulemaking novices may be important sources of
information, they should consider experimenting with user-friendly interfaces for
commenting on their NRPMs. As discussed more fully below in Part IX.F, however,

447

See id. at 28-29.
Telephone Interview with Department of Energy (Jan. 19, 2018). The interviewee
formerly worked for the Department of Commerce and had personal experience with the
NMFS’s efforts.
449
See Farina et al., Rulemaking 2.0, supra note 65, at 397-98, 397 n.10.
448

103

the Regulation Room supplemented its user-friendly web architecture with robust
outreach and active moderation by students trained in law and group facilitation
techniques. 450 In addition, the researchers summarized the on-line discussion during
the final weeks of the comment period and invited participants to suggest revisions
before submitting the summary as a formal comment using Regulations.gov. 451
Nevertheless, even without active moderation, encouraging the public to
comment on highlighted issues should focus attention on the most useful information
the public can provide and things within the power of the agency to do in the
rulemaking. In addition, if other public comments on the same section of the NPRM
are grouped together, subsequent visitors may be more likely to engage in a dialogic
exchange. Breaking up NPRMs in this way and encouraging issue-by-issue
comments may produce more comments, but it should also produce more useful
comments and be easier for the agency to review if grouped by topic.
Of course, agencies may be hesitant to encourage robust public participation in
notice and comment and to make it easier to submit public comments for fear that
they will receive more comments than they have the resources to review and
adequately consider. Moreover, reviewing a large number of low-value comments
may cause the agency to miss a few high-value comments. 452 Therefore, agencies
that engage in these efforts to obtain information from a broad group of stakeholders
should think carefully about how they will handle a large number of comments from
public. Michael Livermore and his colleagues explain how natural language
processing technologies can be used to help separate the wheat from the chafe. 453 In
addition, similar technologies could be used to address what they call the forest
problem, 454 whereby officials who review a large quantity of individual comments
have a tendency to miss broader themes or lessons that could potentially be drawn
from the rulemaking record as a whole. 455
In addition to undertaking efforts to facilitate more useful and valuable public
input, which would generally be better informed, more deliberative, and focused on
producing the kinds of information and arguments that improve the quality and

450

Id. at 413.
Id. at 414-15.
452
See Livermore et al., supra note 99, at 981 (“The haystack problem occurs when
comments of high substantive value are hidden within a very large set of documents of lower
substantive value, creating the risk that agencies will fail to locate and appropriately consider
high-value comments.”).
453
See id. at 1017-23 (describing potential uses of sentiment analysis and other
techniques for assessing the “gravitas” of individual comments).
454
See id. at 981 (“The forest problem occurs when agencies are unable to identify
aggregate patterns within a large collection of comments because they treat each comment in
isolation rather than drawing connections between them.”).
455
See id. at 1017-18, 1023-30 (describing potential uses of sentiment analysis and other
techniques for “identifying emergent meaning” from large sets of comments).
451

104

legitimacy of agency decision making, agencies should also strive to make the most
of the comments they receive. 456 This means that agencies should consider using the
best available natural language processing technologies to help them identify
comments that contain useful data, situated knowledge, or analytical arguments, so
that they can respond to those comments in an adequate fashion. In addition,
agencies should consider using this technology to identify broader themes or lessons
from the rulemaking record as a whole, which might inform their decision-making in
appropriate circumstances.
A few simple upgrades to Regulations.gov would make it easier for the public to
comment and may have the added benefit of improving the quality of comments.
Nevertheless, they are unlikely to be enough to obtain robust participation and
valuable information from unsophisticated stakeholders. Rather, in those cases in
which agencies identify rulemaking novices as potentially unique sources of relevant
information, agencies should consider creating rulemaking portals designed for these
groups.
D.

Effective Commenting Tutorials

In addition to organizing NPRMs by issue, summarizing the issues in plain
language, highlighting key questions, and creating a better user-interface for
commenting, effective commenting tutorials can help rulemaking novices understand
the goals of the rulemaking process more generally and the nature of effective
comments. Regulations.gov has made some effort in this direction, but more could be
done.
Visitors to Regulations.gov who select “Comment Now!” to open the
commenting page may then select a “View Commenter’s Checklist (PDF)” link to a
three-page “Tips for Submitting Effective Comments.” 457 Unfortunately, the “Tips”
begin unhelpfully by suggesting, “[a] comment can express simple support or dissent
for a regulatory action.” 458 Although the next sentence explains that an “informationrich comment that clearly communicates and supports its claims is more likely to
have an impact” and Recommendation No. 8 explains that “[t]he comment process is
not a vote,” the first sentence suggests that comments merely supporting or opposing
a proposal have some value. 459 In fact, such comments likely have little to no value
for the agency. 460 The document does contain some helpful advice, such as “Base

456

See id. at 992-93 (suggesting that agencies can deal with the challenges posed by
mass comments both by undertaking efforts “to improve their sophistication” and by
attempting “to extract more meaning from comments in their current form”).
457
Tips
for
Submitting
Effective
Comments,
REGULATIONS.GOV,
https://www.regulations.gov/docs/Tips_For_Submitting_Effective_Comments.pdf
(last
visited Nov. 17, 2018).
458
Id. at 1.
459
Id. at 1-2.
460
See supra Part III.C (discussing debate over the value of mass comments).

105

your justification on sound reasoning, scientific evidence, and/or how you will be
impacted”; “Identify credentials and experience that may distinguish your comments
from others”; and “support your comment with substantive data, facts, and/or expert
opinions. 461 You may also provide personal experience … as may be appropriate.”462
But these are mixed in with tips that may be beyond the capacity of many
unsophisticated stakeholders (e.g., “[a]ttempt to fully understand each issue” in the
NPRM); or are misleading (e.g., “agencies … appreciate all comments”). 463
The Regulation Room YouTube video, “What is Effective Commenting?,” is
more helpful. First, visual media are likely more engaging for rulemaking novices
than a three-page, text-based pdf. Second, the video immediately directs viewers to
consider the information that the agency needs to make its decision—i.e., “facts,
logical argument, good reasons, things that Congress tells [the agency] it must pay
attention to”—and then explains how an effective comment states a position and
provides evidence through statistics, personal experience, or a story about why a
regulation is or is not useful. The video repeatedly points viewers to the importance
of relevant experience and provides examples of the types of experiences that might
be relevant to certain rulemakings. In addition, the video repeatedly emphasizes that
the commenting process is not a vote.
Similarly, the text version of “What is effective commenting?” begins by
highlighting that agencies are not “allowed to decide based on majority vote. Instead,
they are supposed to study the problem, collect information, and use expertise,
experience, and good judgment to come up with the overall best answer.”
Accordingly, Regulation Room advises: “the best comments explain not only what
the agency should do, but why.”
Agencies should consider developing their own effective commenting tutorials
using visual media connected to their e-rulemaking portals. Although the basic
principles of effective commenting are generalizable across diverse subject areas,
agencies that develop their own tutorials could highlight examples of effective
comments from their own experiences. Such examples would likely illustrate the
principles of effective commenting to the agency’s stakeholders more successfully
than examples drawn from distinct and unrelated regulatory areas. Agencies could
also use the videos to highlight aspects of their NPRMs that they use to seek
information from rulemaking novices or absent stakeholders. For example, if an
agency highlights key questions for different groups at the top of their NPRMs, the
effective commenting tutorial could tell the viewer to look for this section of the
NPRM.

461

Tips for Submitting Effective Comments, supra note 457, at 1-2.
Id. at 2.
463
Id. at 1-2.
462

106

E.

Public Meetings

Agencies have long used in-person public meetings to enhance public
engagement in notice-and-comment rulemaking. The APA does not require agencies
to hold a public meeting in connection with informal rulemaking; rather the agency
can fulfill its obligation to “give interested persons an opportunity to participate in the
rule making . . . with or without opportunity for oral presentation.” 464 But many
agencies held public meetings in connection with some of their rulemaking prior to
the enactment of the APA, and they continue to do so today. In addition, an agency’s
organic statute may require the agency to hold a public hearing in connection with
certain rulemakings. 465
Although a public meeting can take a variety of forms, this section addresses
meetings that, using the IAP2 Levels of Public Engagement, are held to “inform,”
“consult,” and “involve” (in a limited way) the public in rulemaking. Thus, they
allow for more involved public participation than the paper notice-and-comment
process itself. Nevertheless, we address public meetings seeking even more robust
public involvement and collaboration in Part IX.F (Supplemental Deliberative
Exercises) and XI (Enhanced Forms of Deliberation).
Agencies generally supplement the paper rulemaking process with public
meetings when the rulemaking is expected to have a significant impact on the public
or may prove politically controversial. 466 A public meeting provides a more informal
setting for the agency to explain what it is doing, field questions about the NPRM,
and grapple with the views and concerns of stakeholders. Clarifying ambiguities in
the NPRM also helps the public to submit better comments and may lead the agency
to revise and clarify the final rule. When the issues are “big and complicated,” public
meetings also allow the agency to hear about potential unintended consequences and
other stakeholder concerns. In-person meetings offer a dynamic environment in
which the agency can probe and clarify public comments and ask participants for
examples and evidence to support their assertions or concerns. 467
Today, agencies often use conference calls and/or “virtual” online meetings.
These enable the agency to reach people beyond Washington, D.C., without going
into the field, especially if the agency does not have a network of field offices. For
example, GSA held a “town hall” after publishing an NPRM concerning the disposal
of electronic materials. A private company, Broadnet, helped the agency with the
logistical setup for the town hall. The GSA alerted stakeholders about the town hall
using e-mail blasts and attempted to reach a broad spectrum of people. The town hall
took place on the phone and was similar to calling into a radio talk show. Thousands

464

Administrative Procedure Act, 5 U.S.C. § 553(c) (2012).
See, e.g., 15 U.S.C. § 57a(b)(1) (2012) (requiring the Federal Trade Commission
(FTC) to hold an informal hearing in connection with rulemaking).
466
See KERWIN & FURLONG, supra note 2, at 83.
467
Telephone Interview with Pension Benefit Guarantee Corporation (Nov. 21, 2017).
465

107

of people participated. GSA began the call by explaining what it was trying to
accomplish. Screeners then answered calls from participants, recorded basic
information about the participants, and placed them in an electronic queue. The GSA
moderator selected which calls to answer based on their perceived value to the rest of
the participants. The town hall lasted about an hour. Those who did not have an
opportunity to ask questions at the town hall were encouraged to submit comments or
to reach out to the GSA with their questions. GSA felt the town hall was helpful
because it confirmed what the GSA knew from other discussions.
The Department of Energy conducts formal meetings in connection with noticeand-comment rulemaking pursuant to 42 U.S.C. § 6393(a)(3). The agency gives
presentations on the NPRM and encourages participants to ask questions. The agency
uses a facilitator and typically obtains robust public participation, both in person and
via webinar. The transcripts of the meetings are included in the rulemaking record.
As discussed above, the Forest Service conducted National and Regional Forums
as part of the 2012 Planning Rule process. 468 Participants were told to submit any
formal comments in writing as part of the notice-and-comment rulemaking, but the
agency received feedback on different aspects of the proposed rule that may have had
some impact on the final rule. For example, several participants wanted more
concrete standards and guidelines in the Forest Plans.
Agencies that hold a public meeting must decide whether to conduct the meeting
in-person or remotely using the web and/or a call-in system. “Virtual” meetings have
the advantage of being able to reach a broader audience than a few in-person
meetings. Even if in-person meetings are held at different locations around the
country, there will likely be many people who cannot attend. Moreover, if the agency
can queue questions remotely it may be able to field more questions than would be
possible in a public meeting of the same length held in person. The town hall format
used by GSA is sufficient to understand stakeholders’ questions and clarify what the
agency is doing and why. It may also allow the agency to probe comments and
concerns and ask for more information. It probably does not lend itself as well to
deliberative exercises in which participants respond to each other.
Finally, public meetings are an additional opportunity for the agency to
encourage the public to submit comments in the rulemaking itself and provide
rulemaking novices with resources for submitting public comments. Agencies should
always remind participants that they need to submit their comments in the notice-andcomment proceeding to make them part of the record.
F.

Supplemental Deliberative Exercises

Agencies may also want to conduct deliberative exercises to supplement the
notice-and-comment process. Some of the best examples of online deliberative

468

See text accompanying supra note 243.

108

exercises conducted in connection with rulemaking come from Regulation Room, the
pilot e-rulemaking platform created by the Cornell eRulemaking Initiative.
The basic online commenting interface of Regulation Room is discussed above in
Part IX.C. The Regulation Room team summarized several key issues in connection
with each rulemaking and participants were able to comment issue-by-issue and
respond to the comments of others on the same topic. These summaries and the
interface helped to overcome the barrier to participation for rulemaking novices when
they are confronted with a long, dense, and complex NPRM and then must remember
what they have read when submitting comments on Regulations.gov. The Regulation
Room website also provided visitors with helpful and engaging tutorials on the
comment process and the nature of effective comments.
This web-based
infrastructure was at the heart of the project. But there were several other
components that helped it succeed at furthering the information and democratic
functions of rulemaking.
First, Regulation Room engaged in substantial targeted outreach to attract
rulemaking novices with relevant knowledge to the Regulation Room website.
Unfortunately, even if agencies build the most user-friendly commenting interface
imaginable, “[b]uilding it will not be enough to make new participants come.” 469
Accordingly, once agencies have identified missing stakeholders, they must tailor
their outreach efforts to reach the particular group. The best practices and challenges
of designing outreach efforts are discussed below in Part XII.
Second, a Regulation Room team member trained in law and group facilitation
moderated the public comments on the proposed rules. The moderators facilitated the
conversation in a number of ways. They expressed appreciation for comments; asked
for clarifications, more details, personal experiences, or other factual support for
comments; provided additional information relevant to comments that the participant
might not know; focused the discussion on the issues in the rulemaking; referred the
participant to other rulemakings when the comments were off-topic; encouraged
participants to respond to the comments of others; monitored the discussion for
inappropriate content; and helped resolve technical difficulties with the website. 470
Third, because the agencies did not want the comments on the Regulation Room
website submitted wholesale into the rulemaking record, the Regulation Room team
created a draft summary of the discussion during the final weeks of the comment
period. They then emailed the draft summary to registered users and invited them to
suggest revisions. The summary was then finalized and submitted as a formal
comment using Regulations.gov. 471

469

FARINA & NEWHART, IBM CENTER, supra note 22, at 21.
See id. at 34; Farina et al., Rulemaking 2.0, supra note 65, at 413-14; Farina et al.,
Rulemaking in 140 Characters or Less, supra note 8, at 391.
471
Farina et al., Rulemaking 2.0, supra note 65, at 414.
470

109

These aspects of Regulation Room went a long way towards overcoming the
incentive and capacity barriers to commenting by rulemaking novices. But extensive
outreach, active moderation, and synthesizing and summarizing diverse comments is
also very resource intensive. It likely requires substantially more resources than
merely breaking an NPRM into digestible units with plain language summaries. In
addition, agencies themselves may not be comfortable conducting these exercises or
summarizing the outcome of a deliberative online discussion with a diversity of
views. Agencies may therefore want or need to hire an outside facilitator to provide
these services.
Agencies should consider whether these additional resources are well spent in
each rulemaking they undertake. In most cases such extensive efforts will likely not
be worth the costs. In some cases, however, they will be. Regulation Room did
enhance participation by missing stakeholders in the rulemaking process. More than
ninety percent of the participants reported that they had never participated in
rulemaking before. 472 In addition, the agencies received helpful information and
perspectives from these stakeholders that they might not otherwise have heard. For
example, during the DOT rulemaking concerning the accessibility of airport check-in
kiosks and travel websites to individuals with physical and cognitive disabilities,
organizations representing persons with disabilities uniformly sought the increased
independence offered by accessible technologies. But some individuals with
disabilities were concerned that requiring accessible technologies would result in
fewer airline agents who could assist travelers and adapt to their particular needs. 473
In the DOT rulemaking proposing the installation of EOBR equipment, large trucking
companies, most of whom already used the technology, generally supported the
proposal, while small trucking companies and independent commercial motor vehicle
drivers, most of whom did not use technology, generally opposed the proposal. 474
Among other things, the commercial drivers and small companies noted that it would
be more difficult for them to absorb the costs of installing the equipment and
suggested that the agency’s cost-benefit analysis was “skewed toward a big business
model.” 475 In addition, independent commercial drivers and small trucking
companies conveyed “rich and nuanced detail of individual experiences and
operations[, illuminating] a variety of business practices around completing driving
logs and … concerns about inflexibility.” 476
Other agencies have conducted more limited deliberative exercises online to
supplement the notice-and-comment process. In 2001, EPA worked with Information
Renaissance to conduct a two-week online consultation experiment using “computer

472

See Farina et al., Rulemaking in 140 Characters or Less, supra note 8, at 392.
See Farina et al., Knowledge in the People, supra note 5, at 1204-05; FARINA &
CERI, IBM REPORT, supra note 70, at 17.
474
See Farina et al., Knowledge in the People, supra note 5, at 1200, 1214 n.106.
475
Id. at 1200.
476
FARINA & CERI, IBM CENTER, supra note 70, at 17.
473

110

bulletin boards to foster a deliberative dialogue among a limited public interested in
participating in the rulemakings at issue.” 477
In 2010, FCC used OpenInternet.gov and the IdeaScale application to facilitate
public engagement with open Internet rulemaking. 478 The FCC held workshops and
meetings “in a way that facilitated broad and geographically dispersed access,” using
social media, and using its special websites on broadband policy and open Internet as
a way to generate ideas. 479 OpenInternet.gov featured a “video portal for public
workshops on key aspects of the open Internet rulemaking.” 480 The workshops could
be watched live and those watching could ask questions using email or twitter. The
workshops were also available by calling a toll-free telephone number and accessible
to persons with hearing or sight difficulties. In addition, IdeaScale allowed the public
to provide ideas, comment on other people’s ideas, and vote for ideas and other’s
thoughts. These comments were then included as part of the rulemaking record. 481
The Nuclear Regulatory Commission’s RuleNet Pilot project in 1996 was an
early online consultation experiment using a “computer bulletin board[] to foster a
deliberative dialogue among a limited public interested in participating in the
rulemakings at issue.” 482
G.

Reply Comment Periods

Agencies generally have discretion to provide a “reply comment period” to give
interested parties an opportunity to address the information, arguments, and evidence
submitted by others during the comment period required by section 553 of the
APA. 483 Reply comment periods are useful because of the tendency for stakeholders
to submit materials on the last day of the comment period. This may be strategic—to
avoid having their arguments or evidence challenged by other participants in the
rulemaking—or merely the predictable effect of action-forcing deadlines. Regardless
of the causes, the inability of stakeholders to reply to the comments of others can
impair the quality of information generated by the public comment period.
477

Noveck, The Electronic Revolution, supra note 48, at 470 n.181 (citing THOMAS C.
BEIERLE, DEMOCRACY ON-LINE: AN EVALUATION OF THE NATIONAL DIALOGUE ON PUBLIC
INVOLVEMENT IN EPA DECISIONS 12 (2002)).
478
See Peter M. Shane, Empowering the Collaborative Citizen in the Administrative
State: A Case Study of the Federal Communications Commission, 65 U. MIAMI L. REV. 483,
490, 495-96 (2011).
479
Id. at 495.
480
Id. at 496.
481
The
IdeaScale
website
for
this
project
can
be
visited
at
http://openinternet.ideascale.com/. One proposal received 467 votes. Quite a few ideas
received several comments.
482
Noveck, The Electronic Revolution, supra note 48, at 470 n.181.
483
The APA does not require or even mention reply comment periods, but they are
occasionally required by any agency’s organic statute. See, e.g., Clean Air Act, 42 U.S.C.
§ 7607(d)(5) (2012).

111

Reply comment periods should generally be offered as a matter of course for
several reasons. First, agencies are likely to receive better information and evidence
in the rulemaking process when submissions can be contested during a reply
comment period.
Without a reply comment period, parties in possession of
information that supports or refutes comments submitted at the end of the comment
period may not have a chance to add their insights to the rulemaking record.
Moreover, commenters who know others cannot challenge them may make less
persuasive or reliable claims than they would if they knew that other parties could
present counter-evidence. Thus, a reply comment period not only adds relevant
information to the record but also may improve the quality or veracity of the
information that is otherwise received. At a minimum, a reply comment period
should provide the agency with a better record from which to assess the strength of
the different claims made and evidence submitted during the notice-and-comment
process.
Second, reply comment periods can create a more dialogic process, refining
stakeholder positions and creating opportunities to forge compromise or reach
consensus. There is some evidence that commenters make maximalist claims when
they know others will not have an opportunity to respond to them in the
rulemaking. 484 If initial comments may be challenged during a reply period,
stakeholders are more likely to focus on their core interests. This makes it easier for
agencies to understand how regulations might impact stakeholders and how they will
respond to various regulatory alternatives.
The Administrative Conference has addressed reply comment periods in several
prior recommendations. 485 The most recent survey suggests that reply comment

484

See STEVEN J. BALLA, PUBLIC COMMENTING ON FEDERAL AGENCY REGULATIONS:
RESEARCH ON CURRENT PRACTICES AND RECOMMENDATIONS TO THE ADMINISTRATIVE
CONFERENCE OF THE UNITED STATES 12 (2011).
485
See Admin. Conf. of the U.S., Recommendation 2011-2, Rulemaking Comments, 79
Fed. Reg. 48,789, 48,791, 48,791 (Aug. 9, 2011); Admin. Conf. of the U.S.,
Recommendation 76-3, Procedures in Addition to Notice and the Opportunity for Comment
in Informal Rulemaking, 41 Fed. Reg. 29,653, 29,654, 29,655 (July 19, 1976) (explaining that
agencies should consider “a second cycle of notice-and-comment or by notice providing an
opportunity for additional comment . . . when comments filed in the proceeding, or the
agency’s response to such comments, present new and important issues or serious conflicts of
data”); Admin. Conf. of the U.S., Recommendation 72-5, Procedures for the Adoption of
Rules of General Applicability, 38 Fed. Reg. 19,782, 19,792 (July 23, 1972) (recommending
that agencies should consider providing an “opportunity for parties to comment on each
other’s written or oral submissions”); see also Memorandum from Cass R. Sunstein, Adm’r,
Office of Info. & Regulatory Affairs to Heads of Executive Departments and Agencies, and
of
Independent
Regulatory
Agencies
(Feb.
2,
2011),
https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/memoranda/2011/m11-10.pdf
(“[Executive Order 13,563] seeks to increase participation in the regulatory process by
allowing interested parties the opportunity to react to (and benefit from) the comments,
arguments, and information of others during the rulemaking process itself.”).

112

periods are used in a “rather small fraction” of rulemakings. 486 The Federal
Communications Commission (FCC) conducts a significant number of these. 487 Yet
agencies that have used reply comment periods report significant benefits. Professor
Balla’s study found that parties use reply comments both to agree with and to
challenge prior arguments and information. His interviewees at the FCC offered
positive assessments of reply comment periods. Our own interviews confirmed their
general value.
One downside to reply comment periods is that they prolong the rulemaking
process and require additional resources for the agency to manage. 488 The first
concern is unlikely to be a problem in most cases. Professor Balla found that most
reply comment periods only extended the notice-and-comment process a few weeks,
perhaps one month beyond the end of the initial comment period. None extended the
comment period more than six weeks. Given that rulemaking proceedings take, on
average, about two years from publication of the NPRM until publication of the Final
Rule, 489 an extension of a few weeks seems negligible. More time may be necessary
to give reply commenters a meaningful opportunity to review and respond to initial
comments. But in most cases a month would likely suffice unless the reply
commenters are doing original research.
The second concern is perhaps more acute. The agency will need to review the
second set of comments and formulate its responses. This takes additional time on
the part of agency staff. Theoretically, the agency would only have to review
comments germane to the topics raised in the original comment period, rather than on
new or unrelated topics. But such a policy may be unworkable, and the agency will
need to review all the comments submitted during a reply period as a practical matter.
Still, agencies may be able to utilize some of the same natural language processing
technologies discussed above in Part IX.C. In addition, the additional time and
resources required to manage reply comment periods would be offset by reply
comments that help the agency draft its own responses to comments in the preamble
to the Final Rule. Thus, prolonging the comment period by a few weeks on the front
end may save the agency time on the backend.
Therefore, agencies should consider whether a reply comment period will help
them assess the information they receive in the required APA comment period or
otherwise prove beneficial. When the agency seeks to obtain information from absent
stakeholders and members of the general public at this stage, the agency should
highlight in plain language the issues raised during the APA comment period upon
which it seeks additional comments and why.

486

See BALLA, supra note 484, at 10.
See id.
488
See id. at 12-13.
489
Anne Joseph O’Connell, Political Cycles of Rulemaking: An Empirical Portrait of the
Modern Administrative State, 94 VA. L. REV. 889, 959 (2008).
487

113

H.

Status Reports and Notifications

Several agencies have begun to take advantage of new technology to keep
stakeholders informed of developments in their rulemaking proceedings. For
example, DOT allows members of the public to sign up for monthly status reports and
notifications with links to relevant rulemaking documents. DOT’s groundbreaking
efforts are discussed above, 490 and we recommend that other agencies consider
adopting similar practices.
I.

Acknowledgment and Impact Reports

Agencies should acknowledge the comments they receive from rulemaking
novices and members of the general public during the notice-and-comment process.
Such acknowledgement reinforces the value of public participation and encourages
newcomers to remain engaged with the agency. Therefore, if the agency solicits
feedback from rulemaking novices during the notice-and-comment process, the
agency should acknowledge their contributions in the preamble to the final rule. This
includes addressing the substance of their comments and highlighting how the
comments had an impact on the final rule, either by effecting changes to the final rule
or confirming the appropriateness of the agency’s proposed solution to the regulatory
problem.
X. RETROSPECTIVE REVIEW
Retrospective review seeks to determine whether existing agency rules are
achieving their intended goals or are obsolete, ineffective, or excessively burdensome
due to faulty assumptions, changed circumstances, or unanticipated consequences. In
addition, retrospective review may reveal that a rule is redundant or
counterproductive because it overlaps or conflicts with another rule enforced by the
same or a different agency. 491 Depending on what is learned from reviewing a
particular rule, the agency may decide to modify, streamline, expand, or repeal the
rule.
As early as 1946, the APA granted the public a “right to petition for the issuance,
amendment, or repeal of a rule.” 492 But interest in more systematic and regular
review of existing regulations has grown since the 1970s. Beginning with Jimmy
Carter, retrospective review or periodic “regulatory lookbacks” became a staple of

490

See supra Part VII.E.2.
See, e.g., Cary Coglianese, Taking Regulation Seriously, REG. REV. (Jan. 28, 2012),
http://www.theregreview.org/2012/01/28/taking-regulation-seriously (describing EPA’s
repeal of a regulation of dairy farmers that overlapped with the regulations of the Department
of Agriculture).
492
Administrative Procedure Act, 5 U.S.C. § 553(e) (2012).
491

114

presidential campaigns to improve the function of the administrative state. 493 In
addition, Congress has required retrospective review of certain types of regulations.
For example, the Regulatory Flexibility Act of 1980 (RFA) requires agencies to
review rules that have a “significant economic impact” upon small businesses within
ten years of the publication of the final rule. 494 The Act directs agencies to consider:
(1) the continued need for the rule;
(2) the nature of complaints or comments received concerning the rule
from the public;
(3) the complexity of the rule;
(4) the extent to which the rule overlaps, duplicates or conflicts with
other federal rules, and, to the extent feasible, with state and local
governmental rules; and
(5) the length of time since the rule has been evaluated or the degree to
which technology, economic conditions, or other factors have
changed in the area affected by the rule. 495
Furthermore, several laws require specific agencies to conduct periodic reviews of
specific types of rules. 496 More generally, the Government Performance and Review
493

See JOSEPH E. ALDY, LEARNING FROM EXPERIENCE: AN ASSESSMENT OF THE
RETROSPECTIVE REVIEWS OF AGENCY RULES AND THE EVIDENCE FOR IMPROVING THE
DESIGN AND IMPLEMENTATION OF REGULATORY POLICY 4 (2014) (“Every administration
dating back to the Carter Administration in 1978 has implemented some form of regulatory
look-back.”); see also Exec. Order No. 13,771, 82 Fed. Reg. 9,339 (Jan. 30, 2017) (requiring
agencies under the Trump Administration to identify two existing regulations to be repealed
for every new regulation proposed); Exec. Order No. 13,563, 76 Fed. Reg. 3,821 (Jan. 18,
2011) (requiring agencies to develop a plan to “periodically review its existing significant
regulations”); Exec. Order No. 12,866, 58 Fed. Reg. 51,735 (Sept. 30, 1993) (directing
agencies to periodically review their existing significant regulations); Memorandum from
George H.W. Bush, President of the U.S., to Certain Department and Agency Heads (Jan. 28,
1992),
https://www.gpo.gov/fdsys/pkg/PPP-1992-book1/pdf/PPP-1992-book1-doc-pg1662.pdf (requiring agencies to review existing regulations and eliminate those that impose any
unnecessary regulatory burden); Exec. Order No. 12,498, 50 Fed. Reg. 1,036 (Jan. 4, 1985)
(requiring agencies to identify existing regulations they plan to rescind or revise); Exec.
Order No. 12,044, 43 Fed. Reg. 12,661 (Mar. 24, 1978) (requiring agencies to “periodically
review their existing regulations”).
494
Regulatory Flexibility Act of 1980, 5 U.S.C. § 610(a) (2012) (“[E]ach agency shall
publish in the Federal Register a plan for the periodic review of the rules issued by the
agency which have or will have a significant economic impact upon a substantial number of
small entities.”).
495
Id. § 610(b).
496
See, e.g., Dodd-Frank Wall Street Reform and Consumer Protection Act, 12 U.S.C.
§ 5512(d) (2012) (directing the CFPB to conduct a 5-year review of “each significant rule or
order adopted by the Bureau under Federal consumer financial law”); Clean Air Act

115

Act of 1993 (GPRA) requires agencies to develop strategic and performance plans to
establish and track progress towards their regulatory goals. 497
Finally, many agencies have promulgated their own policies and procedures
requiring regulatory lookbacks. 498 These reviews typically focus on “the age of a
rule, its economic impact, the burden it poses on industry, whether it duplicates
another federal requirement, and whether it is inconsistent with a change in a law or
administration policy.” 499
The Administrative Conference sponsored studies of retrospective review and
issued recommendations based on those studies in 1995 500 and 2014. 501 Although
this work focused largely on analytical approaches to retrospective review, it included
several recommendations concerning public involvement in regulatory lookbacks. In
2014, ACUS recommended:
•

leveraging outside expertise in retrospective review;

•

“using social media, as appropriate, to learn about actual experience
under the relevant regulations(s)”;

•

disclosing relevant data analyzing existing regulations on
“Regulations.gov, [agencies’] Open Government webpages, and/or other
publicly available websites[,]” and doing so in a way that “allow[s]
private parties to recreate the agency’s work and to run additional
analyses”; and

•

encouraging private parties to submit their own information and analyses
and integrating this information, where relevant, into the agency’s
review. 502

Amendments, 42 U.S.C. § 7409(d)(1) (2012) (requiring review every five years of national
ambient air quality standards).
497
See, e.g., 5 U.S.C. § 306; 31 U.S.C. § 1115(a) (2012).
498
See SHAPIRO, supra note 67, at 416 (describing some examples); see also Regulatory
Review of Existing DOT Regulations, 76 Fed. Reg. 8,940, 8,941 (Feb. 16, 2011) (“The
Department follows a repeating 10-year plan for the review of our existing regulations . . . .
[The reviews] comply with section 610 of the Regulatory Flexibility Act. . . . Generally, the
agencies have divided their rules into 10 different groups and analyze one group each year,
then start over again. We regularly invite public participation in those reviews and seek
general suggestions on rules that should be revised or revoked.”). The FAA, NHTSA, and
other agencies within DOT conduct additional periodic reviews of existing regulations. Id.
499
SHAPIRO, supra note 67, at 416.
500
See generally id.; Admin. Conf. of the U.S., Recommendation 95-3, Review of
Existing Agency Regulations, 60 Fed. Reg. 43,108, 43,109 (Aug. 18, 1995).
501
See generally ALDY, supra note 493; Admin. Conf. of the U.S., Recommendation
2014-5, Retrospective Review of Agency Rules, 79 Fed. Reg. 75,114, 71,114 (Dec. 17, 2014).
502
Retrospective Review of Agency Rules, 79 Fed. Reg. at 75,117.

116

In 1995, ACUS recommended:
•

soliciting public input using requests for comment, agency ombudsmen,
federal advisory committees, press releases and public notices, roundtable
discussions, and “requesting comments through electronic bulletin boards
or other means of electronic communication.”

•

ensuring adequate and timely responses to petitions for review under
APA section 553(e). 503

The ABA has also made recommendations regarding retrospective review. Most
relevant for purposes of this Report, in 2016 the ABA House of Delegates
recommended that Congress amend the APA to promote retrospective review by
requiring agencies:
a. When promulgating a major rule, to publish a plan (which would
not be subject to judicial review) for assessing experience under
the rule that describes (i) information the agency believes will
enable it to assess the effectiveness of the rule in accomplishing its
objectives, potentially in conjunction with other rules or other
program activities, and (ii) how the agency intends to compile such
information over time; [and]
b. On a continuing basis, to invite interested persons to submit, by
electronic means, suggestions for rules that warrant review and
possible modification or repeal. 504
Ensuring that regulations remain necessary and continue to provide cost-effective
benefits is undeniably important. Nevertheless, retrospective review inevitably pulls
resources from other agency activities. Even if the agency identifies a rule in need of
amendment or repeal, in the process it may delay or lose the opportunity to proceed
with a more important regulatory initiative. Thus, agencies must carefully weigh the
potential benefits of retrospective review compared with other agency activities, at
least when it is within their discretion to choose how to use their limited resources.
We endorse the recommendations of ACUS and the ABA described above and
offer additional recommendations focused on enhancing participation in retrospective
review by traditionally absent stakeholders and members of the general public. These
groups are a potentially rich source of information for retrospective review. Those
who live with a regulation, whether as beneficiaries or regulated stakeholders, are
likely to have the best information about whether the rule is understandable, the ease
or difficulty of complying with the rule, whether the rule is achieving its intended
goals, and any unintended consequences that may have emerged over time.

503
504

Review of Existing Agency Regulations, 60 Fed. Reg. at 41,110.
ABA Res. 106b (2016).

117

Before turning to the tools of public engagement, it is important to note a
difference between retrospective review and prior stages in the rulemaking process.
Unlike a proposed rule, a “well-run agency is constantly, ‘informally’ reviewing its
regulations” as part of its daily operations. 505
Informal reviews are a routine, daily occurrence in which, during the general
operations of the agency, problems with existing rules are identified that
may warrant further action. Investigators and others who work with the
regulated parties may note a continuing problem in implementing rules;
attorneys may note problems in enforcing, interpreting, or litigating over
rules; and accidents, congressional interest, media interest, and other events
may result in discussions within an agency that may, in turn, result in a
decision to change rules. 506
Thus, even without a formal process for retrospective review, most agencies
continually receive feedback on how their regulations function. Indeed, some
agencies report they learn more about how a rule is working from their day-to-day
operations than from formal requests for comments on the effectiveness of their
rules. 507
Accordingly, we focus on ways that agencies can foster public review of existing
regulations outside of agencies’ day-to-day work with regulated entities and other
members of the public. Some of these modes of engagement supplement informal
review by providing on-going opportunities for public input. Others are focused on
enhancing public participation in formal reviews or lookbacks that an agency
conducts in connection with a legal mandate or on its own initiative.
A.

“Open” or “Living” Rulemaking Dockets

If agencies want to institutionalize a culture of retrospective review, they should
consider utilizing “open” or “living” rulemaking dockets. 508 This is consistent with
the ABA recommendation that agencies provide an ongoing electronic means for the
public to submit “suggestions for rules that warrant review and possible modification
or repeal.” 509 Open rulemaking dockets can be designed in a number of ways and
mounted either on an agency’s website or a re-designed Regulations.gov. The goals
are to provide an ongoing way for the public to comment on existing regulations and
to connect those comments to the rulemaking docket established during the noticeand-comment process.

505

Eisner & Kaleta, supra note 336, at 146 (emphasis added); see also Wendy Wagner et
al., Dynamic Rulemaking, 92 N.Y.U. L. REV. 183 (2017) (reporting that most rules
promulgated by agencies revise or update existing regulations).
506
Eisner & Kaleta, supra note 336, at 147.
507
See id. at 148-49.
508
See ARGIVE, supra note 423, at 8.
509
ABA Res. 106b (2016).

118

The close of comment periods before the promulgation of final rules precludes
comments on rules as implemented. Moreover, the limited time frame during which
agencies accept public comments on retrospective review using RFIs, NPRMs, and
similar devices can discourage meaningful assessments of the effectiveness of
existing regulations. As noted in other contexts, the announcement of a deadline for
comments prior to an agency decision whether to amend or repeal a regulation may
encourage commenters to take maximalist positions for or against the regulation
rather than focusing on how the regulation is working, providing examples of
problems or unanticipated consequences, acknowledging what works, or articulating
precise recommendations on how the rule might be improved.
In addition, seeking comments on existing regulations during discrete time
periods may discourage stakeholders affected by a rule from commenting when they
have relevant experiences. In the case of stakeholders who do not routinely
participate in the regulatory process, the likely result of having to wait to comment
until a subsequent regulatory lookback is that they will never comment at all. Thus,
the public should have an open and accessible means to comment on regulations
when they have experiences or other information that would be relevant.
Moreover, general requests for suggestions on which regulations should or
should not be amended or repealed can result in blanket statements complaining about
the burdens of regulations rather than specific actions that an agency might take to
improve existing regulations. 510 Maintaining an open rulemaking docket on existing
regulations encourages the public to comment on the specific regulations in need of
attention and why.
Furthermore, connecting retrospective review with the notice-and-comment
rulemaking docket already created provides the public with a wealth of existing
information and focuses attention on whether the assumptions or evidence upon
which the rule was based remain accurate. This should encourage a more dialogic
commenting process. For example, if a rule was based on certain predicted costs and
benefits, commenters could submit evidence directed at whether or not those
predictions have proved correct.
For such engagement to be realistic for most rulemaking novices, the erulemaking dockets maintained by Regulations.gov or on the agency’s website must
be made more user friendly in the ways recommended in Part IX.C. But this is just a
start. A few additional steps are required to convert the e-rulemaking dockets
recommended in Part IX.C into open rulemaking dockets that facilitate retrospective
comments.

510

See, e.g., ARGIVE, supra note 423, at 8 (describing how the Department of
Commerce’s RFI seeking information on the impact of federal permitting requirements on the
construction and expansion of domestic manufacturing facilities and on regulations that
adversely impact domestic manufacturers encouraged commenters to respond categorically
and repeat “goals of the current administration by referencing international competitiveness
and job rates, or criticizing a rule formed during the Obama administration.”).

119

First, the e-rulemaking docket for a regulation should highlight the specific types
of information the agency believes would be helpful to assess the success of the rule.
Agencies already highlight the basis and purpose for their rules in the preambles to
their final rules. Just as we recommend that agencies summarize their regulatory
proposals and identify the information they seek in plain language accessible to the
different segments of the public they seek to reach, we recommend that agencies
summarize the basis and purpose of their final rules in plain language form that can
be understood by those who may have relevant information for retrospective review.
In particular, agencies should highlight the goals and assumptions upon which the
rule is based and the information the agency believes will enable it to assess the
effectiveness of the rule going forward. 511 The agency might then encourage the
public to address their retrospective comments to whether these goals and
assumptions remain valid, whether the rule has had unintended consequences, and
any other experiences relevant to the success or shortcomings of the rule. This is
consistent with the ABA recommendation that agencies publish with their final rules
“information the agency believes will enable it to assess the effectiveness of the rule
in accomplishing its objectives.” 512
Second, agencies should commit to periodically reviewing these comments and,
to the extent appropriate and realistic in light of the agency’s resources, responding to
comments they receive after the final rule becomes effective. At a minimum, if the
agency encourages comments about existing rules it should commit to reading them.
We recognize, however, that agencies will not always have the resources to respond
to every comment. Nevertheless, they may be able to respond to some. As with the
notice-and-comment process, the number of comments an agency receives concerning
a given regulation is likely to vary dramatically from rule to rule. Whether an agency
has the resources to respond to large numbers of comments likely depends on the
agency. The CFPB has done a remarkable job responding to thousands of consumer
complaints it has received through its website.
Open rulemaking dockets could be implemented using Regulations.gov or the
agency’s own website. We suspect that the agency’s website may be preferable,
however, as it is likely the first place that members of the public will go if they want
to comment on the agency’s regulations. In addition, this would allow for greater
experimentation than is currently possible with Regulations.gov. Thus, agencies
could utilize the same webpage interfaces we recommend adopting for the noticeand-comment process. The focus would merely change after the promulgation of the
final rule. Instead of highlighting questions and information the agency seeks
regarding a proposed rule, the agency would summarize the final rule and highlight
questions and information relevant to retrospective review of the regulation going
forward.

511
512

See ABA Res. 106b.
Id.

120

We recommend that agencies experiment with living rulemaking dockets
prospectively as they promulgate new rules. For older rules, and to the extent
agencies cannot fully implement open e-rulemaking dockets, we recommend agencies
utilize “hotlines” or on-line “suggestion boxes” that allow stakeholders to comment
on what is working or not working with existing rules. 513 Consistent with our other
recommendations, such hotlines or on-line portals should ask the public to identify
the specific rule they are addressing rather than encourage general comments about
the agency’s work. In addition, the agency should explain how it evaluates existing
regulations, identify the types of information or experiences that are most helpful for
the agency to know, and ask commenters to be as detailed as possible in describing
their experiences. On-line portals might also provide the public with examples of
helpful comments on existing regulations.
B.

Petitions for Rulemaking

Agencies review specific regulations in connection with petitions for rulemaking
pursuant to APA § 553(e). The number of petitions for rulemaking to amend or
repeal a rule varies quite a bit by agency. 514 Nevertheless, rulemaking petitions
provide an important way for members of the public, although in most cases
sophisticated stakeholders, to ask the agency to consider changes to existing
regulations. 515
Our recommendations concerning petitions for rulemaking set forth above in
connection with agenda setting are equally applicable to petitions for rulemaking
concerning existing regulations. Of course, the multiple uses of petitions for
rulemaking will shape an agency’s efforts to educate the public on their availability
and uses. For example, if an agency provides examples of “model” rulemaking
petitions that can be used by the public for guidance, the agency should include
examples of petitions to create a rule, to amend a rule, and to repeal a rule, and
explain the types of information that is most useful in each context.
In addition, agencies should be particularly mindful of soliciting comments from
absent stakeholders who benefit from the existing regulation when they receive
petitions to amend or repeal a rule. Petitions to amend or repeal a rule are likely to be
filed by sophisticated stakeholders and to focus on compliance costs, which drive
much of the interest in retrospective review. Agencies should not lose sight of the
benefits the rule may continue to generate. Thus, we recommend that agencies invite
public comment on petitions to amend or repeal a rule as a matter of course, unless
the agency for good cause finds that public comment is unnecessary. In addition,
agencies should conduct education and outreach efforts to solicit comments from
potential beneficiaries of the rule under review.
513

See Eisner & Kaleta, supra note 336, at 164 (proposing the use of “electronic bulletin
boards to ease the process for submitting suggestions”).
514
Id. at 150 (describing examples).
515
See Admin. Conf. of the U.S., Recommendation 2014-6, Petitions for Rulemaking,
79 Fed. Reg. 75,114, 75,117, 75,117-18 (Dec. 17, 2014).

121

As explained below, some of the other modes of public engagement with
retrospective review may also be used in conjunction with petitions to amend or
repeal a rule.
Requests for Information 516

C.

Requests for Information (RFIs) are important tools for engaging the public in
retrospective review. As discussed above in Part VI.D, RFIs are typically published
in the Federal Register and on Regulations.gov. They describe a matter under
consideration by the agency and request opinions, data, and other information from
the public that will assist the agency in deciding how to proceed. 517 RFIs are
particularly useful when (1) the agency is open-minded about whether and how to
address a regulatory matter and (2) the public is likely to have useful information
about the matter, including situated knowledge, data, preferences, and concerns,
which will help the agency decide on an appropriate course of action. Thus, RFIs
lend themselves to retrospective review in which agencies seek to determine whether
a rule is achieving its intended goals or should be amended, expanded, or repealed.
Agencies routinely use RFIs when developing retrospective review plans and
identifying candidates for review. 518 For example, during the Obama Administration,
the Department of Transportation issued several requests for comments from the
public on its “plan for periodically analyzing existing significant rules to determine
whether they should be modified, streamlined, expanded, or repealed and identify
specific rules that may be outmoded, ineffective, insufficient, or excessively
burdensome.” 519 During the Clinton Administration the FAA asked members of the

516

We use the term Request for Information (RFI) here but agencies may sometimes
refer to this tool as an Advanced Notice of Proposed Rulemaking (ANPRM) or some other
name to describe their requests for comments connected with retrospective review. As we
explained above in Part VIII.A, we conceive of ANPRMs as being used at a more advanced
stage of rule development or retrospective review. For example, an agency might issue an
ANPRM when it has decided to amend a rule and is testing different alternatives or gathering
the information it needs to craft its proposal. See, e.g., Certain Preventive Services Under the
Affordable Care Act, 77 Fed. Reg. 16,501, 16,501 (Dep’t of the Treasury Mar. 21, 2012)
(seeking comment on the agency’s plans to amend its regulations to establish alternative ways
to meet certain health coverage requirements by religious organizations that object to the
coverage of contraceptive services for religious reasons and that is not exempt under the
existing regulations).
517
See supra Part VI.D.
518
As noted above, agencies do not always denote such request as RFIs. Regardless of
the nomenclature, however, these requests for comments serve the same purpose. They seek
the public’s views on conducting retrospective review generally or retrospective review of a
specific regulation.
519
Regulatory Review of Existing DOT Regulations, 76 Fed. Reg. 8,940, 8,940 (Feb. 16,
2011). See also generally Notice of Retrospective Review of DOT Existing Regulations, 76
Fed. Reg. 11,699 (Mar. 3, 2011); Next Phase of the Regulatory Review of Existing DOT
Regulations, 79 Fed. Reg. 11,051 (Feb. 27, 2014).

122

public to identify “the top three rules that they believe[d] need[ed to be] review[ed]
(rather than asking them to list everything without requesting priority).” 520
Agencies also routinely use RFIs when conducting regulatory lookbacks
concerning specific rules. For example, the Dodd-Frank Wall Street Reform and
Consumer Protection Act requires the CFPB to review “each significant rule or order
adopted by the Bureau under Federal consumer financial law” five years after it
becomes effective. 521 The CFPB routinely issues RFIs one-year in advance to outline
its proposed approach to the review process, including the scope of the review and the
data needed to assess the effectiveness of the rule. In addition, soon after the agency
was established, the CFPB issued RFIs to solicit feedback on its inherited regulations.
The retrospective review process initiated with RFIs has resulted in tangible changes
to rules. 522
There are many other examples. The Department of Energy uses RFIs in
connection with the regulatory lookbacks it conducts six or seven years after issuing a
rule. The agency uses RFIs to ask questions about the issues it thinks are “at play”
based on its experience with the rule. 523 The Federal Trade Commission regularly
uses RFIs to ask the public to comment on specific questions regarding the economic
impact of existing rules. 524 And the PBGC routinely uses RFIs to solicit feedback on
its deregulatory actions. These are just a few examples.
RFIs are useful for retrospective review when the agency is open-minded about
how to proceed and the public is likely to have useful information or experiences
concerning the effectiveness of the existing regulation(s) and any unintended or
underappreciated consequences. As we have repeatedly emphasized, however,
merely issuing a request for comments is not enough. The agency must also engage
in planning and outreach to obtain participation by a broad range of stakeholders,
including regulated parties, the beneficiaries of the existing regulations, individuals
with situated knowledge, and unaffiliated experts who may have useful information to
share.
Finally, RFIs tend to be most valuable for retrospective review when they ask
specific questions. 525 Thus, RFIs seeking comments from traditionally absent
stakeholders should speak directly to them and ask them to share their experiences
living with the regulations. For example, when the CFPB conducts a regulatory
lookback of its mortgage disclosure rules it should address borrowers directly and ask

520
521

Eisner & Kaleta, supra note 336, at 164.
Dodd-Frank Wall Street Reform and Consumer Protection Act, 12 U.S.C. § 5512(d)

(2012).
522

Telephone Interview with Consumer Financial Protection Bureau (Dec. 5, 2017).
Telephone Interview with Department of Energy (Jan. 18, 2018). The Department of
Energy also sometimes uses negotiated rulemaking as part of the lookback process.
524
See Eisner & Kaleta, supra note 336, at 165.
525
See, e.g., id. at 149, 156.
523

123

them if the disclosures they received pursuant to the rule were clear and transparent or
whether they were surprised or confused by any closing costs. The RFI should also
ask borrowers to describe their relevant experiences in as much detail as possible.
The CFPB can then compare this information with the baseline established by the
comments it received from borrowers before the mortgage disclosure rules were
adopted.
D.

Public Meetings

Agencies hold public meetings at all stages of rulemaking, including
retrospective review. As discussed above in Part VI.E, public meetings can be
conducted in person, telephonically, on-line and live-streamed, or using some
combination of formats. They may also be recorded so that the public can view them
after the event. Both in person and remotely accessible meetings have advantages,
and we recommend that agencies consider using elements of both depending on their
specific goals.
Agencies can use meetings to encourage public participation in retrospective
review in several different ways. First, several agencies periodically hold open
meetings in which the public can raise questions or concerns about a particular topic.
These may include questions or concerns about existing regulations. Indeed, it is
common for agencies to hold public meetings when they are implementing new
regulations to field compliance questions on the part of regulated parties. Any
meeting that permits members of the public to raise their own agenda items with the
agency, even if within a defined subject area, will provide an opportunity for the
agency to learn how the public understands existing regulations, whether they are
achieving their goals, and any unintended consequences.
Second, agencies can design public meetings to focus on soliciting information
relevant to retrospective review. The reverse industry days (RIDs) discussed above in
Part VI.F offer a useful model for providing agencies with information about the
success or shortcomings of existing regulations. To briefly recap, RIDs are organized
and run by stakeholders to educate the agency on their interests and how they
experience the regulatory environment. Our research suggests that RIDs have
provided GSA and DHS with helpful information for improving their acquisitions
processes. We believe similar types of public meetings could provide agencies with
useful information regarding the effectiveness of their regulations in particular areas
and reveal ambiguities and unintended or underappreciated problems that could be
addressed through subsequent rulemaking.
Third, agencies can hold public meetings after receiving a petition to amend or
repeal an existing regulation, perhaps in conjunction with issuing an RFI.
Fourth, agencies can use public meetings when they conduct a regulatory
lookback concerning a specific rule or rules pursuant to a legal mandate or on their
own initiative. For example, the PBGC held an open hearing in connection with
retrospective review of its Reportable Events Rule. An in-person meeting offered a
more dynamic environment in which the agency could probe public comments in

124

ways that would be difficult using only a paper hearing such as notice and comment.
The agency was able to ask participants for examples and evidence to support their
comments and concerns. In addition to publishing a notice in the Federal Register,
the PBGC worked hard to get a balanced group of stakeholders to participate in the
meeting. It also allowed participants to submit material after the meeting and
extended the comment period to allow people to respond to each other’s
comments. 526 This offers a good model for fostering a more deliberative process
using public meetings.
Finally, agencies can hold public meetings in connection with designing
retrospective review plans, undertaken on their own initiative or in response to a
legislative or executive mandate. For example, EPA held twenty public meetings and
nineteen town halls and listening sessions on specific topics in connection with the
plan it developed in response to President Obama’s call for agencies to establish
retrospective review plans and policies. 527
Regardless of precisely how a public meeting is used, it is critical that the agency
undertake thoughtful outreach and planning to obtain balanced participation from a
broad range of stakeholders, including the beneficiaries of the existing regulations.
Agencies should also consider whether unaffiliated experts could provide additional
useful information on the regulation(s) under review, and engage in targeted outreach
efforts to secure their participation in appropriate circumstances.
E.

Federal Advisory Committees

Federal advisory committees provide an important means of public engagement
in retrospective review or regulatory lookbacks. They can provide agencies with
advice from a balanced group of well-informed stakeholders and unaffiliated experts
willing to give the agency their focused attention and able to engage in a deliberative
discussion concerning existing regulations and how to assess them.
Regulatory lookbacks sometimes originate in an agency’s federal advisory
committees. 528 Problems or unanticipated consequences of existing regulations can
come to light during the committee’s work on related topics. But agencies also can

526

Telephone Interview with Pension Benefit Guaranty Corporation (Nov. 21, 2017).
Cass R. Sunstein, Empirically Informed Regulation, 78 U. CHI. L. REV. 1349, 1389
n.170 (2011). See also Regulatory Review of Existing DOT Regulations, 76 Fed. Reg. 8,940,
8,940 (Feb. 16, 2011) (announcing a public meeting held by DOT to discuss its “plan for
periodically analyzing existing significant rules to determine whether they should be
modified, streamlined, expanded, or repealed and identify specific rules that may be
outmoded, ineffective, insufficient, or excessively burdensome”).
528
See, e.g., Eisner & Kaleta, supra note 336, at 147, 149; HCV Lookback for Blood
Products Proposed, Food Drug Cosm. L. Rep. ¶ 45,964 (2000), 2000 WL 36704962
(explaining that a lookback was conducted by DHHS in response to recommendation of
Public Health Service Advisory Committee on Blood Safety and Availability).
527

125

and do make more purposeful use of advisory committees in retrospective review. 529
First, agencies can consult advisory committees when considering whether or how to
undertake a retrospective review or a regulatory lookback. Second, agencies can
consult advisory committees when they receive petitions to amend or repeal existing
regulations or public comments in response to an RFI regarding retrospective
review. 530 Thus, agencies may ask advisory committees to generate their own
candidates for regulatory review, evaluate candidates for review proposed by other
members of the public in a petition or in response to an RFI, or help the agency
design a process for conducting lookbacks.
Advisory committee can provide advice on:
•

Which existing regulations are in need of review?

•

How the agency should prioritize among candidates for regulatory
lookbacks?

•

How the agency should approach retrospective review generally or
lookbacks of specific regulations?

•

What should be the scope of a given retrospective review?

•

What is the appropriate baseline for measuring the success of a
regulation?

•

What types of information or data does the agency need to conduct its
regulatory review?

•

Which stakeholders are likely to have critical information for
retrospective review, and how can the agency ensure that it receives
balanced and unbiased information?

•

What procedural steps the agency should undertake as part of its
retrospective review?

In addition, advisory committees might provide more specific advice on particular
regulations, such as:
•

Potential ambiguities or unintended consequences of the rule;

•

Changes in the industry or other developments since the promulgation of
the rule; and

529

See, e.g., Eisner & Kaleta, supra note 336, at 165.
See, e.g., id. at 165 (describing FDA’s use of an advisory committee to narrow a list
of regulations that the public proposed for review in response to an RFI).
530

126

•

Alternatives to the existing regulatory process.

When deciding whether to use an advisory committee for retrospective review,
the agency must decide whether an existing committee has the appropriate
composition and expertise to review the regulation(s). The advantage of advisory
committees is that a well-informed group of stakeholders can provide focused
attention on the regulatory matter. If the agency does not have a group with the
relevant expertise, it will have to decide whether to charter a new advisory committee
for these purposes. Theoretically, the agency could charter a new advisory committee
to review a specific regulation or regulatory area. We suspect in most cases,
however, this would prove too burdensome given the challenges of chartering new
committees. 531 If the agency routinely engages in retrospective review it might
charter an advisory group for this purpose. There may be advantages to having a
distinct committee focused on retrospective review. But such a group may not have
the relevant subject-matter expertise for all the regulations an agency must review.
Thus, it will generally make more sense for agencies to use existing advisory
committees in the context of retrospective review. Because advisory committees
provide a useful mode of public engagement in a variety of contexts—including
agenda setting and rule development—and because they are relatively difficult to
charter in the first instance, 532 it is likely more efficient for agencies to maintain
committees with relevant expertise that can be consulted for a variety of purposes.
Finally, agencies should routinely share the briefing materials prepared for their
advisory committees and the advice of their committees with the broader public. An
advisory committee’s report and recommendations could be posted on the agency’s
website as part of the e-rulemaking portal described above in Part IX.C.

531

There are several procedural hurdles to establishing new advisory committees. See,
e.g., Federal Advisory Committee Act, 5 U.S.C. app. § 9(a)(2) (2012) (the head of an agency
must find that the establishment of the advisory committee is “in the public interest in
connection with the performance of duties imposed on that agency by law”); id § 9(c)
(requiring advisory committee charter to be filed with, among others, the standing
committees of the House and Senate with jurisdiction over the agency); id. § 14(a)(1) (setting
a two-year limit for advisory committees unless created by statute); Exec. Order No. 12,838,
58 Fed. Reg. 8,207 (Feb. 10, 1993) (“[E]xecutive departments and agencies shall not create or
sponsor a new advisory committee subject to FACA unless the committee is required by
statute or the agency head (a) finds that compelling considerations necessitate creation of
such a committee, and (b) receives the approval of the Director of the Office of Management
and Budget. Such approval shall be granted only sparingly and only if compelled by
considerations of national security, health or safety, or similar national interests. These
requirements shall apply in addition to the notice and other approval requirements of
FACA.”). These requirements operate as a constraint on establishing new advisory
committees. Eisner & Kaleta, supra note 336, at 152.
532
See supra Part VI.B.

127

F.

Focus Groups

Agencies may also want to use focus groups in retrospective review to gauge the
reactions of certain stakeholders to various aspects of existing regulations or potential
alternatives. We describe the basic process and challenges of conducting a focus
group above in Part VI.C. In the context of retrospective review, the agency might
use focus groups in a variety of ways. For example, the agency might use focus
groups to test whether the regulated industry or the public shares concerns or supports
proposals contained in a petition to amend or repeal a rule. Or the agency might use
focus groups to determine how consumers experience a particular regulation, such as
a disclosure requirement, under review. Similar to the use of focus groups in rule
development, the agency might also use a focus group to test alternatives to an
existing regulation. Federal agencies have frequently used focus groups to ask
questions about different approaches to consumer disclosure and product labeling. 533
As we discuss above in Part VII.B, agencies should give careful consideration to
whom to invite to participate in focus groups based on the nature of the regulation
under review and the type of feedback they seek. They should provide skilled
facilitation and conduct careful planning to maximize the likelihood of getting the
most productive input from the group. This includes providing participants with
briefing materials that clearly explain the relevant issues and possible alternatives.
Finally, agencies should prepare a report after the session that summarizes the
feedback and identifies issues for further consideration. The agency should post these
reports on their websites, if possible in the e-rulemaking dockets described in Part
IX.C.
G.

Public Notice and Comment

The APA requires agencies to utilize public notice and comment to amend or
repeal a rule promulgated using notice and comment. Thus, if an agency decides to
proceed with amendment or repeal of a rule based on retrospective review or a
regulatory lookback, the agency will need to provide an opportunity for public notice
and comment. Our recommendations concerning the notice-and-comment process
discussed above in Part IX also apply to proposals to amend or repeal a rule.
XI. ENHANCED FORMS OF DELIBERATION
Most of the tools of public engagement that we have discussed so far provide
opportunities for agency officials and interested members of the public to
communicate with each other by sharing information and ideas about potential
courses of action. These forms of public engagement generally involve efforts to
“inform” or “consult” under IAP2’s spectrum of public participation. 534 While these
are meaningful and potentially valuable forms of public participation, they generally

533
534

See supra Part VI.C.
See supra notes 141-142 and accompanying text.

128

do not involve much in the way of reasoned deliberation or interactive dialogue about
what should be done. In other words, they generally do not rise to the level of
“involving” or “collaborating” with the public under IAP2’s spectrum, 535 and thus
arguably fall short of the ideal requirements of deliberative democratic theory. 536
Deliberative democracy fundamentally aims to facilitate legitimate collective
decisions about what should be done that take into account all of the relevant interests
and perspectives that emerge from a reasoned deliberative process. The deliberative
process should reflect the perceived interests and views of ordinary people, and
should ultimately consider and respond in a reasoned fashion to the interests and
perspectives of everyone who will be affected by the decision. 537 Deliberative
democratic theory maintains that the exercise of governmental authority is only
legitimate if public officials adequately consider everyone’s interests and perspectives
and if they give reasoned explanations for their decisions that could reasonably be
accepted by people with fundamentally competing views. 538
An ideal deliberative process should therefore provide mechanisms for agencies
to “[w]ork directly with the public throughout the process to ensure that public
concerns and aspirations are consistently understood and considered,” and could in
appropriate circumstances involve partnering “with the public in each aspect of the
decision including the development of alternatives and the identification of the
preferred solution,” even if the agency retains ultimate decision-making authority. 539
An ideal deliberative process would also provide opportunities for interested
members of the public to engage in a dialogue with each other as well as with
responsible agency officials. While deliberative processes ideally seek to achieve
consensus on the best course of action under the circumstances, they can also
incorporate voting or other closure devices when reasoned disagreement remains after
discussions have concluded. 540 The enhanced deliberative techniques briefly
discussed in this section would facilitate these more robust forms of participation, and

535

See id.
For influential discussions of the theory of deliberative democracy, see generally
GUTMANN & THOMPSON, WHY DELIBERATIVE DEMOCRACY?, supra note 62; AMY
GUTMANN & DENNIS THOMPSON, DEMOCRACY AND DISAGREEMENT (1996). For a
comprehensive extension of deliberative democratic theory to the realm of administrative
law, see generally RICHARDSON, supra note 14.
537
See Philip Pettit, Republican Freedom and Contestatory Democratization, in
DEMOCRACY’S VALUE 163, 173-80 (Ian Shapiro & Casiano Hacker-Cordón eds., 1999)
(discussing the “electoral” and “contestatory” dimensions of republican democracy).
538
See Joshua Cohen, Deliberation and Democratic Legitimacy, in THE GOOD POLITY:
NORMATIVE ANALYSIS OF THE STATE 17 (Alan Hamlin & Philip Pettit, eds., 1989).
539
See LUKENSMEYER & TORRES, supra note 141, at 7 (providing IAP2’s definitions of
“involve” and “collaborate”).
540
See Staszewski, Political Reasons, supra note 15, at 893-96 (discussing the role of
voting in deliberative democratic theory and the possible relevance of these ideas for agency
decision making).
536

129

could therefore help agencies reach legitimate collective decisions on which courses
of action would promote the public good. However, the use of these tools is timeconsuming and expensive and will not always produce a substantial amount of useful
new information. Accordingly, we will also provide some guidance on when the use
of enhanced deliberative exercises is most likely to be worth the additional effort.
These deliberative tools would, after all, supplement rather than replace the
opportunities for public engagement described elsewhere in this report, including
public notice and comment, which we view as a baseline legal and democratic
requirement.
A.

Deliberative Public Engagement in Government Decision-making

The best example of the use of enhanced deliberative tools in federal rulemaking
is almost certainly the efforts associated with Regulation Room, which we have
already discussed at length. 541 However, the nonprofit world and other governmental
bodies have developed a number of other methods to produce enhanced deliberation
on policy issues, which could be used to supplement the federal rulemaking process.
One of the most well-known of these methods is the “deliberative polls”
pioneered by James Fishkin, the Director of the Center for Deliberative Democracy at
Stanford University. 542 These polls are designed “to combine random sampling with
deliberation” in an effort to ascertain what the general public would think about a
problem if they were fully informed about the relevant issues and had a chance to
engage in reasoned deliberation about what should be done. 543 Deliberative polls
typically involve the participation of approximately 500 randomly selected citizens
who agree to participate in an in depth discussion of a specified topic over the course
of two days. According to Fishkin, the entire process “is designed to facilitate
informed and balanced discussion,” including the provision of carefully prepared
briefing materials that provide an initial basis for discussion, random assignment to
small groups where trained moderators seek to facilitate respectful and balanced
discussion, and the opportunity for participants to pose questions that arise from the
small group discussions to policy makers and other experts at larger plenary
sessions. 544 Participants are asked to complete a confidential survey that expresses
their views on a range of relevant questions both before and after these deliberations,
and Fishkin reports that “it is routine to find large and statistically significant changes
of opinion over the weekend.” 545 The challenges associated with conducting
deliberative polls include persuading a sufficient number of citizens to participate,
and the extensive resources necessary to provide balanced briefing materials, valid
surveys, expert commentators, and trained facilitators. Of course, because the views

541

See supra Part IX.A.6.
See generally JAMES F. FISHKIN, WHEN THE PEOPLE SPEAK: DELIBERATIVE
DEMOCRACY & PUBLIC CONSULTATION (2009).
543
See id. at 25-26.
544
See id. at 26.
545
See id.
542

130

or recommendations produced by deliberative polls are likely to diverge from the
views of the general public (precisely because most citizens are not well-informed
about the relevant problems and they have not engaged in reasoned deliberation with
their neighbors), regulations that are based on recommendations from deliberative
polls may not receive enhanced support from the broader public. 546
Citizen advisory committees provide another example of a tool that could be
used to provide enhanced deliberation to supplement the traditional notice-andcomment rulemaking process. Building on the success of deliberative polling and
similar models, Reeve Bull has recommended that agencies consider establishing
federal advisory committees composed of demographically diverse and otherwise
balanced groups of ordinary citizens to provide federal agencies with thoughtful
advice on their potential rulemaking options in appropriate circumstances. 547 While
the ideal size of these committees would depend on the situation, Bull recommends
that they should generally include roughly a few dozen members to ensure a
sufficient variety of perspectives without becoming unwieldy. 548 Once the citizen
advisory committee was formed, the agency would provide its members with
balanced briefing materials on the nature of the problem and the range of potential
solutions, and would provide them with an opportunity to engage in reasoned
deliberation on the best course of action over the following weeks or months. At the
close of the deliberations, the committee would provide the agency with advice on the
best course of action under the circumstances (either by reaching consensus or
pursuant to a majority vote), and the agency could consider this advice in deciding
how to proceed. 549 Bull points out that some of the financial and logistical challenges
associated with these efforts could be mitigated by conducting at least some of the
committee’s proceedings online, 550 but he acknowledges that agencies will likely
only devote the resources necessary to establish and use such committees for
especially important decisions where informed public opinion is likely to be useful. 551
Carolyn Lukensmeyer’s and Lars Hasselblad Torres’s report, Public
Deliberation: A Manager’s Guide to Citizen Engagement, provides a wealth of
information about different methods of enhanced deliberation that could be used by
federal agencies during the rulemaking process. 552 These include the Jefferson
Center’s “citizen juries,” “citizen assemblies” of the kind used in British Columbia to
546

See id. at 28.
See Bull, Making the Administrative State “Safe”, supra note 13, at 640-47; see also
John S. Applegate, Beyond the Usual Suspects: The Use of Citizen Advisory Boards in
Environmental Decisionmaking, 73 IND. L.J. 903, 926-31 (1998) (describing and endorsing
the use of citizen advisory boards to provide advice on cleaning up the environment at
particular sites).
548
See Bull, Making the Administrative State "Safe”, supra note 13, at 641-42.
549
See id. at 644.
550
See id. at 644-45.
551
See id. at 647.
552
See LUKENSMEYER & TORRES, supra note 141, at 19, 24-32.
547

131

make recommendations on the best ways to reform the electoral process, the 21st
Century Town Meetings hosted by AmericaSpeaks, 553 and the “participatory
budgeting” process that was pioneered in Porto Alegre, Brazil, and is currently being
used in a number of major cities. 554 Lukensmeyer and Torres explain that all of these
methods of enhanced deliberation:
•

“use ‘balanced’ or ‘neutral’ background materials;”

•

“are structured around small group dialogue;”

•

emphasize “learning through exploration of competing perspectives on an
issue;”

•

expect new knowledge “to inform individual and group recommendations
on the issue or problem at hand;” and

•

issue “findings” from the exercise in a final report “made available to
community members and leaders.” 555

Lukensmeyer and Torres point out that while most of these methods for
facilitating enhanced deliberation have an established track record, the federal
government has rarely used methods of this nature to inform its rulemaking decisions.
They contend that “[t]he critical next step in the evolution of deliberative democracy
in administrative decision making will be to experiment with, adapt, and
institutionalize these techniques” within federal agencies. 556 Such efforts would
reflect an emerging new role for federal agencies as “convener of the public” 557—or
as “steward of an infrastructure of engagement.” 558
While this vision may seem like a substantial departure from existing practice, it
is important to emphasize that the foregoing tools are merely supplemental techniques
for potentially enhancing public engagement in a limited subset of rulemakings in
which the benefits justify the costs. Moreover, some federal agencies already use
relatively simple methods of public engagement that could involve “enhanced
deliberation” in some situations. For example, federal advisory committees regularly
provide advice to agencies on rulemaking-related issues. Moreover, agencies
sometimes engage in negotiated rulemaking with a balanced group of interested
stakeholders. Both of these methods of public engagement have the potential to
involve the type of reasoned deliberation that is under consideration here. The

553

See, e.g., id. at 24-25.
See generally HOLLIE RUSSON GILMAN, DEMOCRACY REINVENTED: PARTICIPATORY
BUDGETING AND CIVIC INNOVATION IN AMERICA (2016).
555
LUKENSMEYER & TORRES, supra note 141, at 25.
556
Id.
557
Id. at 12.
558
Id. at 35.
554

132

question, once again, is when to use these methods of public engagement and how to
execute them in the appropriate circumstances.
B.

In-person Versus Online Deliberative Exercises

A major question agencies must answer when considering the use of enhanced
deliberative methods is whether to conduct the proceedings online, in person, or
both. 559 Regulation Room, once again, is a prime example of an enhanced
deliberative tool that was conducted entirely online. Most of the other methods of
enhanced deliberation discussed above have traditionally been conducted in person.
Fishkin has, however, recently conducted deliberative polling online, 560 and ACUS
has recommended that agencies consider holding asynchronous virtual meetings of
their federal advisory committees. 561 Nearly any deliberative method can be
conducted at least partly online based on existing technology, 562 and the best practice
may generally be to design some sort of “hybrid” deliberative process that includes
both in person meetings and virtual discussion. Participatory budgeting in various
major cities has been cited as an example of a process that has made particularly
effective use of hybrid deliberation of this nature. 563
The primary advantages of meeting in person include an enhanced ability to
establish trust and build relationships, to facilitate more respectful and empathetic
interactions, and to ensure participants have adequate knowledge about the relevant
issues to participate effectively—all of which are essential for approximating the
deliberative ideal. In particular, it is easier to overcome the capacity and information
barriers to effective public participation in rulemaking identified in Part III.B in
person than when the unlimited distractions of the Internet are just a click away. The
primary disadvantage of in-person meetings mirrors the primary advantage of their
virtual counterparts—virtual meetings can save substantial resources in the time and
money that is required to attend meetings in person. Moreover, virtual deliberation
can produce more thoughtful or well-considered responses if the meetings are
“asynchronous,” because participants can spend more time thinking about a matter
and providing a more polished response. Finally, although the anonymity that is
provided or enhanced by virtual communications can severely undermine the tenor of
those discussions, anonymity does provide the advantage of minimizing potential

559

For a useful discussion of efforts to conduct deliberation online, see generally
ONLINE DELIBERATION: DESIGN, RESEARCH, AND PRACTICE (Todd Davies & Seeta Peña
Gangadharan eds., 2009).
560
See id. at 39 (discussing this effort).
561
See Admin. Conf. of the U.S., Recommendation 2011-7, The Federal Advisory
Committee Act—Issues and Proposed Reforms, 77 Fed. Reg. 2,257, 2,261, 2,263 (Jan. 17,
2012).
562
For a useful overview of “online deliberation,” see LUKENSMEYER & TORRES, supra
note 141, at 33-43.
563
See generally GILMAN, supra note 554.

133

distortions in evaluating the persuasiveness of a message based on the identity or
personal characteristics of the speaker.
The primary perceived advantage of online deliberative exercises—their
accessibility, however, may be overstated and create its own challenges. When
online deliberative platforms are open to anyone and thus rely on “voluntary”
participation, it is substantially more difficult to obtain balanced or representative
feedback or advice. 564 First, merely placing exercises online does little to overcome
the motivational barriers to greater public engagement in rulemaking identified above
in Part III.B. As Farina and Newhart explain, Regulation Room had to engage in
extensive targeted outreach directed toward missing stakeholders to obtain
meaningful participation in the online platform. 565 Second, interest groups and
advocacy organizations can orchestrate “mass comment” campaigns in this setting (or
even prompt fraudulent comments), which raises further concerns about the
representativeness of public comments (or even their authenticity), and tend not to
provide information that is especially useful to agencies. 566 Agencies may therefore
have valid concerns that the “quantity” of feedback produced by online deliberation
will routinely exceed its “quality.” 567
Deliberative exercises conducted in person typically seek to overcome these
difficulties by recruiting targeted stakeholders or a balanced group (or even a
stratified random sample) of ordinary citizens, providing them with balanced and
objective briefing materials, using trained facilitators to moderate the discussions, and
providing access to government officials or other experts to answer any questions that
arise during the proceedings. While it can still be challenging to achieve a
deliberative ideal, enhanced deliberative exercises of this nature are certainly capable
of generating useful advice for policy makers.
The problem is that producing this advice using in-person exercises is undeniably
time-consuming and expensive, and requires careful planning by experts with
experience in conducting such events. Accordingly, the use of such enhanced
deliberative exercises by federal agencies should be limited to situations where the
potential benefits are likely to exceed the costs. The next section suggests some
factors that agencies should consider when making this decision, as well as
recommendations on how to design these exercises.
C.

Best Practices for Enhanced Deliberative Exercises

Agencies must give careful consideration to when enhanced deliberative
exercises of this nature are most likely to be worth the effort. The scholars who
designed and operated Regulation Room once again provide useful advice for
thinking about this question. Their first major recommendation on this score is for
564

See, e.g., Fung, Varities of Participation, supra note 143, at 67.
Cf. FARINA & NEWHART, IBM CENTER, supra note 22, at 21-26.
566
But cf. Mendelson, Torrents of E-mail, supra note 13, at 1,375.
567
See LUKENSMEYER & TORRES, supra note 141, at 35-36.
565

134

agencies to focus their efforts “on rulemakings in which (a) new participants are
likely to have useful information and (b) it is feasible to provide the participation
support necessary to elicit this information from them.” 568 Farina and Newhart
suggest that agencies should carefully consider whether specific groups of missing
stakeholders or unaffiliated experts could provide added value to a particular
rulemaking effort, and they suggest that agencies should only actively solicit
participation by ordinary citizens if they are confident that those efforts will produce
information that is useful to the agency. 569 Farina and Newhart also recommend
giving careful consideration to the “information load” associated with efforts to
generate useful input from novice participants, by which they mean the effort
required to provide sufficient information to rulemaking newcomers to enable them to
participate effectively. 570 The lower the information load, the more likely it is that
efforts to facilitate enhanced deliberation will prove worthwhile, and vice versa. In a
recommendation that we regard as crucial for these purposes, Farina and Newhart
suggest that agencies consider the possibility of using enhanced deliberative methods
“selectively—that is, of targeting only certain types of potential new participants or
only certain issues in the rulemaking.” 571 We strongly agree that enhanced
deliberative efforts will generally be most worthwhile, regardless of the precise
method used, if they are carefully targeted at designated groups of participants and
focused on relatively specific questions, issues, or problems.
Indeed, Farina’s and Newhart’s proposed conceptual framework is generally
transferable to thinking about when other types of enhanced deliberation would be
worthwhile. Nonetheless, there are also a few caveats or other factors to keep in
mind. First, as explained above, efforts to facilitate enhanced deliberation will
generally be more efficient and productive when they are limited to a carefully
selected and manageably sized group, as is the case with all of the traditional “in
person” methods for producing reasoned deliberation. While such exercises could be
conducted solely with missing stakeholders or unaffiliated experts, ordinary citizens
could also make useful contributions to deliberative exercises of this nature in certain
circumstances.
Second, agencies plainly need to consider which types of rulemaking
proceedings are appropriate for enhanced deliberation. While Farina and Newhart
correctly suggest that these exercises will be most useful when agencies are
confronted with information gaps that could likely be filled by missing stakeholders,
unaffiliated experts, or ordinary citizens, there are certainly other variables to
consider. For example, enhanced deliberative exercises will generally be most useful
when rulemaking proceedings are more rather than less important, more rather than
less politically salient, and more rather than less likely to turn on the resolution of
conflicting public values.

568

FARINA & NEWHART, IBM CENTER, supra note 22, at 38.
See id.
570
See id. at 39.
571
Id.
569

135

Third, we think that it is vital for agencies to give careful consideration to
whether efforts to facilitate enhanced deliberation makes sense at each stage of the
rulemaking process, and then to choose the appropriate method or methods of
producing enhanced deliberation at each respective stage. As we have suggested
elsewhere in this report, efforts to facilitate public engagement in rulemaking will
often make the most sense at the agenda setting stage or during rule development, and
this is particularly true of enhanced deliberative methods because ordinary citizens
(and, to a lesser extent, missing stakeholders) will typically be most adept at weighing
in on the proper ordering of the agency’s priorities and helping determine which
general regulatory directions will best promote the public good, as opposed to
providing useful input on the relatively detailed legal, technical, or empirical
questions the agency will be required to resolve later in the rulemaking process.
Once an agency decides to conduct an enhanced deliberative exercise, it must
also think carefully about its design. According to Lukensmeyer and Torres,
enhanced deliberative exercises should:
•

Provide accessible information to citizens about the issues and choices
involved, so that they can articulate informed opinions.

•

Offer an unbiased framing of the policy issue in a way that allows the
public to struggle with the same difficult choices facing decision makers.

•

Involve a demographically representative group of citizens reflective of
the affected community.

•

Facilitate high-quality discussion that ensures all voices are heard.

•

Produce information that clearly highlights the public’s shared priorities.

•

Achieve commitment from decision makers to engage in the process and
use the results in the policy process.

•

Support ongoing involvement by the public on the issue, including
feedback, monitoring, and evaluation. 572

While enhanced deliberative exercises that adhere to these principles have
traditionally been composed of a representative cross-section of the community, we
believe that agencies might also find it useful in some situations to experiment with
efforts to seek input from deliberative bodies constructed primarily or even
exclusively of missing stakeholders or unaffiliated experts, particularly when
agencies seek the type of situated knowledge or expertise that these groups possess.
Finally, agencies should also consider whether to conduct enhanced deliberative
exercises themselves or hire a facilitator or contractor to manage such efforts. The
practice of deliberative democracy has produced a cottage industry of consultants

572

LUKENSMEYER & TORRES, supra note 141, at 44.

136

with expertise in planning and implementing the available techniques. Given the
effort and potential challenges involved in running these exercises effectively, it will
often make sense for agencies to hire an experienced consultant to carry out this work
in cooperation with agency officials, and to produce a report that summarizes the
findings and recommendations of the deliberative body when the process has
concluded. This is how Regulation Room was conducted, and the cooperating
agencies told us they would not have had the resources or expertise to replicate this
process on their own. This approach also provides advantages associated with
conducting deliberative exercises separately from the agency’s traditional rulemaking
process, while simultaneously allowing the agency to benefit from the views and
perspectives of the deliberative body. The primary disadvantage of hiring a
consultant to conduct enhanced deliberation is obviously the likelihood of additional
costs—and that is, of course, a major factor that federal agencies with limited (and
diminishing) resources routinely need to consider.
Notice-and-comment rulemaking is widely understood to have the potential to
serve as an ideal deliberative process. 573 Yet it is widely recognized that regulated
entities and other organized groups are disproportionately represented in this
process. 574 Deliberative democratic theory does not require participation by every
individual with a potential stake in the agency’s decision. It does require, however,
that agencies give adequate consideration to all of the relevant interests and
perspectives, and provide reasoned explanations for their decisions that could
reasonably be accepted by citizens with fundamentally competing views. A
prerequisite to achieving this deliberative ideal, then, is for all of the relevant interests
and perspectives to be forcefully articulated during the decision-making process and
given careful consideration by agency officials. The rulemaking process falls short of
this ideal if some relevant interests or perspectives are missing or ignored. This
suggests the fundamental importance of enhanced efforts by agencies to involve
missing stakeholders, unaffiliated experts, and even ordinary citizens at some stage of
the rulemaking process in appropriate circumstances.
We have already suggested some potential ways to improve the notice-andcomment rulemaking process to further this goal. When those efforts are insufficient,
however, agencies should consider undertaking additional efforts to facilitate
enhanced deliberation to supplement the traditional legislative rulemaking process
and thereby remedy those deficiencies. The briefing materials that are prepared for
enhanced deliberative exercises, and the recommendations and other feedback that
are provided by deliberative bodies, can also be provided to other interested members
of the public as a basis for further deliberation and commentary. Thus, enhanced

573

See DAVIS, supra note 2, at 65-66 (“Rule-making procedure which allows all
interested parties to participate is democratic procedure.”); KERWIN & FURLONG, supra note
2, at 31 (“Rulemaking adds opportunities for and dimensions to public participation that are
rarely present in the deliberations of Congress or other legislatures.”).
574
See Johnson, supra note 6, at 78; Walters, supra note 6, at 4.

137

deliberative exercises will typically produce additional material that can also be used
to further improve the value and legitimacy of notice-and-comment rulemaking.
XII. THE IMPORTANCE OF PLANNING AND OUTREACH
It is widely recognized that early planning for public engagement efforts is
essential. 575 Agencies should therefore develop general policies for public
engagement in rulemaking and establish mechanisms to ensure that those policies are
consistently followed. Agencies should also develop specific plans for public
engagement for each rulemaking initiative they undertake or seriously consider.
These plans should include internal and external situation assessments, and consider
(1) why the agency wants to engage with the public, (2) who the agency is trying to
reach, (3) what type of information the agency is seeking, (4) how this information is
likely to be obtained, (5) when these efforts should occur, and (6) what the agency
will do with the information. These public engagement plans should also address
issues of outreach and communication.
A.

General Public Engagement Policies

Agencies should adopt general public engagement policies that express their
support for public engagement efforts and provide a framework for involving the
public in particular rulemaking initiatives. EPA’s Public Involvement Policy is one
of the best examples of this practice. 576 EPA has explained that the Policy’s “overall
goal is for excellent public involvement to become an integral part of EPA’s culture,
thus supporting more effective Agency actions.” 577 The Policy provides guidance to
agency managers and staff on specific steps that should be followed to promote
effective public engagement, and provides information about other resources that are
available to facilitate those efforts. The Policy’s seven steps for effective public
engagement are as follows:
1. Plan and budget for public involvement activities;
2. Identify the interested and affected publics;
3. Consider providing technical/financial assistance to support public
involvement;

575

See, e.g., CREIGHTON, supra note 220, at 27 (“Public participation that isn’t an
integrated part of the decision-making process is a waste of time for both the organization and
the public.”); EPA, BETTER DECISIONS MANUAL, supra note 278, at 5 (“Good stakeholder
involvement processes should be planned early enough to allow both [agency] staff and the
stakeholders to obtain the necessary resources and data to interact effectively.”).
576
See EPA PUBLIC INVOLVEMENT POLICY, supra note 35.
577
ENVTL. PROT. AGENCY, INTRODUCING EPA’S PUBLIC INVOLVEMENT POLICY 2
(2003) (describing EPA’s public involvement policy).

138

4. Provide information and conduct outreach;
5. Conduct public consultation and involvement;
6. Review and use input and provide feedback; and
7. Evaluate public involvement activities. 578
The Policy explains the underlying goal of each of these steps, and provides
recommendations for how to achieve those goals. 579 The Policy seeks to “support
and encourage public involvement excellence” by providing well-publicized training
opportunities, fostering “public involvement networks” within the agency “for
sharing information and experiences,” and providing tools for evaluating the success
of the agency’s public engagement efforts. 580
The National Park Service has also adopted a strong Civic Engagement Policy
pursuant to a pair of Director’s Orders. 581 The purpose of the orders is to set forth the
agency’s “commitment to civic engagement, and to have all National Park Service
units and offices embrace civic engagement as the essential foundation and
framework for creating plans and developing programs.” 582 The latest version of the
order articulates the Service’s philosophical commitment to civic engagement, and
states that “[t]he public has a right to know about the challenges that confront the
NPS and to participate in the process by which we find solutions to those
challenges.” 583 The order wisely emphasizes that the relevant public includes
external stakeholders as well as agency employees who must be given “an
opportunity for meaningful involvement during the decision-making process” to
utilize their special knowledge and expertise. 584 After discussing the document’s
scope and providing definitions, the order recognizes that “[t]he public will have a
greater appreciation of, and support for, our management if they recognize that we
seek, and are receptive to, their contributions to and involvement in the important
decisions that are made.” 585 It proceeds to set forth an extensive list of policies and
standards to achieve those ends. 586 While those policies and standards are worthy of
reading (and generally emulating) in their entirety, several of NPS’s adopted policies
focus directly on the importance of planning and therefore merit special attention
here:

578

Id. at 3-5.
See id. at 2-5.
580
Id. at 5-6.
581
See generally DIRECTOR’S ORDER #75A, supra note 35.
582
Id. at 1.
583
Id. at 3.
584
Id. at 4.
585
Id. at 6.
586
See id. at 6-10.
579

139

•

We will plan in advance and be clear at what stages, and how, we will invite
the public to participate in our decision-making processes. It is important to
make a clear and early decision about the extent of the public’s involvement
in each project or decision-making process. The extent of the public’s role
can vary from issue to issue, and at different stages in the process.

•

We will plan early for appropriate opportunities for public involvement in
our decision-making process when the decisions will lead to actions or
policies that may significantly affect or interest them.

•

Managers are encouraged to be resourceful and employ a wide variety of
methods and techniques to obtain the opinions of individuals and groups.

•

On potentially controversial issues, we will be particularly mindful to plan
and design public involvement opportunities at the earliest opportunity, and
to use specialized techniques . . . to minimize potential for conflict and
achieve a solution smoothly. As issues arise, managers should already be
familiar with a range of alternative dispute resolution techniques and
resources, including the use of facilitators or mediators, to help resolve
controversial issues. If a controversy pertains to a rule-making activity (i.e.,
adopting a regulation), “negotiated rulemaking” should be considered.

•

We will call upon individuals with expertise about how to create and manage
opportunities for public involvement activities.

•

We will develop capacity in public involvement strategies and will encourage
Service employees to become knowledgeable about civic engagement and
public involvement techniques and principles. Interdisciplinary training
materials and opportunities will be developed to help park managers and
others who are responsible for public involvement activities understand and
apply “best practices.”

•

We will design public involvement processes that are as open and inclusive
as possible so that diverse publics, including those who typically do not
participate, have opportunities to share their views, values, and concerns.

•

We will maximize the use of computer and Internet technologies to expand
public access to information and opportunities to participate. 587

The order describes the roles and responsibilities of various NPS officials for
implementing the order, and significantly provides that the Director and Deputy
Directors:
[w]ill ensure that the Office of Policy coordinates and implements this
DO, assists in developing further guidance and training to build
587

Id. at 6-8.

140

organizational capacity, and serves as a liaison to the other offices of
the Director, the associate and regional directors, the Department [of
the Interior], and other federal agencies on civic engagement and public
involvement opportunities and issues. 588
The order closes by stating that NPS will develop tools to evaluate the Service’s
public engagement efforts and “take the necessary steps to ensure that adequate
public involvement processes are developed and implemented.” 589 To build the
requisite internal capacity, the order states that NPS will, among other things, “[t]rain,
devise incentives, and recognize employees for practicing civic engagement and
public involvement” and “[p]lan and budget early for public involvement
activities.” 590
The most significant challenge or limitation associated with public involvement
policies of this nature is the difficulty of ensuring that they are consistently followed
or enforced, particularly in administrations that place a lower priority on civic
engagement efforts or when agencies face substantial budget cuts. Agencies can,
however, make “pre-commitments” to designated public engagement efforts by
promulgating rules that at least presumptively require them to take certain action. For
example, the Department of Energy has used stakeholder engagement to adopt a
“process rule” that sets forth the agency’s procedures for promulgating consumer
appliance efficiency standards. 591 As the preamble explains, this process rule
“provides for greatly enhanced opportunities for public input, improved analytical
approaches, and encouragement of consensus-based standards.” 592

588

Id. at 10-11.
Id. at 11.
590
Id. at 12. For additional information about NPS’s public engagement efforts, see
generally MOLLY RUSSELL, PRINCIPLES OF SUCCESSFUL CIVIC ENGAGEMENT IN THE
NATIONAL PARK SERVICE (2011), REBECCA STANFIELD MCCOWN ET AL., NAT’L PARK
SERV. CONSERVATION STUDY INST., BEYOND OUTREACH HANDBOOK: A GUIDE TO
DESIGNING EFFECTIVE PROGRAMS TO ENGAGE DIVERSE COMMUNITIES (2011), NAT’L PARK
SERV. CONSERVATION STUDY INST., LEADING IN A COLLABORATIVE ENVIRONMENT: SIX
CASE STUDIES INVOLVING COLLABORATION AND CIVIC ENGAGEMENT (Jacquelyn L. Tuxill
& Nora J. Mitchell eds., 2010), and MICHAEL DUFFIN ET AL., NAT’L PARK SERV.
CONSERVATION INST., ENGAGING YOUNG ADULTS IN A SUSTAINABLE FUTURE: STRATEGIES
FOR NATIONAL PARKS AND OTHER SPECIAL PLACES (2009).
591
See generally Energy Conservation Program for Consumer Products: Procedures for
Consideration of New or Revised Energy Conservation Standards for Consumer Products, 61
Fed. Reg. 36,974 (U.S. Dep’t of Energy July 15, 1996) (to be codified at 10 C.F.R. pt. 430);
Procedures, Interpretations, and Policies for Consideration of New or Revised Energy
Conservation Standards for Consumer Products, 82 Fed. Reg. 59,992 (U.S. Dep’t of Energy
Dec. 18, 2017) (to be codified at 10 C.F.R. 430) (issuing an RFI seeking comments on ways
to improve DoE’s process rule and noticing a public meeting on this topic).
592
Energy Conservation Program for Consumer Products: Procedures for Consideration
of New or Revised Energy Conservation Standards for Consumer Products, 61 Fed. Reg. at
36,974.
589

141

DOE’s process rule includes some valuable features that go beyond the bare
requirements of notice-and-comment rulemaking, particularly during the agenda
setting and rule development stages. The process rule has four steps, beginning with
“a screening analysis” that identifies “the product categories and technologically
feasible design options and then narrow[s] the range of design options being
considered for the development of candidate standard levels.” 593 The process rule
states that DOE “will seek expert input to conduct the necessary analyses,” and
consider, “wherever feasible, data, information and analyses received from
stakeholders.” 594 To this end, DOE regularly issues RFIs and holds public workshops
to gather information and solicit feedback on the agency’s preliminary analyses of
potential design options. The agency claims that,
[t]his emphasis on the early stages of the process is designed to enable
interested parties and DOE to engage in a more productive,
informative interaction on standards issues prior to the publication of
the [ANPRM], so that the standards development process starts with
the best possible foundation of common understanding. 595
After completing this screening analysis, DOE will typically make a preliminary
decision and issue an ANPRM to solicit public comments as the second step of the
process. The process rule emphasizes, however, that DOE “will provide interested
parties with opportunities to provide data, recommendations and other comments”
throughout the process, and the agency “will share with the public both analyses and
preliminary decisions to inform interested parties as to the progress of standards
development” and “enable the public to provide informed input to DOE at each step
of the process.” 596 DOE established an Advisory Committee on Appliance Energy
Efficiency Standards in conjunction with the process rule, and the agency regularly
uses this committee to “provide an official, organized forum for interested parties to
provide the Department with advice, information, and recommendations” throughout
the rulemaking process. 597 Moreover, DOE recognized that “consumers have rarely
participated directly in standards development,” and therefore committed to
“strengthen its efforts to inform and involve consumers and consumer representatives
in the process of developing standards.” 598 The process rule emphasizes that DOE
“encourages efforts to develop consensus among interested parties on proposals for
new or revised standards as an effective mechanism for balancing the economic,
energy, and environmental interests affected by standards.” 599 Accordingly, the rule
provides that “notwithstanding any other policy on selection of proposed standards, a

593

Id. at 36,976.
Id.
595
Id.
596
Id. at 36,977.
597
Id. at 36,979-80.
598
Id. at 36,978.
599
Id. at 36,977.
594

142

consensus recommendation on an updated efficiency level submitted by a group that
represents all interested parties will be proposed by the Department if it is determined
to meet the statutory criteria.” 600
If DOE ultimately decides to promulgate a proposed rule, it issues a notice of
proposed rulemaking (step 3), and provides the public with an opportunity to
comment before issuing its final rule (step 4). DOE is required by statute to hold a
public hearing during the NPRM stage, 601 and agency officials told us that DOE
routinely holds public hearings at each stage of the rulemaking process. These
meetings are “run formally” and feature a facilitator. Agency officials give
presentations before being “examined” or “grilled” by interested members of the
public. We were told that there is typically extensive public participation at these
hearings, and everything that is said is included in the rulemaking record. The public
hearings are accessible via webinar, so that interested persons can log on and
participate in real time from a remote location.
We believe that other agencies should consider promulgating process rules of this
nature, and that it would also be worthwhile for agencies to consider making binding
pre-commitments to other more extensive public engagement efforts in appropriate
circumstances. Agencies could, for example, promulgate rules that commit them to
developing specific plans for public engagement for each rulemaking initiative they
undertake or seriously consider. As explained in the following section, we think that
agencies should regularly adopt specific plans for public engagement for each of their
rulemaking initiatives regardless of whether they are compelled by process rules to do
so.
B.

Specific Plans for Public Engagement for Each Rulemaking Initiative

Early and thoughtful planning for public engagement in rulemaking is crucial to
its success. 602 There are good resources that provide detailed guidance on the best
ways to plan for public engagement. 603 These resources uniformly recognize that
there is no single approach to public engagement that will work for every type of
decision. There are, however, systematic ways to think about whether and how to
conduct public engagement for any particular decision. 604 Thus, these resources

600

Id.
See 42 U.S.C. § 6393(a)(3) (2012).
602
See EPA, PUBLIC PARTICIPATION TOOLKIT, supra note 221, at 17 (“The success of a
public participation program is largely determined by how thoroughly and thoughtfully it is
planned.”).
603
See, e.g., CREIGHTON, supra note 220, at 27-87; EPA, BETTER DECISIONS MANUAL,
supra note 278, at 19-63; EPA, PUBLIC PARTICIPATION TOOLKIT, supra note 221, at 8-25;
DOUGLAS MISKOWIAK, CTR. FOR LAND USE EDUC., CRAFTING AN EFFECTIVE PLAN FOR
PUBLIC PARTICIPATION (2004).
604
See CREIGHTON, supra note 220, at 27 (“There is no such thing as a one-size-fits-all
public participation plan. But there is a systematic way of thinking through the issues that
will help produce a successful plan that fits the unique requirements of a particular decision
601

143

typically divide the planning process into various stages, and identify the types of
questions that one should address at each stage. They also provide sample
worksheets that one could use to conduct the recommended analyses.
James Creighton, the founding president of IAP2, suggests that planning for
public participation involves three distinct stages. 605 The first stage, decision
analysis, involves clarifying the decision at issue, specifying the stages of the
decision-making process and establishing a schedule, and deciding “whether public
participation is needed and for what purpose.” 606 The second stage, process
planning, involves establishing the goals of public participation at each stage of the
decision-making process, identifying the relevant internal and external stakeholders,
selecting the modes of public engagement that will be used at each stage, and
developing an integrated public engagement plan. 607 The third stage, implementation
planning, involves working out the logistics necessary to conduct the selected forms
of public participation effectively. 608 Creighton emphasizes that,
[b]efore we ask anyone to spend their time participating in our
decision-making processes, we owe it to them to ensure that we are
offering them the opportunity to participate in a manner and at a time
that gives them the greatest opportunity to have a useful influence on
the decisions being made. 609
He points out that during the decision analysis stage, it is important to develop a
broad consensus within the organization on the nature of the decision and the extent
to which public participation is necessary. 610 It is also important to identify any
constraints that could undermine the feasibility or effectiveness of the agency’s public
engagement efforts. 611 Public engagement in rulemaking can only succeed if both the
agency and interested members of the public are truly committed to the effort.
Similarly, a report prepared by the Center for Land Use Education in
collaboration with USDA on “Crafting an Effective Plan for Public Participation,”
provides a useful outline of what a “model public participation plan” should entail.612

or issue.”); see also Nash & Walters, supra note 218, at 25-28 (suggesting five principles of
effective public engagement, including (1) early and frequent efforts to engage the public, (2)
fair and neutral treatment of participants, (3) efforts to facilitate a broad range of views and
perspectives, (4) the selection of appropriate methods, and (5) a spirit of experimentalism and
self-evaluation).
605
See CREIGHTON, supra note 220, at 27-28 (summarizing the stages of planning).
606
Id. at 28-44.
607
See id. at 28, 45-77.
608
See id. at 28, 78-83.
609
Id. at 27.
610
See id. at 29.
611
See id. at 41.
612
See MISKOWIAK, supra note 603, at 4.

144

This report recommends providing an introductory section that describes the relevant
decision, identifies the intended audience and explains how they should use the public
participation plan, and provides “an overarching vision for public participation” for
the decision at issue. 613 The bulk of the public participation plan would center on the
“categories of information” that are necessary “to craft an effective public
participation plan,” or what the report calls the “Four Cornerstones of the PPP.”614
These four cornerstones include (1) the purpose cornerstone – “[W]hat the public is
involved to do and when”, (2) the people cornerstone – who should be involved in
those efforts, (3) the methods cornerstone – which modes of public engagement
should be used, and (4) the evaluation cornerstone – how will the public engagement
efforts be documented and evaluated. 615 After a public participation plan carefully
addresses each of these cornerstones, the report recommends that the plan should
provide an integrated public engagement strategy “for implementing and evaluating
public participation activities” based on this information. 616 The report also explains
that the people cornerstone requires a stakeholder analysis that identifies the people or
groups that should be targeted for participation based on a careful assessment of who
would likely be interested in or affected by the relevant issues, 617 and points out that
“planning consultants” can be hired in appropriate circumstances “to provide a full
range of technical planning and public participation products and services.” 618
EPA’s Public Participation Toolkit also includes useful information on
planning. 619 The relevant portion of this report begins with a discussion of “situation
assessments,” which are “conducted for the purpose of understanding the needs and
conditions of your project and stakeholder community in order to design an effective
public participation process.” 620 A situation assessment involves “gathering
information to determine the public participation program and techniques that are
feasible and most appropriate for the circumstances.” 621 The report distinguishes
between organized and grassroots stakeholders, and recognizes that “sponsoring
agencies often have to be highly proactive in reaching out to and engaging”
traditionally absent stakeholders. 622 The report emphasizes that “[i]t is important to
identify and seek out the full range of interests and perspectives that are potentially
affected by a project and ensure that their voices are heard.” 623 To this end, the report
details the “key findings” that should result from a situation assessment, the rationale
613

Id. at 4-5.
Id. at 4.
615
See id. at 7.
616
Id. at 4.
617
See id. at 12.
618
Id. at 13.
619
See EPA, PUBLIC PARTICIPATION TOOLKIT, supra note 221, at 8-25.
620
Id. at 8.
621
Id.
622
Id.
623
Id.
614

145

for engaging in this exercise, and the manner in which it should be conducted. 624
Situation assessments are generally conducted in two phases, including (1) an
internal assessment, which is intended “to clarify the problem or opportunity, the
decision to be made, available resources and commitment for public participation,
and the sponsor agency’s expectations about the appropriate level of public
participation,” and (2) an external assessment, which is intended “to identify the full
range of external stakeholders that should be engaged and to learn from the public to
understand how stakeholders perceive the situation and decision to be made.” 625
After discussing how to conduct a situation assessment, the report explains how the
agency should use the results of its analysis. 626 This includes an agency’s selection of
the appropriate level of public participation for the decision based on the spectrum
provided by IAP2. 627
The Public Participation Toolkit breaks the process of public participation
planning into five steps. 628 The first step involves organizing for public
participation. 629 This step includes ensuring that meaningful public participation is
possible and that the agency is committed to the requisite efforts. 630 The report
wisely recognizes that “[i]f there is little or no room for public influence over the
decision, then public participation is not a reasonable option for your project.” 631
This initial step also includes identifying and securing the necessary resources for
creating and implementing the public engagement plan, and identifying “where public
input is desired and possible.” 632 The report emphasizes the importance of clearly
specifying “the specific issues and questions where public input is desired and where
the public can have influence,” and notes that “[t]he more clearly you articulate the
areas for input, the more meaningful the ultimate input will be.” 633 The second step
of public participation planning involves identifying and getting to know the relevant
stakeholders. 634 As discussed above, the report emphasizes the importance of
conducting a situation assessment “to understand who might be impacted, who should
be involved, and what concerns they bring to the process,” and claims that it is vital to
“identify all of the viewpoints and interests that must be heard to create a fully

624

See id. at 8-11.
Id. at 9.
626
See id. at 11.
627
See LUKENSMEYER & TORRES, supra note 141, at 7 (describing IAP2’s spectrum of
public participation).
628
See EPA, PUBLIC PARTICIPATION TOOLKIT, supra note 221, at 17.
629
See id. at 17-18.
630
See id.
631
Id. at 17.
632
Id. at 18.
633
Id.
634
See id. at 19.
625

146

participatory process.” 635 When this situation assessment is completed, the agency
should have produced a comprehensive list of stakeholders that will provide the
foundation for its outreach efforts and ensure that the agency is “reaching the full
range of community interests throughout the project.” 636 The report also recommends
building relationships with stakeholders to develop an understanding of their views
and perspectives on the project. It suggests that the best way to achieve this goal is to
conduct interviews during the project planning stage with a broad range of
stakeholders who are representative of the competing interests at stake. 637 The third
step of the planning process involves selecting the appropriate levels of public
participation for the relevant decisions. 638 The fourth step involves “integrat[ing]
[p]ublic [p]articipation in the [d]ecision [p]rocess” by identifying, among other
things, the key stages and relevant timeline of the decision-making process and
precisely when public input will be sought and used. 639 The final step in the planning
process is to select the appropriate modes of public participation to use at each of the
respective stages. 640
EPA’s Conflict Prevention and Resolution Center has also produced a manual
entitled “Better Decisions through Consultation and Collaboration” that provides
detailed advice on designing and implementing public engagement processes. 641 The
Better Decisions manual breaks the process of “preparing for involving stakeholders”
into five stages. 642 The first stage involves conducting an internal situation
assessment, which includes an analysis of the precise nature of the decision facing the
agency, the agency’s goals and concerns, and an assessment of how the decision fits
within the agency’s broader agenda. At this stage, the agency should make a
preliminary determination of what level of stakeholder involvement seems most
appropriate for the decision at issue. The manual points out that agency management
and staff from other offices are important “internal stakeholders,” and recommends
integrating them into the decision-making process through early involvement,
obtaining their buy-in along the way, and keeping them engaged throughout the
rulemaking process. 643 The second stage involves conducting an external situation
assessment, which is intended to secure input and advice from stakeholders outside

635

Id.
Id. at 20.
637
See id.
638
See id.
639
See id. at 24.
640
See id. at 25.
641
See EPA, BETTER DECISIONS MANUAL, supra note 278, at 1 (“This document is a
resource guide on public involvement best practices and strategies for EPA staff who are
tasked with designing and/or implementing public involvement processes for various EPA
activities.”).
642
See id. at 17 (providing a concise summary of the five stages).
643
See id. at 21-22.
636

147

the agency about the proposed public engagement process. 644 The primary
components of the external situation assessment include “identifying stakeholders,
interviewing representatives of affected interests, identifying issues to discuss in a
stakeholder involvement process, assessing the willingness of stakeholders to
participate, projecting likely outcomes, and recommending a detailed stakeholder
involvement process.” 645 The chapter on external situation assessments provides
helpful information on when to begin this aspect of the planning process, when to use
a neutral convener, and how to identify potentially interested stakeholders, along with
detailed advice on how to conduct the external assessment process. 646 When agencies
use a convener, this person should prepare a report with findings and
recommendations for the design of the public engagement process, which the agency
can use to finalize its public engagement plan. 647
Once the internal and external situation assessments have been completed, the
third stage of planning for public engagement involves designing the details of the
process. 648 As indicated above, the agency and its facilitator must consider (1) why
the agency wants to engage with the public, (2) who the agency is trying to reach, (3)
what type of information the agency is seeking, (4) how this information is likely to
be obtained, (5) when these efforts should occur, and (6) what the agency will do with
the information. 649 The fourth stage of the process involves conducting the public
engagement, 650 and the final stage of the process involves “benefitting from the
results.” 651 The Better Decisions manual emphasizes the importance of incorporating
the results of public engagement efforts into the agency’s decision and providing
meaningful feedback to participants regarding how their input influenced the agency
decision. 652 Agencies should also learn from their experiences with public
engagement efforts, and “tell their stories” so that others can learn from their efforts
and the best practices for public engagement in rulemaking will continue to emerge,
improve, and evolve. 653
Based on a review of the foregoing resources, it is striking how much time and
effort is required to create and implement an effective public engagement plan. We
are confident, however, that careful planning is essential to effective public
engagement, and that making this investment in the rulemaking context will routinely

644

See id. at 29.
Id. at 17.
646
See id. at 30-37.
647
See id. at 37-39.
648
See id. at 43.
649
Cf. id. at 17 (designing the process “includes the who, what, when, and how”).
650
See id. at 67.
651
Id. at 75.
652
See id. at 75-79. See also supra Part VII.F (discussing the importance of providing
feedback and explaining how this can be done).
653
See EPA, BETTER DECISIONS MANUAL, supra note 278, at 79-81.
645

148

be worth the effort. Accordingly, we recommend that agencies develop general
policies for public engagement in rulemaking and establish mechanisms to ensure that
those policies are consistently followed. In this regard, agencies should consider
adopting “process rules” that require them to consider conducting certain public
engagement efforts for each rulemaking initiative they undertake or seriously
consider, and provide reasons or “good cause” when they decide to dispense with
these procedures. Such process rules could include, for example, provisions requiring
agencies to issue RFIs and conduct listening sessions early in the rule development
process. Those process rules could also require agencies to consult with federal
advisory committees at various stages of the rulemaking process, to hold public
meetings in appropriate situations, and to issue ANPRMs on a regular basis. Perhaps
most importantly, we recommend that agencies should develop specific plans for
public engagement for each rulemaking initiative they undertake or seriously
consider. Agencies should seriously consider hiring neutral conveners or facilitators
to carry out these planning efforts. These public engagement plans should include
internal and external situation assessments and address (1) why the agency wants to
engage with the public, (2) who the agency is trying to reach, (3) what type of
information the agency is seeking, (4) how this information is likely to be obtained,
(5) when these efforts should occur, and (6) what the agency will do with the
information.
There are also a few simple structural reforms that would likely improve these
efforts to facilitate planning for public engagement in rulemaking. First, agencies
could assign a public engagement specialist or advocate to each rulemaking team to
ensure that these efforts are taken seriously. Second, agencies could provide advance
notice of public engagement opportunities in their Regulatory Plans and the Unified
Agenda whenever possible. They could also routinely provide more detailed
information and as much advance notice as possible regarding those opportunities on
agency websites and through other communications with stakeholders. Finally,
agencies could establish units of public engagement experts (or at least employees
who have been trained in these processes), which could be available as a resource to
help rulemaking teams plan their public engagement efforts. Indeed, we think that it
may be more efficient and effective for Congress to establish and fund a new federal
agency or division of an existing agency that is specifically responsible for helping
other agencies plan their public engagement efforts. As explained further below, this
is conceivably a role that could be provided by ACUS, if it were provided with the
funding and personnel necessary to perform this function effectively.
C.

Outreach and Communication

Technological advances have not yet revolutionized public engagement with
rulemaking and there are still many challenges to overcome. One of the most
significant challenges is for agencies to identify missing stakeholders with situated
knowledge of the subject of regulation, and to persuade them to participate in the
rulemaking process in beneficial ways. This requires both careful advance planning
and effective outreach strategies. As explained below, we think this is perhaps the
area where technological advances could potentially have the greatest payoff.

149

EPA’s Better Decisions manual provides a list of useful “suggestions for finding
stakeholders.” 654 For example, it recommends checking the dockets from prior
versions of a proposed rule or related policies to identify interested parties, contacting
officials from relevant offices throughout the agency for recommendations,
consulting directories of public interest groups and trade associations, posting notices
on the agency’s website and in the Federal Register, providing press releases or
placing advertisements in trade journals and other relevant publications, and
“[s]earch[ing] the Internet.” 655 While these are plainly good ideas that should
routinely be followed, they are most likely to bring rulemaking proceedings and other
opportunities to participate to the attention of relatively sophisticated parties who are
already known by the agency or pay attention to traditional sources of information
about agency activities. Agencies will likely need to pursue other more creative
outreach efforts to recruit missing stakeholders, and those efforts are the primary
focus of the remainder of this section. In any event, we agree with the manual’s
authors that agencies can “stop searching for stakeholders” when they are “confident”
they “have discovered all the sides of the issue and all the major players.” 656
Agencies “don’t have to find everyone, just representatives of the different points of
view.” 657
Cynthia Farina and her colleagues have also made useful suggestions for
conducting effective outreach, focusing in particular on efforts to involve missing
stakeholders based on their experiences with Regulation Room. 658 The goals of
effective outreach are to bring the agency’s rulemaking initiative and available
opportunities to participate to the attention of members of the targeted audiences and
“craft messages that motivate them to respond.” 659 This requires, at a minimum, that
agencies undertake proactive efforts to engage in communications that are likely to
reach their targeted audiences, emphasize in those communications that the agency is
seeking public input and clearly explain how to participate, and persuade members of
the targeted audiences why they should care and why their participation is worth the
effort. 660 These efforts can be achieved, first, by “[d]eveloping an outreach plan,”
and, second, by “[c]rafting message content that motivates engagement.” 661
The outreach plan should be designed “to put information about the rulemaking
in places where members of the targeted participant groups are likely to come across

654

See id. at 32-33.
See id.
656
Id. at 33.
657
Id.
658
See FARINA & NEWHART, IBM CENTER, supra note 22, at 21-26. For a more detailed
discussion of the outreach efforts associated with Regulation Room, see Farina et al.,
Rulemaking in 140 Characters or Less, supra note 8, at 393-416.
659
Farina & Newhart, IBM CENTER, supra note 22, at 21.
660
See id. at 21-22.
661
Id. at 22.
655

150

it.” 662 This means that the plan should be “tailored to the specific rule and to the
targeted types of participants.” 663 It should include direct communication with
targeted stakeholders, where possible, as well as the proactive use of social and
conventional media, and efforts to communicate with groups that are likely to pass
messages along to members of the targeted audiences. An effective outreach plan
requires “significant human effort,” and “the barrier of unawareness” that afflicts
most missing stakeholders cannot easily be overcome solely by social media or other
technological innovations. 664 Thus, Farina and her colleagues explain:
On Regulation Room, we try to identify places where targeted
participants are likely to go for information, including membership
associations; subject-matter, recreational, and trade publications; and
influential individual opinion leaders such as bloggers and newsletter
authors. We reach out to these sources through e-mail, phone, social
networking, and other online communication, asking them to publicize
information about the rulemaking and how individuals can
participate . . . . We also develop a list of keywords and phrases
related to the rule that are likely to have impact for the targeted groups.
We use these both proactively in daily tweeting, Facebook posting,
and Facebook, Google, and Twitter ads; and reactively by
continuously monitoring Internet activity and responding with
comments and tweets about the opportunity to participate whenever
the rulemaking or its issues appears on blogs, news sites, or Twitter. 665
While agencies routinely inform the public about their rulemaking activities,
effective outreach also requires conscious efforts to develop messages that will
persuade members of targeted audiences to participate. 666 In other words, agencies
need to “recruit” as well as to “inform.” Farina and her colleagues provide three
primary suggestions for how this can be done. First, they emphasize the importance
of making “the process part of the message” so that whenever and wherever the
agency’s rulemaking initiative is discussed, citizens are told of their opportunities to
participate (and ideally provided with links that either provide more information or
directly allow them to provide input). 667 Second, they emphasize the importance of
personalizing the impact of the rulemaking initiative by clearly and concretely
explaining how it could positively or negatively affect the targeted audiences’
interests. 668 Third, they recommend seeking to “[m]otivate organizations to help
spread the word” by enlisting organized groups to pass along messages to members or

662

Id.
Id. at 22.
664
Id. at 22-23.
665
Id. at 22 (emphases in original).
666
See id. at 24.
667
Id.
668
Id. at 24.
663

151

allies on behalf of the agency whenever possible, and they provide some helpful
advice on how to overcome the reluctance that organized groups sometimes exhibit
when asked to perform this function. 669
It is difficult to go beyond these recommendations and provide a more detailed
set of uniform best practices for effective outreach for several reasons. First, the best
approach to outreach will naturally vary depending on the circumstances, including
the identities of the targeted participants, and the most effective ways of reaching
them and persuading them to participate. Second, figuring out how to carry out the
requisite tasks successfully is itself a specialized form of expertise. Part of what it
means to be an expert on civic engagement is being able to conduct effective outreach
in any particular situation. Just as one generally needs to be a baseball player to use
the right launch angle and hit a home run, one may need training and experience in
public engagement efforts to understand how to conduct outreach effectively. Third,
the technology that exists to assist with outreach efforts is not always well known or
widely available, can be difficult or complicated to understand, may be controversial
in some respects, and is constantly evolving.
Accordingly, preparing and
implementing an effective outreach strategy for any particular rulemaking proceeding
will necessarily require access to a certain amount of technical expertise.
We therefore believe that structural or institutional reforms may be necessary to
help agencies identify, reach, and involve missing stakeholders in their rulemaking
proceedings on a regular basis. The simplest strategy would be for agencies to ensure
that their rulemaking teams include an expert on public engagement with experience
in conducting outreach that targets missing stakeholders and unaffiliated experts.
Agencies could hire outside consultants to perform this task on an as needed basis or
hire or provide training for a designated group of agency staff who could be available
to perform this function regularly. Agencies would also need to ensure that their
rulemaking teams have the technical support necessary to help them use the best
available information technology to carry out their outreach activities. If Congress
established and funded a new federal agency or division of an existing agency that
was responsible for helping other agencies plan their public engagement efforts, this
entity could also be responsible for facilitating effective outreach. Alternatively,
Congress could establish a new federal agency or division of an existing agency that
is charged, more narrowly, with helping other agencies conduct effective outreach
regarding their rulemaking activities. For example, New York City established the
Public Engagement Unit (“PEU”) as a new division of its city government in 2015.
PEU includes a team of outreach specialists who are responsible for meeting with
otherwise hard-to-reach constituents to ensure they have access to vital city services
and build long-term relationships between those constituents and city staff. 670
According to Hollie Russon Gilman and Sabeel Rahman,

669

Id. at 25-26.
See HOLLIE RUSSON GILMAN & K. SABEEL RAHMAN, NEW AM., BUILDING CIVIC
CAPACITY IN AN ERA OF DEMOCRATIC CRISIS 13 (2017).
670

152

PEU serves as an interface through which residents engage with local
government to better enable city agencies to identify and resolve
individual cases, as well as large community issues. PEU works
across agencies to build capacity among outreach teams and is
implementing new outreach tools, technology, and best practices to
integrate an accessible, door-to-door community engagement approach
throughout the city. These tools support strong partnerships with city
agencies to help maximize outreach for new services and engage New
Yorkers. 671
Regardless of precisely who designs the outreach, there is little doubt that
information communication technologies will play an increasingly prominent role.
Beth Simone Noveck, who served as the first United States Deputy Chief Technology
Officer and director of the White House Open Government Initiative under President
Obama, has written extensively about the emerging possibilities. 672 Noveck
recognizes that open calls for participation have severe limitations, and that a
common challenge for conducting effective outreach is “the lack of clear, costeffective, and reliable ways to find those with the right expertise, insights,
information, and innovative solutions.” 673 She therefore emphasizes the importance
of developing and tapping “the ability to match people and problems” and thereby
“targeting expertise,” 674 and she defines expertise broadly to include what we have
characterized throughout this report as “situated knowledge.” 675 And this is where
technology comes in: Noveck claims that “[a]lthough the tools and approaches are
still evolving, it may soon become easy to identify with precision who knows what
and match them to opportunities to serve.” 676
There are at least two related ways in which agencies could use the latest
technology to target expertise in this fashion. The first involves using expert
networking platforms to identify people with relevant knowledge, information,
experiences, or interests, 677 and then engaging in the kinds of “micro-targeting” that

671

Id.
See generally NOVECK, SMART CITIZENS, supra note 259.
673
Id. at 22.
674
Id. at 42-43.
675
See, e.g., id. at 102 (“Ground-level knowledge of cultural context and the sensitivities
and priorities of relevant stakeholders, as well as of physical and natural conditions, is an
important kind of expertise that emerges from practical experience and that bureaucrats often
have a hard time replicating.”); see also id. at 104-05 (“Professionals inside an organization
lack adequate access to diverse and innovative sources of expertise, especially in situations
where situational awareness, local know-how, craft ability, disciplinary diversity, and the
gritty ‘deep smarts’ of lived experience are critical to good outcomes.”); id. at 251
(discussing three relevant kinds of knowledge: “expertise within government; credentialed
expertise outside of government; and nontraditional forms of distributed know-how”).
676
Id. at 101.
677
See id. at 106-11.
672

153

are conducted by advertisers to invite them to participate in an agency’s public
engagement efforts. 678 Noveck recognizes that while profiling consumers based on
their online behavior “has become part of everyday commerce, these applications of
user segmentation and targeting make some people uncomfortable.” 679 Nonetheless,
she argues that “if we can develop the algorithms and platforms to target consumers,
can we not also target citizens for the far worthier purpose of undertaking public
service?” 680 The second involves using similar technology, and the enhanced ability
that it creates to identify people with nontraditional, experiential forms of expertise
that are enabled by social media and other Internet-related activities, to develop
advanced databases of citizens with special expertise on a broad and diverse range of
subjects. 681 Noveck claims that the development of a searchable “directory of
directories” or “Brain Trust” of civic knowledge would allow agencies to match the
supply of available expertise with their particular demands or needs in any particular
case. 682 This information and related capacity would have the potential to facilitate
and greatly improve the effectiveness of an agency’s outreach efforts at each stage of
the rulemaking process. Noveck thus concludes that “[w]hen we can see with
precision who knows what, we can harness that know-how for the public good—that
is, make our democracy, starting with the administrative state . . . more
participatory.” 683
Noveck acknowledges there is still a long way to go before her vision could be
fully realized, 684 but some of her ideas could already be used to begin to improve
public engagement with rulemaking. For starters, agencies should consider using
expert networking platforms and other information communication technologies to

678

See id. at 109-11 (describing how companies like Facebook and Google “are learning
to micro-target ads with even more precision to exactly the desired demographic, using
proprietary modeling and segmentation techniques,” and suggesting that public agencies
could use similar methods).
679
Id. at 110.
680
Id.
681
See id. at 106-36 (providing an in depth discussion of how “technology is changing
expertise”); id. at 111-12 (explaining that “[t]he Internet is changing our relationship to
expertise” by (1) “making a broad range of expertise—including more forms of craft
knowledge as well as skills, experience, interests, and credentials—more visible, independent
of institutions;” (2) “permitting expertise to be identified by means of manual and automatic
data collection, thus enabling an infinite diversity of expressions of expertise based on
different traits and characteristics;” (3) expanding “the number and types of expertise
markers, including the shift from certification to badging;” and (4) enabling the quantification
of expertise in different and more diverse ways).
682
See id. at 210-11, 218-22.
683
Id. at 271.
684
See, e.g., id. at 132-33 (discussing “technological obstacles” and acknowledging that
“[s]earching for and targeting specific expertise, even within a single organization such as a
government agency, is not yet something we know how to do well or comprehensively or
consistently”); id. at 184-85 (describing potential legal obstacles).

154

identify people with relevant knowledge, information, experiences, or interests.
Agencies should notify those individuals or groups of available opportunities to
participate in the rulemaking process, and provide them with the information that is
necessary for them to do so effectively. More work needs to be done to determine the
best ways of communicating this information to missing stakeholders, and agencies
should therefore consider experimenting with different approaches to determine what
works best for their particular constituencies. We fully agree with Noveck, however,
that the relevant constituencies will routinely include experts within government,
unaffiliated experts outside of government, and citizens with situated knowledge or
what she calls “distributed know-how.” 685 We also agree that agencies should at least
consider using the best available technology in a responsible manner to target citizens
for the purpose of undertaking public service. “[J]ust as an advertiser wants to be
able to target the right audience with ads matched to their interests, a policymaker
should be able to invite those with something to contribute.” 686
Agencies should also consider using information communication technologies to
begin to develop databases, directories, or “brain trusts” of public officials,
unaffiliated experts, and ordinary citizens with relevant forms of expertise. Those
directories could then be tapped by agencies when they conduct outreach for their
public engagement efforts. Although Noveck seems to envision a master directory of
virtually unlimited forms of expertise throughout the country, we think agencies
could benefit simply from creating their own searchable, digital directories of public
employees (which should include state, local, and tribal officials), unaffiliated
experts, and ordinary citizens with various forms of expertise that are relevant to their
delegated statutory authority. Such directories could specifically identify people who
have proven especially helpful or active in prior rulemaking proceedings, and they
should receive recognition from the agency and become candidates for “repeat
business.” This would create an incentive for interested members of the public to
participate effectively in an agency’s public engagement efforts. While agencies have
likely been conducting similar forms of outreach throughout American history—
albeit through rolodexes and more primitive forms of communication—modern
information communication technologies open up the possibility for more creative
and effective forms of outreach that can also be more open, egalitarian, and
meritocratic, and could therefore further enhance democracy. 687 Accordingly, this is
where technological innovations may potentially have the greatest payoff for
improving public engagement with rulemaking.
At the end of the day, there may be interests or perspectives that are not
adequately represented in the rulemaking process despite an agency’s best efforts to
conduct targeted outreach. Therefore, it is also worthwhile to consider appointing

685

See id. at 251.
Id. at 111.
687
See id. (“The Internet is democratizing expertise and opening up the possibility of
connecting anyone to an opportunity to engage in the life of our democracy that speaks to his
or her talents.”).
686

155

ombudspersons (or similar individuals or entities) to represent the concerns of
missing stakeholders, particularly when collective action problems are likely to prove
most intractable. The National Taxpayer Advocate, who was established by Congress
to represent the interests of low income taxpayers on various matters involving the
IRS, is widely viewed as a successful example of this model. 688 Scholars have
pointed out that public interest advocates of this nature can improve the quality of
information that is available to agencies and help to prevent regulatory capture. 689
These representatives can be formally established or designated by Congress, the
White House, or by the agencies themselves, and they can be part of a larger office
that is charged with promoting good government in ways that may be less central to
the agency’s substantive statutory mission. 690 Of course, such offices must be
designed with care to ensure that they have an appropriate degree of influence and
stay true to their assigned functions. 691 However, designated representatives of
missing stakeholders could provide agencies with a more complete and well-balanced
array of views and perspectives and thereby further improve the democratic
legitimacy, effectiveness, and accountability of the rulemaking process.

688

See generally Leslie Book, A New Paradigm for IRS Guidance: Ensuring Input and
Enhancing Participation, 12 FLA. TAX REV. 517 (2012) (discussing the success of the
National Taxpayer Advocate and proposing reforms that would enhance the Taxpayer
Advocate Service’s role in influencing IRS decision making).
689
See Rachel E. Barkow, Insulating Agencies: Avoiding Capture Through Institutional
Design, 89 TEX. L. REV. 15, 62 (2010) (suggesting that agency capture resulting from
asymmetrical political pressure can be counteracted in part by establishing “a formal position
of public advocate who is charged with representing the public’s interest before the agency”);
Wagner, Administrative Law, supra note 39, at 1414 (claiming that one strategy for
addressing imbalanced participation is to deploy agency-selected ombudspersons, advocates,
or other government intermediaries “to stand in for significantly affected interests that might
otherwise be underrepresented in rulemakings”).
690
See generally Brett McDonnell & Daniel Schwarcz, Regulatory Contrarians, 89 N.C.
L. REV. 1629 (2011) (describing existing variations on these offices and their potential for
improving financial regulation); Margo Schlanger, Offices of Goodness: Influence Without
Authority in Federal Agencies, 36 CARDOZO L. REV. 53 (2014) (exploring subsidiary offices
within agencies that are charged with promoting normative values that go beyond and
potentially even cut against their primary statutory missions).
691
See Schlanger, supra note 690, at 103 (“Offices of Goodness cannot increase the
amount of Goodness in an agency without two capacities: influence and commitment.”).

156

XIII. RECOMMENDATIONS
A.

Public Engagement Planning and Infrastructure
1.

Public Engagement Policies

Agencies should develop and make publicly available general policies for public
engagement in rulemaking. The policies should require the agency to consider the
following questions at the beginning of each rulemaking initiative it undertakes or
seriously considers: (1) the agency’s goals and purposes in engaging the public; (2)
the types of individuals or organizations with whom the agency seeks to engage,
including experts and any affected interests that may be absent from or insufficiently
represented in the notice-and-comment rulemaking process; (3) how such types of
individuals or organizations can be motivated to participate; (4) what types of
information the agency seeks from its public engagement; (5) how this information is
likely to be obtained; (6) what the agency will do with the information; (7) when
public engagement should occur; and (8) the range of methods of public engagement
available to the agency. Agencies should consider the full range of stakeholders that
may have information, views, or data relevant to the rulemaking and how to engage
them. In particular, the agency should seek to identify individuals and groups who
traditionally are absent from the notice-and-comment process; any information,
views, or experiences they might possess that is relevant to the rulemaking; how best
to reach such stakeholders and involve them in the rulemaking process; and the
resources available to do so. Planning for public engagement for specific rules would
best take place at the earliest feasible part of the rulemaking process. Agencies
should also consider adopting process rules that presumptively require certain types
of public engagement efforts at certain stages of the regulatory process for each
rulemaking initiative they undertake or seriously consider. At the same time,
agencies should consider employing a wide variety of modes and techniques to obtain
the views and insights of different individuals and groups. Agencies should plan for
public involvement that is as open and inclusive as possible so that diverse publics,
including those who typically do not participate, have opportunities to share their
views, values, and concerns.
Agencies should establish mechanisms to ensure that their public engagement
policies are consistently followed.
2.

Public Engagement Plans

Based on the framework established by their public engagement policies,
agencies should develop specific public engagement plans (PEPs) for each
rulemaking initiative they undertake or seriously consider.
The PEP should identify how, and at what stages, the agency will invite the
public to participate in its decision-making process. The extent of the public’s role
may vary from issue to issue, and at different stages of the rulemaking process. The
agency should provide opportunities for public involvement early in the decisionmaking process when the rulemaking may significantly affect or interest the public.

157

The PEP should also identify whether the rulemaking is likely to involve
controversial issues or require specialized techniques to handle potential conflicts.
Such techniques may include the use of facilitators, mediators, or other persons
trained in alternative dispute resolution to help resolve controversial issues. In such
cases the agency should also consider using negotiated rulemaking.
The PEP should identify whether and if so how the agency will use the Internet
and social media to expand public access to information and opportunities to
participate.
Agencies should publish their PEPs in the Federal Register and on their
websites. Whenever possible, agencies should also provide advance notice of
specific public engagement opportunities in their Regulatory Plans and the Unified
Agenda. Agencies should routinely provide more detailed information and as much
advance notice as possible regarding those opportunities on their websites and
through other communications with stakeholders.
3.

Enhancing Agency Capacity for Public Engagement

Agencies should develop or maintain their capacity for public engagement in
rulemaking by encouraging employees to become knowledgeable about civic
engagement and public involvement techniques and principles. Agencies should
maintain interdisciplinary training materials and support opportunities to train
employees responsible for public involvement activities to understand and apply
recognized “best practices” in the field.
Agencies should designate certain employees or establish new positions with
responsibility for supporting and fostering efforts to engage a broad and diverse
public in their rulemakings. These employees should be trained in procedures and
practices aimed at involving rulemaking novices and unaffiliated experts in the
regulatory process, and serve as resources for rulemaking teams planning and
executing their public engagement efforts. Agencies should also consider assigning a
public engagement specialist or advocate to each rulemaking team to ensure that the
commitment to public engagement is taken seriously. In addition, agencies should
consider using personnel with public engagement training and experience to
participate in both the development of their general public engagement policies and
planning for specific rules.
Finally, Congress should consider establishing and funding a new federal agency
or division of an existing agency that is specifically responsible for helping other
agencies plan their public engagement efforts. Congress should also consider
establishing ombudspersons (or similar individuals or entities) to represent the
concerns of missing stakeholders in rulemaking proceedings, particularly when
collective action problems are likely to prove most intractable. Agencies should
consider establishing or designating public interest advocates of this nature when
appropriate in the absence of legislation.

158

4.

Targeted Outreach.

When agencies believe that their public engagement may not reach all affected
interests, they should consider conducting outreach that targets experts not already
likely to be involved, individuals with knowledge germane to the proposed rule who
do not typically participate in rulemaking, and members of the public with relevant
views that may not otherwise be represented. These targeted outreach efforts should
include:
a. proactively bringing the rulemaking to the attention of affected interests that
do not normally monitor the agency’s activities;
b. overcoming or minimizing possible geographical, language, resource, or other
barriers to participation;
c. motivating participation by explaining the nature of the rulemaking process
and how the agency will use public input; and
d. providing information about the issues and questions raised by the rulemaking
in an accessible and comprehensible form and manner, so that potential participants
are able to provide focused, relevant, and useful input.
B.

Agenda Setting
1.

Petitions for Rulemaking

Agencies should post in a prominent place on their websites a plain language
explanation of the opportunity to submit rulemaking petitions. Agencies should
include instructions and guidance on filing petitions for rulemaking that are
understandable to their stakeholders and members of the general public who may be
unfamiliar with the rulemaking process. This guidance should include the types of
data, arguments, and information the agency finds most helpful for evaluating and
deciding whether to grant petitions for rulemaking. Agencies should also include
examples of successful (and unsuccessful) petitions for rulemaking along with
explanations of why the petitions succeeded (or failed). The examples should include
petitions to create a new rule, petitions to amend an existing rule, and petitions to
repeal a rule.
Agencies should announce petitions for rulemaking in the Federal Register, on
Regulations.gov, and on their websites. Agencies should also post petitions for
rulemaking and related documents, including any agency action on the petitions, on
Regulations.gov and on their websites to allow the public to monitor the progress of
petitions. Agencies should note on Regulations.gov and on their websites whether
they are seeking public comments on petitions for rulemaking.
Agencies should consider whether to solicit public comments on their petitions
for rulemaking as a matter of course or on a case-by-case basis.

159

Agencies should consider affirmatively soliciting comments from regulatory
beneficiaries and unaffiliated experts who do not traditionally participate in the
notice-and-comment process when they are seriously considering petitions filed by
regulated entities and these groups may have relevant information, views, or
experiences to share. Agencies should be particularly mindful of soliciting comments
from regulatory beneficiaries when they are seriously considering a petition to amend
or repeal a rule filed by a regulated entity seeking regulatory relief.
When agencies seek comments on petitions for rulemaking from individuals and
groups traditionally absent from the rulemaking process, agencies should summarize
the key issues raised by the petition and describe the specific types of information,
views, and experiences that would be helpful for the agency to hear about in order to
decide whether to grant the petition. Agencies should also conduct targeted outreach
to encourage these groups to participate in the consideration of the petition.
2.

Hotlines and Suggestion Boxes

Agencies should consider establishing hotlines or suggestion boxes on their
websites to help rulemaking novices and members of the general public raise issues
or concerns and submit suggestions related to the agency’s regulatory agenda. When
establishing hotlines or suggestion boxes, agencies should ensure that staff is trained
to answer the calls and respond to the messages in a reasoned and reasonably prompt
manner.
3.

Requests for Information (RFIs)

Requests for Information (RFIs) are an important way for agencies to solicit data,
views, or other information from the public when agencies need to determine the
existence, magnitude, or nature of a regulatory problem and evaluate potential
strategies to address the issue, which may involve rulemaking. Agencies should
presumptively use RFIs to solicit public comments on their regulatory agendas when
they have some discretion over their agenda or priorities and are open-minded about
which matters to pursue or when to pursue them. When using RFIs for agenda
setting, agencies should (1) remain neutral regarding at least part of their agenda and
(2) pose detailed questions aimed at soliciting the information they need to make
informed decisions about their priorities; and (3) indicate that they are open to input
on other questions and concerns.
When using RFIs in connection with agenda setting, agencies should identify
(1) any individuals or groups who may have relevant data, views, or other
information but are traditionally absent from the notice-and-comment process, and
(2) any unaffiliated experts who may have special expertise on the relevant issues.
Agencies should address these groups directly in their RFIs and engage in targeted
outreach to encourage participation by these groups.

160

Agencies should review any comments they receive in response to RFIs and
explain in subsequent public notices how these comments informed or influenced the
agency’s course of action.
4.

Listening Sessions

Agencies should consider holding listening sessions related to their agendas
when they seek informed public input or a more deliberative or informal exchange.
Agencies should consider conducting listening sessions in multiple locations and
using available technology to facilitate remote attendance. Agencies should also
engage in robust outreach to bring listening sessions to the attention of missing
stakeholders, unaffiliated experts, and other interested parties. In addition, agencies
should provide attendees with the background and information they need to offer
constructive feedback. Finally, agencies should consider using moderators or
facilitators to plan and run their listening sessions, particularly when they may
involve controversial issues or groups with different interests at stake. Facilitators
can ensure that targeted stakeholders are included and the discussion is balanced and
productive.
5.

Advisory Committees

Agencies may consult with advisory committees at all stages of the rulemaking
process, from agenda setting, to rule development, as part of the notice-and-comment
process, and in connection with retrospective review.
Agencies should consider using advisory committees to inform their rulemaking
agendas when they have discretion over which matters to pursue or the order in which
to pursue them. Agencies should ensure that their advisory committees are composed
of a balanced group of stakeholders and that all significant interests are adequately
represented. Agencies whose stakeholders closely mirror the general public should
consider consulting advisory committees that are composed entirely of a
representative sample of ordinary citizens.
Agencies should consider using their advisory committees in conjunction with
other more open forms of public engagement. For example, agencies should consider
consulting with their advisory committees about their agenda and priorities after
receiving public comments on a petition for rulemaking or in response to an RFI
related to agenda setting.
To the extent possible and practical, agencies should re-purpose briefing
materials prepared for their advisory committees for other audiences and other forms
of public engagement.
6.

Focus Groups

Agencies should consider using focus groups to obtain meaningful feedback on
their regulatory agendas from individuals and groups with relevant views, data, and

161

other information but who are difficult to reach and traditionally do not participate in
public engagement that is self-selective.
7.

Public Complaint Databases

Agencies with public complaint databases should establish a process for
identifying issues raised in the complaints that may be appropriate for rulemaking.
When seriously considering rulemaking in connection with an issue raised in public
complaints, agencies should use other public engagement tools to solicit broader input
on whether the matter is appropriate for rulemaking.
8.

Public Notice and Comment

Agencies should publish their proposed regulatory agendas for public comment
after using other methods of public engagement described herein.
C.

Early and Advanced Rule Development
1.

Traditional or Negotiated Rulemaking

At the beginning of each rulemaking an agency undertakes or seriously
considers, the agency should identify the interests that may be affected by the
rulemaking and whether organized groups represent them adequately. ACUS
Recommendation 2017-2, Negotiated Rulemaking and Other Options for Public
Engagement (June 16, 2017), sets forth conditions under which negotiated
rulemaking may be appropriate for drafting a proposed rule.
Negotiated rulemaking is likely not appropriate when the agency identifies (1)
individuals or groups who may be affected by the rulemaking but (2) traditionally do
not participate in the rulemaking process and (3) may not be adequately represented
by other groups that do. Nor is negotiated rulemaking appropriate where there is a
large group of diverse interests affected in different ways or who hold a variety of
different perspectives. When agencies identify individuals or groups who may be
affected by a rulemaking but whose interests or preferences may not be adequately
represented by organized groups, agencies should identify the types of information
they need from these groups and how to reach them as early in the process of
developing a rule as possible.
2.

Requests for Information (RFIs)

Agencies should presumptively use RFIs to solicit data, views, or other
information from the public early in the process of developing a rule when they are
open to different approaches to a matter and need additional information or data
before choosing the best regulatory approach. When using an RFI in early rule
development, agencies should (1) remain neutral regarding how it would or should
resolve the matters on which it seeks public comments, (2) pose detailed questions
aimed at soliciting the data, views, or other information the agency needs to make an
informed decision, and (3) indicate that they are open to input on other questions and
concerns.

162

When using RFIs during rule development, agencies should identify (1) any
individuals or groups who may have relevant data, views, or other information but are
traditionally absent from the notice-and-comment process and (2) any unaffiliated
experts who may have special expertise on the relevant issues. Agencies should
address these groups directly in their RFIs and engage in targeted outreach to
encourage participation by these groups.
Agencies should review any comments they receive in response to RFIs and
explain in subsequent public notices how these comments informed or influenced the
development of the rule.
3.

Internet and Web-Based Outreach

Agencies should also follow ACUS’s previously issued recommendations on best
practices for using the Internet and social media in rulemaking.
ACUS
Recommendation 2013-5, Social Media in Rulemaking (Dec. 5, 2013). In addition,
agencies should routinely use the Internet and social media to inform the public of (1)
an agency’s interest in a potential rule, (2) rules currently under development, and
(3) opportunities to participate in the process of developing rules.
Agencies should also consider using the Internet and social media as platforms
for public discussion and input concerning potential regulatory initiatives and rules
under development. It may be particularly useful to use the Internet and social media
to facilitate public input concerning regulatory initiatives when agencies seek a more
informed or deliberative process than possible with an RFI and input from a broader
or more dispersed group than possible using in-person listening sessions. When
paired with targeted outreach, the Internet and social medial may provide a means of
reaching individuals and groups with relevant views, experiences, or other
information who are traditionally absent from the notice-and-comment rulemaking
process.
When creating web-based opportunities to participate in rule development,
agencies should conduct targeted outreach (1) to inform stakeholders and members of
the general public traditionally absent from the rulemaking process about web-based
opportunities to participate in rule development, (2) to persuade these individuals and
groups to participate, and (3) to provide the targeted audience with the information
and guidance they need to participate effectively in rule development. Such
information might include a description in plain language of the of the issues under
consideration and background material; an explanation of the rulemaking process, the
role of public participation, and the qualities of a useful comment; and summaries of
public engagement efforts to date, including any information received from the public
to date.
Agencies should consider the information they receive from the public through
the Internet and social media when developing their proposed rules. Agencies should
also inform participants of how the public input influenced the development of the
rule and any decisions the agency made.

163

4.

Advance Notices of Proposed Rulemaking (ANPRMs)

Agencies should presumptively use ANPRMs to solicit public comments when
they need additional information to choose between more than one regulatory
alternative or develop and refine a proposed rule. When using ANPRMs during
advanced rule development, agencies should pose detailed questions aimed at
soliciting the data, views, or other information they need to make an informed
decision, and ask for potential revisions to their tentatively favored approach.
In addition, agencies should identify (1) any individuals or groups who may have
relevant data, views, or other information but are traditionally absent from the noticeand-comment process and (2) any unaffiliated experts who may have special
expertise on the relevant issues. Agencies should address these groups directly in
their ANPRMs and engage in targeted outreach to encourage participation by these
groups.
Agencies should consider the information received from the public through
ANPRMs when drafting an NPRM or making another decision. In addition, agencies
should inform the public of how its input influenced the development of the proposed
rule and any decisions the agency made. The agency should include this information
in the preamble to an NPRM published after an ANPRM or in some other manner if
the agency does not proceed promptly with an NPRM.
5.

Advisory Committees

Agencies should consider consulting with their advisory committees during rule
development. It is generally most appropriate to consult advisory committees in
conjunction with other more open forms of public engagement. For example,
agencies may want to consult with their advisory committees when framing RFIs or
ANPRMs, after receiving public comments in response to an RFI or ANPRM, or
when drafting a proposed rule.
To the extent possible and practical, agencies should re-purpose briefing
materials prepared for their advisory committees for other audiences and other forms
of public engagement.
6.

Listening Sessions and Public Meetings

Agencies should presumptively hold in-person, online, or telephonic listening
sessions during rule development to educate interested persons about the regulatory
process and obtain informed public input on the development of their rules. Listening
sessions are particularly helpful when agencies seek informed public input or a more
deliberative or informal exchange with stakeholders. Agencies should consider
conducting listening sessions in multiple locations and using available technology to
facilitate remote attendance. Agencies should also engage in robust outreach to bring
listening sessions to the attention of missing stakeholders, unaffiliated experts, and
other interested parties. In addition, agencies should provide attendees with the
background and information they need to offer constructive feedback. Finally,
agencies should consider using moderators or facilitators to plan and run their
164

listening sessions, particularly when they may involve controversial issues or groups
with different interests at stake. Facilitators can ensure that targeted stakeholders are
included and the discussion is balanced and productive.
7.

Focus Groups

Agencies should consider using focus groups to obtain feedback during rule
development from individuals and groups with relevant views, data, and other
information but who are difficult to reach and traditionally do not participate in public
engagement that is self-selective.
D.

The Notice-and-Comment Process
1.

Plain Language NPRMs

Agencies should ensure the preambles to their NPRMs speak in plain language to
individuals and groups the agency identifies as sources of relevant views, data and
other information but may be unfamiliar or inexperienced with the notice-andcomment process. When drafting NPRMs, agencies should highlight issues that
would benefit from public ventilation and pose specific questions to individuals and
groups the agency identifies during rule development as sources of relevant views,
data, experiences and other information. Questions directed to rulemaking novices
and members of the general public should be featured prominently and as early as
possible in the preamble to the NPRM. Agencies should also include questions
directed to rulemaking novices and members of the general public on their websites
along with links to Regulations.gov.
Agencies should consider employing visual elements, such as numbered lists,
bullet points, tables, Q&A formats, and more colloquial language to reach rulemaking
novices and members of the general public.
Agencies should look to ACUS Recommendation 2017-3, Plain Language in
Regulatory Drafting (Dec. 14, 2017), when drafting their NPRMs. In addition,
agencies should consider including personnel who do not have subject-matter
expertise in the rulemaking or routinely draft regulations to review draft NPRMs for
clarity and accessibility to rulemaking novices and members of the general public.
2.

Regulations.gov

Regulations.gov needs to be updated to provide a more user-friendly interface for
reviewing rulemaking dockets and submitting public comments. The website should
allow visitors to search comments in different ways, including by author/submitter
and key words in the text of the comments. In addition, users should be able to scroll
easily through the public comments and the results of their searches. Users should
also be able to toggle more easily between the text of an NPRM and both the public
comments already submitted and the comment a user is in the process of drafting.
Regulations.gov should revise its “Tips for Submitting Effective Comments” to
emphasize the importance of including facts, relevant experiences, logical argument,

165

good reasons, and things Congress tells the agency to consider in public comments.
Regulations.gov should also emphasize that rulemaking is not a voting process and
comments that merely express approval or disapproval of a proposal generally have
little value.
Regulations.gov should consider producing a more accessible and engaging
video aimed at the general public to provide advice on effective commenting.
3.

User-Friendly Rulemaking Portals

When agencies identify individuals and groups who do not traditionally
participate in the notice-and-comment process as important potential sources of
views, data, experiences, and other information relevant to a rulemaking, they should
set up a user-friendly portal to the rulemaking on their websites. The e-rulemaking
portal might include: (1) specific questions in plain-language form and directed to the
individuals and groups identified by the agency as potential sources of important
views and information, (2) plain-language summaries of all or at least key parts of the
NPRM, (3) a means for the public to comment on specific issues summarized on the
e-rulemaking portal rather than on the whole NPRM, (4) a means for the public to
reply to the comments of others, (5) a video tutorial on submitting effective
comments, (6) examples of helpful and unhelpful public comments, (7) links to
Regulations.gov and other useful information and documents related to the
rulemaking, and (8) a means for interested persons to sign up for updates on the
status, progress, and major developments in the rulemaking.
Agencies should include the comments received through their e-rulemaking
portals in the rulemaking docket along with other public comments. If an agency
expects a large number of public comments through the web and the issue is
particularly important or controversial, the agency should consider using a third-party
to moderate and facilitate a web-based discussion and summarize the discussion for
including in the rulemaking docket.
When agencies set up e-rulemaking portals to enhance public engagement by
absent stakeholders they should conduct targeted outreach to solicit participation by
these groups.
When publishing a final rule, agencies should summarize in plain language on
their web portals the basis and purpose of the final rule, including an explanation of
how the public input influenced the final rule or otherwise had an effect on the
agency’s decision.
4.

Supplemental Deliberative Exercises

Agencies should consider using enhanced deliberative methods in appropriate
circumstances to supplement their traditional rulemaking processes. There are
numerous examples of mechanisms for enhanced deliberation that could be used as a
supplement to the traditional rulemaking process, including Regulation Room,
Citizen Juries, Citizen Advisory Committees, Citizen Assemblies, and Deliberative
Polls. These methods commonly (1) are more dialogic in nature than typical public
166

meetings, (2) provide participants with balanced and objective briefing materials, (3)
include opportunities for small group discussion, (4) provide participants with
opportunities to consider and respond to competing perspectives, (5) include
opportunities to ask questions of experts and/or agency officials, (6) produce new
information that should be incorporated into resulting recommendations, and (7)
result in a final report with findings and recommendations. They allow agencies to
facilitate collective decisions about a policy decision that take into account all of the
interests and perspectives that emerge from a reasoned deliberative process.
In assessing when such efforts are likely to prove worthwhile, agencies should
(1) focus on rulemakings in which new participants are likely to have useful
information and it is feasible to provide the participation support necessary to elicit
this information; (2) consider the “information load” or effort required to provide
sufficient information to rulemaking newcomers to enable them to participate
effectively; and (3) consider using enhanced deliberative techniques selectively—that
is, of targeting only certain types of potential new participants and only certain issues
in the rulemaking. Agencies should also favor enhanced deliberative methods when
rulemaking proceedings are more rather than less important, more rather than less
politically salient, and more rather than less likely to turn on the resolution of
conflicting public values.
Agencies should also consider (1) hiring facilitators to plan and conduct
enhanced deliberation; (2) engaging in enhanced deliberation with pre-existing
groups with balanced participation, such as federal advisory committees or negotiated
rulemaking committees; (3) combining in-person deliberative exercises with online
communication; and (4) recruiting participants for enhanced deliberation, rather than
relying on open, self-selected participation.
5.

Reply Comment Periods

Agencies should routinely consider using reply comment periods when they
receive a large number of public comments near the end of the section 553 comment
period, when new and important issues are raised during the section 553 comment
period, when serious conflicts in data and other descriptive or predictive information
need to be resolved, or when agencies otherwise believe additional information from
the public will help them evaluate the comments received during the section 553
public comment period.
When using reply comment periods, agencies should highlight in plain language
any questions or issues they believe individuals or groups traditionally absent from
their rulemaking may be able to address, including unaffiliated experts with special
expertise on the relevant issues. In addition, agencies should conduct targeted
outreach to encourage participation by these groups.
When publishing a final rule after a reply comment period, agencies should
explain in plain language how the public input during the reply comment period
influenced the final rule or otherwise had an effect on the agency’s decision.

167

E.

Retrospective Review
1.

Preambles to Final Rules

When publishing preambles to final rules, agencies should strive to include a
description of the information that will help them to assess the effectiveness of the
rule in accomplishing its objectives.
When conducting retrospective review, agencies should seek to identify
individuals or groups traditionally absent from the rulemaking process who may
possess relevant information on the effectiveness of the rule in accomplishing its
objectives. When agencies believe rulemaking novices may have information, views
or experiences relevant to retrospective review, agencies should highlight these in
plain language in the preamble and speak directly to these individuals and groups.
2.

Open or Living e-Rulemaking Dockets

Agencies should provide the public with open or living e-rulemaking dockets—
portals on their websites for submitting comments on existing regulations and
suggesting rules that may warrant review and modification or repeal.
When summarizing final rules on their websites, agencies should highlight the
goals and assumptions upon which the rule is based and the information the agency
believes will enable it to evaluate the effectiveness of the rule going forward. In
particular, agencies should highlight in plain language the specific information,
views, or experiences that different groups may possess or come to possess that will
help the agency evaluate the effectiveness of the rule.
When agencies cannot create an e-rulemaking docket dedicated to a single rule or
collection of rules, they should utilize “hotlines” and/or on-line “suggestion boxes”
that allow the public to comment on existing regulations. Agencies should ask the
public to identify the specific rule or regulation that is the subject of their comments
and be as specific as possible in describing their experiences with the rule, what is
working and what is not working, and the most effective remedy consistent with the
agency’s statutory mandate.
Open e-rulemaking dockets or online suggestion boxes should explain how the
agency evaluates existing regulations, identify the types of information or experiences
that are most helpful to the agency, and offer other tips on submitting effective
retrospective comments. Open e-rulemaking dockets or general suggestion boxes
should also provide the public with examples of helpful and unhelpful comments on
existing regulations.
3.

Requests for Information (RFIs)

Agencies should presumptively use RFIs to solicit public comments when they
are conducting regulatory lookbacks and are open-minded concerning whether or how
to amend or repeal an existing regulation.

168

When using an RFI for regulatory lookbacks, agencies should (1) remain neutral
regarding the best way to proceed and (2) pose detailed questions aimed at soliciting
the information needed to conduct the retrospective review.
Before publishing an RFI in connection with retrospective review, agencies
should identify (1) any individuals or groups who do not traditionally participate in
the rulemaking process but may possess data, views, experiences, or other
information relevant to retrospective review and (2) any unaffiliated experts who may
have special expertise on the relevant issues. Agencies should address these groups
directly in the RFI, ask them specific questions, and conduct targeted outreach to
encourage their participation.
4.

Advisory Committees

Agencies should consider consulting with their advisory committees when
deciding whether or how to conduct a regulatory lookback. Agencies should ensure
that advisory committees used for this purpose are composed of a balanced group of
stakeholders and that all significant interests are adequately represented. Agencies
whose stakeholders closely mirror the general public should consider consulting
advisory committees that are composed entirely of a representative sample of
ordinary citizens.
Agencies should consider using their advisory committees in conjunction with
other more open forms of public engagement. For example, agencies should consider
consulting their advisory committees after receiving a petition to amend or repeal a
regulation or public comments in response to an RFI related to retrospective review.
Advisory committees may generate their own candidates for regulatory review,
evaluate reviews proposed by other members of the public, and help the agency
design a process for evaluating the effectiveness of a rule.
5.

Focus Groups

Agencies should consider using focus groups to obtain meaningful feedback on
existing regulations from individuals and groups with relevant views, data,
experiences, and other information but who are difficult to reach and traditionally do
not participate in public engagement that is self-selective.
6.

Listening Sessions and Public Hearings

Agencies should presumptively hold in-person, online, or telephonic listening
sessions early in the process of conducting a regulatory lookback to educate interested
persons about the process and obtain informed public input on potential changes to
their rules. Listening sessions are particularly helpful when agencies seek informed
public input or a more deliberative or informal exchange with stakeholders.
When planning listening sessions, agencies should provide attendees with the
background and information they need to offer constructive feedback. Agencies
should consider using moderators or facilitators to run the listening sessions,

169

particularly when they may involve controversial issues or groups with different
interests at stake.

170

APPENDIX A
SURVEY ON PUBLIC ENGAGEMENT WITH AGENCY RULEMAKING
Dear Respondent,
Thank you for taking the time to complete this Survey. The results will inform a
report to the Administrative Conference of the United States (ACUS) on agency
strategies to enhance public engagement at various stages of the rulemaking
process—including during early agenda setting stages, during the informal
rulemaking process, or when undertaking retrospective review.
This Study seeks to help agencies invest resources in ways that maximize the
probability that rulewriters obtain high quality information from commenters—
including from traditionally absent stakeholders, unaffiliated experts, and members of
the general public—as early in the process as possible. Among other things, the
Study considers efforts to promote public education and participation through print or
web-based media, use plain writing and clear visual formatting for rule text and
supporting material, and take advantage of in-person engagement opportunities to
solicit stakeholder input and support future informed participation. Your answers to
the following questions will be invaluable to this Study.
To avoid confusion, we have defined the following terms used in the Survey:
“Absent stakeholders” means individuals or groups that a regulation may
benefit or burden, or otherwise have a direct stake in the outcome of the agency’s
rulemaking, but who do not traditionally participate in the agency’s rulemaking
process.
“Unaffiliated experts” means scientific, technical, or other professionals
with expertise relevant to the agency’s rulemaking who are neither direct
stakeholders nor employed or retained by a stakeholder.
Agenda Setting
1. Does the agency utilize any procedures or practices (other than petitions for
rulemaking) to engage the public in setting its rulemaking agenda?
If so, please describe any such procedures or practices, including the types of
parties the agency tries to engage and how the agency seeks to engage them.
In addition, please describe how, if at all, these efforts have focused on: (a)
“absent stakeholders;” (b) “unaffiliated experts” in the field; and (c) members of
the general public, including citizens with situated knowledge of the conduct or
activities being regulated.

171

Informal Rulemaking Proceedings
2. Which types of stakeholders typically participate in the agency’s rulemaking
proceedings, for example, by participating in town halls and workshops or by
submitting comments on proposed rules?
3. Aside from the solicitation of public comments in connection with notices of
proposed rulemaking, does the agency utilize any procedures or practices, formal
or informal, to enhance public involvement in the agency’s rulemaking process
(including rule formation prior to issuance of the NPRM)?
If so, please describe any such procedures or practices, including the types of
parties the agency tries to engage and how the agency seeks to engage them.
In addition, please describe how, if at all, these efforts have focused on: (a)
“absent stakeholders;” (b) “unaffiliated experts” in the field; and (c) members of
the general public, including citizens with situated knowledge of the conduct or
activities being regulated.
4. Does the agency have any experience conducting web-based or in-person
deliberative exercises (using, for example, citizen assemblies or deliberative
juries) or any other practices to engage the public in its rulemaking that the
agency considers “innovative”?
If so, please briefly describe the nature of the exercise(s).
Retrospective Review
5. Does the agency utilize any procedures or practices (other than petitions for
rulemaking) to engage the public in retrospective review?
If so, please describe any such procedures or practices including the types of
parties the agency tries to engage and how the agency seeks to engage them.
In addition, please describe how, if at all, these efforts have focused on: (a)
“absent stakeholders;” (b) “unaffiliated experts” in the field; and (c) members of
the general public, including citizens with situated knowledge of the conduct or
activities being regulated.
Value of Public Engagement in Rulemaking
6. How would you characterize the value of public engagement in the agency’s:
(a) agenda setting?
(b) rulemaking (including rule formation)?
(c) retrospective review?
172

7. In general, what types of information or views of the general public are most
useful to the agency in pursuing its statutory mandate?

Please include your name and contact information for any follow-up questions:

Thank you for your help with this important Study!

173

APPENDIX B
As part of our study, we received survey responses and/or interviewed current
or former officials at the following federal agencies:
Consumer Financial Protection Bureau
Consumer Product Safety Commission
Department of Agriculture
Department of Education
Department of Energy
Department of Justice
Department of Labor
Department of Transportation
Environmental Protection Agency
Federal Energy Regulatory Commission
Federal Maritime Commission
Federal Trade Commission
General Services Administration
National Park Service
Nuclear Regulatory Commission
Pension Benefit Guaranty Corporation
Occupational Safety and Health Commission
Office of Government Ethics
Social Security Administration
United States Coast Guard
United States Forest Service

174

Appendix C: Modes of Public Engagement
Stages of Selection
RM
Method
AS, RR Open

Mode of
Collective
Communication Recommend IAP2 Level
Strengths or Benefits
Express
No
Consult
Provide ideas for regulatory change, especially when
Preferences
petitioners have diffuse information that is otherwise
unavailable or difficult for agencies to collect.

Mode
Rulemaking
Petitions

Function/Goal
Give interested persons the right to petition for the issuance,
amendment, or repeal of a rule.

Advisory
Committees

Allow agencies to solicit and obtain advice from formally
All
established groups of stakeholders, unaffilated experts, and/or
ordinary citizens.

Professional Express and
or Lay
Develop
Stakeholders Preferences

Yes

Collaborate Provide relatively inexpensive advice from formally
Relatively heavily regulated by law, time-consuming
established and balanced groups of outside experts (or and expensive to charter, and composition may not truly
citizens).
be representative.

Focus Groups

Facilitated, small group discussions of prepared questions by
individuals or members of targeted demographic groups.

All

Random or
Targeted
Recruitment

Express and
Develop
Preferences

No

Consult

Requests for
Information

Published requests for written information, data, or comments
on a designated problem or issue of potential regulatory
interest.

AS, ERD, Open
RR

Express
Preferences

No

Consult

Listening
Sessions

Public meetings to gather information, data, or comments on a AS, ERD, Open
designated problem or issue of potential regulatory interest.
ARD, RR

Express and
Develop
Preferences

No

Consult

Professional Express
Stakeholders Preferences

No

Involve

Provide feedback about agencies' practices from the
Ensuring adequate and balanced participation from the
entire range of relevant stakeholders, given RIDs'
perspective of the regulated community and builds
stronger relationships with stakeholders. Brings
traditional focus on the regulated community.
unappreciated problems to light and suggests avenues
for reform.

Reverse Industry Results in invitations to carefully selected industry
Days
representatives to tell agency officials about their needs and
challenges in the procurement process.

AS, RR

Weaknesses or Challenges
Reviewing petitions may divert agency from its own
priorities or provide a relatively low visibility mechanism
for regulatory capture.

Provide relatively inexpensive sounding board to guage Requires skilled facilitation and careful planning,
participants' reactions to information, ideas, messages, participants may face steep learning curve, and views
or proposals, and identify preferred alternatives and
expressed may not be representative (even in an
potential concerns.
informal sense).
Provide information about issues or problems when
Requires robust outreach efforts to generate
agency lacks knowledge and is open-minded about
participation by missing stakeholders and unaffiliated
how to proceed.
experts; may be difficult to secure representative or
balanced feedback.
Provide benefits similar to RFIs, but allow for more
Requires robust outreach and skilled facilitation to
informal and interactive exchange than typically
secure attendance from a broad range of interested
possible through written communications.
stakeholders, achieve balanced participation, and
obtain sufficiently detailed or focused advice.

Hotlines or
Suggestion
Boxes

Provides mechanism for interested persons to contact agencies AS, RR
informally by telephone or web-based communication with
questions, comments, or suggestions.

Open

Express
Preferences

No

Consult

Provide ideas for regulatory change, especially when
petitioners have diffuse information that is otherwise
unavailable or difficult for agencies to collect. More
informal, open, and accessible than rulemaking
petitions.

Public
Complaints

Provides mechanism for interested persons to lodge complaints AS, RR
with agency that regulated entity engaged in potentially unlawful
behavior.

Open

Express
Preferences

No

Involve

Provide mechanism for public to express concerns and Requires broad advertisement and sufficient resources
for the agency to assess the frequency or magnitude of to respond to complaints in a timely and substantively
problems, which could inform its rulemaking agenda or adequate fashion.
provide an impetus for rule development.

Web-Based
Outreach

Online tools that facilitate two-way communication,
All
collaboration, interaction, or sharing between agencies and the
public.

Open

Express and
Develop
Preferences

No

Involve

Facilitates outreach and public education, and may be a If you build it, the public will not necessarily come: even
useful mechanisms for obtaining situated knowledge
e-rulemaking may require robust outreach and skilled
and other valuable input during rule development from facilitation to obtain meaningful input. Conversely,
missing stakeholders, unaffiliated experts, and perhaps mass participation presents logistical challenges.
ordinary citizens.

ANPRMs

Published requests for public comments or proposed
ARD, RR Open
alternatives to a tentative proposal before agency's issuance of
an NPRM.

Express
Preferences

No

Consult

Provide public feedback on preliminary or tentative
proposals, including the likelihood of compliance,
potential costs and benefits, and unintended
consequences.
Allow agency to explain proposals, hear views and
concerns of stakeholders, describe how agency has
responded to public input, and publicize futher
opportunities for participation.

Public Meetings Live and/or remote public forum for agency to explain its
proposals and solicit the views and concerns of interested
stakeholders and other citizens.

All

Listen and
Express
Preferences

No

Consult

Shuttle
Diplomacy

ARD, N&C Professional Express and
Stakeholders Develop
Preferences

No

Collaborate Provides agency with candid views of certain
staekholers when developing solutions or addressing
challenges identified during rule development.

Private meetings with specific stakeholders to gain a deeper
understanding of their views or perspectives.

Open

Requires broad advertisement, and sufficient staffing
and resources to answer calls or respond to messages
in a reasonably prompt and substantively adequate
fashion.

Participation tends to be imbalanced and limited
primarily to relatively sophisticated stakeholders.
Participation may not be representative or balanced, the
audience may not be well-informed about the process
or the agency's proposals, and opportunities for
reasoned deliberation are limited.
Typically involve sophisticated stakeholders who
routinely participate in rulemaking. Because such
meetings generally occur behind closed doors, they can
raise transparency and capture concerns.

Appendix C: Modes of Public Engagement
Technical
Workshops

Workshops with stakeholder representatives with special
ARD, N&C Professional Express and
expertise or unaffiliated experts to obtain feedback on agencies'
Stakeholders Develop
technical data or analyses.
or Targeted Preferences
Recruitment

No

Collaborate Provide a relatively informal alternative to using
Stakeholders may be biased and present agency with
advisory committees for obtaining technical advice from unbalanced information. Agency should therefore also
experts outside the agency.
consider soliciting information from unaffiliated experts
as part of this process.

Negotiated
Rulemaking

Advisory committees composed of representatives of key
ARD
stakeholders that collaborate on proposed rules for public notice
and comment.

Yes

Collaborate Provides a deliberative, consensus-driven process that Generally inappropriate when agency identifies
is appropriate when there are a limited number of
stakeholders not adequately represented by other
identifiable interests that will be significantly affected by parties or where diverse interests are affected in
a rule and there is a reasonable likelihood that a
different ways or hold a variety of different preferences
balanced committee can be convened to adequately
or perspectives.
represent those interests and negotiate in good faith.

Public Notice
and Comment

Publication of a proposed rule or other course of action for
written public comments.

Express
Preferences

No

Consult

Enhanced
Deliberative
Exercises

A variety of mechanisms that facilitate reasoned deliberation
All
about what should be done by agency officials in collaboration
with well-informed citizens.

Random,
Develop
Targeted
Preferences
Recruitment,
or
Professional
and Lay
Stakeholders

Yes

Collaborate Facilitates robust levels of participation, and provides Resource intensive to design and implement, and will
detailed information about what stakeholders or the
not always produce a substantial amount of useful new
general public would think about a problem if they were information.
fully informed about the relevant issues and had a
chance to engage in reasoned deliberation about what
should be done. Provides a valuable supplement to
notice and comment in appropriate circumstances.

Professsional Express and
and Lay
Develop
Stakeholders Preferences

AS, N&C, Open
RR

Provides public input on proposed rules or other
Participation typically limited to sophisticated
courses of action. Sometimes described as "one of the stakeholders, may be reluctance to make major
greatest inventions of modern government."
changes at this stage, and mass comments from the
general public raise logistical concerns and tend to be
of limited value.