SMALL BUSINESS 

ADMINISTRATION

Office of Advocacy 

Needs to Improve 

Controls over 

Research, Regulatory, 

and Workforce 

Planning Activities

Report to the Chairman, Subcommittee 

on Financial Services and General 

Government, Committee on 

Appropriations, United States Senate

July 2014

GAO-14-525

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-14-525, a report to the 

Chairman, Subcommittee on Financial 

Services and General Government, 

Committee on Appropriations, United States 

Senate

July 2014

SMALL BUSINESS ADMINISTRATION

Office of Advocacy Needs to Improve Controls over 

Research, Regulatory, and Workforce Planning 

Activities 

Why GAO Did This Study

Congress created Advocacy within

SBA in 1976 as an independent voice 

for small businesses. Questions have 

recently been raised about Advocacy’s 

efforts to represent small businesses in 

regulatory activities and some of its 

research on small business issues. In 

light of these questions, GAO was 

asked to review Advocacy’s activities. 

This report examines Advocacy’s (1) 

research, (2) regulatory activities, and 

(3) workforce planning efforts. GAO 

analyzed Advocacy’s research, 

comment letters, and other regulatory

information for fiscal years 2008-2013; 

assessed Advocacy’s policies and

procedures against federal standards 

for internal control and information 

quality; and interviewed agency 

officials, and small business and 

industry representatives. GAO also 

analyzed the applicability of the 

Federal Advisory Committee Act to 

Advocacy roundtables. 

What GAO Recommends

GAO makes several recommendations 

to improve the Office of Advocacy’s

controls over the quality of its research, 

the documentation of its regulatory 

activities, and workforce planning. In 

commenting on a draft of this report, 

Advocacy agreed with our 

recommendations and noted some 

steps it will take to address them. 

What GAO Found

The Office of Advocacy (Advocacy) within the Small Business Administration 

(SBA) fulfills its mission by researching small business issues and providing input 

into federal rulemaking and related regulatory activities. However, GAO identified 

key areas in Advocacy’s system of internal control that could be improved. 

• Research. GAO found that Advocacy did not ensure that its staff 

monitored the quality of the information the office disseminated, as 

required. GAO reviewed 20 selected research products and found that in 

16 cases a required quality review had not been documented. Advocacy 

recently established a review policy for its research, but it does not 

include procedures for selecting the reviewers or documenting that a 

review occurred and how reviewer comments are addressed. GAO also 

found that Advocacy staff had not followed federal information quality 

guidelines to retain data and could not substantiate the quality of

information in two cost-estimation reports—a research product it has 

contracted for every 5 years. Without better controls over its quality 

review process and efforts to substantiate the information it 

disseminates, Advocacy cannot ensure the validity of one of its core 

activities—research in support of small businesses. 

• Regulatory activities. Advocacy recently updated procedures for its 

regulatory activities, but these could be strengthened. GAO found the 

extent to which individual staff maintained records varied, in part, 

because the procedures lacked policies for documentation. For instance, 

the procedures state that when staff decide to intervene in the 

rulemaking process, they must follow up as appropriate with the 

interested groups to ensure that Advocacy has sufficient information and 

data to support its case. However, there is no policy that these 

interactions be documented. Federal internal control standards state that 

documentation and records should be maintained. If key procedures are 

not being documented, managers do not have an institutional record that 

agency goals and objectives in this area are being met. GAO also found 

that the Federal Advisory Committee Act’s transparency and other

requirements do not apply to Advocacy’s meetings with stakeholders to 

get input on regulations (roundtables). 

• Workforce planning. Advocacy’s workforce efforts include training and 

mentoring for new staff, but do not include succession planning, which is 

recommended by the Office of Personnel Management. According to 

federal internal control standards, effective management of a workforce 

is essential to achieving program results. Officials told GAO that 

Advocacy was a small office and that additional staff were hired on an 

as-needed basis. However, some key staff have been with Advocacy for 

many years and their experience will be difficult to replace. If Advocacy 

does not incorporate succession planning strategies into its workforce 

planning efforts, it is at risk of not having the skills and expertise to meet 

its mission when key staff leave or retire. View GAO-14-525. For more information, 

contact Cindy Brown Barnes at (202) 512-

8678 or brownbarnesc@gao.gov.
Page i GAO-14-525 Office of Advocacy

Letter 1 

Background 4

Advocacy’s Internal Controls Do Not Ensure the Quality of Its 

Research and That Required Information Quality Practices Are 

Followed 10

Documentation of Key Regulatory Activities is Inconsistent and 

Roundtables Are Not Publicized According to Advocacy’s 

Policies 17

Advocacy’s Workforce Planning Efforts Lack Succession Planning 21

Conclusions 23

Recommendations for Executive Action 24

Agency Comments 25

Appendix I Objectives, Scope, and Methodology 27

Appendix II Comments from the Office of Advocacy, Small Business 

Administration 32

Appendix III Legal Analysis of the Applicability of the Federal Advisory Committee 

Act to the Office of Advocacy’s Roundtables 34

Tables

Table 1: Advocacy Comment Letters by Policy Area, Fiscal Years 

2009-2013 8

Table 2: Number of Roundtables by Policy Area, Fiscal Years 

2009-2013 9

Figures

Figure 1: Office of Advocacy Organizational Chart, Fiscal Year 

2014 6

Figure 2: Office of Advocacy Organizational Chart, Fiscal Year 

2014 21

Contents
Page ii GAO-14-525 Office of Advocacy

Abbreviations

ADA Americans with Disabilities Act 

AAPS Association of American Physicians and Surgeons

APA Administrative Procedure Act

BATF Bureau of Alcohol, Tobacco, and Firearms

CFPB Consumer Financial Protection Bureau

EPA Environmental Protection Agency

FACA Federal Advisory Committee Act

FOIA Freedom of Information Act 

GSA General Services Administration

HCAAF Human Capital Assessment and Accountability Framework

HEW Department of Health, Education, and Welfare

JAG Judge Advocate General of the Army

OIRA Office of Information and Regulatory Affairs

OMB Office of Management and Budget

OPM Office of Personnel Management

OSHA Occupational Safety and Health Administration

OST Executive Office of the President’s Office of Science and 

Technology

RFA Regulatory Flexibility Act

SBA Small Business Administration

SBREFA Small Business Regulatory Enforcement Fairness Act of 

1996

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-14-525 Office of Advocacy

441 G St. N.W.

Washington, DC 20548

July 22, 2014

The Honorable Tom Udall

Chairman

Subcommittee on Financial Services and General Government

Committee on Appropriations

United States Senate

Dear Senator Udall:

Congress created the Office of Advocacy (Advocacy) within the Small 

Business Administration (SBA) in 1976 as an independent voice for small 

businesses within the federal government.1 The Chief Counsel for 

Advocacy, appointed by the President and confirmed by the U.S. Senate, 

directs the office and is tasked with representing the views and interests 

of small businesses to other federal agencies, including during 

rulemaking and related regulatory activities. Advocacy’s mission is to 

encourage the development and growth of America’s small businesses by

• producing research to keep policymakers and other stakeholders 

informed about the impact of federal regulatory burdens on small 

businesses, document the role of small businesses in the economy, 

and explore and explain issues of concern to the small business 

community;

• intervening early in federal agencies’ regulatory development process 

on proposals that affect small businesses and providing training in 

complying with the Regulatory Flexibility Act (RFA) to agency 

policymakers and regulatory development officials; and

• fostering two-way communication between federal agencies and the 

small business community.

Questions have recently been raised about Advocacy’s efforts to 

represent small businesses. For example, some stakeholders questioned 

how Advocacy staff gather and use information to develop comment 

letters on proposed regulations and whether that information actually 

conveys the views of small businesses. Also, some parties have 

expressed concerns that Advocacy’s information-gathering events 

(“roundtables”) are not fully transparent and open to the public, and 

 

1

P. L. No. 94-305, §201, 90 Stat. 668 (1976).
 

Page 2 GAO-14-525 Office of Advocacy

others have criticized the methodology of some of the research 

conducted by contractors hired by Advocacy.

You asked us to review the Office of Advocacy. This report examines 

Advocacy’s (1) research activities; (2) regulatory activities, including the 

applicability of the Federal Advisory Committee Act’s (FACA) 

transparency and other requirements to Advocacy’s roundtables; and (3) 

workforce planning efforts.

To understand Advocacy’s mission, operations, and participation in the 

rulemaking process, we interviewed Advocacy officials, and reviewed 

Advocacy documents, including its policies and procedures, budget and 

strategic planning documents, annual reports to Congress on the RFA, 

and other publications. To evaluate Advocacy’s research activities, we 

selected a nonprobability sample of 20 of its products and assessed them 

against Advocacy’s review process for its research (“peer review”), the 

Office of Management and Budget’s (OMB) Peer Review guidance, and 

OMB and SBA’s Information Quality Guidelines.2 Our sample was 

designed to include a variety of topics and report types and focused on 

more recent publications in order to maximize the possibility that the 

research staff involved would be available for interviews. In addition, we 

evaluated three of Advocacy’s most recent external studies on the 

economic costs of regulation and assessed the studies against OMB and 

SBA Information Quality Guidelines. We also reviewed peer reviewers’ 

comments and other external reviews on the studies to understand 

questions that had been raised about the studies. Lastly, we interviewed 

the Advocacy officials responsible for this research to obtain information 

on the steps taken to substantiate the quality of these studies.

 

2

OMB, Final Information Quality Bulletin for Peer Review, December 16, 2004. This 

Bulletin established government-wide guidance aimed at enhancing the practice of peer 

review of government science documents. OMB. Guidelines for Ensuring and Maximizing 

the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal 

Agencies. 67 Fed. Reg. 8452 (Feb. 22, 2002). OMB’s information quality guidelines 

implement section 515 of the Treasury and General Government Appropriations Act for 

Fiscal Year 2001 (P. L. No. 106-554). Section 515 directed OMB to issue governmentwide guidelines that “provide policy and procedural guidance to Federal agencies for 

ensuring and maximizing the quality, objectivity, utility, and integrity of information 

(including statistical information) disseminated by Federal agencies.” Agencies were 

required to issue their own implementing guidelines by October 1, 2002. For SBA’s 

guidelines, accessed April 24, 2014, see http://www.sba.gov/aboutsba/sba_performance/open_government/information_quality/information_quality_guideline

s. Advocacy officials told us they follow SBA’s information quality guidelines.
 

Page 3 GAO-14-525 Office of Advocacy

To evaluate Advocacy’s regulatory activities, we analyzed comment 

letters and information-gathering events (roundtables) from fiscal years 

2009 through 2013 as well as relevant policies and procedures. We 

interviewed Advocacy staff who were responsible for these activities and 

reviewed supporting documentation to understand their origins and 

purposes. In addition, we attended three roundtables. To analyze whether 

FACA (5 U.S.C. App. II) applies to Advocacy’s roundtables, we reviewed 

relevant statutes, case law, regulations, and guidance. We also 

considered Advocacy’s views on the issue. Lastly, we interviewed officials 

from the entities that interact with Advocacy in rulemaking—the 

Environmental Protection Agency (EPA), Occupational Safety and Health 

Administration (OSHA), Bureau of Consumer Financial Protection, also 

known as the Consumer Financial Protection Bureau (CFPB), and OMB’s 

Office of Information and Regulatory Affairs (OIRA). Also, we interviewed 

officials from six industry associations that represent small and large 

businesses; one large corporation; and two nonprofit organizations whose 

missions relate to policies or rules under consideration, to understand 

their perspectives on Advocacy’s roundtables and other work. We made 

our selections to include representatives from a variety of sources, and 

while the results from our interviews cannot be projected to all entities 

that interact with Advocacy, the information we gathered provides insights 

into how the selected groups view Advocacy and its work in representing 

small businesses to federal policymakers.

To assess Advocacy’s workforce planning efforts, we interviewed senior 

management to discuss any workforce planning efforts that were in place. 

We evaluated those efforts and any related documents against applicable 

federal standards for internal control and the Office of Personnel 

Management’s Human Capital Assessment and Accountability 

Framework (HCAAF).3

We conducted this performance audit from August 2013 to July 2014 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

 

3

Human Resources Management in Agencies, HCAAF and HCAAF Systems, Standards, 

and Metrics, 73 Fed. Reg. 23012 (Apr. 28, 2008). 
 

Page 4 GAO-14-525 Office of Advocacy

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

Under legislation that established Advocacy in its current form in 1976, 

Advocacy’ duties are to

• serve as a focal point for small businesses’ concerns about the 

federal government’s policies and activities;

• advise small businesses on how to interact with the federal 

government;

• develop proposals for federal agencies on behalf of small businesses;

• represent the views and interests of small businesses before federal 

agencies;

• enlist the cooperation and assistance of public and private agencies, 

businesses, and other organizations in disseminating information 

about the federal government’s programs and services that benefit 

small businesses.4

Since its establishment, a series of laws and executive orders has 

increased Advocacy’s roles and responsibilities. First, in 1980 the White 

House Conference on Small Business made recommendations that led 

directly to the passage of the RFA, which requires government agencies 

to consider the effects of their regulatory actions on small entities and, 

where possible, mitigate them.5 Under the RFA, agencies provide a small 

business impact analysis, known as an initial regulatory flexibility 

analysis, with every proposed rule published for notice and comment and 

a final regulatory flexibility analysis with every final rule.6 The Chief 

Counsel for Advocacy was charged with monitoring federal agencies’ 

compliance with the act and with submitting an annual report to 

Congress.

 

4

Prior to establishing a separate Office of Advocacy within SBA in 1976, Congress created 

the position of Chief Counsel for Advocacy within SBA. See Pub. L. No. 93-386, § 10, 88 

Stat. 749 (1974).

5

Under the RFA, the term “small entity” includes small businesses, small governments, 

and small organizations. 5 U.S.C. § 601(6). For the purposes of this report, we use the 

term “small businesses” since that is our focus.

6

When an agency can determine that the rule would not have a “significant economic 

impact on a substantial number of small entities,” the head of the agency may certify to 

that effect and forego the regulatory flexibility analysis requirements. (5 U.S.C. § 605). 

Background
 

Page 5 GAO-14-525 Office of Advocacy

Second, in 1996 the Small Business Regulatory Enforcement Fairness 

Act (SBREFA) provided for the judicial review of agency compliance with 

key sections of the RFA. It also established a requirement that EPA and 

OSHA convene panels whenever these agencies are developing a rule 

for which an initial regulatory flexibility analysis would be required 

(SBREFA panels).7 These panels consist of the agency, OIRA, and 

Advocacy. The 2010 Dodd-Frank Act added the newly created CFPB to 

the agencies required to convene SBREFA panels.8 The SBREFA panels 

meet with representatives of the affected small businesses to review the 

agencies’ draft proposed rules, identify alternative approaches to the 

rules, and provide insight on the anticipated impact of the rules on small 

entities. The panels issue a report, including any recommendations for 

minimizing the economic impact of the rule on small entities.

Third, Advocacy’s responsibilities were further expanded by Executive 

Order 13272, which was issued in 2002. The order required each agency 

to establish procedures and policies to promote compliance with the RFA 

and to publish a response in the Federal Register to any written comment 

received from Advocacy on rules published.9 This requirement was 

codified by the Small Business Jobs Act of 2010.10 Executive Order 

13272 also directs Advocacy to provide training to federal agencies on 

how to comply with the RFA.

Until 2010, Advocacy’s budget was part of SBA’s. As of fiscal year 2010, 

however, Advocacy was given statutory line-item funding in a Treasury 

account separate from other SBA funding, with Congress setting the 

amount available for Advocacy’s direct costs.11 In fiscal year 2014, 

Advocacy’s enacted budget was $8.75 million. Its fiscal year 2015 budget 

request was $8.46 million. Of that amount, $7.75 million (92 percent) is to 

be used to fund compensation and benefits for Advocacy’s professional 

staff, with the balance of Advocacy’s budget split almost equally between 

 

7

The Small Business Regulatory Enforcement Fairness Act of 1996 (P. L. No. 104-121, 

title II).

8

The Dodd-Frank Wall Street Reform and Consumer Protection Act (P. L. No. 111-203).

9

Executive Order 13272, Proper Consideration of Small Entities in Agency Rulemaking, 

2002.

10Pub. L. No. 111-240, § 1601(4), 124 Stat. 2504, 2551 (codified at 5 U.S.C. § 604(a)(3)).

11See Pub. L. No. 111-240, sec. 1602(b) 
 

Page 6 GAO-14-525 Office of Advocacy

external research and all other direct expenses.12 Advocacy is organized 

in five offices, as shown in figure 1.

Figure 1: Office of Advocacy Organizational Chart, Fiscal Year 2014

Advocacy’s Office of Economic Research produces both internal and 

external research, which is publicly disseminated, on a variety of smallbusiness issues. More specialized research—requiring proprietary data or 

econometric analysis—typically is conducted by contractors (external). 

SBA handles the contracting process for Advocacy. The contracts 

generally last for 1 year. Advocacy economists act as the official 

contracting officers’ representatives (contracting officer) overseeing and 

coordinating the work of the contractors. The contracting officers maintain 

a contract file for each external research product. Each year the Office of 

Economic Research solicits research topics from Advocacy staff and 

small business stakeholders, such as associations comprised of small 

businesses. In addition, Congress requests studies, either formally (by 

putting the requirement into a law), or informally (through discussions with 

Advocacy staff). A final list of potential research is presented to the Chief 

Counsel before the beginning of the fiscal year, and the counsel chooses 

the topics for the year. In fiscal year 2013, Advocacy produced 22 

 

12Under Public Law 111-240, SBA must provide support to Advocacy, including office 

space and equipment, communication and IT services, and equipment and facilities 

maintenance. Therefore any costs of those services, as well as centralized indirect 

expenses shared with other SBA offices, would appear in SBA’s budget request, not 

Advocacy’s.

Research Activities
 

Page 7 GAO-14-525 Office of Advocacy

research products on topics that included access to capital, small 

business exporters, entrepreneurship, and minority- and women-owned 

businesses. Other Advocacy research addresses the concerns 

highlighted in Advocacy’s authorizing statute, such as examining the role 

of small business in the American economy, assessing the effectiveness 

of existing federal subsidies and assistance programs for small 

businesses, and evaluating efforts to assist small veteran-owned small 

business concerns.13

Advocacy follows a peer review process that was revised and formalized 

in 2013 and applies to all internal and external research products. 

Advocacy staff, including the Director of the Office of Economic 

Research, conduct peer reviews for internal products. For external 

research products, the peer review is initiated when Advocacy receives 

the first draft of the product, typically 6 to 8 months after the contract has 

been awarded, and generally includes external reviewers. In addition to 

the peer review process, all of Advocacy’s research products are required 

to pass Advocacy’s internal clearance process, which involves review by 

the Director of Economic Research, editors in Advocacy’s Office of 

Information, and individuals in Advocacy’s Office of Chief Counsel (Senior 

Advisor, Deputy Chief Counsel, and Chief Counsel).

Advocacy’s Office of Interagency Affairs oversees the office’s regulatory 

activities, which aim to convey the views of small businesses on the 

impact of federal regulations and related costs. These activities generally 

fall into three categories—developing and issuing comment letters, 

convening information roundtables, and providing RFA training.14

Attorneys in this office (“regulatory attorneys”) are expected to become 

experts in the policy areas they oversee and to establish and maintain 

broad and effective networks of small business experts (e.g., trade 

associations) in their policy area. The regulatory attorneys are 

encouraged to go to trade association and other industry meetings in 

order to maintain and expand those networks. In addition to maintaining 

working relationships with industry members and experts, the staff are to 

 

13P. L. No. 94-305, §202, 90 Stat. 668 (1976), as amended. 

14The regulatory attorneys also engage with rulemaking agencies through the interagency 

review process, which is run by OMB’s Office of Information and Regulatory Affairs 

(OIRA). However, OIRA and its operations are outside the scope of this report. 

Regulatory Activities
 

Page 8 GAO-14-525 Office of Advocacy

establish and maintain relationships with the regulatory staff within each 

agency who write the rules.

One of the primary ways Advocacy provides input to agencies that are 

issuing rules and regulations of concern to small businesses is through 

public comment letters. Our review of comment letters from fiscal years 

2009 through 2013 found that they covered a wide range of rulemakings 

on issues such as food labeling, designations for critical habitat, and 

emission standards. Advocacy made a number of recommendations in its 

comment letters that included creating an exemption for small businesses 

or strengthening economic analyses required by the RFA. Advocacy also 

issued “nonrule” letters that involved the agencies’ other activities, such 

as agencies’ scientific research. These letters constitute a small 

proportion of Advocacy’s comment letters. Table 1 below shows the 

number of comment letters on rulemakings by fiscal year.

Table 1: Advocacy Comment Letters by Policy Area, Fiscal Years 2009-2013

Policy area

Fiscal year 

2009

Fiscal year 

2010

Fiscal year 

2011

Fiscal year 

2012

Fiscal year 

2013 Total 

Environment 11 9 10 6 7 43

Natural resources and agriculture 6 4 8 5 3 26

Safety and transportation 5 9 6 2 3 25

Banking and economic regulation 1 2 7 3 5 18

Food, drug, and health 3 4 6 0 3 16

Labor and immigration 1 3 6 1 1 12

Procurement and small business 1 4 1 3 1 10

Rulemaking issues (no specific policy area) 2 1 3 3 0 9 

Telecommunications and energy 3 2 2 2 0 9 

Taxes, pensions, and securities 2 0 4 1 1 8 

Intellectual property 0 1 0 0 2 3 

Not applicable 0 0 0 2 0 2 

Total 35 39 53 28 26 181

Source: GAO analysis of Advocacy data. | GAO-14-525

Regulatory attorneys also convene information-gathering roundtables to 

discuss the regulatory concerns of small businesses. Roundtables are 

convened on a regular basis in two policy areas—environment and labor 

safety—while events covering other areas are convened on an ad hoc 

basis, depending on which regulatory or rulemaking issues might be 

forthcoming. According to the regulatory attorneys, the most typical 

reason for convening a roundtable was an upcoming rule or legislation 
 

Page 9 GAO-14-525 Office of Advocacy

that would impact small businesses. The regulatory attorneys use the 

information gathered from the roundtables, together with other 

information, to inform Advocacy’s positions on the issues involved, and to 

give Advocacy direction on proposed rules’ economic impacts and 

possible regulatory alternatives. The attorneys also told us roundtable 

discussions help them set priorities, and broaden their knowledge base. 

The roundtables sometimes resulted in a comment letter, although not 

always. Table 2 shows the number of roundtables by policy area for fiscal 

years 2009 through 2013.

Table 2: Number of Roundtables by Policy Area, Fiscal Years 2009-2013

Policy areaa Fiscal year 

2009

Fiscal year 

2010

Fiscal year 

2011

Fiscal year 

2012

Fiscal year 

2013 Total

Safety and transportation 11 9 9 9 8 46

Environment 7 10 11 9 7 44

Taxes, pensions, and securities 2 7 6 6 2 23

Banking and economic regulation 0 1 0 4 1 6 

Labor and immigration 1 1 3 1 0 6 

Veteran issues 1 1 3 0 0 5 

Intellectual property 0 1 1 1 0 3

Natural resources 0 1 0 1 1 3 

Labor safety/environment 0 0 0 2 0 2 

Rulemaking issues (no specific policy area) 1 0 0 1 0 2 

Procurement and small business 0 1 0 0 0 1 

Telecommunications 0 0 1 0 0 1 

Total 23 32 34 34 19 142

Source: GAO analysis of Advocacy data. | GAO-14-525

a

There are more policy areas for the roundtables since there are instances where Advocacy hosted 

roundtables in areas that did not have comment letters, such as veteran issues. 

As discussed previously, Executive Order 13272 requires Advocacy to 

provide training to the agencies on how to comply with the RFA.

According to data provided by Advocacy officials, in 2013, Advocacy staff 

provided training on the RFA to 159 officials at nine agencies and to 22 

congressional staff. In addition to the formal training sessions, regulatory 

attorneys told us they were encouraged to interact regularly with the 

relevant rulemaking officials at the agencies as rules were developed in 

order to communicate the concerns of the small business advocates.
 

Page 10 GAO-14-525 Office of Advocacy

Producing research products, both internally and externally, on issues of 

importance to small businesses is one of Advocacy’s primary 

responsibilities. However, we found that Advocacy’s quality review 

process lacked some key controls to substantiate the quality of the 

research and did not take steps to ensure that staff were adhering to

existing controls.

According to Advocacy officials, peer review is the main quality control 

over the research it disseminates. Advocacy’s current Chief Counsel 

recently directed the office to strengthen its peer review process with the 

intent to make it more rigorous and consistent. As a result, during the 

course of our review, Advocacy finalized a written peer review process. 

We found that the written guidance discussed the various levels of review 

for internal and external products as well as a process for initiating peer 

review. However, it did not specify how the economists who managed the 

research products were to identify peer reviewers. Instead, Advocacy 

officials told us that they relied on their own expertise and professional 

contacts to identify appropriate peer reviewers and provide 

recommendations through the Director of Research and to the Chief 

Counsel for Advocacy. The officials told us that within a specific subject 

matter there is often a small group of available peer reviewers, in part 

because their expertise is specialized and there are not many other 

alternatives.

OMB’s peer review guidance calls on agencies to select peer reviewers 

with the appropriate knowledge and expertise and to take into account 

their independence and lack of conflicts of interest.15 Advocacy managers 

told us that, in practice, the economists recommend peer reviewers based 

on knowledge and experience in both subject matter and databases—as 

discussed in the OMB guidelines. However, they did not provide specific 

written guidance to the economists on how to identify peer reviewers. 

Federal internal control standards state that internal control activities help 

ensure that management’s directives are carried out and in implementing 

 

15 OMB, Final Information Quality Bulletin for Peer Review, December 16, 2004. 

Advocacy’s Internal 

Controls Do Not 

Ensure the Quality of 

Its Research and That 

Required Information 

Quality Practices Are 

Followed

Advocacy’s Peer Review 

Process Lacks Guidance 

for Identifying Reviewers
 

Page 11 GAO-14-525 Office of Advocacy

those standards management is responsible for developing the detailed 

policies, procedures and practices for their agency’s operations.16 With 

additional guidance, Advocacy would be in a better position to help 

ensure that the economists fully understand how best to identify qualified 

peer reviewers and carry out the Chief Counsel’s directive to improve the 

peer review process.

Advocacy does not have consistent documentation showing whether a 

peer review occurred for all of its research products. Our review of 20 

recent research products—10 internal and 10 external—revealed that 16 

did not have documented peer reviews in the research files.17 According 

to interviews with Advocacy economists who managed the research, all of 

the 20 products underwent some form of peer review. The economists 

said that the type of review was commensurate with the methodological 

complexity of each product, among other factors. However, the Advocacy 

officials were unable to produce any documentation that peer reviews 

occurred for these 16 products. According to Advocacy’s peer review 

guidance, the economists who manage the research should document all 

correspondence pertaining to the peer review and maintain this 

documentation in the research file. In addition, federal internal control 

standards require all transactions and significant events be documented 

and that the documentation be readily available.

Advocacy officials do not have procedures to review the external research 

files to ensure that the peer reviews were documented. Furthermore, they 

noted that for some of the less in-depth internal research products—

typically 2 to 5 pages—such documentation would be administratively 

burdensome.18 However, we note that the documentation could be 

likewise concise, such as a checklist, or a form that reviewers sign, 

 

16GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 

(Washington, D.C.: November 1999). These internal control standards, first issued in 

1983, present the internal control standards for federal agencies for both program and 

financial management.

17Of the 10 external products we reviewed, 6 did not contain documentation on whether a 

peer review occurred. Of the 10 internal products we reviewed, none contained 

documentation on whether a peer review occurred. 

18According to our analysis, of the 70 internal research products produced from fiscal 

years 2008 through 2012, approximately 54 percent were shorter products, between 1 to 5 

pages.

Advocacy Has Not 

Consistently Documented 

Peer Reviews
 

Page 12 GAO-14-525 Office of Advocacy

similar to the one currently used by Advocacy for its internal clearance 

process. Otherwise, absent written documentation, Advocacy managers

are limited in their ability to conduct oversight and ensure that this key 

quality control activity is happening. For example, for one of the internal 

research products we selected for review, the study author no longer 

worked at Advocacy, and therefore no one could tell us with any certainty 

whether the required peer review had occurred, or who had participated 

in it. Without adequate documentation—a key internal control—of its peer 

reviews, Advocacy does not have an institutional record of its activities

and cannot demonstrate that it is following its own peer review process.

In addition, Advocacy has not consistently documented how peer 

reviewers’ comments were addressed by the authors of its external 

research products. Of the 10 external research files we reviewed, 4 had 

documentation that a peer review occurred, and 1 file included evidence 

that the peer reviewer comments were incorporated into the final report. 

Advocacy officials told us that the economists who managed external 

research consolidated the peer reviewers’ comments and forwarded to 

the author those that needed to be addressed, including methodological 

and data issues and other comments, but not those that might change the 

scope of the contracted research.19 They also told us that they did not 

typically maintain documentation showing which comments had been 

addressed and why, but included the final report in the research file. 

However, Advocacy’s peer review process states that the economist 

managing the research will analyze and incorporate, as needed, peer 

reviewers’ suggestions and maintain all related documentation in the 

research file. However, since the economists are not keeping records and 

documenting that comments have been considered and addressed, 

management does not have an institutional record to provide reasonable 

assurance that its quality control process is being followed.

 

19According to Advocacy officials, it is not permissible to expand the scope of the research 

or change the nature of the deliverables specified in the contract after an award has been 

made. Therefore, if the peer reviewer’s comments discuss changes that would result in 

expanding the scope of the research (e.g., analyzing data from additional time periods), 

those comments could not be accommodated. 
 

Page 13 GAO-14-525 Office of Advocacy

While Advocacy has quality review policies for its peer review process, it 

does not have policies and procedures that reflect the federal information 

quality guidelines on retaining data for influential studies or taking other 

steps to substantiate the quality of information in such studies when they 

have not retained the data. Advocacy officials told us that they did not 

retain the original data or underlying computer codes for three external 

studies on the costs of regulation as required by the information quality 

guidelines.20 We focused on external studies on estimating the costs of 

regulation because it is a key research area for Advocacy, according to its 

originating statute and the mission statement of its Office of Economic 

Research. The OMB Information Quality Guidelines require that all 

agencies producing and disseminating “influential statistical information” 

help ensure a high degree of transparency about data and methods to 

facilitate the reproducibility of such information by qualified third parties. 

The SBA guidelines implement this standard for transparency by requiring 

that the underlying data be stored and made available for public review 

for as long as the agency-disseminated information based on the data are 

valid. The guidelines also state that all formulas, calculations, matrixes, 

and assumptions used in processing the data should be available. 

Because Advocacy classified two of the regulatory cost studies as 

“influential” according to the OMB guidelines, those data should have 

been maintained.21

Advocacy officials said that they did not maintain the data or models for 

influential external research because there might be a cost associated 

with obtaining such data, which would raise the costs of the studies, 

possibly making them prohibitively expensive. However, in the case of 

two of the studies, the original data were from publicly available sources 

and involved a relatively small dataset, suggesting the cost would not 

 

20Andy Bollman, E. H. Pechan & Associates, Evaluation of Barrier Removal Costs 

Associated with the 2004 Americans with Disabilities Act (ADA) Accessibility Guidelines, 

prepared for SBA Office of Advocacy, under contract SBAHQ-03C0020 (Washington, 

D.C.: 2007); W. Marc Crain, The Impact of Regulatory Costs on Small Firms, prepared for 

SBA Office of Advocacy, under contract SBHQ-03-M-0522 (Washington, D.C.: 2005); and 

Nicole V. Crain and W. Mark Crain, The Impact of Regulatory Costs on Small Firms, 

prepared for SBA Office of Advocacy, under contract SBAHQ-08-M-0466 (Washington, 

D.C.: 2010). We refer to the latter two studies as “the Crain studies.” 

21The study by Andy Bollman and E. H. Pechan & Associates was also classified as 

“influential.” 

Advocacy Has Not 

Followed Some Required 

Elements of Information 

Quality Guidelines
 

Page 14 GAO-14-525 Office of Advocacy

have been prohibitive.22 The OMB guidelines state that sufficient 

transparency—achieved in part by storing the relevant data—results in 

analyses that can be substantially reproduced. Not retaining the 

underlying information for these influential research papers makes it 

much more difficult to assess the quality of that work, including its 

objectivity—a key goal of the information quality guidelines.

We also found that Advocacy staff had not taken additional steps, in the 

absence of retaining the underlying data, to substantiate the quality of the 

regulatory cost estimates in two of the studies that it sponsored and 

disseminated.23 The OMB guidelines state that when data and methods 

are not retained and made available to the public because of other

compelling interests such as privacy, trade secrets, intellectual property, 

and other confidentiality protections, the agency shall apply rigorous 

checks to the analytical results and document what checks were 

undertaken. Because Advocacy had not retained data on the two cost 

estimation studies that had been criticized, we interviewed senior 

Advocacy officials, including the Director of its Office of Economic 

Research, about the information and methodologies used in the studies. 

We asked them a set of questions related to criticisms of the 

methodologies, data, and models used in the studies that were identified 

in our evaluation and the work of other researchers. Advocacy staff 

declined to answer many of our questions and instead directed us to the 

authors, stating that they, not Advocacy economists, were the experts on 

 

22In the case of the 2010 Crain study, the original underlying data for the economic 

regulation cost regression were publicly available from the World Bank and the 

Organisation for Economic Co-operation and Development, and the study’s regressions 

contained only 118 records and 6 explanatory variables. Furthermore, since the 

regression models employed in the studies consisted of one equation each, we do not 

believe there would have been significant costs associated with requiring the Crains to 

provide Advocacy the data and computer coding written to produce the estimates. While 

Advocacy did not classify the 2005 Crain study as “influential” according to the OMB and 

SBA criteria, because the 2005 study used similar methodologies, we examined them 

together.

23We asked Advocacy officials how they substantiated the two Crain studies. We did not 

ask the Advocacy officials to verify the quality of the third regulatory cost study (Evaluation 

of Barrier Removal Costs Associated with the 2004 Americans with Disabilities Act (ADA) 

Accessibility Guidelines) because our work did not reveal the same level of 

methodological weaknesses that we identified in the Crain studies. 
 

Page 15 GAO-14-525 Office of Advocacy

the issues covered in the studies.24 However, the authors would not 

speak with us, stating that they were no longer contractually obligated to 

respond to our requests for information.

During our discussions with Advocacy officials, they stated that the 

purpose of the studies was not to estimate the overall costs of 

regulations, but rather to estimate the disproportionate share borne by 

small businesses. They also noted that, as with all contract research, the 

external research reports contain a disclaimer indicating that the views 

presented did not necessarily represent those of Advocacy. In addition, 

Advocacy added language to the 2010 report’s cover page about the 

uncertainty of the authors’ estimates of the costs of regulation.25

Advocacy officials noted that the majority of the research it conducts is 

not classified “influential” according to the OMB and SBA guidelines, and 

that they have no plans to engage in such work in the near future. 

However, given Advocacy’s mission, it may do influential research in the 

future as it has in the past even if on a limited basis. We acknowledge 

that these reports may not necessarily be representative of all Advocacy’s 

research efforts, but not substantiating the quality of the information in 

even one study could call into question the credibility of Advocacy’s 

research program. Thus, establishing policies and procedures that reflect 

the federal information quality guidelines on retaining data for influential 

studies or when such data are not retained because of certain compelling 

interests, and requiring additional steps to substantiate the quality of the 

information, would put Advocacy in a better position to provide 

reasonable assurance about the quality of its research program.

OMB Information Quality Guidelines require that agencies develop 

policies to ensure that managers be able to substantiate the quality of 

information they disseminate. The guidelines also discuss narrow 

 

24Advocacy officials also provided us with rebuttals the Crains published in response to 

earlier criticisms by others of the studies. However, the rebuttals do not directly address 

important criticisms. 

25This one-page statement notes, among other things, that “The study is a top-down

analysis of regulatory costs that uses certain assumptions to estimate totals, and is not a 

bottom-up precise accounting of the overall cost of regulations.” Also, the statement says: 

“The overall figure of $1.75 trillion in costs is derived from a number of different 

assumptions and sources to create an estimate. As with almost any academic 

methodology, it was not intended to be considered a precise finding.” For the full 

statement, see the report, available on Advocacy’s website, accessed on April 24, 2014, 

www.sba.gov/advocacy/7540/49291.
 

Page 16 GAO-14-525 Office of Advocacy

circumstances under which an agency does not have to substantiate the 

quality of the information that resulted from a research project of one of its 

contractors or grantees. In those circumstances, the researcher is to 

make clear with an appropriate disclaimer that the views expressed in the 

research are his or her own and do not necessarily reflect those of the 

agency. However, OMB cautions that “if an agency, as an institution, 

disseminates information prepared by an outside party in a manner that 

reasonably suggests that the agency agrees with the information, this 

appearance of having the information represent agency views makes 

agency dissemination of the information subject to these guidelines.”

26

As we noted previously, Advocacy placed a disclaimer on the two studies, 

but its actions indicate its agreement with the information in the studies. 

First, Advocacy made the two studies available on its website, where they 

continue to be available. Second, Advocacy’s disclaimer that appeared on 

the 2010 study noted that it contained information and analysis that was 

reviewed and edited by Advocacy. Advocacy’s description of its review 

role suggests that the agency contributed to the content, if not the 

conclusions of the study. Finally, several Advocacy comment letters have 

cited the 2005 study’s regulatory cost estimates in support of their 

arguments.27 Because Advocacy’s actions raise, at the least, an 

appearance of agreement with the information contained in the studies, 

we believe that the office was required to substantiate the quality of the 

estimates of the economic costs contained in the studies.

 

26OMB. Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and 

Integrity of Information Disseminated by Federal Agencies. 67 Fed. Reg. 8452 (Feb 22, 

2002).

27See Advocacy letter to Environmental Protection Agency, dated November 28, 2008, 

Re: Comments on EPA’s Advance Notice of Proposed Rulemaking “Regulating 

Greenhouse Gas Emissions under the Clean Air Act,” Docket ID No. EPA-HQ-OAR-2008-

0318; Advocacy letter to Environmental Protection Agency, dated June 23, 2009, Re: 

Docket ID No. EPA-HQ-OAR-2009-0171, Comments on EPA’s “Proposed Endangerment 

and Cause or Contribute Findings for Greenhouse Gases Under Section 202(a) of the 

Clean Air Act”; Advocacy letter to National Marine Fisheries Services, Department of 

Commerce, dated October 30, 2009, Re: RIN 0648-AW18 Magnuson-Stevens Fishery 

Conservation and Management Act: Regional Fisheries Management Councils; 

Operations. 
 

Page 17 GAO-14-525 Office of Advocacy

Advocacy has practices and procedures (“policies”) for its regulatory 

activities—comment letters and roundtables—however, documentation of 

these key regulatory activities is inconsistent. Furthermore, while we 

determined that transparency and other requirements in FACA do not 

apply to Advocacy’s roundtables, Advocacy is not following its internal 

policies meant to ensure its roundtables are as open to the public as they 

could be. As a result Advocacy cannot demonstrate that it is always fully 

meeting its mission to foster two-way communication between small 

businesses and federal policymakers.

Our review of Advocacy’s key regulatory activities—developing and 

issuing comment letters and convening information roundtables—found 

that Advocacy staff are inconsistently maintaining documentation of key 

decisions and events. In April 2014, Advocacy provided us with a 

“practices and procedures guide” for the Office of Interagency Affairs, 

dated March 2014. The guide covers staff activities related to initiating 

comment letters and convening roundtables, among other things. 

Advocacy management told us that they update the guidance periodically, 

including amending it during the course of our review to make it clear that 

the decision to issue a public letter or hold a roundtable rests with the 

Chief Counsel for Advocacy. However, our review found that the updated 

guide continued to lack policies for documenting key decisions and 

activities. For example, the guide stated that when regulatory attorneys 

decide to intervene in the rulemaking process, they must follow up, as 

appropriate, with the interested associations to ensure that Advocacy has 

sufficient information and data to make its case for intervening. But there 

is no policy to document these interactions. The following are other 

instances where we found a lack of policies to document key activities.

• Small Business Input into Comment Letters. Advocacy does not have 

any policies requiring that the regulatory attorneys retain 

documentation showing which entities provided input into comment 

letters, and we found that the attorneys do not consistently do so. Our 

analysis of Advocacy’s comment letters found that about 57 percent of 

the letters referenced small business input or concerns. But when we 

interviewed seven regulatory attorneys about how they developed 

comment letters generally, some told us that they maintained a record 

of the entities providing input for specific letters, but others said that 

they did not. Further, when we asked Advocacy staff for 

documentation on the sources of the small business input referred to 

Documentation of 

Key Regulatory 

Activities is 

Inconsistent and 

Roundtables Are Not 

Publicized According 

to Advocacy’s Policies

Advocacy’s 

Documentation for Key 

Regulatory Activities Is 

Inconsistent
 

Page 18 GAO-14-525 Office of Advocacy

in a nonrepresentative sample of 11 comment letters, they were 

unable to provide specific emails or notes of conversations.28

• Reasons for Convening Roundtables. Advocacy’s practices and 

procedures guide states that the Chief Counsel must approve the 

proposed agenda, speakers, and discussion topics for all proposed 

roundtables before participants are invited. But the guide contains no 

policy that this step be documented. We interviewed seven regulatory 

attorneys responsible for the roundtables and their management and 

were told that the attorneys set agendas and selected speakers based 

on their own assessment of the issues and, in some cases, with 

suggestions from small businesses or interested industry parties. 

Advocacy officials emphasized that they only held roundtables when 

there is sufficient interest or need on the part of small businesses.

• Roundtable Discussions and Participants. Information gathered from 

the roundtables is used to inform Advocacy’s positions on issues 

related to small businesses and in comment letters, but Advocacy’s 

guidance contains no policies to document roundtable discussions. 

Our content analysis of Advocacy’s comment letters showed 19

percent of them referred to roundtable discussions. However, staff did 

not routinely take and maintain notes of the discussions, according to 

the interviews we conducted with the regulatory attorneys. In addition, 

not all staff take attendance at the events. In most cases, Advocacy 

staff keep an “RSVP list” of those who have indicated that they will 

attend. However, some of the staff noted that the RSVP lists may not 

include all participants, including those on the phone. As a result it is 

difficult to determine the extent to which small businesses and related 

entities were represented at these events.

Advocacy officials told us that they did not have guidance on maintaining 

documentation on the sources of input to comment letters or on 

roundtable discussions because they did not feel that setting such 

standards was required to fulfill their duties under Advocacy’s statute. 

Furthermore, they noted that, in the case of roundtables, keeping records 

in a manner that identified specific speakers would inhibit discussion and 

limit their ability to gain valuable input. They also cited logistical 

challenges in taking accurate attendance at larger events. We 

acknowledge that specific parties might not want to be publicly named, 

but federal standards for internal control call for agencies to document 

 

28We selected a nonrepresentative sample of recently published comment letters—

published from 2011 through 2013—weighting our choices of letters to reflect the 

distribution of policy areas listed in table 2. 
 

Page 19 GAO-14-525 Office of Advocacy

significant events. Taking steps to balance the need for privacy so 

individuals can speak freely, with a commitment to maintain a basic level 

of documentation of these events—that is, documenting which entities 

provided input into its comment letters and roundtables—could help 

Advocacy demonstrate that it is meeting its mission to represent the 

interests of small businesses.

Key documents for Advocacy’s roundtables—agendas and presentation 

materials—are not made available to the public at-large after the fact. The 

regulatory attorneys cultivate email lists of relevant stakeholders who are 

invited to roundtable events, and these lists are continually updated. It is 

Advocacy’s policy to add any interested parties to the invitation list, if 

asked, and several of the attorneys we interviewed said they did so. They 

also told us they made agendas and presentations available to any 

interested parties who requested them after the roundtable. However, the 

agendas and presentations are not posted to the website or made 

publically available in any systematic way and if small businesses and 

other interested parties do not know about the roundtables, they cannot 

request information from the events.

Advocacy’s policies and procedures state that agendas and presentations 

should be posted on Advocacy’s website. Advocacy officials stated they 

had been unable to post roundtable materials to the website, which SBA 

maintains, because of difficulties in meeting certain readability and 

accessibility requirements in the Americans with Disabilities Act.29

However, a variety of other Advocacy reports and publications are posted 

on its website that also must meet these requirements, and Advocacy 

officials did not explain why information on the roundtables could not be 

likewise included. Making the roundtable materials available on its 

website would strengthen Advocacy’s efforts to inform small businesses 

and other interested parties about its efforts to represent their interests to 

federal decision makers.

 

2942 U.S.C. § 12101 et seq. 

Advocacy’s Policy for 

Publicizing Information on 

Roundtable Events Is Not 

Followed
 

Page 20 GAO-14-525 Office of Advocacy

In addition to evaluating whether Advocacy publicizes and conducts its 

roundtables in accordance with its internal policies and procedures, we 

also analyzed whether Advocacy’s roundtable groups constitute “advisory 

committees” subject to the public notice and other transparency 

requirements of FACA.30 We found that the roundtables are not advisory 

committees, and thus Advocacy is not required by law to follow these 

rules. We first determined that Advocacy is an “agency” covered by 

FACA. It is an “authority of the Government of the United States, whether 

or not it is within or subject to review by another agency.”

31 In addition, 

Advocacy possesses the type of “substantial independent authority” 

required by the courts. While located within SBA, Advocacy is 

independent of SBA, and it has distinct statutory authorities and 

responsibilities, a separate statutory charter, and an appropriation 

account that is separate from the rest of SBA. Furthermore, its duties and 

authorities under statute and Executive Order are substantial and its 

recommendations made in furtherance of small entities’ concerns must, 

by law, be given considerable weight by other agencies.

While we found that Advocacy is a FACA “agency,” we concluded that its 

roundtable groups are not “advisory committees” as defined by the statute 

and interpreted by the courts and implementing regulations. A covered 

advisory committee is a panel, task force, or similar group created or 

used by an agency for the purpose of providing advice or 

recommendations on particular matters. Participants’ input must be 

sought as a group, not as a collection of individuals. The formality and 

fixed structure of the group also is an important factor in determining 

coverage under FACA. Advocacy does not seek roundtable participants’ 

input as a group, however; rather, attendees provide their individual 

perspectives on the agency rule or policy under discussion, do not 

significantly interact with one another, and no attempt is made to reach a 

consensus. Advocacy’s roundtables also do not have the requisite 

organized or fixed structure; rather, as noted, Advocacy’s policy is to 

extend roundtable invitations to anyone who expresses a desire to attend. 

Finally, according to Advocacy, it is the agency itself, not the roundtable 

participants, that develops policy advice and recommendations, albeit 

based in part on the data and information provided by roundtable 

participants (as well as obtained elsewhere). Our detailed legal analysis 

 

305 U.S.C. App. II.

315 U.S.C. App. II, § 3(3), incorporating 5 U.S.C. § 551(1) (Administrative Procedure Act). 

Federal Advisory 

Committee Act 

Requirements Do Not 

Apply to Advocacy’s 

Roundtables
 

Page 21 GAO-14-525 Office of Advocacy

of the applicability of FACA to Advocacy’s roundtables is contained in 

appendix III.

Workforce planning presents some challenges for Advocacy, in part 

because the office is small and has a large number of positions, which, 

according to Advocacy officials, typically are replaced with each new 

administration. While the office currently does some workforce planning, 

its efforts do not include the long-term strategic plans that would help

ensure that it maintains the expertise and skills needed to fulfill its 

mission. While Advocacy now has responsibility for developing its own 

budget, goals, and performance measures, our review of its strategic plan 

and goals did not find goals or objectives that discussed workforce issues 

such as staff development and succession planning. Advocacy officials 

told us that they used their organizational chart as the basis for their 

workforce planning efforts and discussed workforce issues with staff and 

managers at meetings as needed (see fig. 2). Advocacy officials said that 

because Advocacy was a small office with only 46 staff, the 

organizational chart and meetings to discuss staffing issues met their 

workforce planning needs. Advocacy officials told us that to develop staff 

they provided training to regulatory attorney and economist staff, had 

established a mentoring program for newer staff, and were implementing 

a knowledge-sharing database to help staff develop expertise in 

specialized policy areas.

Figure 2: Office of Advocacy Organizational Chart, Fiscal Year 2014

When we asked Advocacy officials about succession planning and their 

plans for addressing the departure of senior or experienced staff, the 

officials told us that most of the turnover in staff occurred during changes 

Advocacy’s 

Workforce Planning 

Efforts Lack 

Succession Planning
 

Page 22 GAO-14-525 Office of Advocacy

in administration. For example, according to Advocacy officials, the 

change of administration in 2009 resulted in the turnover of at least 15 of 

20 senior officials. Among these were the 10 regional advocates, who are 

located in each of SBA’s regional offices across the country, and make up 

the Office of Regional Affairs. Turnover tends to be low among the 

economists and regulatory attorneys, some of whom have been with the 

office for years. Advocacy officials said that because rulemaking often 

took so long, they generally had enough time to hire or realign any staff 

as needed.

According to federal internal control standards, effective management of 

a workforce is essential to achieving program results. Our body of work 

on workforce planning has demonstrated the importance of such planning 

and the need to develop long-term strategies—such as training and 

succession planning—for acquiring, developing, and retaining staff to 

achieve programmatic goals.32 OPM’s Human Capital Assessment and 

Accountability Framework states that agencies and offices with workforce 

planning are better able to manage their staff by, for example, ensuring 

that systematic processes are in place for identifying and addressing any 

gaps between current and future workforce needs. Further, OPM 

recommends that succession planning is needed to ensure continuity in 

leadership positions. In addition, workforce planning can help 

management determine the type of training and other strategies that are 

needed to address factors such as projected retirements and succession 

planning. In past work on succession planning, we have found that in 

addition to focusing on replacing individuals, succession planning 

strategies can also strengthen current and future organizational 

capacity.33

Given the length of time some attorneys and economists have been with 

Advocacy, the loss of their expertise through retirement, among other 

things, could leave significant gaps in needed skills and knowledge, 

according to Advocacy officials. Yet Advocacy has not incorporated 

 

32GAO, Human Capital: Key Principles for Effective Workforce Planning, GAO-04-39

(Washington, D.C.: Dec. 11, 2003), and Securities and Exchange Commission: Improving 

Personnel Managements is Critical for Agency’s Effectiveness, GAO-13-621 (Washington, 

D.C.: July 18, 2013).

33GAO, Human Capital: Selected Agencies Have Opportunities to Enhance Existing 

Succession Planning and Management Efforts, GAO-05-585 (Washington, D.C.: June 30, 

2005).
 

Page 23 GAO-14-525 Office of Advocacy

succession planning into its workforce planning efforts, such as its 

training and mentoring initiatives. Further, Advocacy officials told us that 

they discussed future staffing needs and various options for addressing 

them, including training, but these efforts are not documented in a 

manner that would ensure consistent implementation. However, ensuring 

that activities such as staff training are consistently implemented is 

particularly important when senior management can change significantly 

every 4 or 8 years. Without incorporating long-term succession planning 

into its workforce planning efforts, Advocacy is not in the best position to 

ensure that it has qualified staff to fill leadership and other key positions 

and a skilled workforce able to meet the demands of its mission on behalf 

of small businesses.

Effective internal controls are critical for the Office of Advocacy if it is to 

achieve program outcomes and minimize operational problems. 

Recognizing this, Advocacy recently has taken action to improve some of 

its guidance and controls. However, Advocacy’s research, regulatory, and 

workforce planning functions could be improved by strengthening its 

internal controls as follows:

• Research activities. In its research operations, Advocacy has taken 

some initial steps toward establishing stronger control policies by, for 

example, formalizing its peer review process for internal and external 

research products. However, the guidance provided did not include 

information on how to select appropriate peer reviewers—the experts 

whom Advocacy relies on to assess the quality of the research that it 

disseminates. The guidance also lacks policies for documenting that a 

peer review occurred and how reviewer comments were addressed 

for its external research products. Such guidance would help ensure 

that Advocacy research staff fully understand how best to identify the 

most qualified peer reviewers, and how to document and incorporate 

reviewer comments. In addition, Advocacy did not follow federal 

guidelines for information quality for influential studies that set out 

requirements for retaining data and taking additional steps to 

substantiate the quality of information it disseminates when it has 

certain compelling reasons to not retain the data. Improving policies 

and procedures for its research activities would help support 

Advocacy’s mission to provide quality research on small business 

issues that decision makers and the public can rely on.

• Regulatory activities. Advocacy’s lack of documentation and 

transparency of the regulatory activities we reviewed made it difficult 

to validate Advocacy’s efforts to represent small businesses. 

Conclusions
 

Page 24 GAO-14-525 Office of Advocacy

Specifically, weaknesses exist in Advocacy’s documentation of both 

the sources of the small business input in comment letters and the 

views of small businesses discussed and conveyed at its roundtables. 

As a result, the extent to which the small businesses’ views on 

regulations were being obtained and communicated is unknown. 

Improving its guidance to staff on its regulatory activities and 

emphasizing the importance of documentation would enable 

Advocacy to more effectively demonstrate to decision makers that it 

was obtaining and communicating the interests of small businesses. 

In addition, Advocacy is not following its internal policy to post 

materials from roundtables on its website. As a result, it is missing an 

opportunity to reach out more broadly to small businesses and other 

interested parties and increase the transparency of its activities.

• Workforce planning activities. While Advocacy’s workforce planning 

efforts help in managing its current staff, these efforts do not include 

any strategies to plan for succession, even though several staff have 

been with the office for many years and will eventually need to be 

replaced. Without this important element of workforce planning, 

Advocacy could be in a vulnerable position when critical staff leave 

the agency or staff face new demands. Although Advocacy is a 

relatively small office, having a skilled workforce is critical to meeting 

its mission. Succession planning would help ensure that Advocacy 

was better prepared to maintain qualified staff to conduct its research 

and regulatory activities in support of small businesses.

To improve Advocacy’s system of internal control, and help to provide 

reasonable assurance that the office is meeting its mission, we 

recommend that the Chief Counsel of Advocacy take the following five 

actions:

• Strengthen the accountability of its research activities by taking the 

following two actions:

• Enhance its peer review policies and procedures by including 

written guidance on selecting peer reviewers and documenting all 

steps of the peer review process—whether a peer review occurred 

and how and to what extent peer reviewer comments were 

addressed. 

• Develop policies and procedures that reflect the federal 

information quality guidelines on retaining data for influential 

studies, and when not retaining data, taking additional steps to 

substantiate the quality of information disseminated. 

• Strengthen the accountability of its regulatory activities by developing 

policies and procedures to ensure that key elements of that work—

Recommendations for 

Executive Action
 

Page 25 GAO-14-525 Office of Advocacy

such as sources of input for comment letters and roundtable 

discussions—are consistently documented.

• Coordinate with SBA officials who oversee website administration to 

comply with Advocacy’s roundtable policy to make information on the 

events—agendas, presentation materials—publicly available on its 

website so that its regulatory activities are more transparent to the 

public.

• Improve its workforce planning efforts to be better prepared to meet 

its future workforce needs by incorporating succession planning.

We provided a draft of this report to Office of Advocacy for its review and 

comment. In its written comments (reproduced in app. II), Advocacy 

agreed with our recommendations. Advocacy stated that for its research 

activities, its current effort to further formalize procedures for the peer 

review process will include steps for additional documentation and that it 

also plans to develop written guidelines for determining which research 

products are considered influential, which will clarify when Advocacy 

needs to take additional information quality steps. Advocacy also said that 

as it develops how it will disseminate information about its regulatory 

activities, it will incorporate approaches that are responsive to our 

recommendations. Finally, Advocacy agreed that workforce planning is 

important for ensuring that the office maintains the skills and resources 

needed to fulfill its mission and noted that the office is developing a 

Leadership Succession Plan in response to our recommendation.

Agency Comments 
 

Page 26 GAO-14-525 Office of Advocacy

We are sending this report to the Office of Advocacy and interested 

congressional committees. The report also will be available at no charge 

on the GAO website at http://www.gao.gov. If you or your staff have any 

questions about this report, please contact Cindy Brown Barnes at (202) 

512-8678 or brownbarnesc@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made major contributions to this report are 

listed in appendix V.

Sincerely yours,

Cindy Brown Barnes

Acting Director, Financial Markets and

Community Investment
Appendix I: Objectives, Scope, and 

Methodology

Page 27 GAO-14-525 Office of Advocacy

This report examines Advocacy’s (1) research activities; (2) regulatory 

activities, including the applicability of the Federal Advisory Committee 

Act (FACA) to Advocacy’s roundtables; and (3) workforce planning 

efforts. The scope of our work covered key activities conducted from 

fiscal year 2008 to fiscal year 2013, although in some cases, we 

broadened that scope to include fiscal years 2005 through 2008, as noted 

below.

To understand Advocacy’s mission, operations, and participation in the 

rulemaking process, we reviewed relevant laws and regulations, 

Advocacy documents, including its annual reports to Congress on the 

Regulatory Flexibility Act (RFA), budget and strategic planning 

documents, and other publications.

To assess Advocacy’s research activities, we determined: how Advocacy 

staff chose its research topics; how they conducted the research; and 

what controls they had in place to ensure quality products. We analyzed 

Advocacy’s research products produced in the 5 most recent fiscal 

years—2008 through 2012—and interviewed relevant Advocacy officials 

on the processes by which they assess the quality of its research 

products.1 To evaluate Advocacy’s research activities, we analyzed a

nonprobability sample of research, which included 10 research products 

authored by Advocacy staff (internal), and 10 products produced by its 

contractors (external). To select a sample of internal research products 

that best represented the variety of internal research, we reviewed the 58 

research products issued from fiscal years 2008 through 2012, and 

categorized each as either “routine” (meaning published either quarterly 

or semi-annually, and relying on the same underlying data set), or as 

“nonroutine.” Within the routine category, we selected the mostly recently 

published product from each of the following routine publications: (1) 

Small Business Profiles of the States and Territories, (2) Small Business 

Quarterly Bulletin, (3) Quarterly Lending Bulletin, (4) Small Business 

Lending, and (5) The Small Business Economy. For the nonroutine 

products, we created five categories, such as “analysis” and “fact sheets.” 

We selected, from each of those five categories, the most recently 

published report. Then, for each of the 10 selected internal research 

products, we used a data collection instrument and interviewed the 

 

1

For the period we conducted our audit, not all the internal research publications for fiscal 

year 2013 were available in all cases, so we focused on internal research published from 

fiscal years 2008 through fiscal year 2012.

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 28 GAO-14-525 Office of Advocacy

Advocacy staff who wrote the products in order to see to what extent the 

products adhered to Information Quality Guidelines issued by the Office 

of Management and Budget (OMB) and the Small Business 

Administration (SBA) and Advocacy’s internal quality review process 

(“peer review”).2 To assess the quality of the peer review used for 

Advocacy’s external research, we reviewed the contract files for the 10 

most recently published external research products, as of year-end 2013, 

and compared how the work was produced, including Advocacy’s peer 

review process, against the OMB Peer Review Guidance and OMB and 

SBA information quality guidelines, and applicable federal internal control 

standards.3 The results from our reviews cannot be projected to all 

Advocacy studies, but they provide an indication of how Advocacy staff 

conducted or oversaw the research and what controls were in place to 

ensure quality products.

We also assessed Advocacy’s contract studies that focused on the 

economic costs of regulation. We focused on external studies in these 

areas because they are key research areas for Advocacy, according to its 

originating statute and the mission statement of its Office of Economic 

Research. For the economic costs studies, we assessed them against 

OMB and SBA Information Quality Guidelines, and examined Advocacy’s 

compliance with related data retention policies therein.4 We also reviewed 

peer reviewers’ comments and other external reviews on the studies as 

part of our assessment. Lastly, we interviewed Advocacy officials to 

obtain information on steps they took to substantiate the quality of 

information for influential studies when data are not retained.

 

2

OMB, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and 

Integrity of Information Disseminated by Federal Agencies (Washington, D.C.: February 

2002). 67 Fed. Reg. 8452 (Feb. 22, 2002). Agencies were required to issue their own 

implementing guidelines by October 1, 2002. For SBA’s guidelines, accessed on April 24, 

2014, see http://www.sba.gov/aboutsba/sba_performance/open_government/information_quality/information_quality_guideline

s. 

3

OMB, Final Information Quality Bulletin for Peer Review, December 16, 2004. This 

Bulletin established government-wide guidance aimed at enhancing the practice of peer 

review of government science documents. 

4

Advocacy’s studies examining total costs of regulation to the U.S. economy have been 

conducted approximately every 5 years, according to Advocacy officials. For this reason, 

we broadened the scope to fiscal year 2005, whereas in other research work, we focused 

on fiscal years 2008 through 2012. 
Appendix I: Objectives, Scope, and 

Methodology

Page 29 GAO-14-525 Office of Advocacy

To evaluate Advocacy’s regulatory activities, we assessed: how and why 

Advocacy decides to issue comment letters and convene roundtables; the 

policies and practices in place that pertain to those activities; how 

Advocacy staff solicit input from small businesses and other parties. We 

interviewed relevant staff, as described below; reviewed relevant policies 

and procedures, and analyzed comment letters produced and 

roundtables convened from fiscal years 2009 through 2013.5 Specifically, 

we performed a content analysis on the 181 comment letters issued 

during that period, using NVivo software, to analyzed and categorize the 

content of the letters and the nature and source of the small business 

input provided.6 We also interviewed Advocacy staff who are responsible 

for rulemaking and related activities, weighting our choices of 

interviewees to reflect the distribution of comment letters by policy area, 

and reviewed supporting documentation for comment letters to 

understand how Advocacy staff develop and issue comment letters, and 

compared this information to federal internal control standards.

Similarly, we reviewed available information on the 142 roundtables 

convened during the same 5-year period and interviewed the responsible 

Advocacy staff, weighting our choices of interviewees to reflect the 

distribution of roundtables by policy area. We reviewed Advocacy’s 

policies for roundtables to understand their origin, purpose, and 

documentation requirements, and compared this information to federal 

internal control standards. In order to understand the perspective of those 

who attended the roundtables, we attended three of the events. In 

addition, we selected and interviewed a nonprobability sample of past 

participants from email lists provided by Advocacy’s regulatory attorneys. 

Specifically, we talked to the following representatives: five from industry 

associations that represent small and large businesses; one from a large 

corporation; and one nonprofit organization whose mission relates to 

policies or rules under consideration.7 We made our selections to include 

 

5

Similar to our approach in assessing Advocacy’s research work, we wanted to focus on 

the five most recent fiscal years. In the case of the regulatory activities—comment letters 

and roundtables—information was available through the end of fiscal year 2013. 

6

NVivo is a qualitative data analysis software system that allows organization and analysis 

of information from a variety of sources including complex nonnumeric or unstructured 

data. 

7

Separate from interviewing the officials selected from the roundtable invitation lists, we 

interviewed representatives from an association that represents small businesses and a 

nonprofit whose mission relates to transparency in government. We discussed 

roundtables and other Advocacy activities with these parties.
Appendix I: Objectives, Scope, and 

Methodology

Page 30 GAO-14-525 Office of Advocacy

representatives from a variety of sources, and while the results from our 

interviews cannot be projected to all entities that interact with Advocacy, 

the information we gathered does provide insights into how the selected

groups view Advocacy and its work in representing small businesses to 

federal policymakers.

We also reviewed Advocacy’s training to agency staff in how to comply 

with the RFA and interactions with agency rulemaking officials. In 

addition, we interviewed officials from the entities that interact with 

Advocacy in rulemaking—the Environmental Protection Agency (EPA),

Occupational Safety and Health Administration (OSHA), Bureau of 

Consumer Financial Protection, also known as the Consumer Financial 

Protection Bureau (CFPB), and Office of Management and Budget’s 

(OMB) Office of Information and Regulatory Affairs (OIRA).

To analyze whether FACA (5 U.S.C. App. II) applies to Advocacy’s 

roundtables, we reviewed relevant statutes, case law, regulations, and 

guidance. In addition, we reviewed and considered Advocacy’s written 

views on the issue. See appendix III for more information on the legal 

analysis we conducted.

Finally, we reviewed Advocacy’s workforce efforts. To understand 

Advocacy’s workforce planning, we reviewed Advocacy’s strategic goals 

and other planning and budget documents and interviewed senior 

management to determine what, if any, policies and procedures were in 

place related to workforce and succession planning. We assessed any 

such policies and procedures against applicable federal standards for 

internal control and the Office of Personnel Management’s (OPM) Human 

Capital Assessment and Accountability Framework (HCAAF).8 We also 

reviewed GAO reports on workforce and succession planning to gain 

insights about key practices and how agencies have used them.

We conducted this performance audit from August 2013 to July 2014 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

 

8

Human Resources Management in Agencies, HCAAF and HCAAF Systems, Standards, 

and Metrics, 73 Fed. Reg. 23012 (Apr. 28, 2008). 
Appendix I: Objectives, Scope, and 

Methodology

Page 31 GAO-14-525 Office of Advocacy

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.
Appendix II: Comments from the Office of 

Advocacy, Small Business Administration

Page 32 GAO-14-525 Office of Advocacy

Appendix II: Comments from the Office of 

Advocacy, Small Business Administration
Appendix II: Comments from the Office of 

Advocacy, Small Business Administration

Page 33 GAO-14-525 Office of Advocacy
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 34 GAO-14-525 Office of Advocacy

As part of our review of the Small Business Administration Office of 

Advocacy’s (Advocacy) activities in identifying and communicating the 

concerns of small businesses and other small entities as agencies 

develop regulations and policies, we analyzed whether the Federal 

Advisory Committee Act (FACA), 5 U.S.C. App. II, applies to “roundtable” 

groups that Advocacy convenes for the purpose of conveying and 

soliciting information and viewpoints about agency rules and policies.

Based on our review of relevant statutes and legislative history, case law, 

regulations, and guidance, and consideration of Advocacy’s views, and 

for the reasons detailed below, we conclude that these roundtable groups 

are not “advisory committees” within the meaning of FACA and thus 

Advocacy is not required to comply with the provisions of that statute.1

Advocacy is an office established by statute within the Small Business 

Administration (SBA). 15 U.S.C. § 634a. The office has a separate 

statutory charter2 and, as of 2010, the President must include a separate 

line-item appropriation request for Advocacy designated in a separate 

Treasury account.3 Advocacy is headed by a Chief Counsel who, among 

other things, has statutory authority to appear as an official “friend of the 

court” (amicus curiae) in all court challenges to agency regulations in 

order to “present his or her views with respect to compliance with [the 

Regulatory Flexibility Act, discussed below], the adequacy of the 

rulemaking record with respect to small entities and the effect of the rule 

on small entities.” 5 U.S.C § 612(c). According to recent testimony, 

Advocacy is not required to obtain clearance of its comment letters, 

publications, and testimony as are other executive branch agencies.4

 

1

Our practice when rendering opinions is to contact the relevant agency and obtain its 

legal views on the subject of the request. GAO, Procedures and Practices for Legal 

Decisions and Opinions, GAO-06-1064SP (Washington, D.C.: Sept 2006), available at 

www.gao.gov/legal/resouces.html. We contacted the Chief Counsel for Advocacy who, 

through Advocacy’s Director of Interagency Affairs, provided us with the agency’s views. 

Letter from Advocacy to Managing Associate General Counsel, GAO, April 24, 2014 

(Advocacy Letter to GAO).

2

Pub. L. No. 94-305, Title II (1976), codified as amended at 15 U.S.C. §§ 634a et seq. 

3

Pub. L. No. 111-240 (2010).

4

Testimony of Dr. Winslow Sargeant, Chief Counsel for Advocacy, U.S. Small Business 

Administration, before the Senate Committee on Small Business and Entrepreneurship 

(Apr. 17, 2013).

Appendix III: Legal Analysis of the Applicability 

of the Federal Advisory Committee Act to the 

Office of Advocacy’s Roundtables

Introduction and Summary 

of Conclusions

Background on 

Advocacy’s Duties and 

Use of Roundtables
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 35 GAO-14-525 Office of Advocacy

Among Advocacy’s duties are serving as “a focal point for the receipt of 

complaints, criticisms, and suggestions” concerning federal policies 

affecting small businesses; developing proposals for changes in the 

policies and procedures of federal agencies that further small business 

interests and communicating those proposals to the appropriate federal 

agencies; and “represent[ing] the views and interests of small businesses 

before other Federal agencies whose policies and activities may affect 

small business.” 15 U.S.C. § 634c. Advocacy also has key responsibilities 

and authorities under the Regulatory Flexibility Act (RFA), 5 U.S.C. §§ 

601 et seq., and under Executive Order 13272 which helps implement the 

RFA. The RFA requires Advocacy to assist regulatory agencies during all 

stages of the rulemaking process to mitigate the potential adverse impact 

of the rules on small entities while achieving their regulatory objectives, 

and the Executive Order requires agencies to notify Advocacy of any 

proposed rules expected to have a significant economic impact on a 

substantial number of small entities and to consider any comments

Advocacy submits on the rule. The RFA and Executive Order both require 

agencies to respond to Advocacy comments in their final rules and note 

any resulting changes. Advocacy plays a similar role with respect to 

certain designated agencies as a member of what are known as 

“SBREFA panels.” The Small Business Regulatory Enforcement Fairness 

Act of 1996 (SBREFA)5 amended the RFA to require the Environmental 

Protection Agency, the Occupational Safety and Health Administration, 

and the Consumer Financial Protection Bureau to convene panels with 

officials from Advocacy, the Office of Management and Budget, and the 

rule-writing agency whenever the agencies develop rules for which an 

initial regulatory flexibility analysis is required. The panels must develop 

information solicited from small entity representatives and other sources 

concerning the potential impacts of the agency proposal; consider 

alternatives that minimize burdens on small entities; and prepare a report 

with recommendations to the agency head for consideration in the 

proposed rule. 5 U.S.C. § 609(b)-(e).

One of the ways that Advocacy fulfills these statutory responsibilities is by 

convening roundtable meetings for representatives of small businesses 

and government agencies. Advocacy invites roundtable participants, 

typically by email, but it is Advocacy’s policy to welcome participation by 

 

5

Pub. L. No.104-121, title II.
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 36 GAO-14-525 Office of Advocacy

any interested party other than the press.6 According to Advocacy, it 

hosted 32 roundtables in Fiscal Year 2012 and 21 roundtables in Fiscal 

Year 2013.7 The roundtables typically focus on specific proposed agency 

regulations and stakeholders provide information about the potential 

economic impacts of the proposal and identify less burdensome 

alternatives. Advocacy often uses the results of these roundtables (and 

other information) as the basis of formal comment letters it submits to the 

proposing agency, for example disagreeing with the agency’s impact 

analysis and urging less burdensome alternatives; it submitted 26 such 

letters in FY 20138 to which agencies must, by statute, respond. 5 U.S.C. 

§ 604(a)(3). In Advocacy’s view, the concerns articulated by small 

business representatives at roundtables are influential; for example, 

according to Advocacy, after a roundtable regarding the Employee 

Benefits Security Administration’s guidance on brokerage windows, the 

Administration issued revised guidance addressing the small business 

concerns. See Report on the Regulatory Flexibility Act, FY 2012, above, 

at 7. And after attending an Advocacy roundtable that addressed 

proposed revisions to Occupational Safety and Health Administration 

(“OSHA”) noise standards, OSHA withdrew the rule and pledged to 

conduct additional stakeholder outreach before proceeding. See Report 

on the Regulatory Flexibility Act, FY 2011 at 4.

FACA establishes requirements for “advisory committees” to the federal 

government, defined as relevant here as “any committee, . . . panel, task 

force, or other similar group . . . which is established or utilized by one or 

more agencies, in the interest of obtaining advice or recommendations for 

. . . one or more agencies or officers of the Federal Government . . ..” 5 

U.S.C. App. II, § 3(2). Congress enacted FACA in 1972 “to enhance the 

public accountability of advisory committees and to reduce wasteful 

expenditures” that “result only in worthless committee meetings and 

 

6

Advocacy roundtable invitations and agendas state in part: “Roundtable meetings are 

open to all interested persons, with the exception of the press, in order to facilitate an 

open and frank discussion about small business-related issues. Agendas and 

presentations are available to all, including the press. Anyone who would like to receive 

roundtables agendas or presentations, or be included in the regular distribution, should 

forward such requests to [Advocacy].” 

7

SBA Office of Advocacy, Report on the Regulatory Flexibility Act, FY 2012 at 4; Report 

on the Regulatory Flexibility Act, FY 2013, at i.

8

Report on the Regulatory Flexibility Act, FY 2013, supra, at i.

Applicability of the Federal 

Advisory Committee Act
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 37 GAO-14-525 Office of Advocacy

biased proposals”. Pub. Citizen v. U.S, Dep’t of Justice, 491 U.S. 440, 

459 (1989). Similarly, FACA’s purpose was to “ensure that new advisory 

committees be established only when essential [;] . . . that their creation, 

operation, and duration be subject to uniform standards and procedures; 

that . . . the public remain apprised of their existence, activities, and cost; 

and that their work be exclusively advisory in nature.” Id. at 446.

For advisory committees covered by the act, FACA requires, among other 

things, that their meetings be open to the public; that timely notice of 

meetings be published in the Federal Register; that interested parties be 

allowed to attend, appear before, or file statements with the advisory 

committee; and that detailed minutes of advisory committee meetings be 

kept. In addition, for advisory committees established by agency heads, 

the agency must, as applicable, ensure that committee membership is 

fairly balanced in terms of points of view represented and functions to be 

performed and ensure that the recommendations of the committee are 

not inappropriately influenced by the agency or any special interest but 

rather reflect the committee’s independent judgment. 5 U.S.C. App. II, § 

10. Because Advocacy does not apply these FACA requirements to its 

roundtables, we examined whether the statute applies to these groups 

and their activities.

1. Is Advocacy a FACA “Agency”? 

As noted above, whether Advocacy roundtables are advisory committees 

subject to FACA depends in part on whether they have been “established 

or utilized by one or more agencies . . ..” (emphasis added). Thus as a 

threshold issue, we examine whether Advocacy is an “agency” for 

purposes of FACA.

FACA defines “agency” by incorporating the term’s general definition in 

the Administrative Procedure Act (APA). 5 U.S.C. App. II, § 3(3). With 

exclusions not relevant here, the APA defines “agency” as “each authority 

of the Government of the United States, whether or not it is within or 

subject to review by another agency . . ..” 5 U.S.C. § 551(1).

Advocacy believes it is an “agency” for purposes of FACA. Citing the 

foregoing APA definition, Advocacy states that it is an “authority of the 

Government of the United States” and not covered by any of the 

exclusions. Advocacy Letter to GAO at 2.

Yet as the courts have recognized, “the law on the simple question of 

what is an [APA] agency is quite complex.” Lee Constr. Co. v. Federal 
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 38 GAO-14-525 Office of Advocacy

Reserve Bank, 558 F. Supp. 165, 172 (D. Md. 1982). Likewise, the courts 

have acknowledged that “[a]ny general definition [of “agency” under the 

APA] can be of only limited utility . . . [when] confronted with one of the 

myriad organizational arrangements for getting the business of the 

government done. . . .The unavoidable fact is that each new arrangement 

must be examined anew and in its own context.” Washington Research 

Project, Inc. v. HEW, 504 F.2d 238, 245-46 (D.C. Cir.1974), cert. denied, 

421 U.S. 963 (1975)(citations omitted).

Courts have taken somewhat different approaches to what constitutes an 

“agency” under the APA. The seminal case is Soucie v. David, 448 F.2d 

1067 (D.C. Cir. 1971), which involved what was then called the Executive 

Office of the President’s Office of Science and Technology (OST). The 

lawsuit sought the release of an OST document under the Freedom of 

Information Act (FOIA). Because the FOIA and APA definitions of 

“agency” were identical at the time of the lawsuit, the court looked to 

interpretations of the APA in determining whether OST was an “agency” 

under FOIA. The court found that while the meaning of “agency” under 

the APA was “not entirely clear,” it “apparently confers agency status on 

any administrative unit with substantial independent authority in the 

exercise of specific functions.” 448 F.2d at 1073 (emphasis added). 

Because OST evaluated federal science programs previously evaluated 

by the National Science Foundation, in addition to its non-”independent” 

responsibilities in advising and assisting the President, the court found 

OST had “substantial independent authority” and “must be regarded as 

an agency subject to the APA and [FOIA].” Id. at 1075.

The court in McKinney v. Caldera, 141 F. Supp. 2d 25 (D.D.C. 2001), 

applied the Soucie test and found that the office of the Judge Advocate 

General of the Army (JAG) was not an APA “agency” because it lacked 

“substantial independent authority.” Although JAG supervised the Army’s 

military justice system, performed appellate review of court martial trials, 

and furnished legal services, the court focused on the fact that JAG’s 

duties flowed from the authority of the Secretary of the Army. This 

rendered JAG insufficiently independent, the court ruled, notwithstanding 

that the APA contemplates the possibility of an “agency” within an 

“agency.” 

Other courts have framed the APA “agency” definition slightly differently, 

looking to whether an entity can take “final and binding action.” This 

standard is based in part on the Soucie “substantial independent 

authority” test but also on the APA’s legislative history indicating 

Congress intended covered agencies to have “final and binding” legal 
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 39 GAO-14-525 Office of Advocacy

authority. In Dong v. Smithsonian Institution, 125 F.3d 877 (D.C. Cir. 

1997), cert. denied, 524 U.S. 922 (1998), for example, the court looked to 

whether the Smithsonian Institution was:

“a part of government which is generally independent in the exercise of [its] functions and . 

. . by law has authority to take final and binding action affecting the rights and obligations 

of individuals, particularly by the characteristic procedures of rule-making and 

adjudication.” 

Id. at 881 (emphasis added). The court found that the Smithsonian was 

not an APA “agency” under this standard because while it exercised 

authority, the authority was not sufficiently “substantial.” Although the 

Smithsonian is closely linked with the federal government, receives 

federal funding, has certain police powers to protect its physical plant, 

and publishes regulations in the Code of Federal Regulations, the court 

noted that it does not make binding rules of general application, 

determine rights and duties through adjudications, issue orders, or 

perform regulatory functions.9

Still other courts determining APA agency status have focused on 

whether an entity is one of the “centers of gravity” in the exercise of 

administrative power “where substantial ‘powers to act’ . . .are vested’”

(see, e.g., Lee Constr. Co., above, 558 F. Supp. at 174, holding Federal 

Reserve Bank of Richmond was APA agency based on substantial 

delegated decision-making authority from Federal Reserve System Board 

of Governors), or whether the entity has “authority in law to make 

decisions” (see, e.g., Washington Research Project, Inc. v. HEW, above, 

504 F.2d at 248, holding that the Department of Health, Education, and 

Welfare (HEW) peer review groups for grant applications were not APA 

agencies because they only recommended grants to a National Institutes 

of Health entity, which, in turn, made recommendations to the HEW 

Secretary).

 

9

Dong involved whether the Smithsonian was an “agency” under the Privacy Act, which in 

turn adopts the definition of “agency” under FOIA. See 5 U.S.C. § 552a(a)(1). FOIA 

incorporated the APA “agency” definition until 1974, at which point Congress broadened 

the core APA “agency” definition for FOIA purposes, but the portion of the Dong court’s 

analysis cited here focuses on whether the Smithsonian was agency under the pre-1974 

version of the APA.
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 40 GAO-14-525 Office of Advocacy

Applying these authorities and considering Advocacy’s views, we 

conclude that Advocacy is an “agency” under the APA, and thus under 

FACA, because it possesses the requisite “substantial independent 

authority.” First, it acts with independence. Although Advocacy is an 

“agency” within an “agency” (SBA), as noted, it has distinct statutory 

authorities and responsibilities and a separate statutory charter; a 

separate line-item budget request; its head can appear in court appeals to 

speak to the rulemaking agency’s compliance with the RFA; and it does 

not have to obtain clearance of its comments and testimony. Unlike JAG 

in the McKinney case, above, which the court found was not an “agency” 

because its duties derived from the Army Secretary, the Chief Counsel’s 

duties are related to but distinct from the SBA Administrator.

Second, Advocacy’s authorities and duties under its organic statute, the 

RFA, and Executive Order 13272 are “substantial.” As noted, it is 

responsible among other things for identifying and advocating the views 

and concerns of small entities throughout the Executive Branch and for 

developing proposed changes in the policies and procedures of federal 

agencies in order to further the interests of small business. Although it 

does not issue regulations, conduct adjudications, or otherwise take 

traditional “final and binding action,” its views and recommendations 

must, by law, be accorded considerable weight. Taken as a whole, we 

believe it has substantial authority under the case law.

2. Are Advocacy’s Roundtable Groups FACA “Advisory Committees”? 

Having found that Advocacy is a FACA “agency,” we examine whether its 

roundtables are FACA “advisory committees.” As noted, FACA defines 

this term in part as “any committee, . . . panel, task force, or other similar 

group . . . which is established or utilized by one or more agencies, in the 

interest of obtaining advice or recommendations for . . . one or more 

agencies or officers of the Federal Government . . ..” 5 U.S.C. App. II, § 

3(2). Because of Advocacy’s particular statutory responsibilities, notably 

to identify and communicate concerns of small businesses and entities, 

Congress clearly contemplated that Advocacy would have a close 

relationship with small business. Its mission virtually requires consultation 

with these entities to obtain data, opinions and possible advice, and one 

way it has chosen to do this is by holding roundtable meetings.
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 41 GAO-14-525 Office of Advocacy

In Advocacy’s view, its roundtables are not FACA advisory committees. It 

believes the roundtables help it achieve its mission;10 indeed, Advocacy 

maintains that without the discussions that occur at roundtables, its 

“ability to fulfill its . . . statutory obligations would be severely curtailed.” 

Advocacy Letter to GAO at 2. Explaining its position that these activities 

do not trigger FACA, it states:

“[a] lthough the definition [in] FACA is broad, FACA does not apply to all committees, 

meetings, or groups. . . . In enacting FACA, Congress was concerned with formally 

organized advisory committees which the President or an executive department or official 

directed to make recommendations on identified governmental policy for which specific 

advice was sought. It was not intended to apply to all amorphous, ad hoc group meetings; 

only groups having some sort of established structure and defined purpose constitute 

advisory committees within the act.” 

Advocacy Letter to GAO at 3 (citations omitted).

We agree with Advocacy that under the case law, not every group that 

provides information or advice to a federal agency is a FACA “advisory 

committee.” Although no single aspect of a group’s creation, structure or 

activities is controlling, most courts have focused on the formality, 

structure, and continuity of the group and whether the group is brought 

together to provide advice or recommendations as group rather than as a 

collection of individuals. As the Supreme Court explained in Public 

 

10According to Advocacy: “Advocacy holds numerous roundtable discussions with small 

business stakeholders to educate them about ongoing legal and regulatory developments 

across the federal government and discuss their effect on small entities. Roundtable 

attendance varies from meeting to meeting, and staff encourages wide public participation 

by encouraging invitees to forward roundtable invitations to interested parties to attend or 

to call in if in-person attendance is not practicable. On many occasions, officials from 

federal agencies and Congress attend Advocacy roundtables to provide information 

regarding agency activities and approaches and to pose questions to small entities and 

their representatives directly. Roundtables on specific topics also provide a platform for 

small businesses to provide their views on specific practices, policies and regulations 

promulgated by various government agencies. Roundtables also serve to inform 

Advocacy staff of a wide variety of practical and economic challenges small businesses 

might be facing as a result of federal policies. Advocacy’s roundtables are particularly 

useful for the office because they allow staff to witness and foster dialogue among various 

stakeholders, including small businesses, their representatives, policymakers, and public 

interest groups. These conversations often shed light on what are usually complex 

questions that deserve thorough consideration, on often very short notice, by Advocacy 

staff in their own evaluation of the impact of regulations on small entities.” 

Advocacy Letter to GAO at 1-2.
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 42 GAO-14-525 Office of Advocacy

Citizen, above, Congress did not design FACA to cover every formal or 

informal consultation between a government agency and a group 

rendering advice; such a literalistic reading would cover “far more groups 

and arrangements than Congress could conceivably have intended.” 491 

U.S. at 464-65.

Several early cases addressed whether an informal or “ad hoc” group 

such as Advocacy’s roundtables was an advisory committee. The court in 

Food Chemical News v. Davis, 378 F. Supp. 1048 (D.D.C. 1974), 

considered whether two “informal” meetings of distilled spirits industry 

representatives, convened by the head of the Bureau of Alcohol, Tobacco 

and Firearms (BATF) to obtain advice on draft regulatory amendments, 

were subject to FACA. Noting that the act’s purpose was to control the 

advisory committee process and to open to public scrutiny the manner in 

which government agencies obtain advice from private individuals, the 

court held it was “undisputed that [BATF] utilized an ad hoc committee of 

industry representatives in order to obtain advice. Such a relationship . . . 

clearly comes within the terms of . . . [FACA].” Id. at 1051.

The following year, the same court considered whether bi-weekly 

meetings held at the White House and attended by selected groups 

invited by the President’s Assistant for Public Liaison were subject to 

FACA. In Nader v. Baroody, 396 F.Supp. 1231 (D.D.C. 1975), the court 

concluded that the groups were not advisory committees, relying on two 

characteristics of the meetings: the absence of a specific request for 

advice on a matter of government policy and the lack of formal 

organization or continuity.

The Second Circuit Court of Appeals considered Food Chemical News 

and Nader in National Nutritional Foods Association v. Califano, 603 F. 2d 

327 (2d Cir. 1979). The Califano court determined that FACA applied to a 

single meeting between six officials from the Food and Drug 

Administration and six obesity experts “to assist the FDA in selecting the 

best course of action ‘for regulating the production and promotion of 

[weight reduction products]. . . .’” Agreeing with the rulings of both 

previous cases, the court found the facts of Food Chemical News to be 

closer to the case before it because FDA had called on specific 

individuals to provide group advice on a particular proposed regulation 

and relied on the group advice.

The element of “group advice” continued to play a key role in the seminal 

case of Association of Am. Physicians and Surgeons, Inc. v. Clinton, 997 

F. 2d 898 (D.C. Cir. 1993) (AAPS), involving the Clinton Administration’s 
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 43 GAO-14-525 Office of Advocacy

Health Care Reform Task Force. The D.C. Circuit Court of Appeals held 

in AAPS that “a group is a FACA advisory committee when it is asked to 

render advice or recommendations, as a group, and not as a collection of 

individuals.” Id. at 913 (emphasis in original). “[A]n important factor in 

determining the presence of an advisory committee [is] the formality and 

structure of the group,” the court held, laying out three criteria needed to 

implicate FACA: “an organized structure, a fixed membership, and a 

specific purpose.” Id. at 914. Characterizing variation in these factors as a 

continuum, the court explained that at one end of the continuum is a 

“formal group of a limited number of private citizens who are brought 

together to give publicized advice as a group”—this group is covered—

while at the other end of the continuum is “an unstructured arrangement 

in which the government seeks advice from what is only a collection of 

individuals who do not significantly interact with one another.” Id. at 915.11

See also Freedom Watch v. Obama, 807 F. Supp. 2d 28 (D.D.C. 2011) 

(applying AAPS criteria; finding colorable claim of FACA coverage based 

on fixed group membership and formation for specific policy purpose); 

Citizens for Responsibility and Ethics in Washington v. Leavitt, 577 F. 

Supp. 2d 427, 432 (D.D.C. 2008) (same; finding no FACA coverage 

where “[t]he groups had no formal organizational structure . . . . Attendees 

conveyed their own opinions regarding their individual areas of expertise” 

and “[n]o group report or other collaborative work product was created.”).

But see Aluminum Co. of America v. National Marine Fisheries Service, 

92 F. 3d 902 (6th Cir. 1996) (cautioning against rigid application of AAPS 

criteria).

Implementing regulations issued by the General Services Administration 

(GSA), the agency responsible for administering FACA government-wide, 

rely in part on this case law in defining what constitutes a FACA advisory 

committee. In issuing its final FACA management rule in 2001, GSA 

“recognize[d] that the broad definition in the Act of an ‘advisory 

committee’ might be interpreted to extend coverage by the Act to any 

gathering of two or more persons from whom . . . Federal officers or 

agencies seek advice or recommendations. However, in the cases 

discussed above, the courts have rejected such a broad reading . . . .” 66 

Fed. Reg. 37728, 37730 (July 19, 2001). GSA’s regulations therefore 

exempt from FACA regulation “[a]ny group that meets with Federal 

 

11The AAPS court remanded the case to determine whether a particular working group of 

President Clinton’s Health Care Reform Task Force was a FACA advisory committee 

under these criteria.
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 44 GAO-14-525 Office of Advocacy

official(s), including a public meeting, where advice is sought from the 

attendees on an individual basis and not from the group as a whole. . . .” 

41 C.F.R. § 102-3.40(e). GSA regulations further state that development 

of consensus among meeting participants does not in itself convert the 

group to a FACA committee, nor does the presence of trade association 

or other organization representatives affect whether advice is obtained 

“on an individual basis.” 41 C.F.R. Parts 102-3, Subpart A, App. A, Key 

Points and Principles II, III.

Applying these authorities and considering Advocacy’s views, we 

conclude that Advocacy’s roundtables are not FACA advisory 

committees. First, we recognize that roundtable participants are not 

random attendees but rather invited representatives of a particular 

“group”—the small business community—whose concerns and 

perspective it is Advocacy’s very mission to identify and advocate.12 As 

noted, however, Advocacy invites anyone (except the press) with an 

expressed desire to attend and thus there is no structured or fixed 

membership in a “roundtable group.” 

Nor is group consensus sought from roundtable participants. Under the 

standards in AAPS, 603 F.2d at 913, the group’s activities are not 

expected to, and do not appear to, benefit from the interaction among the 

participants. Rather, as contemplated by the GSA regulations, information 

is solicited from participants as individuals (including individuals 

representing trade associations), although a consensus may develop.

See 41 C.F.R. Part 102-3, Subpart A, App. A, Key Point II (“The 

development of consensus among all or some of the attendees at a public 

or similar forum does not automatically invoke FACA.”). As in Leavitt, 

above, no group report or other collaborative work product is created. And 

as in American Society of Dermatology v. Shalala, 962 F. Supp. 141 

(D.D.C 1996), there is no attempt to create consensus among participants 

either overall or on any aspects of the issue under consideration. See id. 

at 148 (among the “main factors leading to the Court’s determination that 

the panels fall into the non-advisory committee end of the [AAPS] 

continuum [is] . . . the absence of any attempt to achieve consensus.”).

 

12As summarized in recent testimony, Advocacy was “created . . . to be a voice for small 

business within the federal government. . . [and] advances the views, concerns, and 

interests of small business. . ..” Testimony of Dr. Winslow Sargeant, Chief Counsel for 

Advocacy, U.S. Small Business Administration, before the House Committee on Small 

Business, Subcommittee on Investigations, Oversight and Regulations (Mar. 14, 2013).
Appendix III: Legal Analysis of the 

Applicability of the Federal Advisory 

Committee Act to the Office of Advocacy’s 

Roundtables

Page 45 GAO-14-525 Office of Advocacy

Nor does Advocacy’s submission of comment letters based on 

information obtained at roundtables establish that it convened the 

roundtable to obtain a group consensus. While Advocacy’s position in a 

comment letter may represent a consensus of views expressed at a 

roundtable, that consensus appears to be a function of the fact that many 

of the participants, as small entities, share common concerns.

Finally, according to Advocacy, it only obtains data and information from 

roundtable participants, not policy advice or recommendations. It is 

Advocacy itself that develops policy recommendations, it told us, based 

on information it obtains at roundtables and elsewhere and using its own 

judgment and expertise. Even if advice and recommendations were 

offered at roundtables, however (a small business representative 

recommending a regulatory policy change based on its impact on small 

business, for example), in our view, this would not outweigh the other 

factors indicating that the roundtables are not covered advisory 

committees.13

Advocacy is an “agency” under FACA first because it meets the statutory 

definition of this term in FACA and the Administrative Procedure Act 

incorporated by reference: it is an “authority of the Government of the 

United States, whether or not it is within or subject to review by another 

agency” and not subject to statutory exemption. In addition, Advocacy has 

the type of “substantial independent authority” and other indicia of 

“agency” status that courts interpreting the APA definition have required.

However, Advocacy’s roundtable groups are not FACA “advisory 

committees” because they do not have an organized or fixed structure; 

there is no attempt to reach consensus about an agency regulation or 

policy but instead viewpoints are sought from individual participants; and 

it is only data and information, according to Advocacy, rather than advice 

or policy recommendations, that participants provide.

 

13Because we conclude that Advocacy’s roundtables are not FACA advisory committees 

for the reasons discussed here, we did not determine whether the additional FACA 

element was met of the roundtables being “established or utilized by” Advocacy. 5 U.S.C. 

App. II, § 3(2). 

Conclusions
Appendix IV: GAO Contact and Staff 

Acknowledgments

Page 46 GAO-14-525 Office of Advocacy

Cindy Brown Barnes (202-512-8678), brownbarnesc@gao.gov. 

In addition to the contact named above, Kay Kuhlman and Triana McNeil 

(Assistant Directors), Kristeen McLain (Analyst-in-Charge), Emily 

Chalmers, William Chatlos, John McGrail, Lauren Nunnally, Roberto 

Pinero, Jena Sinkfield, Jack Wang, and Carrie Watkins made key 

contributions to this report.

Appendix IV: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(250738)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.