Assessing the Health of the 

Federal Statistical Agencies: 

Vision, Purpose, and Challenges

FCSM Conference

Nancy Potok

October 2023

1
US Federal Data Ecosystem

• Federal Data Strategy

• Evidence Based Policy Act/CHIPS Act

• Administrative Policies and Guidance to Agencies

• OMB Trust Regulation

• Infrastructure

• Tools

• People (workforce, stakeholders)

2
Statutes - Evidence Act of 2018/CHIPS

3

Foundations for Evidence-Based Policy Making Act 

of 2018 includes recommendations from the U.S. 

Commission on Evidence Based Policy Making 

• Contains three titles:

I. Federal Evidence Building Activities

II. Open Government Data Act

III. Confidential Information Protection and Statistical

Efficiency Act . 

CHIPS and Science Act of 2022 (CHIPS Act) authorizes 

NSF-NCSES to establish a pilot project for a National Secure 

Data Service
Assessing U.S. 

Data 

Infrastructure:

2024 onwards

Reactive 

Monitoring 

● Identify/shine light on 

needs, weaknesses, and 

vulnerabilities

● Address before problems 

arise

● Help agencies make their 

cases

Proactive 

Monitoring

4

● Challenging to reverse 

threats underway

● Damage may be done ● Reaction may be perceived 

as partisan
Proactive Monitoring

5

Success hinges on 

engagement of and 

participation of federal 

statistical community 

of stakeholders

Publish report in 2024 

Q1 and annually 

thereafter, expanding 

coverage to more 

agencies

Promote additional 

conversation and 

study of federal 

statistical agencies
Defining the “Health” of Any 

Agency

6

Str

Workforce

Funding

Culture

Processes 

and 

Structure

Collaboration

Leadership 

Strategic 

Goals
High Level Stat Agency Health

7

Str

Timeliness

Relevance

Appropriateness

Useability

Granularity 

Periodicity 

Public Trust
Can Assessment Dive Deeper Quantitatively?

Autonomy Protections Resources

Innovation Host agency support

Workforce Data Usage

8
Deeper Dive Potential indicators

9

Resources (e.g., budget levels, staffing, contracting)

Professional autonomy (e.g., safeguards against political meddling)

Innovation/modernization capabilities (e.g., new products and data sources, pilot projects)

Workforce (e.g., employee satisfaction, professional development, recruitment and retention, diversity)

Host agency support (e.g., budget requests, data governance processes)

Data use and user engagement (e.g., downloads, citations, data linkages)
Gathering Indicator Data for Analysis

10

Collection sources

- Publicly available data: budgets, 

publication schedules, OPM 

employee satisfaction, etc. 

- Agency questionnaire, possible data 

user survey

Analysis plans

- Conduct qualitative and quantitative 

analyses to ensure data and expert 

accounts agree

- Present findings in public report 
Challenges We Have 

Encountered With a 

Deeper Dive

11
What does “Health” Mean?

• No standard definition or vision that is specific enough

• Most principles are high level but don’t define such that a metric is 

obvious (e.g. how much autonomy is enough? What does that look 

like for each agency?)

12
Sensitive Areas

Relationship between statistical agency and parent agency

• Not a lot of transparency – discussions are not public

• Many interactions (budgets, IT plans, staffing levels, Secretarial priorities)

• Need cooperation to be successful

• Varying viewpoints on what is “helpful”

• OMB Trust Regulation not yet in place, thus effectiveness not yet evaluated

13
Major Differences Between Agencies

• Example: Cannot directly compare Statistics of Income at IRS, Bureau of 

Transportation Statistics at DOT, and Census Bureau at Commerce (size, control 

of resources)

• Widely varying statutory authorities that affect autonomy, agenda setting, and 

placement in agency

• Different appointment authorities for Directors (Presidential, political, career)

14
Budgets Can Be Difficult to Decipher

• A flat budget may not be a healthy budget if inflation is high

• No insight available on what an agency requested before its request went 

through the budget process of parent agency and OMB review

• No insight into parent agency instructions to agencies even before requests go 

forward (e.g., all increases must have equal offsets)

• Congress may add work without adding funds during the appropriations 

process

• Larger agencies may be better able to absorb cuts or level funding

15
Collaboration is Not Centrally Collected 

Information

• Cross-agency collaborations may not be noted anywhere in publicly available 

information

• Important to measure collaborations that lead to innovations and evidencebuilding

• Can that information be found without surveying agencies directly?

16
Agencies Track Activities Differently

• Some information requires use of proxies, such as training and travel

• Some training is in the travel budget but some training is virtual or local

• Agencies track these activities differently

• Travel restrictions during the pandemic created anomalies in trend data

17
A Majority of the Agencies Have Limited 

Usage Data

• Most of the agencies can only track website hits or downloads

• Only four agencies currently can track systematically how data are actually 

being used by researchers in scientific journal publications

• Need usage data beyond journal publications (e.g., conference papers, policy 

documents, reports)

• Need to capture agency reports usage but no common nomenclature being 

used for report citation (e.g., DOIs) and users don’t have incentives for 

including citations

18
Attempts to Survey Agencies Directly had 

Varying Results

• Some agencies did not have enough staff available to answer questions

• Questions were interpreted differently by each agency

• May be difficult to collect information annually

• Different agencies have different clearance processes through their parent 

agencies, so information was missing in several instances if not already public

19
So What Were We Able to Collect?

• Public budget data

• Timeliness of data releases

• Some resources data

• Inconsistent and low quality FEVS data

• Inconsistent data on professional development and innovation

• Inconsistent data on usage

Here are some examples to date... (Connie and Jonathan)

20