Assessing the Health of the 

Federal Statistical Agencies

FCSM, October 26, 2023

Connie Citro

Some

Challenges 

Made Visible
2

Measuring “Health”—Indicator Definition Challenges

 We want to measure the “health” of federal statistical agencies: broadly, I 

suggest it is their ability to provide relevant, timely, frequent, granular, 

accurate, objective, and accessible data for the public and policymakers

 Desirable properties of indicators: track a construct related to agency 

health; not incentivize “teaching to the test;” be measurable with available 

or feasible-to-collect data; be consistently measured across time and 

agencies; be intuitive to the public

 Six (potential) areas for defining indicators: autonomy, host agency 

support, resources, workforce, innovation, and data use and usability—first 

five areas enable the sixth
3

Measuring “Health”—Data Collection Challenges

 Fewer data are publicly available and readily accessible than we thought

 Developed a survey of the principal statistical agencies, but framing good 

survey questions is hard (!), and there is unit (agency) and item 

nonresponse

 Agency websites differ greatly, so not easy to find comparable information 

(e.g., end of reference period for surveys for use in developing a timeliness indicator)

 We need a survey of data users, which we are endeavoring to scope and 

obtain funding to carry out
4

Example #1—Innovation 

 Innovations by statistical agencies can and need to occur in concepts and 

topics, data collection, data processing and estimation, data dissemination, 

and data evaluation

 We asked agencies about new products, new data sources, pilot projects, 

recent innovations, planned innovations (if $ permit), et al.

 Agencies gave us many interesting and important examples, but we are not 

sure how to turn their responses into indicators

 Also, when is an “innovation” innovative enough?
5

Example #1—Innovation (cont’d)

AGENCY/

TOTAL

NEW 

PRODUCTS

NEW DATA 

SOURCES

PILOT 

PROJECTS

RECENT 

INNOVATIONS

PLANNED 

INNOVATIONS

BEA 23 4 5 4 2

BJS N.A. N.A. N.A. N.A. N.A.

BLS 36 4 1 9 1

BTS N.A. N.A. N.A. N.A. N.A.

Census 17+ N.A. 2+ 3 N.A.

NASS 3 3 0 3 3

NCHS 19 0 3 2 0

NCES 3 2 2 3 4

NCSES 0 0 5 5 1

NOTES: Responses summed over 5 years (new products and sources), 3 years (recent), or current (pilot and planned); 

+ = agency gave examples and said there were many more; N.A. = Not available (BTS did not answer this section; BJS responded 

but not in a usable format).
6

Example #1—Innovation (cont’d)

Type Historical Recent (past 5 years per agency survey responses)

Concepts/Topics

^NIPAs

^SPM

^Consumer unit consumption (different from expenditures) (BLS)

^Global value chains (BEA, NCSES)

Data Collection

^Probability sampling 

^CASM

^CAPI

^Publicly available data sets from theme parks for experimental price 

calculations (BLS)

^Core Logic Commercial Land Parcel data (NASS)

^National Hospital Care Survey data linked to 2016 and 2017 National 

Death Index (NCHS)

Data Processing 

and Estimation

^Hot deck imputation

^X-11,-12,-13-ARIMA (times 

series seasonal adjustment)

^Household distribution of disposable personal income (revival of series 

discontinued in 1970s) (BEA)

^Official GDP for Puerto Rico (BEA)

^Index of Economic Activity (experimental; monthly; updated daily; 

weighted average of 15 business indicators) (Census Bureau)

^Weekly Business Formation Statistics for states (Census Bureau)

^Adult literacy estimates for states and counties by age and gender using 

survey data with small-area estimation techniques (NCES)

Data Dissemination

^PUMS

^FSRDCs

^Spanish-language Puerto Rico website (BEA)

^Crop Condition and Soil Moisture Analytics Tool (Crop-CASMA) (Webbased mapping tool with NASA imaging data) (NASS)

Data Evaluation

^Dual-system estimation for 

coverage error
7

Example #2—Data Use and Usability 

 Asked agencies about downloads/page views; citations in journals, news 

media, congressional documents; user conferences/sessions with users at 

other conferences; changes made in response to user feedback

 Collected data on timeliness (months/years from end of reference period for 

data collection to first publication)

 Here is where we clearly need a comprehensive user survey
8

Example #2—Data Use and Usability (cont’d)

AGENCY/TOTAL

USER 

CONFERENCES

SESSIONS 

WITH USERS

PROGRAM CHANGES IN 

RESPONSE TO USER SESSIONS

BEA 60+ (conferences and sessions) 

2+ (outdoor recreation economy; county GDP—

users accepted suppression in return for more 

industry detail)

BJS 16 24 N.A.

BLS 10 95+

5 (timing of news releases; task list categories for 

Occupational Requirements Survey)

BTS N.A. N.A. N.A.

Census 2+ 110 2+ (improvements to data.census.gov)

NASS 6 11 N.A.

NCES N.A. N.A. N.A.

NCHS N.A. N.A. N.A.

NCSES 27 10

7 (added questions on CHIPS funding for R&D to 

Survey of Federal Funds for R&D)

NOTES: Responses summed over 3 years (conferences and sessions); + = agency gave examples and said there were

many more; N.A. = Not available (item nonresponse).
9

Example #2—Data Use and Usability (cont’d)

DOWNLOADS 2020 2021 2022

BEA 6.3 5.2 5.6

BJS 1.1 0.4 0.3

BLS 15.1 11.6 23.1

Census 7.6 6.6 6.7

NASS 172.2 219.3 209.8

NCHS 4.0 2.1 1.7

NCSES 0.1 0.2 0.3

PAGE VIEWS 2020 2021 2022

BEA 17.6 14.2 22.7

BJS 9.3 4.1 5.2

BLS 138.3 147.1 170.5

Census 456.8 209.3 167.3

NASS N.A. N.A. N.A.

NCHS 105.9 56.5 41.4

NCSES 2.2 2.4 2.4

NOTE: Numbers rounded to millions

.