Challenges in Collecting and Reporting Race and 

Ethnicity Data in Establishments: 

Examples from Two Federal Surveys

FCSM Any views expressed are those of the authors 

October 26, 2023 and not those of SAMHSA, the US Census Bureau or 

any contractors

Herman A. Alvarado and Kathryn Downey Piscopo

Substance Abuse and Mental Health Services Administration (SAMHSA)

Amy Anderson Riemer and Krysten Mesner

United States Census Bureau
• Introduction: Hybrid Establishment Response 

Process

• SAMHSA: National Substance Use and Mental 

Health Services Survey (N-SUMHSS)

• Census: Survey of Sexual Victimization (SSV)

• Lessons Learned from Both Federal Surveys

2

Agenda
• How does a revised race and 

ethnicity question impact 

establishment data collection and 

reporting?

3

Purpose
1. Encoding/Record Formation

– How is the information collected? (paper, electronic)

2. Respondent Selection/Identification

– Who is reporting: self or other (proxy)?

3. Assessment of Priorities

– Are the collection and reporting required for business 

operations?

4

Hybrid Establishment Response Process (Willimack & Nichols, 2010)
4. Comprehension of the Data Request

– Are questions understood as intended?

– Does the establishment have the requested data?

5. Retrieval of Data

– Where is the information stored? (within premises, 

separate location, cloud?) 

– Who has access to the information? (full or limited?)

– Who is required/allowed to report the information?

5

Hybrid Establishment Response Process (cont.)
6. Judging the Adequacy of the Response

– Is there any missing information?

– How do we handle “Other” and “Unknown” 

responses?

7. Reporting the Response

– Are there any reporting guidelines?

8. Release of the Data

– Is the data submitted within timeframe?

6

Hybrid Establishment Response Process (cont.)
SAMHSA

The National Substance Use and Mental Health 

Services Survey (N-SUMHSS)

7
• Two different data collections/universes:

– Mental health treatment facilities

– State representatives (behavioral health state agencies)

• Initial contacts: cold calling or emails; used internet 

searches

• Interview phase: virtual meetings and telephone calls; 

structured cognitive interview for 30-60 minutes

• Total: 18 interviews (4 in Spanish)

– 11 state representatives

– 7 mental health facilities

8

Methodology for Study
• Started in 2021; prior was two separate SAMHSA surveys

– National Survey of Substance Abuse Treatment Services (N-SSATS)

– National Mental Health Services Survey (N-MHSS)

• Universe: Facilities providing treatment services

• Collects data on treatment modalities, payment options, 

age groups accepted, special programs, medical services, 

clients receiving treatment, and other relevant information

• Information from the N-SUMHSS feeds into SAMHSA’s 

Online Treatment Locator (https://findtreatment.gov/)

9

National Substance Use and Mental Health Services Survey 

(N-SUMHSS)
2021 results for inpatient

(Table MH52b): 

• 38.7% Race unknown 

or not collected;

• 46.7% Ethnicity

unknown 

or not collected

10

National Substance Use and Mental Health Services Survey 

(N-SUMHSS): Race and Ethnicity Items

https://www.samhsa.gov/data/sites

/default/files/reports/rpt39450/202

1%20NSUMHSS%20Annual%20Detailed%2

0Tables_508_Compliant_2_8_2023.

pdf
Willimack and Nichols (2010)

1. Encoding / record formation 

2. Respondent selection / identification 

3. Assessment of priorities 

4. Comprehension of the data request 

5. Retrieval of data 

6. Judging the adequacy of the response 

7. Reporting the response 

8. Release of the data

11

Hybrid Establishment Response Process (Willimack & Nichols, 2010)
• R/E is collected at intake and entered in Electronic Health 

Record (EHR)

• R/E is not required of clients, sometimes leave this blank or 

code as “unknown” or “not collected” 

• Some facilities will follow-up on other demographics (such as 

gender) but not R/E

• Also training-related issues when giving clients the 

demographic forms

12

Findings: Encoding/Record Formation
• Facilities may allow a proxy (e.g., relative) 

• Medical staff may also use other medical charts (“collateral 

information”)

13

Findings: Respondent Selection/Identification
• May be of low priority for facilities and there may be issues 

with satisficing

• One manager said it was “not a good use of our time”

14

Findings: Assessment of Priorities
Select all that apply 

• Mixed approach to reporting multiple races by facilities and 

states

• Pick one race

• “2+” category

• Programmed to not allow for more than one race

• If multiple subcategories/examples within a race were 

checked, would it be classified as multiracial 

• Possible combinations to program in a database would be 

daunting

15

Findings: Comprehension of the Data Request
Differences in terminology

• One did not have a “Native Hawaiian or Other Pacific 

Islander” category and termed “White” as “Caucasian”

• Sometimes clients want examples for R/E, particularly if 

they aren’t English-speaking

• Had significant issues with the examples/subcategories for 

the detailed item

• DEIA (diversity, equity, inclusion, and accessibility) experts 

at the hospital system level provide input on terminology 

used

Use of “Other”

• “Other” is not included in the proposed R/E items (and is 

also not in the 1997 Directive), but it is used by facilities

16 • Use an “other” category when clients check more than one race

Findings: Comprehension of the Data Request (cont’d)
Order of racial categories

• Should be some flexibility in the order of the categories. 

Spanish translation

• One facility pointed out the use of “y/o” (“and/or”) in Spanish 

is incorrect

• Spanish is not parallel to the English version

17

Findings: Comprehension of the Data Request (cont’d)
• Many facilities mentioned that they would try to follow the 

state’s data needs

• For federal reporting, they would comply if given guidance

18

Findings: Reporting the Response
U.S. Census Bureau

Survey of Sexual Victimization (SSV)

19
• Prison Rape Elimination Act (PREA) – Passed by Congress in 2003

• Sponsored by the Bureau of Justice Statistics (BJS) 

• Annual collection on substantiated incidents of sexual abuse and 

harassment from adult correctional and juvenile justice facilities

• PREA Coordinators are the proxy respondents

• Incidents could involve:

– Adult inmates (victim or perpetrator)

– Juveniles (victim or perpetrator)

– Staff perpetrators

• R/E collected about all victims and perpetrators

20

Survey of Sexual Victimization (SSV)
Current SSV 

Race/Ethnicity 

Question
• Evaluation of the revised R/E questions was a subset of 

existing testing happening for the SSV

• Moderated Cognitive Interviews (n=15)

– Virtual meetings

– Concurrent probing

• Unmoderated Cognitive Interviews (n=159)

– 'Web probing’

– Online questionnaire w/subset of questions

– Closed and open-ended probes 

22

Methodology
Willimack and Nichols (2010)

1. Encoding / record formation 

2. Respondent selection / identification 

3. Assessment of priorities 

4. Comprehension of the data request 

5. Retrieval of data 

6. Judging the adequacy of the response 

7. Reporting the response 

8. Release of the data

23

Hybrid Establishment Response Process (Willimack & Nichols, 2010)
• Respondents rely entirely on administrative records

• Respondents worked with 2 different record systems: 

– Inmate/Juvenile

– Staff (often managed by Human Resources)

• Respondents didn’t always have access to the systems that 

contained information about staff perpetrators

– Must work with Human Resources for this information

• Source of inmate/juvenile R/E

– Databases/Systems

– Investigative reports

24

Findings: Encoding/Record Formation
• Systems often do not allow for the collection of 

multiple race categories (inmate, juvenile, or 

staff)

• Middle Eastern and Northern African (MENA) 

categories are not part of the existing record 

structure (inmate, juvenile, or staff)

• Some facilities have the option of a write-in 

within records and could potentially extrapolate 

MENA from this information

25

Findings: Encoding/Record Formation (cont.)
• R/E collected: 

– Self-reported during intake

– Gathered from investigative reports 

• Self-reported or from observation from the 

detective/officer

26

Findings: Respondent Selection/Identification
• Concerns with the order of categories 

• Categories for the minimal version match well 

with the categories in their systems

– Respondents would not be able to provide the 

level of detail requested on the detailed question

27

Findings: Comprehension of the data request 
• Due to protocol or access, respondents cannot 

speak directly with the inmate, juvenile or 

staff member to request R/E data or confirm 

what is within records

28

Findings: Retrieval of Data
• Many respondents would be open to updating 

their record systems to be closer to the 

requested OMB categories in the future

29

Findings: Reporting the Response
• These issues are in addition to any issues related to 

self-reporting issues found on the household testing

• Try to test with many types of establishments

• Although respondents want to provide quality data, 

they are limited by their records

• Respondents are interested in updating their systems 

30

Lessons Learned
• Minimum

• It may take extra effort or time for systems to update 

• Issues with reporting 2 or more R/E 

• Detailed 

• Records don’t have this level of detail

• Issues with reporting 2 or more R/E 

• Issues with text boxes and mark all that apply

• Establishments have competing priorities and may not find 

the value of adapting their collection for these revised 

questions

31

Lessons Learned (cont.)
Challenges in Collecting and Reporting Race and Ethnicity Data in 

Establishments: Examples from Two Federal Surveys

32

Thank You

Herman.Alvarado@samhsa.hhs.gov

Kathryn.Piscopo@samhsa.hhs.gov

Krysten.Mesner@census.gov

Amy.E.Anderson.Riemer@census.gov