RTI International is a registered trademark and a trade name of Research Triangle Institute. www.rti.org

Reflecting a Diverse America: Qualitative Testing of 

Changes to Race and Ethnicity Collection in the 

United States 

FCSM

October 25, 2023

Victoria Dounoucos (RTI International), Y. Patrick Hsieh (RTI International), Alisu SchouaGlusberg (RSS), Aleia Clark-Fobia (U.S. Census Bureau), Jennifer Childs (U.S. Census Bureau) 
Disclosure Review

 This report is released to inform interested parties of research and to encourage 

discussion. The views expressed are those of the authors and not those of the U.S. 

Census Bureau. The U.S. Census Bureau reviewed this data product for unauthorized 

disclosure of confidential information and approved the disclosure avoidance practices 

applied to this release. 

2
Project Overview

 RTI International and Research Support Services (RSS), under the guidance of the 

U.S. Census Bureau, conducted approximately 100 cognitive interviews in English and 

Spanish to evaluate the proposed revisions to the race and ethnicity questions. 

 Cognitive interviews explored: 

– Updates to race and ethnicity question format

– Terminology and wording of questions

– Instructions for respondents and associated guidance 

 The primary goals of cognitive testing were to: 

– Understand how the combined race and ethnicity question format affected response 

distributions, respondent interpretation, self-identification, and understanding 

– Assess whether changes to the question stem and simplification of instructions affected 

respondent understanding and response. 

3
Project Overview: Version A of Race/Ethnicity Combined Question

4
Project Overview: Version B of Race/Ethnicity Combined Question

5
Recruitment Overview 

 Data collection period: April 12 to June 26

 Recruitment was conducted nationally. Some advertisements were targeted at large, diverse 

metropolitan regions, but participants responded from across the nation.

 Participant Recruitment Strategies (1) Online advertisement:

– A general description classified advertisement posted under Craigslist’s “et cetera” job section for the 

following metropolitan areas. But recruits came from anywhere in the U.S. 

– Facebook groups search (did not receive assistance and did not yield meaningful outcomes)

 Participant Recruitment Strategies (2) Community based strategies:

– Community organizations that our team has collaborated with in the past

– Word of mouth referral from staff’s professional and personal networks

6

 Oahu and Maui, HI

 Anchorage and Fairbanks, AK

 Los Angeles and San Francisco, CA

 Denver, CO

 Phoenix, AZ

 Minneapolis/St. Paul, MN

 Boston, MA

 New York, NY

 Washington DC
Recruitment Overview

 Main eligibility screening questions:

– Place of birth

– Race and ethnicity

– Self description of ancestry or ethnic origins (open-ended text question)

 Screening process:

– Review IP addresses to exclude massive repeated entries

– Mark to exclude entries without contact information

– Review to identify the notable self description of ancestry or ethnic origins for recruitment targets

– Code the detailed race and ethnicity categories

– Review IP addresses for the coded cases to exclude scammers from click farms (instituted in May)

– Geographic location of recruits is NOT part of the screening, and we interviewed participants living outside or the 

metro areas we advertised

7
Recruitment Overview

 Target population: General adult population in the U.S.

8

Minimum Category Detailed and Additional Categories Min. # Final #

American Indian or Alaska 

Native

Navajo Nation, Blackfeet Tribe, Mayan, Aztec, Native Village of Barrow 

Inupiat Traditional Government, and Tlingit 5 3

Original peoples of North, Central, and South American not included above 5 6

Asian Chinese, Filipino, Asian Indian, Vietnamese, Korean, and Japanese 5 8

Additional nationalities or ethnic groups originating in East Asia, Southeast 

Asia, or South Asia and not included above. Examples of these groups 

include, but are not limited to, Pakistani, Cambodian, Hmong, Thai, Bengali, 

and Mien etc. 

5 5

Black or African American African American, Jamaican, Haitian, Nigerian, Ethiopian, and Somali 5 14

Additional nationalities or ethnic groups originating in any of the Black racial 

groups of Africa. Examples of these groups include, but are not limited to, 

Ghanaian, South African, Barbadian, Kenyan, Liberian, Bahamian, etc.

5 7
Recruitment Overview

9

Minimum Category Detailed and Additional Categories Min. # Final #

Hispanic or Latino Mexican or Mexican American, Puerto Rican, Cuban, Salvadoran, Dominican, 

and Colombian

2 Eng.

10 Esp.

8 Eng.

10 Esp.

Additional nationalities or ethnic groups originating in Central and South 

America, and other Spanish cultures.

Examples of these groups include, but are not limited to, Guatemalan, 

Honduran, Spaniard, Ecuadorian, Peruvian, Venezuelan, etc.

3 Eng.

10 Esp.

8 Eng.

10 Esp.

People from Central and South American, and Caribbean nationalities or 

ethnicities whose primary language is not Spanish (e.g., Brazil, Surinam, 

Guyana, Belize, French Guiana)

5 7

Middle Eastern or North African Lebanese, Iranian, Egyptian, Syrian, Moroccan, and Israeli 5 10

People of Middle Eastern or North African nationalities or ethnicities not 

included above. Examples of these groups include, but are not limited to, 

Algerian, Iraqi, Kurdish, Tunisian, Chaldean, Assyrian, etc.

5 4

Native Hawaiian or Pacific 

Islander

Native Hawaiian, Samoan, Chamorro, Tongan, Fijian, and Marshallese 5 2

Additional nationalities or ethnic groups originating in the Pacific Islands not 

included above. Examples of these groups include, but are not limited to, 

Palauan, Tahitian, Chuukese, Pohnpeian, Saipanese, Yapese, etc.

5 2
Recruitment Overview

10

Minimum Category Detailed and Additional Categories Min. # Final #

White German, Irish, English, Italian, Polish, and French 5 17

Additional nationalities or ethnic groups originating in Europe. Examples of 

these groups include, but are not limited to, Scottish, Norwegian, Dutch, 

Slavic, Cajun, Roma, etc.

5 9

People with white origins not listed above (e.g., Russian, or eastern European 

groups) 5 11

Additional Interviews in Any 

Category 5 −

Total 100 121 Eng.
Cognitive Interview Findings

11
Ease/Difficulty of Responding 

– Overall, participants clearly understood the race question and thought the question was asking 

about their family heritage, country of origin, or where they were from. 

– Most were able to easily select a response from the available response options for both the 

minimum category and detailed category questions. Even those who wrote in a response 

expressed that it was easy to identify themselves.

– Most participants decided on their response based on the race of their parents as this was how 

they also identified: 

 “My Dad is Puerto Rican, my mom is African-American, and I identify as both.”

– Some participants shared that they were unsure how far back in their family history they should 

go when answering the question. Among those participants: 

 Most decided to answer according to the race and/or ethnicity of their parents. 

 Some thought about the race and/or ethnicity of their grandparents. 

 A few thought about information they had learned from ancestry websites.

12
Ease/Difficulty of Responding (Continued)

– Overall, Spanish participants did not have issues selected a detailed category for 

themselves. Most simply selected their own country of birth, which seemed 

straightforward to them. 

– Spanish participants who selected American Indian or Alaska Native in addition to 

Hispanic or Latino explained that they did so because they considered their racial 

identity to be “mestizo” [mixed race], due to the mixing of indigenous people from 

their country of birth with White Europeans who colonized those countries. 

 However, these participants were basing this on mixing of races at the population level, not 

the individual level. 

13
Ease/Difficulty of Responding (Continued)

– Participants clearly understood the instruction of “Select all that apply” to 

mean that they could select more than one answer. 

 “I think it means to include as many options as you might want because I 

understand that there are some people that are mixed race so maybe I might be a 

Black Hispanic individual so I would be able to check both boxes.”

– Those who selected more than one response expressed appreciation for 

being able to do so. 

– Three participants did not notice the instruction when reading the question 

and only noticed they could select more than one option during probing 

(attention issue rather than comprehension issue)

14
Answering for Other Household Members

– In general, participants experienced slight difficulty in reporting for other household 

members simply because they did not always know the racial or ethnic background 

of others. 

– When participants were unsure, they either made their best guess, or only answered 

what they knew (e.g., reported one race of a multi-racial household member).

– One-fifth of Spanish participants had a particular difficulty reporting for other 

household members when answering about the race/ethnicity of U.S.-born children 

of Latin American immigrants. 

 These participants were often looking for a response option that would make it clear that the 

children were born in the United States

 These participants did not always consider their U.S.-born children to be Hispanic or Latino 

because they were not born in a Latin America. 

15
Interpretation of “and/or” vs. “or”

– Most participants did not express any confusion with the phrase “and/or” in the 

question, “What is your race and/or ethnicity?”

 Only one participant expressed confusion with “and/or.” This participant shared that they 

would just interpret the question as “What is your race and ethnicity?”

– When asked if their answer would change if the question asked, “What is your race 

or ethnicity?” instead of “What is your race and/or ethnicity?” most participants 

reported that their answer would not change.

 Some participants noted that using “and/or” is more inclusive given that some people think of 

“race” and “ethnicity” in the same way and others think about them differently.

– A few participants were unable to explain the difference between “race” and 

“ethnicity;” however, most participants interpreted these two constructs differently: 

 Participants tended to interpret “race” as a broad descriptor of one’s physical appearance 

(e.g., skin color). 

 Participants tended to have a broader interpretation of “ethnicity” that included one’s cultural 

or ancestral heritage. 16
Feedback on Help Text

– Most participants did not use the help text or would have answered in the same way 

regardless of whether they read the help text. 

– All participants were asked to review the help text during probing if they did not read 

it when answering the race/ethnicity question. Some comments on the text included: 

 “Yes, it is helpful because it includes all individuals who identify with one or more nationalities 

or ethnic groups. It is helpful to read it, and I’m sure it’s helpful for people of other races to 

read it.” 

 “I was very curious about if the geography was listed. That was the main thing.” 

 “I clicked on it because I wanted to see if they gave specific identification for each category.”

– Additional feedback included: 

 Help text should define “race” and “ethnicity” 

 Help text categories should be listed in same order as response options

 Help text should be related to technical problems with the survey rather than information 

about the survey response options. 

17
Reactions to MENA Response Option

– Most MENA participants had seen the MENA response option before on other 

forms/applications. 

– MENA participants discussed that in the past they would have selected “White” and 

“Other” and written in their MENA background. 

– MENA participants expressed a positive reaction to this response category and 

appreciated it being separate from the “White” category. 

 “I’m very, very excited that the Census Bureau is doing this. I’ve been active in the 

community, I’ve written articles, I’ve researched the topic of race. Racial justice is one of my 

passions. From that perspective, I’m very excited that the Census Bureau is making this 

change. Even though the label has a colonial context, comparted to not having the option of 

Middle Eastern or North African, it’s a huge step forward.” [This participant preferred 

“Southwest Asian”]

18
Overreporting

– There was little evidence of over-reporting

– The two participants who noted using ancestry tests reported that the test 

results did not impact their responses. 

 Not all participants were asked about use of ancestry tests since this question 

was added mid-data collection 

– The term “originate” made one participant think of distant middle eastern 

ancestry, which could have led to overreporting, but she did not ultimately 

add MENA to her response options

19
Version Preferences

• While participants provided information on their preference, the qualitative study did 

not show participants changing their response based on the alternative version.

o One changed their response only after reading the help text and not due to version 

differences.

• Many participants preferred Version A because it was clear what information the 

question was trying to gather due to the dropdown occurring on the same page. 

o One explained that you could visually register and acknowledge that it was all from the 

same question.

o One liked that it showed subcategories as she could be more descriptive than White.

o One liked everything on the same page. This participant thought that every time there 

was a new page, he had to refocus and assess whether it was a new question. He also 

thought this would be particularly difficult for those who had selected multiple 

responses.

o One thought it might be more difficult for multi-racial groups to have the details on the 

next page. She thought it was cleaner to have them all together.

20
Version Preferences (continued)

• Some participants preferred Version B because getting the questions one at a 

time looked cleaner.

o One thought that Version A had more boxes so it would take more time to complete.

o One thought Version B’s instructions were clearer. She thought the instruction “Provide 

details” meant that she had to elaborate on the category already clicked (i.e., typing 

Northern Italian after checking Italian).

o One thought more response options were listed on the screen for Version B.

• Some participants thought Version A and Version B were the same, so they did 

not have any preference.

21
Additional Anecdotes, Findings, and Feedback

– Information Sharing Concerns: 

 One participant noted that if she was responding to the Census, she would not fill 

in a more specific answer and so chose to leave detailed categories blank. She 

thought it was an invasion of her privacy to ask.

 While another participant shared details for all questions regarding all household 

members, he mentioned that it was not his preference, and he would have liked 

to only provide the topline responses.

– Equity and Inclusion Concerns:

 One did not like the capitalization of the word White because she associated the 

capitalization with the White power movement

22
Questions and Discussion

What questions or comments do you have?

23