Federal Interagency Working Group on

Improving Measurement of Sexual Orientation and Gender Identity in Federal Surveys

Toward a Research Agenda for Measuring 

Sexual Orientation and Gender Identity in 

Federal Surveys: Findings, 

Recommendations, and Next Steps 

October 20, 2016

Contents

Executive Summary......................................................................................................................................2

I. Introduction and Purpose of the Working Paper..................................................................................... 2

II. Definitions and Acronyms........................................................................................................................ 3

III. Methodology...........................................................................................................................................4

IV. Findings ...................................................................................................................................................4

A. Primary Research Priorities.................................................................................................................. 5

B. Secondary Research Priorities ............................................................................................................. 9

C. Practical Recommendations .............................................................................................................. 14

V. Conclusion ..............................................................................................................................................15

VI. References.............................................................................................................................................17

VII. Appendices...........................................................................................................................................20

A. List of Surveys Represented Among Interviewed Survey Directors/Agency Sponsors ...................20

B. List of Non‐Federal Experts................................................................................................................ 20

C. Interview Protocol for Survey Directors/Agency Sponsors.............................................................. 21

D. Interview Protocol for SOGI Non‐Federal Experts ............................................................................ 25

1
   

                           

                         

                             

                                 

                         

                             

                       

                  

                           

                         

                             

                       

                                 

             

                       

                           

                           

                             

     

                                 

                                 

                             

                           

                           

                     

                           

                               

           

             

                                 

                                   

                             

                               

                           

                         

                           

           

                                                            

                                       

                                 

                             

Executive Summary

The Office of Management and Budget (OMB) convened the Federal Interagency Working Group on

Measuring Sexual Orientation and Gender Identity (SOGI) to begin addressing concerns regarding the

availability of data for sexual and gender minority (SGM) populations and the methodological issues in

collecting such data. This document is the third in a series of working papers from the Federal

Interagency Working Group. The first working paper, Current Measures of SOGI in Federal Surveys,

provides an overview of concepts and current measurement in Federal surveys and serves as the

foundation for subsequent documents. The second working paper, Evaluations of SOGI Survey

Measures: What Have We Learned? reviews evaluations of questionnaire measurement.

This third working paper, Toward a Research Agenda for Measuring Sexual Orientation and Gender

Identity in Federal Surveys: Findings, Recommendations, and Next Steps, expands upon the research

framework provided by the earlier working papers. This paper proposes a research agenda to address

unresolved conceptual and methodological topics that currently impede robust survey measurement in

this area. It is intended to assist in decisions that may face Federal programs seeking to collect

information on sexual orientation and gender identity.

We conducted twenty‐three interviews, with eleven Federal Survey Directors, seven Agency Sponsors,

and five non‐Federal experts; the results generated valuable insights that informed the findings and

recommendations in this working paper. The subgroup reviewed and classified research topic areas into

primary critical areas for immediate future research efforts versus secondary topics to be addressed in

later research activities.

The Research Agenda in Table 1 includes three primary or critical areas that we recommend should be

the first areas of focus for immediate research activities about SOGI measurement. They are in order of

priority: (1) question terminology focusing on gender identity; (2) proxy reporting; and (3) translation of

SOGI concepts/questions into other languages. Table 2 includes a set of secondary areas that offer

opportunities for future research, but which are not immediately pressing, including survey fit and

context, mode administration differences, wording of sexual orientation question(s) and longitudinal

data collection. The Research Agenda is supplemented by a series of practical, comparatively simple

ways to improve the accuracy and quality of SOGI measurement in Federal surveys as more complex

research recommended by the agenda is pursued.

I. Introduction and Purpose of the Working Paper

At a time when sexual and gender minority (SGM)1 populations are becoming more visible in social and

political life, there remains a lack of data on the characteristics and well‐being of these groups. In order

to understand the diverse needs of SGM populations, more representative and better quality data need

to be collected. The U.S. Federal Government is taking several steps to coordinate data collection efforts

across its many Departments. The Office of Management and Budget (OMB) convened the Federal

Interagency Working Group on Measuring Sexual Orientation and Gender Identity (SOGI) to begin

addressing concerns regarding the availability of data for these populations and the issues surrounding

methodological issues in collecting such data.

1 This working paper refers to the population of interest as sexual and gender minority (SGM) rather than the more

commonly used reference as lesbian, gay, bisexual, and transgender (LGBT). We believe that SGM is more inclusive

as it would include persons not specifically referenced by LGBT such as genderqueer, among others.

2
                           

                             

                               

                                     

                

                                 

                               

                           

                         

                         

   

       

                             

                           

                           

                               

                           

                 

      

             

                               

                               

                           

                       

                               

                           

                           

                                   

                               

                             

                             

                     

                       

                     

                                 

                           

                         

                           

           

Although a few Federal agencies have collected information describing SGM populations for over a

decade, some aspects of SOGI have been more routinely measured than others. Further, there are

Federal agencies not currently collecting data on SOGI that have expressed interest in doing so. The

purpose of this working paper is to identify the most vital areas of future research in the areas of

operationalizing and measuring SOGI in U.S. Federal surveys.

This document is the third in a series of working papers from the Federal Interagency Working Group.

The first working paper, Current Measures of SOGI in Federal Surveys, provides an overview of concepts

and current measurement in Federal surveys and serves as the foundation for subsequent documents.

The second working paper, Evaluations of SOGI Survey Measures: What Have We Learned? reviews

evaluations of questionnaire measurement. Other working papers may be developed as research in this

area matures.

II. Definitions and Acronyms

The acronym “SOGI” is used to connote the term “sexual orientation/gender identity” and the acronym

“SGM” is used to connote “Sexual and Gender Minority” throughout this working document. As

documented in the extensive review of Federal survey measures and the research literature about

sexual orientation and gender identity in the first and second working papers, Current Measures of SOGI

in Federal Surveys and Evaluations of SOGI Survey Measures: What Have We Learned?, respectively, the

measurement of SOGI is organized around the concepts of sexual orientation, natal sex, gender identity,

and same‐sex and opposite‐sex households.

Sexual orientation has three main dimensions: sexual identity, sexual attraction, and sexual behavior.

Research on sexual identity, defined as the way someone identifies with a given sexual orientation, was

found to be the most prevalent among the studies reviewed. The term “sexual orientation” is used

within this working paper to indicate sexual identity. Although we recognize that attraction and

behavior are additional components of sexual orientation that should be considered in measurement

efforts to identify this population, we have chosen to focus exclusively on the dimension of sexual

identity as our operationalization of sexual orientation for the purposes of this working paper.

Gender identity may be defined as an individual’s self‐identified sense of gender. A person whose

gender identity and sex assigned at birth are consistent may be referred to as a cis‐gender person (that

is, a person who gender is “consistent in sex”). The term transgender describes anyone whose gender

differs from their sex assigned at birth (Spade, 2008); transgender also refers to a diverse population

that departs significantly from gender norms (IOM, 2011). There are several dimensions of gender that

can be measured: gender identity, gender expression, and gender dysphoria. Some surveys might aim to

measure gender identity, or an individual’s self‐identified sense of gender. Others might be more

interested in measuring gender expression, or an individual’s external manifestation of gender.

It should be noted that the subgroup that prepared this paper was in substantial agreement with the

findings of the extensive review of current SOGI measurement and the associated research literature

documented within both of these working papers. Accordingly, these findings provide the theoretical

underpinning the work of the Federal Interagency Working Group on Measuring Sexual Orientation and

Gender Identity (SOGI) Research Agenda subgroup.

3
                               

                         

                         

                             

                      

                           

                           

                         

              

                         

                         

                         

                         

       

                

                        

                      

                         

                       

            

                 

                 

             

                                

                       

                           

                    

   

                           

                           

                                   

                                 

                             

       

III. Methodology

We conducted a series of interviews with Federal Survey Directors and Agency Sponsors as well as non‐

Federal experts about the topic of SOGI operationalization and measurement. We also dialogued

extensively with Federal Interagency Working Group colleagues and reviewed the first and second

working papers, Current Measures of SOGI in Federal Surveys and Evaluations of SOGI Survey Measures:

What Have We Learned?, respectively, in preparation for our data collection process.

A total of twenty‐three structured interviews were completed: eleven with survey directors and seven

agency sponsors and five with non‐Federal experts. Non‐Federal experts in the area of SOGI

measurement were interviewed to supplement our understanding of the issues and possible solutions

around SOGI measurement in large‐scale Federal surveys.

Survey Directors, Agency Sponsors, and non‐Federal experts were interviewed about their concerns with

current SOGI measurement and priorities for future research. Interviewees ranged widely in SOGI

measurement experience and knowledge: some interviewees were SOGI experts or involved in current

SOGI measurement efforts and research; others represented surveys and agencies that have not yet

considered asking SOGI questions.

Survey Directors and Agency Sponsors were asked about:

1. SOGI survey question implementation at their agencies and/or on their surveys;

2. Past, current, and future SOGI question implementation and research; and

3. Areas of SOGI measurement that they perceived as requiring additional research.

Survey Directors and Agency Sponsors provided their perspective and experience about fourteen

Federal surveys listed in Attachment B.

Non‐Federal experts (listed in Attachment C) were asked about:

1. Their areas of expertise in SOGI measurement;

2. Current SOGI measurement research; and

3. Research priorities they would study if they had funding available to pursue these projects.

The structured interview protocols for the Survey Directors/Agency Sponsors and non‐Federal experts

are available in Attachments E and F, respectively. These interviews provided valuable insights that

informed the findings and recommendations included within this working paper.

IV. Findings

Generally, several areas were identified as important areas for research regarding questions about both

sexual orientation and gender identity. These prospective research areas have been classified into two

groups: a priority listing of research areas that we recommend to be the more immediate focus of future

studies about SOGI measurement (see Table 1) and a secondary listing (Table 2) of less critically pressing

research areas that may warrant investigation after the primary areas have been addressed in the

research and practice literature.

4
                             

                             

                           

                          

     

                             

                         

                         

                 

               

                               

                         

                      

                       

  

                   

                           

   

                             

                           

     

                           

                             

                           

                                 

                             

                           

                             

   

                           

                           

                             

                       

                           

                                 

                                   

                         

                       

We have also provided a number of practical and relatively simple opportunities to obtain information

that can potentially improve the collection of accurate SOGI data over time. They also represent low‐

cost or easily adopted investments that Federal survey programs can implement relatively easily and

which have the potential for meaningful gains and improvements in SOGI data collection.

A. Primary Research Priorities

Three research priorities have been identified as important areas for future research efforts devoted to

SOGI measurement in Federal surveys. These areas are (1) Question Terminology Related to Gender

Identity; (2) Proxy Reporting; and (3) Translation of SOGI Concepts/Questions into Other Languages.

These are described in greater detail in this section.

A.1. Question Terminology Related to Sex and Gender Identity

Several issues were raised by interviewees about the wording and ordering of response options for SOGI

questions, particularly regarding questions designed to measure gender identity (GI). Many factors can

influence wording and ordering of GI response options, including the following:

 Terminology used for GI self‐identification can evolve over time, particularly for younger

respondents.

 Some respondents prefer non‐binary gender options, which are not adequately addressed

currently in Federal surveys. For example, many gender identity questions do not include terms

such as “gender‐nonconforming.”

 Some individuals who vary in age, cultural and linguistic groups, etc., may not endorse terms

such as "transgender" when responding to Federal GI questions because they do not identify

with this term.

These factors present potential challenges in collecting SOGI information for both younger and older

respondents, who may not know what various terms mean or who may differ in their comprehension

and perception of GI question terminology. In addition, as new terms for various GI subpopulations

emerge, it is possible that respondents are unfamiliar with or may not fully comprehend what is meant

by these newly emerging terms. Experts from outside of the Federal Government also identified these

issues as potential threats to collecting accurate GI information from Federal surveys, noting the

challenges presented by limitations in how respondents can report their GI on many surveys, both

Federal and non‐Federal.

The Current Population Survey (CPS) from the Bureau of Labor Statistics is currently conducting

qualitative research about the appropriate wording of SOGI questions. Results are pending, but early

glimpses of preliminary findings suggest that this work will yield valuable information that will address

important questions about collection of gender identity information from respondents on Federal

surveys.

A related issue involves the sequencing of a set of survey questions, particularly in health‐related

Federal surveys, where there are questions that only apply to one sex (e.g., pregnancy). The pathing or

sequencing of questionnaires or sets of questions based on sex at birth or gender identity needs to be

considered in Federal surveys. The Environmental Protection Agency (EPA) is conducting pilot research

activities emphasizing the collection of voluntary, self‐disclosed SOGI data from EPA employees and

5
                             

                                 

                               

                               

       

                             

                     

                              

      

                                     

                         

                         

                             

                               

                     

                   

                               

                                 

                             

                             

     

                           

                         

                               

                                 

                           

           

                           

                         

                               

                                 

                                   

                               

                           

                         

       

               

                           

                       

                               

applicants via questions added to existing forms and data systems (Monson and Minoli, 2016). The

intent of this effort is to increase and improve tools to assess equal opportunity and diversity and

inclusion efforts. The results are forthcoming in the near future and early indications suggest that the

pilot study may provide useful insights about effective language that can be used to collect more

accurate SOGI information from respondents.

The above topics offer important and conceptually and semantically vital areas for potential research in

reducing nonresponse and collecting more accurate SOGI information on Federal surveys. Research

efforts are currently underway that are focusing on addressing some of the issues raised above.

A.2. Proxy Reporting

For some Federal surveys, the only way to collect SOGI data is to allow proxy reporting. Very little is

known about whether respondents have the knowledge to report SOGI information about other

household members. This may be a particularly significant and sensitive challenge in parent‐child

relationships, for example. The issue of the appropriateness of proxy reporting for SOGI questions was

mentioned frequently by interviewees and has been a question raised, but not fully explored, in the

research literature; accordingly, there are presently no evidence‐based recommendations that can be

made about proxy reporting of SOGI questions in Federal surveys.

Proxy reporting is recommended as a priority for future research. Currently, not enough is known about

the effects of proxy reporting with respect to SOGI measurement and data collection to feel confident in

the data quality of responses collected via proxy. Two studies are currently underway, however, that

offer the potential for promising research efforts that can identify challenges and solutions around this

important research topic.

The Current Population Survey (CPS) is conducting qualitative research (e.g., focus groups and cognitive

interviews) about the effects of proxy reporting relative to SOGI measurement. The 2016 Practicum

course offered through the Joint Program in Survey Methodology (JPSM) at the University of Maryland is

also conducting research on SOGI proxy reporting. The results of both of these research efforts will likely

be useful in identifying additional research efforts and recommendations for best practices in proxy

reporting of SOGI questions in Federal surveys.

Proxy reporting research has often focused on privacy and sensitivity issues such as whether

respondents can and will be comfortable reporting SOGI information about themselves and potentially

others while other people are in the room. Very little is known presently about whether household

members even have the knowledge about the SOGI status of people for whom they are providing a

proxy report. It is possible that proxy reporters believe they are able to report SOGI status and identity

for fellow household members when in fact they lack this knowledge and provide an inaccurate proxy

report.

The previously mentioned CPS research will also likely yield additional information to help address

research questions identified within the proxy reporting component of the Research Agenda included

within Table 1 below.

A.3. Translation of SOGI Concepts/Questions into Other Languages

Another area of concern and direction for future research identified by interviewees concerns the

translation of SOGI concepts/questions into other languages. The research conducted to this point

indicates that SOGI questions in surveys such as the National Health Interview Survey (NHIS) do not

6
                           

                             

                         

                     

                               

             

                     

                                     

                             

                               

                       

                             

                       

            

                             

                           

                               

                           

                               

                           

                           

                         

                           

                         

                             

                     

                           

                         

                         

      

                           

                         

                             

                       

                         

                                 

                           

                               

      

   

perform as well with speakers of languages other than English. Additional question development efforts

and evaluation of these efforts should be conducted using methods such as cognitive interviewing to

increase accuracy in capturing SOGI data in Federal surveys administered to non‐English speakers.

Comprehensive cognitive interview testing protocols should be conducted with non‐English speakers

who represent different linguistic groups that are part of the Federal survey’s target population as part

of SOGI question development and improvement activities.

Additional research is needed with Spanish‐speaking populations, in particular, because Spanish is

spoken in about 25 nations, has many regional, national and local idioms that may be in use with respect

to SOGI concepts, and large numbers of native Spanish speakers participate in Federal surveys. Focused,

in‐depth cognitive testing is needed to probe these concepts more completely to identify terms that are

culturally and linguistically appropriate and relevant in Spanish translations of Federal surveys.

Translation by committee is a commonly accepted approach to identifying appropriate terms to use with

varied Spanish‐speaking populations and offers an appropriate vehicle to help identify culturally and

linguistically competent SOGI terminology and questions.

Translation of SOGI concepts and questions into other languages is designated as a priority research

area, particularly given the ongoing increase of culturally and linguistically diverse communities in the

United States. Not enough is currently known to have confidence about the data quality of SOGI

responses from non‐English speakers. We recommend that this topic be selected for immediate further

research with Spanish language as a priority for initial research efforts. Federal survey sponsors are also

encouraged to work with local cultural brokers in culturally and linguistically diverse communities to

identify current language and terminology used to identify SOGI populations in local and regional

communities.

Another unresolved issue is how well various SOGI questions perform with respondents from

communities of color. Although there is considerable available anecdotal evidence to suggest that there

are challenges and differences in collecting SOGI information across culturally and linguistically diverse

communities, there is no accepted clarity or standard that currently exists for SOGI measurement in

these communities. It is notable that community‐based organizations that serve SGM communities of

color have issued practice guidelines for how to ask culturally and linguistically competent SOGI

questions of respondents from SGM communities of color. Currently, however, there are insufficient

research data to identify best practices for SOGI measurement in surveys administered to respondents

from communities of color.

Currently, the Centers for Medicare & Medicaid Services (CMS) are conducting limited research studies

to assess the effectiveness of translating SOGI concepts into Spanish; presently, the CMS Spanish‐

language translations of SOGI terms and concepts may offer the best available options for surveying

Spanish‐language respondents about SOGI questions. This early research effort appears poised to yield

promising potential and direction for future research studies of Spanish‐language translation of SOGI

concepts and questions. In the interim before the complete results of the CMS studies are available, as

practical guidance we recommend that the Spanish‐language version of the SOGI questions tested by

CMS using qualitative means be used when Federal survey programs institute an effort to collect SOGI

information from Spanish‐speaking respondents.

7
                

     

   

               

                   

                   

                               

          

                     

     

                 

         

                    

                         

   

                       

                      

                       

      

                     

         

      

                           

                           

                     

                         

               

                     

                         

                       

   

                       

                       

             

   

                         

                         

   

TABLE 1. Research Agenda for Federal Surveys –

Primary Research Priorities

A.1. QUESTION TERMINOLOGY

RESEARCH

QUESTION #1

 Specific recommendations for testing of the GI question(s) include:

o Is the 2‐step question necessary to reduce undercounting among transgender

population?

o Do differences in accuracy and/or nonresponse emerge when the two‐step

sequence is ordered by (1) GI, followed by sex assigned at birth; or (2) sex assigned

at birth, followed by GI?

o Should “transgender male to female” and “transgender female to male” be added

as response categories?

o Should non‐binary categories such as “non‐gender conforming” or “gender queer”

be added as response options?

RESEARCH

QUESTION #2

 Is it necessary to filter questions by gender or sex?

 Specifically, is it preferable and easier operationally to provide just one question across

genders, if appropriate?

RESEARCH

QUESTION #3

 How do SOGI questions perform with respondents from multiple communities of color

that differ in race, ethnicity, language, geographic distribution and other dimensions?

 What challenges and differences exist in collecting SOGI information across culturally and

linguistically diverse communities?

 What are best practices for SOGI measurement in surveys administered with

respondents from communities of color?

A.2. PROXY REPORTING

RESEARCH

QUESTION #1

 How does proxy reporting affect the accuracy of responses to SOGI questions in Federal

surveys?

 Are proxy reporters able to report accurate SOGI information for others, even when they

believe they know SOGI information about individual(s) for whom they are responding?

 What is the relationship between proxy reporting and the privacy and/or sensitivity of

SOGI information about individual(s) for whom they are responding?

 How are data generated by proxy reporting affected by matters of privacy/sensitivity?

RESEARCH

QUESTION #2

 What findings will the focus groups and cognitive interviews conducted for the Current

Population Survey (CPS) yield to more fully understand the challenges associated with

proxy SOGI reporting?

 What findings will the study conducted through the Joint Program in Survey

Methodology (JPSM) at the University of Maryland yield to better understand the

relationship between proxy reporting and SOGI data collection?

RESEARCH

QUESTION #3

 What qualitative differences and source(s) of error emerge in proxy reporting relative to

the relationship between the proxy respondent to the individual for which s/he provides

responses?

8
              

                         

       

                  

                       

               

                     

                     

                     

               

                       

                   

           

     

                             

                         

                               

                             

                     

                             

                               

                                 

       

                             

                           

                                 

                 

                          

                         

                           

                             

                           

                             

               

                             

                             

                               

                     

                           

A.3. TRANSLATION OF SOGI CONCEPTS/QUESTIONS INTO OTHER LANGUAGES

RESEARCH

QUESTION #1

 What level of comprehension do speakers of languages other than English have about

SOGI questions in surveys?

 How accurately are these respondents answering SOGI survey questions?

RESEARCH

QUESTION #2

 Can SOGI measurement in surveys that are administered to culturally and linguistically

diverse communities be improved through collaborative development efforts with these

communities?

 What SOGI terms and questions can be identified for Spanish‐speaking populations

through the use of translation by committee approaches and cognitive testing

approaches to help identify culturally and linguistically competent SOGI terminology and

questions for respondents from over 25 Spanish‐speaking nations?

RESEARCH

QUESTION #3

 What can Federal survey sponsors learn from strategic engagements with local cultural

brokers in culturally and linguistically diverse communities to improve SOGI

measurement and the identification of SOGI populations?

B. Secondary Research Priorities

The following research priorities represent some additional areas that have been identified as areas of

concern for some interviewees. These topic areas have been categorized as secondary research

priorities as they represent less preeminent topics that can be addressed as part of future research

efforts after primary research priorities have been addressed. These secondary areas are (4) Survey Fit

and Context; (5) Question Terminology Related to Sexual Orientation; (6) Mode Administration

Differences; and (7) Longitudinal Data Collection. These are described in greater detail in this section.

Table 2 presents the Research Agenda listing of research questions for this secondary group of research

priorities that could be the subject of research studies once the primary group of research activities is

addressed in the literature.

Topics areas were first reviewed and prioritized by examining the empirical evidence available for each

topic. We applied the following criteria in determining the priority for each research topic:

 Was the topic, per our collective expert opinion, a "show stopper" – in other words, did the

barrier/challenge/concern/unanswered research question impede a survey from collecting data

that meet Federal data quality standards adequate to produce and publish official statistics?

 For some topics, if we determined there was sufficient empirical evidence available to

substantially reduce or eliminate the need to devote financial and staff resources to the

conduct of additional testing activities, the topic was either eliminated or prioritized at a lower

level. This was particularly true if the evidence suggested that SOGI questions were too

sensitive to ask and/or item nonresponse would be sufficiently high, resulting in the decision to

not include the SOGI question(s) within a survey.

 For other topics, we considered whether there was too little or no empirical evidence available

to be able to reasonably recommend a practice or SOGI wording without the availability of

further research. An example of this was the issue of asking SOGI questions of household proxy

respondents; currently, there is insufficient evidence available to reasonably provide guidelines

for practice about this issue. Another example where we were unable to make empirically

9
                         

                     

                                 

                             

                                 

                                 

                                 

              

                             

                               

                             

           

         

                             

                       

                         

                             

                                 

                                   

     

                                 

                           

                               

                               

                                 

     

                     

                       

                                 

                           

                             

                             

                

                         

                         

                           

                     

                             

                               

                               

                               

grounded recommendation due to the current paucity of evidence was the topic of translation

of SOGI questions, specifically, asking SOGI questions of non‐native English speakers.

As additional context, we also prioritized the ordering of topics based in large measure on the influential

nature of a particular survey program, and, subsequently, the methodological issues most salient to that

program. For some surveys, such as the Current Population Survey (CPS), the proxy issue is currently a

higher priority research area than the topic of translation of SOGI questions. On the other hand, for

survey programs that do not incorporate the use of proxy reporting, the topic of translation of SOGI

questions becomes, potentially, the primary research concern.

Certain topics, such as the wording of gender identity, are higher‐priority research areas because they

are preeminent issues for multiple types of surveys (assuming there are plans to add questions about

gender identity). Through this review and decision‐making process, we arrived at the order of research

topics included within this Research Agenda.

B.1. Survey Fit and Context

A potential challenge around SOGI data collection in Federal surveys is whether survey “fit” and

“context” for SOGI questions affects nonresponse and/or the accuracy of respondents’ answers

throughout a questionnaire. There are particular concerns raised by some interviewees when asked

about the potential perception of their survey respondents about the inclusion of SOGI questions in

Federal surveys that are not about health issues or topics. This perceptual challenge may result in having

SOGI questions deemed to be a poor fit and/or out of place with the questionnaire content of some

non‐health‐related Federal surveys.

A second issue about survey fit and context involves the placement of SOGI questions in the survey

itself. Many surveys collect demographic information in household rosters, which are completed at the

beginning of the interview. A concern was raised by interviewers that placement of SOGI questions so

early in the survey will lead to immediate breakoffs and a reduction in response. Another potential

challenge is that the placement of questions in other parts of the survey may seem nonsensical or

perhaps even offensive.

Several interviewees identified a related, but more over‐arching issue regarding respondents’ reception

to SOGI questions in Federal surveys. Interviewees expressed concerns that Federal survey respondents

may “push back” on being asked to respond to SOGI items. This concern was particularly salient for

politically and socially conservative areas of the United States, and for gender identity questions

specifically, regardless of geographic location throughout the U.S. It is important to note that Federal

surveys that have collected SOGI data have not reported evidence that these concerns have translated

into an impact on Federal survey response rates.

Some interviewees also expressed concerns about the impact of including SOGI questions on Federal

surveys on both item‐ and unit‐level nonresponse. Most Federal survey directors and sponsors already

perceive declining response rates to be a major concern more generally. Interviewees raised concerns

around the relationship between unit nonresponse and SOGI measurement, including whether

questions perceived as sensitive could potentially lead to challenges with response rates and the ability

to report critical survey estimates. Interviewees also identified concerns that they might not be able to

publish estimates or use SOGI data in analyses with other variables because of item nonresponse and

small sample sizes in SOGI response categories. It should be noted that interviewees who raised concerns

10
                         

                         

                         

                     

                           

                                   

                               

                                 

                                       

                 

             

                             

                             

                             

                                   

                                 

                         

                                  

                       

                       

                             

                             

                                 

                             

                                 

                           

                 

                           

                                 

                                   

       

                           

                       

                         

                                 

                     

                             

                             

             

                             

                   

about incorporating SOGI questions in Federal surveys were often unaware that several Federal surveys

have administered SOGI questions with no significant impact on nonresponse or “push back” from

respondents, including the National Health Interview Survey (NHIS), National Survey of Family Growth

(NSFG) and the National Survey on Drug Use and Health (NS‐DUH).

Survey length is another potential challenge. Many Federal survey directors and sponsors believe that

their surveys are already very long and are wary about the difficulty associated with adding even just a

few questions to a survey. Some interviewees commented that they would need to be persuaded about

the use and importance of including SOGI questions on their surveys before doing so; in addition, they

would need to be able to identify other questions that could be removed from the survey if they were to

allow SOGI questions to be incorporated within their surveys.

B.2. Question Terminology Related to Sexual Orientation (SO)

Extensive research has been conducted over many years and with several Federal surveys regarding how

to ask Federal survey recipients about their sexual orientation (SO). Research conducted by the National

Center for Health Statistics (NCHS) indicates that the wording of the sexual orientation (SO) question

NCHS has used in extensive cognitive and field testing over a period of over several years appears to

hold up as an accurate and empirically appropriate version of the SO question to include on Federal

surveys. Accordingly, we recommend that this question be strongly considered by Federal Survey

Agency Sponsors and Survey Directors seeking to add a question about respondent SO on their surveys.

Interviewees were asked about other issues related to SO survey questions. Some interviewees

expressed concerns about whether detailed sexual orientation (SO) information from respondents is

potentially missed by excluding a follow‐up question series if a respondent identifies with a sexual

orientation not included in a survey item. Conversations with interviewees also raised the issue of

whether it is necessary or appropriate to filter questions by gender or sex. For example, should female

respondents receive a “lesbian or gay” response option while male respondents receive only a “gay”

response option? Or can all respondents receive a “lesbian or gay” response option? In response to this

as yet unresolved terminology issue, some interviewees noted that it would likely be easier

operationally to provide just one question across genders, if appropriate.

A related issue involves the sequencing of a set of survey questions, particularly in health‐related

Federal surveys, where there are questions that only apply to one sex (e.g., pregnancy). The pathing or

sequencing of questionnaires or sets of questions based on sex at birth or gender identity needs to be

considered in Federal surveys.

Interviewees also raised the issue of whether supplementary definitions about SOGI concepts should be

developed and provided to respondents and/or interviewers, as appropriate, in Federal surveys.

Although it seems likely that using common supplementary definitions with either respondents and/or

interviewers would enhance the accuracy of responses to SOGI questions, this topic has not, as of yet,

been studied using methodologically rigorous protocols. Implementing this strategy would necessitate

determining how these definitions should be identified and what definitions would be acceptable for the

largest proportion of respondents. Decision rules would likely need to be applied to determine how

these definitions are operationalized in Federal surveys.

The above topics offer important and conceptually and semantically vital areas for potential research in

reducing nonresponse and collecting more accurate SOGI information on Federal surveys.

11
       

                         

                         

                           

                           

                       

                        

                               

                           

                             

                     

                     

                             

                       

                             

                         

                       

  

                             

                       

                       

                             

                             

                 

                           

                           

                             

                             

                           

                           

                             

                           

                 

      

     

                         

                             

                       

                               

                               

B.3. Mode Administration Differences

Interviewees identified two important areas relevant to mode differences regarding the performance of

SOGI questions in Federal surveys. The first centers around determining the effect of self‐administered

survey modes on SOGI response. Specifically, does survey mode affect either item‐level response rate

and/or the accuracy of SOGI information? Despite the intuitive appeal of the assumption that

respondents are likely to provide more truthful responses when completing self‐administered SOGI

questions, there is only limited evidence currently available to support this assumption.

A second larger and potentially more fruitful area for exploration focuses on how to ensure the

adherence of Federal survey interviewers to important data collection instructions for SOGI questions, in

general, and gender identity questions, in particular, and how this affects the performance of SOGI

questions. Interviewees reported that, in general, Federal survey interviewer administration of sexual

orientation questions has not been problematic. Gender identity questions, however, require

interviewers to ask respondents their sex at birth and gender identity directly. Many surveys currently

instruct interviewers to record “perceived gender” rather than ask respondents to provide information

about their gender. In other cases, interviewers are instructed to ask respondents about their gender,

but do not follow these instructions. It appears likely that some interviewers are uncomfortable

confirming discrepant answers in two‐part gender identity questions and, accordingly, skip this

confirmation process.

Resolving this data collection challenge will likely require additional training and a strong emphasis on

how interviewers can address barriers and challenges associated with collecting SOGI information from

respondents to successfully collect SOGI data. Interviewer debriefing sessions present an optimal

opportunity to more fully explore these challenges with interviewers. Further, it would be useful to

identify and share best practices interviewers can apply to minimize both their own and respondent

discomfort and increase respondent accuracy in reporting gender identity.

The National Crime Victimization Survey (NCVS) will shortly be debriefing with interviewers about their

experience with SOGI measurement questions and related topics. It would be helpful to incorporate

probes within the interviewer debriefing protocols about barriers to asking SOGI questions as well as

possible solutions that can mitigate the effect of these barriers. We also recommend that interviewer

debriefing sessions for all Federal surveys that include SOGI questions explicitly probe about interviewer

efforts to collect this information, challenges to the successful collection of this information, and

strategies that successfully address these barriers and challenges. It would also be useful to conduct

comparison studies to determine whether the performance of SOGI questions varies based on whether

respondents answer Federal survey SOGI questions via self‐administered or interviewer‐administered

data collection modes.

B.4. Longitudinal Data Collection

Longitudinal data collection on SOGI information from Federal surveys represents a unique opportunity

to evaluate whether respondents maintain or revise their SOGI identity for reporting purposes over the

long‐term. Interviewees suggested that evaluative studies be conducted to examine whether a change

in how respondents self‐report their SOGI status occurs over time. Longitudinal studies can also focus on

whether sex on the respondent’s original birth certificate should be asked more than once for GI

12
                     

                                 

                

     

          

                         

            

                         

            

                             

                     

                        

                           

                    

                           

                       

                         

  

                           

          

                     

  

                         

                         

   

                         

                       

                   

           

              

                         

           

                       

         

                         

                           

     

   

questions. Additional insights could be derived from qualitative follow‐up questioning with respondents

to learn the reason(s) for changes in SOGI identity or use of terminology to describe one’s SOGI identity,

TABLE 2. Research Agenda for Federal Surveys –

Secondary Research Priorities

B.1. SURVEY FIT AND CONTEXT

RESEARCH

QUESTION #1

 Does the placement of SOGI questions early in Federal surveys lead to immediate

breakoffs and/or a reduction in response?

 Where can SOGI questions be placed within Federal surveys to minimize the appearance

of being nonsensical or potentially offensive?

 Is/are there an optimal location(s) in surveys that can be drawn upon to place SOGI

questions within surveys to reduce nonresponse and promote appropriate “fit” and

“context,” or does this issue need to be addressed on a survey‐by‐survey basis?

 Can best practices be developed that will guide survey developers and sponsors in the

placement of SOGI questions to improve survey “fit” and “context?”

 Further, does the inclusion of SOGI questions in Federal surveys that are not about

health issues or topics results in immediate breakoffs and/or a reduction in response?

RESEARCH

QUESTION #2

 What is the impact of including SOGI questions on Federal surveys on item‐level

nonresponse?

 Does the inclusion of SOGI questions on Federal surveys contribute to an overall decline

in Federal survey response rates?

RESEARCH

QUESTION #3

 Does the inclusion of SOGI questions in Federal surveys affect survey administration

costs?

 Given the likelihood of item nonresponse and small sample sizes in SOGI response

categories, is there empirical support for the ongoing collection of SOGI information in

Federal surveys?

RESEARCH

QUESTION #4

 What implications do privacy concerns have for both specific SOGI questions and the

larger survey that includes the SOGI questions (i.e., do privacy concerns cause

respondents to potentially withdraw from completing the Federal survey?), particularly

in the context of interviewer‐administered surveys?

B.2. QUESTION TERMINOLOGY RELATED TO SEXUAL ORIENTATION (SO)

RESEARCH

QUESTION #1

 Additional testing is needed to identify the most appropriate wording of the following

areas relative to questions about SO:

o Should the “straight/heterosexual” response category be placed first in the set of

response categories for the SO question?

o Should the categories for “gay” and “lesbian” be presented to respondents of all

sexes, with no concomitant use of branching of this aspect of the question according

to sex of respondent?

13
  

                     

                     

                     

          

                   

     

                       

                 

   

                   

             

                   

                       

                       

           

                         

                           

                   

   

                     

   

                         

       

      

                           

                

                       

            

                           

     

                             

  

                       

                           

       

  

                       

                             

                         

                             

B.3. MODE ADMINISTRATION DIFFERENCES

RESEARCH

QUESTION #1

 What information can survey interviewer debriefing sessions for Federal surveys that

include SOGI questions extract relative to interviewer efforts to collect this information?

 What challenges and barriers do survey interviewers experience that impede the

successful collection of SOGI information?

 What strategies do survey interviewers implement that successfully address these

barriers and challenges?

RESEARCH

QUESTION #2

 Does the performance of SOGI questions vary based on whether respondents answer

Federal survey SOGI questions via self‐administered or interviewer‐administered data

collection modes?

 Specifically, do respondents answer SOGI questions more truthfully when completing

self‐ versus interviewer‐administered questions? Further, does survey nonresponse vary

based upon what mode is used to ask respondents SOGI questions?

RESEARCH

QUESTION #3

 How does the adherence of Federal survey interviewers to important data collection

instructions for SOGI questions, in general, and gender identity questions, in particular,

affect the performance of SOGI questions?

 Does the use of flexible interviewing by survey interviewers improve the accuracy of

SOGI information?

RESEARCH

QUESTION #4

 What is the potential effect of privacy concerns on obtaining a true and accurate

response from Federal survey recipients, particularly in the context of interviewer‐

administered surveys?

 Do respondents provide honest responses to SOGI questions when they are concerned

about privacy?

 Do privacy concerns result in an undercount in the number of respondents who

complete SOGI questions honestly?

B.4. LONGITUDINAL DATA COLLECTION

RESEARCH

QUESTION #1

 Should SOGI questions be asked only once, at intermittent data collection points, or at

every wave of data collection in Federal surveys?

 What variations in response, if any, emerge when respondents respond to SOGI

questions at multiple data collection points?

 What are the reason(s) for these variations in response to SOGI questions at multiple

data collection points?

RESEARCH

QUESTION #2

 Should sex on the original birth certificate be asked more than once for gender identity

questions?

 If responses to SOGI questions, particularly around gender identity, change over time,

can these changes be attributed to changes in respondents’ identity or to faulty or

inadequate SOGI question wording?

C. Practical Recommendations

We have identified Practical Recommendation(s) that represent relatively simple opportunities to obtain

information that can potentially improve the collection of accurate SOGI data over time. They also

represent low‐cost investments that Federal survey programs can implement relatively easily and that

have the potential for meaningful gains and improvements in SOGI data collection. The following three

14
                     

                        

       

                               

                                 

                                 

                       

                               

                           

                                     

                         

  

       

                           

                               

                          

                             

                                   

                             

               

                     

                         

                           

                         

                     

                       

                             

                         

                         

                             

               

   

                           

                             

                               

                           

                           

              

   

practical recommendations are offered for consideration by Federal survey programs that are

implementing (or plan to implement) SOGI data collection activities within their programs.

C.1. Reintroduce the write‐in option

The Research Agenda subgroup recommends that a write‐in option be added back in to SOGI questions

that ask the respondent to “specify” in cases when “other” is selected. Allowing respondents to write in

a response if they select “Other” allows analysts to examine the write‐in over time to look for

new/common terms. Experts outside of the Federal Government frequently articulated concerns about

the lack of a write‐in option for SOGI questions to allow respondents to self‐identify using terminology

representing their own SOGI status. Accordingly, this small and low‐impact change may yield useful

results and has little in the way of a downside; further, this practical step does not necessarily need to

be tested or researched, but could still yield useful information, particularly over repeated

administrations.

C.2. Sequence SOGI questions

For health‐related Federal surveys that contain questions that apply only to one sex (e.g., pregnancy),

how the question path should be designed, integrated and sequenced within the larger survey is a

frequent concern. The recent studies conducted about the sequencing of SOGI questions for the

California Health Interview Survey (CHIS) offer useful guidance and may be considered a best practice

about how to design question paths that apply to only one sex so the collection of accurate SOGI

information can be optimized. (CHIS 2017‐2018 SOGI Questions Working Group, 2016; Grant, D., et al.,

2015)

C.3. Use common definitions in standard operating processes

Additional issues that emerged during interviews included whether supplementary definitions about

SOGI concepts for respondents and/or interviewers should be provided as appropriate in Federal

surveys with the purpose of ensuring shared understanding and consistency in operationalization. If so,

where and how should these definitions be identified? What would constitute adequate agreement

among various constituencies (e.g., survey researchers, members of SGM communities, non‐members of

SGM communities, etc.) to represent a consensus on commonly accepted definitions? Further, what

decision rules would need to be applied to identify these commonly accepted definitions? Examples of

best practices to address these issues concerning definitions are offered by standard operating

processes and guidelines implemented within the National Health Interview Survey (NHIS), the National

Survey of Family Growth (NSFG) and the National Crime Victimization Survey (NCVS). (See overview in

Center for Behavioral Health Statistics and Quality, 2015.)

V. Conclusion

Our proposed Research Agenda for SOGI measurement and data collection in Federal surveys includes

the seven areas below that represent primary and secondary priorities and areas of emphasis. This

extensive set of research areas offers important foci for the collection of empirical evidence and the

identification of best practices around SOGI measurement in Federal surveys. We provide this research

agenda to assist Federal programs when considering how to improve measurement of sexual orientation

and gender identity in their information collections.

15
         

       

                 

     

               

     

         

             

       

     

   

TABLE 3. Recommended Research Priorities

A. PRIMARY RESEARCH PRIORITIES

1. Question Terminology Related to Sex and Gender Identity

2. Proxy Reporting

3. Translation of SOGI Concepts/Questions into Other Languages

B. SECONDARY RESEARCH PRIORITIES

4. Survey Fit and Context

5. Question Terminology Related to Sexual Orientation

6. Mode Administration Differences

7. Longitudinal Data Collection

16
   

                             

                 

  

                                       

                         

          

                             

                     

   

                         

       

                          

                       

  

                               

                         

       

                             

        

                               

            

                                   

                                 

                   

                           

                       

   

                                     

                                 

                               

       

                         

                               

     

VI. References

Baker, K, and Hughes, M. 2016. “Sexual Orientation and Gender Identity Data Collection in the

Behavioral Risk Factor Surveillance System.” Center for American Progress.

https://cdn.americanprogress.org/wp‐content/uploads/2016/03/29090401/BRFSSdatacollect‐brief‐

03.31.16.pdf

Cahill, S, Singal, R, Grasso, C, et al. 2014. “Do Ask, Do Tell: High Levels of Acceptability by Patients of

Routine Collection of Sexual Orientation and Gender Identity in Four Diverse American Community

Health Centers.” PLoS One 9: e107104.

Center for Behavioral Health Statistics and Quality. (2015). National Survey on Drug Use and Health:

2014 and 2015 Redesign Changes. Substance Abuse and Mental Health Services Administration,

Rockville, MD.

CHIS 2017‐2018 SOGI Questions Working Group (2016). CHIS Methodology and Sample Design Memo.

CHIS‐METH‐006. September, 16 2016.

Interagency Working Group on Sexual Orientation and Gender Identity. 2016. “Current Measures of

Sexual Orientation and Gender Identity in Federal Surveys.” Accessed July 2016 at:

https://fcsm.sites.usa.gov/files/2014/04/WorkingGroupPaper1_CurrentMeasures_08‐16.pdf

Dahlhamer, J, Galinsky, A, Joestl, S, and Ward, B. "Sexual Orientation in the 2013 National Health

Interview Survey: A Quality Assessment." Vital and Health Statistics. Series 2, Data Evaluation and

Methods Research 169 (2014): 1‐32.

Gates, G. 2011. “How Many People are Lesbian, Gay, Bisexual, and Transgender?” The Williams Institute.

Accessed July 2016. https://escholarship.org/uc/item/09h684x2

Grant, D, and Jans, M. No date. “Assessing Sexual Identity & Behavior in California’s Diverse Population”

UCLA Center for Health Policy Research.

Grant, D., Jans, M., Royce, P., Ninez, P.A., Jane, K., Gates, G., Bianca D.M., W., & Herman, J.L.. “Putting

the "T" in LGBT: A Pilot Test of Questions to Identify Transgender People in the California Health

Interview Survey.” American Association of Public Opinion Research, May 1, 2015.

Grant, LA, Mottet, JD, Tanis, J., Herman, J, Harrison, J, Keisling, M. 2010. “National Transgender

Discrimination Survey Report on Health and Health Care.” National LGBTQ Task Force.

http://www.thetaskforce.org/static_html/downloads/resources_and_tools/ntds_report_on_health.pdf

Jans, M, Viana, J, Grant, D, Cochran, S, Lee, A, and Ponce, N. 2015. "Trends in Sexual Orientation Missing

Data over a Decade of the California Health Interview Survey." American Journal of Public Health 105 (5):

E43‐E50.

Joloza, T, Evans, J., and O’Brien, R. 2010. “Measuring Sexual Identity: An Evaluation Report.” Office for

National Statistics, United Kingdom.

Kim, H, and Fredriksen‐Goldsen, K. 2013 “Nonresponse to a Question on Self‐Identified Sexual

Orientation in a Public Health Survey and Its Relationship to Race and Ethnicity.” American Journal of

Public Health, 103(1): 67‐69.

17
                                 

              

                           

                      

                           

             

                           

         

                           

             

                             

                       

           

                     

           

                       

                   

                                 

                             

          

                         

                         

                               

                 

                           

                 

                         

                       

                         

               

                                   

               

Lombardi, E, and Banik, S. 2016. “The Utility of the Two‐Step Gender Measure within Trans and Cis

Populations.” Sexuality Research and Social Policy 13(3): 288–296.

Magliozzi, D, Saperstein, A, and Westbrook, L. 2016. “Scaling Up: Representing Gender Diversity in

Survey Research.” Socius: Sociological Research for a Dynamic World 2: 1‐11.

Martinez, M, Henderson, A, and Luck, J. 2016. “Cognitive Pretesting of the National Crime Victimization

Survey Supplemental Victimization Survey.” U.S. Census Bureau (unpublished report)

Miller, K. 2001. “Cognitive Testing of the NHANES Sexual Orientation Questions.” National Center for

Health Statistics. Q‐Bank. Retrieved from http://www.cdc.gov/qbank.

Miller, K and Ryan, JM. "Design, Development and Testing of the NHIS Sexual Identity

Question." National Center for Health Statistics (2011): 1‐33.

Monson, M and Minoli, K. 2016. “The Importance of Being Counted: Collecting Sexual Orientation and

Gender Identity (SOGI) Data in Federal Government.” Presentation at Out & Equal Workplace Summit,

Orlando, Florida on October 6, 016.

National Center for Education Statistics. Forthcoming. “Cognitive Testing Items HSLS:09 Second Follow‐

Up – Field Test Version Findings.”

National Opinion Research Center (NORC). 2016. “Testing Alternative Response Options for Spanish

Translations of Sexual Identity Items for National Surveys.” (unpublished report)

Reisner, SL, Conron, KJ, Tardiff, LA, et al. 2014. “Monitoring the Health of Transgender and Other Gender

Minority Populations: Validity of Natal Sex and Gender Identity in a U.S. National Cohort of Young

Adults.” BMC Public Health 14: 1224.

Ridolfo, H, Miller, K, and Maitland, A. 2012. "Measuring Sexual Identity using Survey Questionnaires:

How Valid are our Measures?" Sexuality Research & Social Policy 9 (2): 113‐124.

Ridolfo, H., Perez, K, and Miller, K. (2010). Testing of Sexual Identity and Health Related Questions:

Results of Interviews Conducted May‐July 2005. Q‐Bank. Retrieved from http://www.cdc.gov/qbank.

Ridolfo, H, and Schoua‐Glusberg, A. (2009). Testing of NHANES ACASI Sexual Behavior Questions: Results

of Interviews Conducted November 2008–February 2009. Q‐Bank. Retrieved from

http://www.cdc.gov/qbank.

Sexual Minority Assessment Research Team (SMART). 2009. “Best Practices for Asking Questions about

Sexual Orientation on Surveys.” Williams Institute at UCLA School of Law, November.

http://williamsinstitute.law.ucla.edu/wp‐content/uploads/SMART‐FINAL‐Nov‐2009.pdf

Smirnova, M and Scanlon, P. 2013. “2013 Census Test Cognitive Interview Report.” Survey Methodology

Paper 2013‐18. Suitland, MD: U. S. Census Bureau.

Stern, MJ, Michaels, S, Milesi, C, et al. 2016. “Task 1.32.a: Develop and Test Sexual and Gender Minority

Status (LGBT) Items Final Summary Report.” (unpublished report)

18
                           

                         

   

                       

                     

                         

                        

   

The GenIUSS Group. 2014. Best Practices for Asking Questions to Identify Transgender and Other

Gender Minority Respondents on Population‐Based Surveys. J.L. Herman (Ed.). Los Angeles, CA: The

Williams Institute.

Vrangalova, Z, and Savin‐Williams, RC. 2012. “Mostly Heterosexual and Mostly Gay/Lesbian: Evidence

for New Sexual Orientation Identities.” Archives of Sexual Behavior 41(1): 85‐101.

Westbrook, L, and Saperstein, A. 2015. “New Categories are Not Enough: Rethinking the Measurement

of Sex and Gender in Social Surveys.” Gender and Society 29(4): 534‐560.

19
   

                   

    

       

                   

       

           

         

           

           

               

           

           

             

             

  

         

             

           

       

                   

             

   

VII. Appendices

A. List of Surveys Represented Among Interviewed Survey Directors/Agency Sponsors (N=13)

CE: Consumer Expenditure Survey

CPS: Current Population Survey

FHWAR: National Survey of Fishing, Hunting, & Wildlife Associated Recreation

NCVS: National Crime Victimization Survey

NHES: National Household Education Surveys Program

NHIS: National Health Interview Survey

NSCG: National Survey of College Graduates

NSCH: National Survey of Children's Health

NSDUH: National Survey on Drug Use and Health

NSFG: National Survey of Family Growth

NTPS: National Teacher and Principal Survey

SIPP: Survey of Income and Program Participation

YRBS: Youth Risk Behavior Surveillance System

B. List of Non‐Federal Experts (N=5)

 Lee Badgett, Williams Institute and University of Massachusetts

 Kellan Baker, Center for American Progress

 Ryan Coventry, Northwestern University ‐ IMPACT Center

 Joanne Keatley, Director of Center of Excellence for Transgender Health

 Sari Reisner, National LGBT Health Education Center

20
       

                               

                               

             

                           

                          

                             

                     

                                 

                            

                             

                                       

         

                           

                           

   

                               

                               

                       

          

           

                             

   

                           

                

  

               

                       

            

                       

          

  

C. Interview Protocol for Survey Directors/Agency Sponsors

INTRO: I am a member of the OMB interagency group working to improve measures of sexual

orientation (SO) and gender identity (GI) in Federal surveys. I am on the subgroup tasked with

identifying a research agenda for the future.

We are having conversations with Survey Directors and Agency sponsors to better understand what

challenges (if any) you have encountered when collecting data on SO and GI.

 For those agencies that already collect these data: We want to understand if there are additional

areas of research you would like to see the agencies conduct.

 For those agencies that do not currently collect these items: We want to understand if you foresee

any barriers or challenges to asking these items (should you be tasked with adding them).

 IMPORTANT: I’d like to ask you about whether you are comfortable with us potentially identifying

(1) your name and role relative to the [INSERT NAME OF SURVEY] and (2) the name of your survey in

our written report of findings?

o NOTE TO INTERVIEWER: Please discuss this issue with the respondent, as needed. It may be

that respondents who do NOT want to be identified will decline to respond to some

protocol questions.

Our ultimate goal is to create a prioritized inventory of research projects we recommend be conducted

in the near future. In addition to conversations with Federal agency contacts, we are also having

discussions with academics and think‐tank representatives who conduct policy and survey methods

research on sexual minority populations.

About the Respondent and the Survey

1. How long have you been in this role as the Survey Director for this survey?

Sexual Orientation

2. To your knowledge, does your agency currently sponsor any survey or serve as a data

collection agent for a survey that asks Sexual Orientation?

If YES:

 What are the surveys? What question(s) is/are asked?

 Where did these survey item(s) originally come from? What was/were the source(s)?

 When did the data collection begin?

 Did the question undergo any testing prior to implementation (for example, cognitive

interviewing or a field test)?

 Was implementation problematic?

21
                         

                       

           

                     

   

         

                     

                          

       

                            

                         

                     

                               

   

                     

                         

   

   

                               

                

  

               

            

                       

          

  

             

           

                     

 For example, was the approval process within your agency difficult? What about the

OMB clearance process? Did other stakeholders weigh in on this approval process?

 Did interviewers express concerns? What were they?

 How did you address the question in training? Was training difficult?

 Were respondents uneasy?

 Were there any other problems encountered?

3. How confident are you in the data quality of this item?

4. Do you feel additional research could improve the data quality of this item?

 If yes, please explain.

5. Do you have any plans to sponsor or conduct further tests of this item?

6. For agencies currently collecting Sexual Orientation data: Where any of the questions or

response categories originally developed and/or implemented and then changed to the

current item(s)? For example, perhaps a change from a 2‐step to a single question to assess

gender identity?

 If YES: What were these original question(s)? Why were these question(s)? What

issues were involved in the decision to revise these question(s) and implement (a)

new question?

Gender Identity

1. To your knowledge, does your agency currently sponsor any survey or serve as a data collection

agent for a survey that asks Gender Identity?

If YES:

 What are the surveys? What question(s) is/are asked?

 When did the data collection begin?

 Did the question undergo any testing prior to implementation (for example, cognitive

interviewing or a field test)?

 Was implementation problematic?

 For example, was the approval process difficult?

 Did interviewers express concerns? What were they?

 How did you address the question in training? Was training difficult?

22
   

         

                     

                          

       

                            

                         

                     

                             

     

                     

                         

   

                 

                           

                             

  

                            

                           

  

           

                               

                 

                         

                                

                            

                           

  

 Were respondents uneasy?

 Were there any other problems encountered?

 How confident are you in the data quality of this item?

 Do you feel additional research could improve the data quality of this item?

 If yes, please explain.

 Do you have any plans to sponsor or conduct further tests of this item?

 For agencies currently collecting Gender Identity data: Where any of the questions or

response categories originally developed and/or implemented and then changed to the

current item(s)? For example, perhaps a change from a 2‐step to a single question to

assess gender identity?

 If YES: What were these original question(s)? Why were these question(s)? What

issues were involved in the decision to revise these question(s) and implement (a)

new question?

If answer is NO to collecting/sponsoring Sexual Orientation question:

 Has a sexual orientation question been considered on your survey(s)? Why or why not?

 If your survey was required to add sexual orientation, do you anticipate any challenges or

barriers?

 Is there any research you would conduct prior to adding this item to production?

 Are there certain test outcomes you require before you would add this question into

production?

 Have you considered adding Sexual Orientation question(s)?

IF YES: In light of this response, are there reasons why these questions not been added your

survey?

If answer is NO to collecting/sponsoring Gender Identity question:

 Have gender identity question(s) been considered on your survey(s)? Why or why not?

 If your survey was required to add gender identity, do you anticipate any challenges or barriers?

 Is there any research you would conduct prior to adding this item to production?

 Are there certain test outcomes you require before you would add this question into

production?

23
           

                               

                                 

       

 Have you considered adding Gender Identity question(s)?

IF YES: In light of this response, are there reasons why these questions not been added your

survey?

Debriefing

 Is there anything else you’d like to mention that we haven’t had a chance to discuss yet?

Thank you for your time!

24
       

                                     

                                   

       

                               

                                           

                 

                                         

                                 

                       

                         

         

                               

         

                               

                 

                                 

               

                                 

         

D. Interview Protocol for SOGI Non‐Federal Experts

INTRO: I am a member of the Office of Management and Budget interagency group working to improve measures of

sexual orientation and gender identity in Federal surveys. I am on the subgroup tasked with identifying a research

agenda for the future.

We are having conversations with researchers to better understand what challenges you may encounter when collecting

and/or analyzing data on SO and GI. We want to understand if there are additional areas of research you would like to

see funded (by academics, think‐tanks, Federal agencies, or whomever.)

Our ultimate goal is to create a prioritized inventory of research projects that we can recommend to be conducted in the

near future. In addition to conversations with researchers and experts about SOGI measures, we are also having

discussions with Federal agency survey directors who oversee nationally representative population surveys.

1. What is your area of expertise regarding sexual minority (including sexual orientation/SO) and transgender

(including gender identity/GI and gender non‐conforming) populations?

2. Are you currently involved in any research that informs the measurement of sexual orientation (SO)? PROBE

AS NEEDED TO OBTAIN DETAILS

3. Are you currently involved in any research that informs the measurement of transgender, gender identity (GI)

or gender non‐conforming populations? PROBE AS NEEDED TO OBTAIN DETAILS

4. If grant money were to become available, what are the top 3‐4 research projects related to SOGI

measurement that you would recommend be pursued? Why?

5. Is there anything else you’d like to mention that we haven’t had a chance to discuss yet?

Thank you for your time!

25