Sexual Violence Data: Actions Needed to Improve Clarity and Address Differences Across Federal Data Collection Efforts
Published: 2016-07-19T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Concerns have grown about sexual violence—in general, unwanted sexual acts—in the United States, particularly involving certain populations such as college students, incarcerated individuals, and military personnel. Data on the occurrence of sexual violence are critical to preventing, addressing, and understanding the consequences of these types of crimes. GAO was asked to identify and compare federal efforts to collect data on sexual violence.
This report addresses two questions: (1) What are the federal efforts underway to collect data on sexual violence, and how, if at all, do these efforts differ? (2) How do any differences across the data collection efforts affect the understanding of sexual violence, and to what extent are federal agencies addressing any challenges posed by the differences? GAO reviewed agency documentation and academic literature, and interviewed agency officials.
section_title: What GAO Found
paragraphs: Four federal agencies—the Departments of Defense, Education, Health and Human Services (HHS), and Justice (DOJ)—manage at least 10 efforts to collect data on sexual violence, which differ in target population, terminology, measurements, and methodology. Some of these data collection efforts focus on a specific population that the agency serves—for example, the incarcerated population—while others include information from the general population. These data collection efforts use 23 different terms to describe sexual violence. Data collection efforts also differ in how they categorize particular acts of sexual violence. For example, the same act of sexual violence could be categorized by one data collection effort as “rape,” whereas it could be categorized by other efforts as “assault-sexual” or “nonconsensual sexual acts,” among other terms. In addition, five data collection efforts—overseen by Education, HHS, and DOJ—reflect inconsistencies between their measurements and definitions of sexual violence. Further, these data collection efforts do not have publicly-available descriptions of what is included in their respective measurements to allow persons using the data to understand the differences, which may lead to confusion for data users. Publicly-available measurement information could enhance the clarity and transparency of sexual violence data. Data collection efforts also differ in terms of the context in which data are collected, data sources, units of measurement, and time frames.
Differences in data collection efforts may hinder the understanding of the occurrence of sexual violence, and agencies' efforts to explain and lessen differences have been fragmented and limited in scope. Differences across the data collection efforts may address specific agency interests, but collectively, the differences lead to varying estimates of sexual violence. For example, in 2011 (the most recent year of available data), estimates ranged from 244,190 rape or sexual assault victimizations to 1,929,000 victims of rape or attempted rape. These differences can lead to confusion for the public. Officials from federal agencies and entities GAO spoke with who use federal data on sexual violence emphasized that the differences across the data collection efforts are such that the results are not comparable, and entities reported using data that best suited their needs. Agencies have taken some steps to clarify the differences between the data collection efforts. For example, two DOJ entities coauthored a statement that describes the differences between their two efforts. In addition, agencies have taken some steps to harmonize the data collection efforts—that is, coordinate practices to achieve a shared goal. However, actions to increase harmonization have been fragmented, generally only involving 2 of the 10 data collection efforts at a time, and limited in scope. The Office of Management and Budget (OMB) through its authority to coordinate federal statistics has previously convened interagency working groups, such as the Interagency Working Group for Research on Race and Ethnicity, to improve federal statistics. OMB has no plans to convene a working group on sexual violence data. Additional collaboration, facilitated by OMB, between agencies that manage data collection efforts about which differences help or hinder the overall understanding of sexual violence could help to clarify the scope of the problem of sexual violence in the United States.
section_title: What GAO Recommends
paragraphs: GAO recommends that Education, HHS, and DOJ make information that is included in their measurements of sexual violence publicly available. GAO also recommends that OMB establish a federal interagency forum on sexual violence data. Education, HHS, and DOJ agreed with the recommendation. OMB stated that convening a forum may not be the most effective use of resources at this time, in part because the data collection efforts are not far enough along in their research. However, OMB said it will consider convening or sharing information across agencies in the future.

section_title: Letter
paragraphs: There is concern about sexual violence in the United States, particularly  involving college students, incarcerated individuals, and military  personnel. Research has shown that sexual violence has long-lasting  effects on victims, including sexually-transmitted diseases, eating  disorders, anxiety, depression, and post-traumatic stress disorder.  Further, the economic costs of rape, including medical and social  services, loss of productivity, decreased quality of life, and law  enforcement resources, are estimated to range from $41,247 to $150,000  per incident.
According to research, victims often do not report sexual violence to law  enforcement officials due to feelings of guilt, shame, and embarrassment;  fear of the perpetrator; or fear of not being believed, among other  reasons. Therefore, the occurrence of sexual violence is considered to be  underestimated.
Congress, state and local governments, and the public depend on data  collected by the federal government to prioritize resources, design  policies and programs that prevent and address crimes of sexual  violence, and evaluate those policies and programs. Federal data on  sexual violence are also used in crafting initiatives to mitigate the effects  of these experiences on victims and in gauging the effectiveness of these  initiatives.
You asked us to identify the primary sources of information on sexual  violence collected by the federal government and to identify any  challenges that may result from the differences across the data collection  efforts. This report addresses the following questions: (1) What are the  federal efforts underway to collect data on sexual violence, and how, if at  all, do these efforts differ? (2) How do any differences across the data  collection efforts affect the understanding of sexual violence, and to what  extent are federal agencies addressing any challenges posed by the  differences?
For the purpose of our analysis, we selected data collection efforts which  focus on data that: provided information on the extent to which acts of sexual violence  occur in the United States in a particular year (for example, the  number of times an act of sexual violence occurred or the number of  victims of sexual violence); were collected since 2010; were collected at least once every 2 years; were reported publicly; and were not focused primarily on minors.
We set these criteria to focus our analysis on data collection efforts that  provide current and regular information that is available to the public on  sexual violence. To narrow our scope, we chose to not include data  collection efforts that primarily focused on sexual violence against minors.  To identify data collection efforts that met our criteria, we reviewed past  GAO reports and a Federal Bureau of Investigation (FBI) list of federal  agencies that may collect crime data. We asked officials at those  agencies if they had any data collection efforts that met our criteria. We  also asked academic experts and officials from entities that use federal  data on sexual violence about any additional federal data collection  efforts they were aware of that met our selection criteria. We identified  experts and officials from entities that use federal data on sexual violence  by conducting background research and by asking those experts and  officials that we interviewed to recommend additional experts and entities  that use federal data on sexual violence, including victim advocacy  groups, other special interest groups, and other federal agencies.
To identify and describe differences across the data collection efforts, we  obtained information on the purpose, scope, and methodology of each  data collection effort. We obtained this information through a review of  documents, such as user manuals and program descriptions. We also  conducted interviews with senior agency officials, senior officials at  entities that use federal data on sexual violence, and academic experts.
To identify how the differences affect understanding of sexual violence,  we obtained and reviewed federal reports and interviewed and reviewed  relevant documentation from agency officials, experts, and  representatives from entities that use federal data on sexual violence. We  also reviewed articles, conference papers, and government reports that  discuss differences across federal sexual violence data collection efforts.  To identify these articles, our research librarian conducted a search of  several bibliographic databases, such as ProQuest, Embase, and  Scopus.
To describe the extent to which federal agencies are addressing any  challenges posed by differences across the data collection efforts, we  interviewed senior agency officials and academic experts and obtained  relevant agency documentation.
We conducted this performance audit from March 2015 to July 2016, in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The federal government has implemented a number of initiatives to  address sexual violence or mitigate its effects. For example, the  Department of Justice’s (DOJ) Office on Violence Against Women  (OVW), created in 1995 in order to help implement the Violence Against  Women Act (VAWA), sponsors grant programs for local law enforcement  agencies, prosecutors and judges, health care providers, and other  organizations that assist victims of sexual violence by providing, for  example, forensic medical services in sexual violence cases in rural areas  and specialized counseling services for victims from underserved  populations. Another office within DOJ, the Office for Victims of Crime  (OVC), convened crime victim advocates and experts in 2013 as part of  the Vision 21 Initiative and recommended in their report, among other  things, that federal agencies collaborate and expand the collection and  analysis of data on all forms of criminal victimization. The Department of  Health and Human Services’ (HHS) Family Violence Prevention and  Services Program supports two national resource centers on domestic  violence and special-issue and culturally-specific resource centers. In  addition, HHS’s College Sexual Assault Policy and Prevention Initiative  was launched in 2016 and is intended to provide support for organizations  that are implementing policies and practices at postsecondary schools to  prevent sexual assault on their campuses. The Department of  Education’s Office for Civil Rights (OCR) issued guidance to colleges and  universities, one in 2011 and another in 2014, concerning the  responsibilities of those institutions under Title IX of the Education  Amendments of 1972 with regard to addressing sexual violence against  students. OCR’s guidance sets standards for the grievance procedures  institutions must adopt and publish to promptly and equitably resolve  complaints brought by students alleging sex discrimination (including acts  of sexual violence and sexual harassment), and recommends preventive  education and training programs designed to reduce the occurrence of  sexual violence on campus and improve institutions’ responses to sexual  violence on campus when it does occur. In 2004, Congress passed a law  that required the Secretary of Defense to develop, among other things, a  comprehensive policy for the Department of Defense (DOD) on the  prevention of sexual assaults involving servicemembers. In response to  that statutory requirement, DOD established its sexual-assault prevention  and response program in 2005, and in 2008, DOD published its first  sexual assault prevention strategy.
Several of the federal government’s responses to sexual violence involve  data collection on the occurrence of sexual violence. For example, the  Prison Rape Elimination Act (PREA) of 2003 directed DOJ to carry out  studies of the incidence and effects of prison rape. The Jeanne Clery  Disclosure of Campus Security Policy and Campus Crime Statistics Act,  as amended by VAWA in 2013, requires that all institutions of higher  education that participate in federal student financial assistance programs  disclose statistics on certain crimes, including those related to sexual  violence, to the Department of Education (Education). Since 2005,  National Defense Authorization Acts have directed DOD to, among other  things, collect and report information on sexual assaults against service  members.
Under the Paperwork Reduction Act (PRA), the Office of Information and  Regulatory Affairs (OIRA) in the Office of Management and Budget  (OMB) is charged with improving the efficiency and effectiveness of  federal information resources, which includes functions relating to  statistical policy and coordination. Specifically, with regard to statistics,  OMB’s responsibilities include:
Oversight and approval of data collection: OMB reviews statistical  information collections as part of its responsibility under the PRA to  approve all federal agency information collections that will be  administered to 10 or more people to ensure adherence with PRA  standards for minimizing information collection burdens and  maximizing the practical utility of information collected by federal  agencies, including eliminating unnecessary duplication.
Guidance and standards: OMB develops and oversees  governmentwide policies, principles, standards, and guidelines for  collecting and disseminating statistical information.
Coordination: OMB coordinates the activities of the federal statistical  system, including ensuring the integrity, objectivity, and utility of  federal statistics.
Oversight of budgets: OMB ensures that statistical agencies’ budget  proposals are consistent with systemwide priorities for maintaining  and improving the quality of federal statistics.
Other entities also provide guidance to agencies that conduct statistical  work. For example, the National Academy of Sciences’ Committee on  National Statistics (CNSTAT) publishes Principles and Practices for a  Federal Statistical Agency for newly appointed cabinet secretaries at the  beginning of each presidential administration. Principles and Practices  outlines basic principles for statistical agencies to carry out their missions  effectively, as well as practices designed to help implement them.
Different entities use federal data on sexual violence, including, for  example victim advocacy groups, other special interest groups, and other  federal agencies. Officials at victim advocacy groups we spoke with  publish reports on topics related to sexual violence and lobby Congress  for laws and programs designed to address the needs of victims. Other  groups include law enforcement associations and campus safety groups  that provide training and educational materials for law enforcement and  campus safety personnel. Federal agencies also use data on sexual  violence, for example to inform grant making decisions regarding  research and program development.
section_title: Four Federal Agencies Manage Ten Efforts to Collect Data on Sexual Violence, which Differ in Target Population, Terminology, Measurements, and Methodology
subsections: section_title: Federal Agencies Collect Sexual Violence Data on the General Population and on Segments of the Population
paragraphs: Four federal agencies manage at least 10 data collection efforts that  include data on sexual violence, among other things. Some of these  data collection efforts focus on a target population that the agency serves.  For example, Education’s Clery Act data collection effort obtains  information on the occurrence of sexual violence at institutions of higher  education. DOD’s Defense Sexual Assault Incident Database (DSAID)  and the Workplace and Gender Relations Survey of Active Duty Members  (WGRA) include and collect data on sexual violence involving military  service members. Others, such as the FBI’s Uniform Crime Reporting  Program (UCR) data collection efforts compile data from law enforcement  agencies on the general population. Those data collection efforts that  include information from the general population differ in terms of the ages  of respondents or individuals from whom reports of sexual violence are  taken. For example, the National Intimate Partner and Sexual Violence  Survey (NISVS) collects data from individuals who are 18 and older while  the National Crime Victimization Survey (NCVS) collects data on  household members who are 12 and older; both the Uniform Crime  Reporting Program-Summary Reporting System (UCR-SRS) and the  Uniform Crime Reporting Program-National Incident-Based Reporting  System (UCR-NIBRS) include data from law enforcement agencies on  criminal incidents involving people of all ages. Table 1 includes  information about each of the 10 data collections discussed in this report,  including their respective target populations.
Data collection efforts that are focused on target populations—such as,  the military population, institutions of higher education, and the  incarcerated population—provide information on the problem of sexual  violence within those groups and thus may be helpful for informing policy  affecting those groups. For example, Education officials told us that  Education’s Office of Federal Student Aid administers inquiries to specific  campuses if Clery Act data show unusually high incidences of certain  crimes, including rape. The Bureau of Justice Statistics (BJS) reports that  the National Inmate Survey (NIS) and the Survey of Sexual Victimization  (SSV) data provide helpful information for understanding and addressing  the problem of sexual violence in prisons, jails, and juvenile correctional  facilities. In March 2015, we reported on the importance of using military  data on sexual violence to inform program decisionmaking.
The 5 data collection efforts whose target population is a segment of the  national population are the result of specific congressional mandates,  while the 5 data collection efforts that focus on the general population are  discretionary initiatives arising from broad agency missions. For  example, BJS is mandated under the terms of the Prison Rape  Elimination Act (PREA) of 2003 to collect data on sexual violence in  prisons, jails, and other detention facilities. BJS conducts both the NIS  and SSV in response to that mandate. In contrast, according to HHS’s  Centers for Disease Control and Prevention (CDC) officials, the agency  launched NISVS as part of its public health mission, and with support  from the National Institute of Justice and DOD as a result of requests  from organizations in the field of sexual violence prevention.
The extent to which the data collection efforts focus on sexual violence  also varies. Some of the data collection efforts collect information solely  or primarily on the occurrence of sexual violence, such as the SSV or  DSAID. Other data collection efforts have a larger focus. For example, the  UCR-SRS, UCR-NIBRS, and NCVS include information on a broad  spectrum of crimes, and the National Electronic Injury Surveillance  System–All Injury Program (NEISS-AIP) includes information on a wide  variety of types of injuries.
section_title: Data Collection Efforts Differ in the Terms They Use to Describe Sexual Violence, and Measurements and Definitions for Some Efforts are Inconsistent
paragraphs: Data collection efforts use a range of terms to describe sexual violence in  publicly-available agency documentation. Specifically, the 10 data  collection efforts use a total of 23 different terms to describe sexual  violence. Table 2 shows the terms that data collection efforts use to  describe sexual violence.
Given the variation in terminology, data collection efforts may  characterize the same sex act using different terms. For example,  regarding sexual violence involving vaginal penetration of a victim, 6 data  collection efforts include this act of sexual violence in their measurement  of “rape,” 2 include it in their measurements of “nonconsensual sexual  acts” or “staff sexual misconduct,” 2 include it in their measurements of  “sexual assault” or “assault-sexual,” 1 includes it in its measurement of  “sexual coercion,” 1 includes it in its measurement of “penetrative sexual  assault,” and 1 includes it in its measurement of “sexual assault with an  object.” See tables 5 through 7 in app. II for additional information on  acts of sexual violence included in measurements of sexual violence by  data collection effort.
It is also the case that one data collection effort may use multiple terms to  characterize a particular act of sexual violence, depending on the  contextual factors that may be involved, such as whether the perpetrator  used physical force. For example, if a victim is penetrated vaginally,  NISVS may characterize that particular act as either “rape” or “sexual  coercion,” and the decision as to which term is most appropriate is based  on the contextual factors surrounding the act. As such, NISVS  characterizes vaginal penetration of a victim as “rape” if the act involves  the use of physical force or threats to physically harm the victim. On the  other hand, NISVS characterizes this same act as “sexual coercion” if the  act occurs after the victim is verbally pressured in a nonphysical way, for  example if the perpetrator uses their influence or authority.
Based on our analysis, data collection efforts rarely use the same  terminology to describe sexual violence; however, when they do, there  are some differences in the particular acts of sexual violence and  contextual factors that they include in their measurements of those terms.  For example, 4 of the 6 data collection efforts that use the term “rape”  consider whether actual physical force was used and the other two do  not. Three of the 6 that use the term “rape” consider whether the threat of  physical force was used and the other 3 do not. See tables 8 through 10  in app. II for additional information on contextual factors included in  measurements of sexual violence by data collection effort.
In general, measurements of sexual violence closely relate to definitions  of sexual violence in federal data collection efforts. For 5 of the data  collection efforts we reviewed, the acts of sexual violence and contextual  factors that are included in the measurements generally align with the  acts of sexual violence and contextual factors that are included in the  definitions.
However, for the remaining 5 data collection efforts, some of the acts of  sexual violence or contextual factors are included in both their  measurements and definitions and others are not. Further, these data  collection efforts do not have publicly-available descriptions of what is  included in their respective measurements to allow persons using the  data to understand the differences. Specifically, Clery Act data include  attempted rapes in its measurement of rape, but do not include attempted  rapes in its definition of rape. NEISS-AIP includes acts of sexual violence  involving penetration of a victim with an object and acts of sexual violence  involving a victim being made to penetrate someone else with an object in  its measurement of assault-sexual, but does not explicitly include these  acts of sexual violence in its definition of assault-sexual or in the  description of assault-sexual in the NEISS-AIP coding manual.  Additionally, NCVS includes the contextual factors of “victim unable to  consent or refuse” and “victim alcohol/drug facilitated” in its  measurements of rape and sexual assault, but does not include these  contextual factors in its definitions of rape and sexual assault. Similarly,  SSV includes attempted nonconsensual sexual acts in its measurement  of nonconsensual sexual acts, but does not include attempts in its  definition of nonconsensual acts. NIS includes the act of victim  penetration with an object in its measurements of nonconsensual sexual  acts and staff sexual misconduct, but does not include it in its definitions  of nonconsensual sexual acts and staff sexual misconduct.
The National Academy of Sciences’ Principles and Practices for a Federal  Statistical Agency states that data releases from a statistical program  should include the methods and assumptions used for data collection and  reporting. Similarly, OMB guidelines regarding the federal Information  Quality Act call for agencies that disseminate government information to  ensure its utility, objectivity, and integrity, which refers to information  reproducibility and transparency. Additionally, federal internal control  standards state that an agency’s information requirements should  consider the expectations of both internal and external users and that  reliable internal and external information sources should provide data that  faithfully represent what they purport to represent. Education officials  told us that they are updating The Handbook for Campus Safety and  Security Reporting and expect to issue the updated handbook in summer  2016. This may provide an opportunity for Education to eliminate  discrepancies between the Clery Act’s sexual violence measurements  and definitions. Regarding NEISS-AIP, CDC officials told us that the  definition of “assault-sexual” is intended to include the range of sexual  assault experiences that victims presenting to the emergency department  may report. BJS officials told us that it is not possible to enumerate every  act of sexual violence that is included under NCVS’s terms of sexual  violence and that data users can make their own determinations about  what acts of sexual violence are included in the measurements. BJS  officials also told us that the definitions included in the NIS and SSV  summary reports are intended for the general public, and acknowledged  that the reports present ambiguity regarding which acts of sexual violence  and contextual factors are included in the measurements. CDC and BJS  officials told us that researchers who are interested in descriptions of  what is included in the measurements for NEISS-AIP, NIS, and SSV  could access coding information, which is available at the University of  Michigan’s National Archive of Criminal Justice Data of the Interuniversity  Consortium for Political and Social Research. However, a layperson who  is not a researcher may not know how to access this information.
If data users are seeking to understand what sexual acts and contextual  factors are included in a data collection effort’s measurement of sexual  violence, they may read the definitions of terms contained in reports of  those data collections. If the definitions of the terms are different from  what the data collection effort includes in its measurement of those terms,  data users may lack clarity about what acts of sexual violence and  contextual factors the efforts are including in their measurements of  sexual violence.
section_title: Federal Sexual Violence Data Collection Methodologies Differ by Context, Data Source, Units of Measurement, and Time Frames
subsections: section_title: Context
paragraphs: Federal agencies generally collect data on sexual violence within one of  two contexts—criminal justice or public health. For the purposes of this  report, “criminal justice” describes data collection efforts that refer to acts  of sexual violence as “crimes” or “offenses.” “Public health” describes  data collection efforts that seek to understand the health implications of  acts of sexual violence. Of the 10 data collection efforts within our scope,  7 collect data primarily in a criminal justice context, and 2 collect data  primarily in a public health context, and 1 data collection effort combines  both contexts. Table 11 in app. II outlines which data collection efforts fall  into each category.
According to agency officials, context can determine how each data  collection effort is designed and how the data are collected—and context  may inform what is included in the measurements of sexual violence in  the data collection efforts. The data collection efforts of BJS and FBI  included in this report have a criminal justice focus, intended to collect  information on crimes, victims, or trends. CDC officials stated that their  data collection efforts—NEISS-AIP and NISVS—which both have a public  health focus, are more concerned with assessing the health impacts of  victimization and informing violence prevention efforts and are less  concerned with categorizing incidents as crimes. In some public health  surveys, interviewers begin by asking basic health and lifestyle questions  to establish a rapport with the interviewee and introduce the concepts of  public health and experiences rather than crimes and criminal events.  According to BJS and CDC officials, some studies with a focus on  criminal justice may ask questions about sexual violence as crimes that  have occurred, whereas some studies with a public health focus may ask  questions about violent sexual experiences by describing specific acts of  sexual violence while avoiding criminal terminology. For instance, NISVS  does not use the word rape when questioning interviewees, whereas  NCVS asks respondents if anyone has been attacked for example by  rape, attempted rape, or other type of sexual attack. NISVS program  documentation states that the term rape may carry a stigma or have  different meanings to different people, so the survey poses multiple  questions about behaviorally specific sexual acts, without using the term  rape.
section_title: Data Sources
paragraphs: Federal sexual violence data primarily come from two sources:  information reported to authorities and information obtained from victim  surveys. Some data collection efforts compile information reported to  relevant authorities. Other federal data collection efforts obtain their data  from surveys where agencies attempt to identify victims from a larger  population and invite them to share information about their experiences  with sexual violence. Table 12 in app. II outlines which data collections  fall into each category.
Information reported to authorities may originate from situations in which  a victim or observer reports an alleged act of sexual violence to law  enforcement, campus, or prison authorities or to military officials. Data  collection efforts vary in how and to whom the information is submitted.  For DSAID, restricted and unrestricted reports of sexual violence are  reported by victims to sexual assault response coordinators or victim  advocates who input information on the incident into DSAID. Whereas for  Clery Act data, all institutions of higher education that receive federal  student financial aid are required to report campus security data  (including information on sexual violence) to the Department of  Education.
Some data collection efforts obtain information on sexual violence  through surveys. Each survey uses different methods to collect data from  their subjects. For instance, NISVS employs a random digit dialing  telephone survey while the NCVS uses a mix of face-to-face and  telephone interviewing.
Both types of data sources involve tradeoffs. With respect to information  reported to authorities, according to agency documentation and a senior  official from a law enforcement special interest group, data collection  efforts that provide information on crimes reported to authorities are  useful for administrative and funding decisions related to law  enforcement. For example, the Bureau of Justice Assistance uses UCR- SRS data, in part, to determine how much grant funding should be  awarded to state, local, and tribal governments through the Edward Byrne  Memorial Justice Assistance Grant program, which made $255.7 million  in funding available to states, territories, and localities in fiscal year 2015.  However, one limitation is that these data may underestimate the scope  of the problem, since acts of sexual violence are historically  underreported to authorities.
As such, obtaining information through a survey may identify more  instances of sexual violence than efforts that rely on information reported  to authorities. However, surveys also have their limitations. Surveys are  subject to variable response rates over time, and different surveys may  have different response rates, which may affect the resulting estimates  and the validity of the data. For example, response rates of the data  collection efforts included in our review range from 24 percent for WGRA  in 2012 to 33 percent for NISVS in 2011 to 84 percent for NCVS in  2014. Survey results may also be subject to response biases, for  example the tendency for a respondent to provide untruthful but socially  acceptable responses or the tendency for individuals who either have or  have not experienced the action (e.g., sexual violence) to not participate  in the survey (which can lead to nonresponse bias). Also, according to  BJS officials, obtaining information directly from victims creates a burden  on survey respondents and interview subjects, and research and officials  from an entity that uses federal data on sexual violence stated that such  surveys may face difficulties in getting people to discuss victimization  experiences with strangers during interviews. Furthermore, administrative  costs associated with surveys and interviews can affect the practical  frequency of data collection.
section_title: Units of Measurement
paragraphs: Different federal data collection efforts measure and report different  aspects of the occurrence of sexual violence. Some data collection efforts  report the number of incidents that involved an act of sexual violence,  some report the number of unique victims of sexual violence, and some  report information about the number of times an act of sexual violence  occurred. On the surface it may appear that the number of incidents that  involve an act of sexual violence and the number of times an act of sexual  violence occurred are synonymous, but that is not necessarily the case.  Multiple offenses could occur in the same incident, and one incident could  involve multiple victims. For example, a perpetrator could both rob and  sexually assault someone in the same incident, or a perpetrator could  carry out sexually violent acts against multiple victims in the same  incident.
Table 3 outlines the units of measurement that each data collection effort  reports.
The 4 data collection efforts that report the number of times an act of  sexual violence occurred—NCVS, NIS, UCR-SRS, and UCR-NIBRS— may not record the exact number of acts of sexual violence from each  individual incident in some cases. These data collection efforts count one  act of sexual violence per victim per incident, meaning they would capture  acts against multiple victims in a single incident but not multiple acts  against each individual victim.
The data source each federal data collection effort uses is an important  determinant of the units of measurement that each study uses in reporting  its results. Data collection efforts that use information reported to  authorities generally measure the number of incidents or reports that  involve sexual violence. Those efforts may or may not publish a count of  the number of acts of sexual violence that occurred. Such efforts are  generally not set up to count the number of unique victims across  incidents; they may report a count of victims, but have no mechanism to  avoid the double-counting of victims who experience multiple incidents  and are thus not measuring the same quantity that surveys that seek to  report a number of unique victims measure. By contrast, data collection  efforts that use information obtained from victim surveys generally  measure and report the number of victims rather than the number of  incidents that involved sexual violence. These efforts also collect data on  the number of separate times each individual respondent has been a  victim of sexual violence. However, the agencies operating those studies  are cognizant of the challenges associated with asking respondents about  multiple incidents, particularly with respect to the respondent’s ability to  accurately recall multiple experiences of sexual violence. For example, a  DOD official stated that WGRA asks questions about how many times a  crime has been committed against the respondent, but DOD does not  report a total number of times that an act of sexual violence occurred  because respondents’ recall of multiple events may be subject to memory  biases and confirming details of each act may be overly burdensome in a  survey.
section_title: Time Frames
paragraphs: Federal agencies collect sexual violence data for different periods of time  and report the data at different frequencies. Table 13 in app. II outlines  the time frames for each federal data collection effort.
The data collection efforts cover different intervals of time for which an act  of sexual violence occurred. For instance, NISVS asks whether each  respondent has experienced sexual violence during the previous 12  months and during the respondent’s lifetime. NCVS asks whether each  respondent has experienced sexual violence during the previous 6  months. By contrast, the data collection efforts that compile reports from  authorities may capture information at the point in time when the event  was reported to those authorities.
Some data collection efforts release their results annually, whereas  others do so less often. For example, NISVS collected data annually  except in 2014 and issues reports periodically. NISVS most recently  reported results to the public in 2014 using data collected in 2011. Most  data collection efforts that compile reports from authorities release their  results annually.
One expert we interviewed stated that the period of time for which data on  sexual violence are collected and how often data are publicly reported  affects each data collection effort’s results regarding the occurrence of  sexual violence, an observation also found in academic literature and a  nongovernmental report. For example, data on lifetime experience of  sexual violence may yield larger numbers of rape and sexual assault than  data on experiences of sexual violence in the last 6 or 12 months.  According to officials from 2 entities that use federal data on sexual  violence, data that are reported annually may be more useful for trend  analysis than data reported less frequently.
section_title: Differences in Data Collection Efforts May Hinder the Understanding of the Occurrence of Sexual Violence; Agencies Have Taken Steps to Explain and Lessen Differences, but Efforts Have Been Fragmented and Limited in Scope
paragraphs: The differences across the data collection efforts may hinder  understanding of the extent of sexual violence, and agencies have taken  steps to clarify differences and harmonize the data collection efforts.  However, these efforts have been fragmented and more could be done to  increase understanding of the problem of sexual violence.
subsections: section_title: Differences in Data Collection Efforts Collectively May Hinder Understanding of the Extent of Sexual Violence
paragraphs: Collectively, the differences across federal data collection efforts lead to  differing estimates of sexual violence, for example rape, in the United  States, as shown in selected data collection efforts on the general  population in table 4.
According to research and individuals we spoke with who are familiar with  the data, differences in federal data on sexual violence may confuse the  public. A National Academy of Sciences’ Panel that studied the  incidence of rape in the United States reported in 2014 that the data  collection efforts’ different purposes and methodologies produce different  results, which creates confusion for the public, law enforcement,  policymakers, researchers, and victim advocacy groups. Additionally,  officials from four entities that use federal data told us that they believe  the public does not understand data from federal sources on sexual  violence. Officials from three entities that use federal data on sexual  violence stated that they—and the media—cite a range of sources and  may not always, or adequately, explain the details of the data collection  efforts. In addition, the public may not take the time to understand the  differences among the data collection efforts. For example, an official  from one entity told us that the entity frequently uses the results from one  particular data collection effort to educate the public on sexual violence,  but the official was not aware of certain methodological details and  limitations of the data. Also, as previously discussed, some data  collection efforts’ measurements and definitions do not align and  information on what is included in these measurements is not publicly  available, which may lead to confusion for data users. Further, officials  from the federal agencies and entities we spoke with that use federal data  on sexual violence emphasized that the differences across the data  collection efforts are such that the results are not comparable.
Officials we spoke with who use the data stated that differences in  measurements, definitions, and methodology across the data collection  efforts can lead to confusion. Officials at one entity stated that they found  challenges in using federal data on sexual violence because varying  measurements and definitions across the data collection efforts make it  difficult to compare data. However, even in instances where the acts of  sexual violence and contextual factors that are included in measurements  and definitions are similar across data collection efforts, other differences  create challenges. For example, officials at the National Center for  Campus Public Safety stated that whereas the Clery Act data program  and UCR-SRS use the same definition for rape, the methodologies are  different—the Clery Act data program collects information on allegations  made in “good faith,” whereas the UCR-SRS includes only information on  incidents resulting in a police report—which results in different estimates  of rape and may lead to confusion for users who try to compare the data.
Because there is wide variation in the results, entities that use federal  data on sexual violence have a choice of which data to use, and entities  reported using data that best suited their needs. For example, officials  from one entity told us that they use NCVS data because it includes  information on incidents not reported to the police and is user-friendly,  and officials from another entity told us they use NCVS data because it  has a larger sample size than other data collection efforts. Officials from  another entity stated that they use NISVS because it includes the most  expansive set of acts of sexual violence and contextual factors in its  measurement of “rape” and estimates lifetime prevalence rates.
section_title: Agencies Have Taken Some Steps to Clarify Differences across Sexual Violence Data Collection Efforts, but Efforts to Harmonize the Data Collection Efforts Have Been Fragmented
paragraphs: Federal agencies have acknowledged that differences exist among data  collection efforts, for example in terms of methodology, context, and data  sources, which has led some agencies to take steps to identify and  explain differences across the data collection efforts. In addition, some  federal agencies have taken steps to lessen the differences among data  collection efforts by focusing on harmonization—that is, coordination of  practices to enhance data collection to achieve a shared goal. However,  such efforts have been fragmented—that is, they are limited in scope and  generally involve two data collection efforts at a time.
Two ongoing efforts are intended to clarify the differences across two  data collection efforts:
BJS and FBI coauthored a statement describing the differences  between UCR and NCVS. In 1995, BJS and FBI coauthored  statements entitled “The Nation’s Two Crime Measures,” which  describe similarities and differences between the UCR program and  NCVS. The statement was updated in September 2014. BJS and FBI  post similar but different versions of the statement on their websites.  BJS’s statement provides a side-by-side description of FBI’s UCR  program and BJS’s NCVS including, for example, information on  historical background, data sources, and time frames for data  collection and reporting. Both statements also include a section on  comparing UCR and NCVS data, which describes the data collection  efforts’ similarities (e.g., they both have somewhat similar subsets of  serious crimes, such as rape, robbery, aggravated assault, burglary,  theft, and motor vehicle theft ) and key differences (e.g., definitions of  certain crimes). The statements conclude with a description of the two  data collection efforts’ strengths (e.g., UCR provides data on the  number of crimes reported to law enforcement and NCVS provides  data on the number and types of crimes not reported to law  enforcement).
CDC and BJS have discussed publishing a statement that  compares sexual violence statistics in NISVS and NCVS. At a  meeting in November 2015, CDC and BJS discussed coauthoring a  statement about NISVS and NCVS that would describe the  differences and similarities of the two data collection efforts.
There are five efforts underway or recently implemented that are intended  to increase harmonization across the data collection efforts, including:
Education adopted FBI’s UCR-SRS definition of rape for use in  the Clery Act data. Education, in its 2014 rule implementing the  VAWA 2013 reauthorization, changed the definition of rape that is  used in the Clery Act data to match UCR-SRS’ definition. The  definition used by both UCR-SRS and the Clery Act data is  “Penetration, no matter how slight, of the vagina or anus with any  body part or object or oral penetration by a sex organ of another  person, without the consent of the victim.” Education began collecting  data using the new definition of rape for calendar year 2014.
BJS sponsored the National Academy of Sciences’ CNSTAT  Panel on Estimating the Incidence of Rape and Sexual Assault in  BJS Household Surveys. In March 2011, BJS charged the panel to  “assess the quality and relevance of statistics on rape and sexual  assault from NCVS and other surveys contracted for by other federal  agencies as well as surveys conducted by private organizations,”  examining issues such as the “legal definitions in use by the states for  these crimes, best methods for representing the definitions in survey  instruments so that their meaning is clear to respondents, and best  methods for obtaining as complete reporting as possible of these  crimes in surveys, including methods whereby respondents may  report anonymously.” The panel, which was comprised of experts in  research and sexual assault response, held five in-person meetings  and issued 15 recommendations in a final report published in 2014.  For example, the panel recommended that BJS’s definitions of sexual  violence be expanded to include victimizations when the victim does  not have the capacity to consent to the sexual actions of the offender  and that this research be conducted in a coordinated manner because  many of the issues to be investigated are interrelated. The panel also  recommended that the survey questionnaire should have a neutral  context, such as a health survey. BJS officials told us that some of  their current work addresses some of the panel’s recommendations.  For example, BJS is currently conducting a methodological  comparison of NCVS with a public health approach that includes a  five-city comparison study and consultation with CDC. The five-city  comparison study involves an expanded scope of sexual violence,  which includes questions regarding consent, and BJS is testing a  range of behavior-specific questions. BJS officials told us that they are  planning to issue a report on the progress of the project in spring  2016.
BJS and FBI commissioned a National Academy of Sciences’  CNSTAT Panel on Modernizing the Nation’s Crime Statistics.  Commissioned in 2013, the panel will assess and make  recommendations for the development of a modern set of crime  measures in the United States and the best means for obtaining them.  The review will focus, among other things, on full and accurate  measurement of criminal victimization events and their attributes,  considering types of crime (and their definitions), including the current  scope of crime types covered by existing FBI and BJS data  collections; gaps in knowledge of contemporary crime; development  of international crime classification frameworks that should be  considered in increasing international comparability; and the optimal  scope of crime statistics to serve the needs of the full array of data  users and stakeholders—-federal agencies, other law enforcement  agencies, Congress, other actors in the justice system (such as the  courts and corrections officials), researchers, and the general public.  Panel membership includes academics in the field of criminal justice  and statistics and stakeholders who use and provide the data that the  government collects. According to the chair of the panel, as part of the  first phase of work, the panel has developed an initial  conceptualization for classifying all types of crime, including rape and  sexual assault. The recommended classification and its justification  appear in the panel’s first report, which was released in May  2016. The panel has also begun the second phase of its work, which  will suggest the means for gathering data for the comprehensive  crime classification, including information from non-BJS or FBI  sources, and recommend how crime data collection should proceed in  practice. The panel plans to consider possible coordination among  agencies to produce more comprehensive reports on data, instead of  one agency producing one report and another agency producing a  separate but related report. The panel intends to finish its second  phase of work in 2016 and issue a final report in early 2017.
CDC, partnering with BJS, plans to convene a Technical Expert  Panel to examine ways to improve NISVS. As part of OMB’s review  of a CDC information collection proposal, OMB requested that CDC  and BJS officials convene a panel of experts in survey methods to  improve NISVS’s methodology, including increasing the response rate  and minimizing non-response bias. The agencies identified and  recruited a panel of experts and plan to meet in spring 2016.
CDC provided DOD with an adapted dataset from NISVS. In 2010,  NISVS included two random samples of active duty women and wives  of active duty men in addition to a random sample of the general U.S.  population. Using identical survey methods, data were collected in the  first two quarters of 2010. According to DOD officials, CDC provided a  subset of NISVS data to DOD so that DOD received information on  crimes that fell under military law to enable DOD to do an “apples to  apples” comparison of CDC and DOD data. CDC officials informed us  that they are working on a follow-up military population study in 2016.
However, these various federal efforts to clarify and harmonize sexual  violence data have been fragmented. CNSTAT’s Principles and Practices  for a Federal Statistical Agency (2013) calls for federal agencies that  produce similar federal statistics, with different missions, to coordinate  and collaborate with each other to meet current information needs and  provide new or more useful data than a single system can provide. While  the guidance applies primarily to the 13 federal statistical agencies, OMB  officials stated that the report provides best practices for all federal data  collection activities. The guidance encourages collaborative interagency  efforts and highlights the importance of agencies developing standard  definitions as a way to maximize the value and comparability of data.
However, the coordination that has occurred across the agencies that  collect data on sexual violence has been limited. Specifically, coordination  has been bilateral—generally involving only 2 of the 10 data collection  efforts at a time and limited in scope. Agency officials expressed  skepticism that broader harmonization efforts could benefit federal data  on sexual violence, stating that each data collection effort is designed— through target population, measurements and definitions, and  methodology—to fulfill a certain purpose. Agency officials told us that  changes to any of these differences may undermine the specific purpose  of each data collection effort. However, harmonization does not  necessarily entail making the data collection efforts identical; instead, it  could entail agencies considering how they could make their efforts more  complementary and, as appropriate, more alike without compromising  their programmatic needs.
The Paperwork Reduction Act (PRA), among other things, established a  process for OMB to oversee agency information collection efforts in order  to improve the quality and use of federal information while reducing  collection burdens, including through the coordination of federal  statistics. Per its authority under the PRA, OMB has convened  interagency working groups to assess differences across data collection  efforts and determine which of those differences are beneficial and which  are unnecessary. For example, OMB has convened the following  interagency groups:
The Interagency Working Group for Research on Race and Ethnicity  was formed in 2014 to exchange research findings, identify  implementation issues, and collaborate on a shared research agenda  to improve federal statistics on race and ethnicity.
The Interagency Working Group on Measuring Relationships in  Federal Household Surveys, which was established in 2010,  convenes representatives from a variety of federal agencies involved  in the collection, dissemination, or use of household relationship data  to address the challenges in measuring household relationships,  including same sex couples.
The Federal Interagency Forum on Child and Family Statistics was  formally established in 1997 to develop priorities for collecting  enhanced data on children and youth, improve the reporting and  dissemination of information on the status of children to the policy  community and the general public, and produce more complete data  on children at the state and local levels.
We asked OMB if there are plans to convene a similar group for  harmonizing data on sexual violence. OMB staff stated that they did not  have plans to form an interagency group on the topic, but instead they  plan to invest limited resources strategically by engaging with BJS on its  redesign of NCVS and with CDC on its information quality of NISVS.  OMB staff also stated that their plans are to ensure that both agencies  are taking advantage of the insights gained as each agency undergoes  redesign and technical consultations in the next couple of years.  However, other data collection efforts, in addition to NCVS and NISVS,  also influence policy decisions on sexual violence. Depending upon the  outcome of the work being conducted by BJS and CDC, OMB may  encourage other data collection efforts to adapt or adopt insights gained  as appropriate to their respective programmatic missions. In the absence  of broader harmonization efforts, agency sexual violence data continue to  be inconsistent and incomparable, leading to confusion about the data  and lack of clarity about the scope of the problem of sexual violence in  the United States.
section_title: Conclusions
paragraphs: Differences in data collection efforts—particularly in terms of what is  included in measurements and definitions of sexual violence and  methodologies—collectively can lead to confusion. Without publicly- available information on which acts of sexual violence and contextual  factors are included in the measurements of sexual violence, data users  may lack clarity about what each data collection effort’s results represent.  Additionally, entities that use federal data may misunderstand the data  and develop policies that may not be based on the full extent of the  problem. In the absence of collaboration among agencies that manage  data collection efforts, it is unclear which differences enhance and which  impair the overall understanding of sexual violence, and as a result, policy  makers and the public lack coordinated information by which to address  the problem.
section_title: Recommendations for Executive Action
paragraphs: To enhance the clarity and transparency of sexual violence data that is  reported to the public, we recommend that the Secretary of Education  direct the Assistant Secretary for the Office of Postsecondary Education,  the Secretary of Health and Human Services direct the Director of CDC,  and the Attorney General direct the Director of BJS to make information  on the acts of sexual violence and contextual factors that are included in  their measurements of sexual violence publicly available. This effort could  entail revising their definitions of key terms used to describe sexual  violence so that the definitions match the measurements of sexual  violence.
To help lessen confusion among the public and policy makers regarding  federal data on sexual violence, we recommend that the Director of OMB  establish a federal interagency forum on sexual violence statistics. The  forum should consider the broad range of differences across the data  collection efforts to assess which differences enhance or hinder the  overall understanding of sexual violence in the United States.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a copy of our report to DOD, Education, HHS, DOJ and  OMB for their review and comment. The agencies provided technical  comments, which we incorporated as appropriate. Education, HHS, and  DOJ also provided written comments, which are reprinted in appendices  IV, V, and VI, respectively. The OMB liaison to GAO provided us with  comments via email, which are summarized below.
DOJ, Education, and HHS agreed with our recommendation that, in order  to enhance the clarity and transparency of sexual violence data, they  should make information on the acts of sexual violence and contextual  factors that are included in their measurements of sexual violence publicly  available. In their written comments, DOJ and Education described  actions they have recently taken or plan to take to implement the  recommendation. DOJ stated that beginning in calendar year 2017, BJS  will provide the exact computer code used to construct its measures of  sexual violence as well as additional information on how sexual violence  is defined and measured. Education stated that in June 2016, the  department released an updated version of The Handbook for Campus  Safety and Security Reporting, which provides additional details on what  acts of sexual violence and contextual factors are included in the data  collection effort’s sexual violence measurements. HHS stated that the  department is committed to improving the quality of the data and the  clarity of the descriptions and definitions of sexual violence.
In an email responding to our recommendation that OMB establish a  federal interagency forum on sexual violence statistics, OMB stated that it  did not believe convening a forum at this time was the most strategic use  of resources. OMB stated that other interagency groups it has convened  were typically about statistical methods or measurement issues that  would affect a wide swath of government and for which OMB guidance or  a best practice working paper would be forthcoming. OMB noted that  there are only four agencies involved in collecting sexual violence data,  and regarded none to be conducting or far enough along in its research  for OMB to develop guidance or identify best practices at this time. OMB  does, however, plan to follow closely and participate in CDC’s and BJS’s  ongoing technical work, and will consider convening or sharing  information across agencies when that work is further along.  We  understand the importance of allowing time for a data collection effort to  mature before providing guidance or best practices. However, considering  that 7 of the 10 data collection efforts have been in place for more than 10  years, and several have been in place for multiple decades, we disagree  with OMB’s assertion that none of the data collection efforts are far  enough along for OMB to provide guidance and best practices. DOJ and  Education also commented on the recommendation to OMB. DOJ stated  that BJS welcomes OMB efforts to coordinate data collection and  reporting on sexual violence and stands ready to participate in an  interagency forum. Education stated that efforts to “harmonize” definitions  should not be pursued solely to achieve symmetry for its own sake, but  that the focus should be on the needs of each individual program. We  agree that the data collection efforts should continue to meet the needs of  individual agencies; however, considering the number of federal data  collection efforts, the range of differences across them, and the potential  for causing confusion, it would be beneficial for agencies to discuss these  differences and determine whether they are, in fact, necessary.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the Director  of OMB, the Attorney General, the Secretaries of Defense, Education,  and Health and Human Services, and other interested parties. The report  will also be available at no charge on the GAO web site at  http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-8777 or goodwing@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found   on the last page of this report. GAO staff who made key contributions to  this report are listed in appendix VII.
section_title: Appendix I: Objectives, Scope and Methodology
paragraphs: Our objectives for this report were to address the following questions:  (1) What are the federal efforts underway to collect data on sexual  violence, and how, if at all, do these efforts differ?  (2) How do any differences across the data collection efforts affect the  understanding of sexual violence, and to what extent are federal agencies  addressing any challenges posed by the differences?
To address the first question, we identified federal efforts to collect data  on sexual violence, for which the data: provided information on the extent to which acts of sexual violence  occur in the United States in a particular year (for example, the  number of times a rape or sexual assault has occurred or the number  of victims of rape and sexual assault); were collected recently (i.e., 2010 or after); were collected periodically (i.e., at least once every 2 years); were reported publicly; and were not focused primarily on minors.
To identify efforts that met these criteria, we reviewed past GAO reports  and a Federal Bureau of Investigation (FBI) list of federal agencies that  may collect crime data. We asked officials at those agencies if they had  any data collection efforts that met our criteria. Additionally, we asked  experts in the field (for example, academic researchers) and officials from  victim advocacy groups and other special interest groups about any  additional federal data collection efforts they were aware of that met our  selection criteria. We initially identified experts and entities that use  federal data on sexual violence by conducting background research.  Then, we interviewed experts and officials from the entities identified in  our research, and from these contacts we identified additional entities that  use federal data on sexual violence. In all, we spoke with officials from  three victim advocacy groups, five other special interest groups (for  example, law enforcement associations and campus safety groups), three  other federal agencies, and two academic experts. Based on this work,  we identified 10 data collection efforts that met our criteria across four  federal agencies, including the Departments of Defense, Education,  Health and Human Services, and Justice.
To identify and describe differences across the data collection efforts, we  obtained information on the purpose, scope, and methodology of each  data collection effort. We obtained this information through a review of  documents, such as user manuals and program descriptions. We also  conducted interviews with senior agency officials, senior officials at  entities that use federal data on sexual violence, and academic experts.  Using documentary and testimonial information, we compared the  similarities and differences of the data collection efforts with respect to  target population; context in which data were collected; source of the  data; unit of measurement; time frames (for example, when data are  collected and how often data are reported by the federal agency); and  terminology and measurements of sexual violence. To compare  similarities and differences of terminology of sexual violence, we used  agency documents to identify for each data collection effort the terms  used to describe sexual violence. To identify for each data collection  effort what acts of sexual violence and contextual factors are included in  measurements of sexual violence, we reviewed agency documentation  and interviewed agency officials.
To identify how the differences affect understanding of sexual violence,  we obtained and reviewed federal reports and interviewed and reviewed  relevant documentation from agency officials, experts, and officials from  entities that use federal data on sexual violence. We asked these officials  and experts whether, in their experience, any of the differences made it  difficult for people who may use the data (e.g., Congress, policy makers,  academics, the general public) to understand the extent to which sexual  violence occurs in the United States. We also asked them to identify any  difficulties or challenges, of which they were aware, that have resulted  from the differences across federal efforts to collect data on sexual  violence. Because these officials and experts were not selected as a  representative sample, the information obtained from these interviews  applies solely to this set of officials and experts, and cannot be  generalized to others.
We also reviewed articles, conference papers, and government and  nongovernment reports that discuss differences across federal sexual  violence data collection efforts. To identify articles, a research librarian  conducted a search of several bibliographic databases, such as  ProQuest, Embase, and Scopus, using terms such as “rape data” or  “sexual assault statistics,” among others. The search looked for peer- reviewed articles, books, and conference papers published during or after  2005. This search yielded 36 publications, 16 of which were relevant to  our research objective on the impact of the differences across the data  collection efforts and 20 which were not. In reviewing the identified  publications, we found and reviewed an additional 9 articles and reports  that were pertinent. See app. III for a list of articles and reports that we  reviewed and determined to be relevant for our analysis. The librarian- assisted literature search was conducted in August 2015, and we  reviewed literature from that search and identified additional sources from  August 2015 to May 2016.
To describe the extent to which federal agencies are addressing any  challenges posed by differences across the data collection efforts, we  interviewed senior agency officials and academic experts and obtained  relevant documentation. We asked agency officials about what, if any,  steps their agency has taken, or planned to take, to address some of the  difficulties or challenges that may have resulted from the differences  across the data collection efforts. We also asked agency officials as well  as officials from entities that use federal data on sexual violence if they  were aware of any additional steps being taken by other federal agencies,  state agencies, or other national entities, etc., to address some of the  difficulties or challenges that may have resulted from the differences  across the data collection efforts.
We conducted this performance audit from March 2015 to July 2016, in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Descriptive Tables of Federal Data Collection Efforts
paragraphs: Tables 5 through 10 provide information on the acts of sexual violence and the contextual  factors included in the measurements of sexual violence by federal data collection efforts  identified by GAO.
section_title: Appendix III: List of Relevant Articles and Reports
paragraphs: National Academy of Sciences, Committee on National Statistics,  Estimating the Incidence of Rape and Sexual Assault. Washington, DC,  US: National Academies Press, Washington, DC, 2014.  . “Overcoming Challenges Related to Data Collection and  Measurement.” Forced Migration Review, vol. 27, no. Jan (2007): 28-29,.
Addington, L. and C. Rennison. “Rape Co-Occurrence: Do Additional  Crimes Affect Victim Reporting and Police Clearance of Rape?” Journal  of Quantitative Criminology, vol. 24, no. 2 (2008): 205-226.
Bachman, R. “Measuring Rape and Sexual Assault: Successive  Approximations to Consensus.” A paper commissioned for the National  Academy of Sciences, June 6, 2012.
Campbell, R, A. E. Adams, and D. Patterson. “Methodological Challenges  of Collecting Evaluation Data from Traumatized Clients/Consumers - A  Comparison of Three Methods.” American Journal of Evaluation, vol. 29,  no. 3 (2008): 369-381.
Chen, Y. and S. E. Ullman. “Women’s Reporting of Sexual and Physical  Assaults to Police in the National Violence Against Women Survey.”  Violence Against Women, vol. 16, no. 3 (2010): 262-79.
Clay-Warner, J. and J. McMahon-Howard. “Rape Reporting: ‘Classic  Rape’ and the Behavior of Law.” Violence and Victims, vol. 24, no. 6  (2009): 723-743.
Cohen, M.A. and A. R. Piquero. “New Evidence on the Monetary Value of  Saving a High Risk Youth,” Journal of Quantitative Criminology, vol. 25  (2008).
Cohn, A. M., H. M. Zinzow, H. S. Resnick, and D. G. Kilpatrick.  “Correlates of Reasons for Not Reporting Rape to Police: Results from a  National Telephone Household Probability Sample of Women with  Forcible Or Drug-Or-Alcohol Facilitated/Incapacitated Rape.” Journal of  Interpersonal Violence, vol. 28, no. 3 (2013): 455-473.
Cook, S. L., C. A. Gidycz, M. P. Koss, and M. Murphy. “Emerging Issues  in the Measurement of Rape Victimization.” Violence Against Women, vol.  17, no. 2 (2011): 201-18.
Delisi, M., A. Koloski, M. Sweeny, E. Hachmeister, M. Moore, A. Drury.  “Murder by Numbers: Monetary Costs Imposed by a Sample of Homicide  Offenders,” The Journal of Forensic Psychiatry & Psychology, vol.21, no.  4 (2010).
Fang, H., M. T. French, and K. E. McCollister. “The Cost of Crime to  Society: New Crime-Specific Estimates for Policy and Program  Evaluation.” Drug Alcohol Dependence, vol. 108, no.1-2 (2010).
Du Mont, J., K. Miller, and T.L. Myhr, “The Role of “Real Rape” and “Real  Victim” Stereotypes in the Police Reporting Practices of Sexually  Assaulted Women.” Violence Against Women, vol. 9, no. 466 (2003).
Fisher, B. S. “The Effects of Survey Question Wording on Rape  Estimates: Evidence from a Quasi-Experimental Design.” Violence  Against Women, vol. 15, no. 2 (2009): 133-47.
Gardella, J. H., C. A. Nichols-Hadeed, J. M. Mastrocinque, J. T. Stone, C.  A. Coates, C. J. Sly, and C. Cerulli. “Beyond Clery Act Statistics: A Closer  Look at College Victimization Based on Self-Report Data.” Journal of  Interpersonal Violence, vol. 30, no. 4 (2015).
Lynch, J. P. “Clarifying Divergent Estimates of Rape from Two National  Surveys.” The Public Opinion Quarterly, vol. 60, no. 3 (1996) 410-430.
Lynch, J. P. and J. L. Lauritsen. “Modernizing the Nation’s Crime  Statistics.” The Criminologist, vol. 40, no. 2 (2015).
Miller, T.R., M. A. Cohen, and B. Wiersema, “Victim Costs and  Consequences: A New Look.” A final summary report presented to the  National Institute of Justice, January 1996.
Murphy, S. B., K. M. Edwards, S. Bennett, S. J. Bibeau, and J. Sichelstiel.  “Police Reporting Practices for Sexual Assault Cases in which ‘the Victim  does Not Wish to Pursue Charges’.” Journal of Interpersonal Violence,  vol. 29, no. 1 (2014): 144-156.
Palermo, T. and A. Peterman. “Undercounting, Overcounting and the  Longevity of Flawed Estimates: Statistics on Sexual Violence in Conflict.”  Bulletin of the World Health Organization, vol. 89, no. 12 (2011): 924-925.
Saltzman, L. E., K. C. Basile, R. R. Mahendra, M. Steenkamp, E. Ingram,  and R. Ikeda. “National Estimates of Sexual Violence Treated in  Emergency Departments.” Annals of Emergency Medicine, vol. 49, no. 2  (2007): 210-7.
Simon, T. R., M. Kresnow, and R. M. Bossarte. “Self-Reports of Violent  Victimization among U.S. Adults.” Violence and Victims, vol. 23, no. 6  (2008): 711-726.
Violence Against Women: A Statistical Overview, Challenges and Gaps in  Data Collection and Methodology and Approaches for Overcoming Them.  A report of the expert group meeting organized by the UN Division for the  Advancement of Women in collaboration with the Economic Commission  for Europe and World Health Organization, April 11-14, 2005.
Weiss, K. G. “‘You Just Don’t Report that Kind of Stuff’: Investigating  Teens’ Ambivalence Toward Peer-Perpetrated, Unwanted Sexual  Incidents.” Violence and Victims, vol. 28, no. 2 (2013): 288-302.
Wolitzky-Taylor, K. B., H. S. Resnick, J. L. McCauley, A. B. Amstadter, D.  G. Kilpatrick, and K. J. Ruggiero. “Is Reporting of Rape on the Rise? A  Comparison of Women with Reported Versus Unreported Rape  Experiences in the National Women’s Study-Replication.” Journal of  Interpersonal Violence, vol. 26, no. 4 (2011): 807-832.
section_title: Appendix IV: Comments from the Department of Education
section_title: Appendix V: Comments from the Department of Health and Human Services
section_title: Appendix VI: Comments from the Department of Justice
section_title: Appendix VII: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, individuals making key  contributions to this report were Kristy Love, Assistant Director; Meghan  Squires, Analyst-in-Charge; Tim Young; Kirsten Leikem; Janelle House;  David Alexander; and David Plocher. Diana Maurer, Tom Jessor, Janet  Temko-Blinder, Tovah Rom and Eric Hauswirth also provided valuable  assistance.