Title IX Data Collection: 

Technical Manual for 

Developing the User’s Guide

Alan F. Karr and Ashish P. Sanil 

Technical Report Number 150 

February 2005 

National Institute of Statistical Sciences 

19 T. W. Alexander Drive 

PO Box 14006 

Research Triangle Park, NC 27709-4006 

www.niss.org 
ii

Contents

Introduction......................................................................................................................... 1 

Chapter 1: Background ....................................................................................................... 3 

1.1 Introduction to Title IX............................................................................................. 3 

1.2 The Three-Part Test .................................................................................................. 3 

1.3 The Title IX Commission and the Assistant Secretary’s Letter ............................... 4 

Chapter 2: Analysis of OCR Cases..................................................................................... 5 

2.1 Summary of the Data ................................................................................................ 5 

2.2 Results....................................................................................................................... 7 

2.2.1 Comparisons Between OCR and COMPARISON ............................................ 7 

2.2.2 Comparisons of PART 3 to NON-PART 3...................................................... 13 

2.2.3 Comparisons of SURVEY to NO SURVEY ................................................... 17 

2.2.4 Comparisons of COMPLAINT to COMPLIANCE MONITORING.............. 18 

2.3 Time Effects in the Use of Part 3 and Non-Part 3 .................................................. 19 

Chapter 3: The Data Collection Instruments .................................................................... 22 

3.1 Characteristics of the Data Collections and Instruments ........................................ 25

3.2 Notable Items.......................................................................................................... 33 

3.3 Generic Issues ......................................................................................................... 37 

Chapter 4: Other Data Collection Instruments.................................................................. 40 

4.1 Characteristics of the Instruments........................................................................... 40 

4.2 Notable Items.......................................................................................................... 45 

Chapter 5: Implementation of Part 3 Data Collections..................................................... 46 

5.1 Problem Formulation .............................................................................................. 46 

5.2 Process Specification .............................................................................................. 48 

5.3 Data Collection Process .......................................................................................... 52 

5.4 Web-Based Data Collection.................................................................................... 54 

5.4.1 Data Collection Instrument.............................................................................. 54 

5.4.2 Software ........................................................................................................... 65 

5.5 Data Analysis.......................................................................................................... 66 

5.5.1 Preprocessing ................................................................................................... 67 

5.5.2 Data Analysis in the Absence of Non-Response ............................................. 68 

5.5.3 Data Analysis in the Presence of Non-Response............................................. 71 

5.6 Precautions.............................................................................................................. 73

5.7 Pre- and Post-Data Collection Procedures.............................................................. 73 

Acknowledgments............................................................................................................. 76 

References......................................................................................................................... 78 

Appendix A: List of OCR Populations ............................................................................. 80 

Appendix B: The Institutional Characteristics.................................................................. 84 

Appendix C: Complete Set of Institutional Comparisons................................................. 88 

C.1 Comparisons of OCR to COMPARISON.............................................................. 88 

C.2 Comparisons of PART 3 to NON-PART 3............................................................ 94 

C.3 Comparisons of SURVEY to NO SURVEY........................................................ 101 

C.4 Comparisons of COMPLAINT to COMPLIANCE MONITORING .................. 107 

Appendix D: Data Collection Classification Characteristics.......................................... 114 
iii

Appendix E: Data Collection Instruments in Chapter 4 ................................................. 116 

Appendix F: Complete Analysis in the Presence of Non-Response............................... 118 
iv

List of Figures and Tables 

Figure 2.1: The populations of institutions appearing in this report................................... 6 

Table 2.1: Results of statistical tests comparing OCR to COMPARISON. ....................... 9 

Table 2.2: Tabular summary comparing OCR to COMPARISON for the characteristic 

Sector......................................................................................................................... 9 

Table 2.3: Tabular summary comparing OCR to COMPARISON for the characteristic 

Region...................................................................................................................... 10 

Table 2.4: Tabular summary comparing OCR to COMPARISON for the characteristic 

Carnegie Classification.......................................................................................... 10 

Table 2.5: Tabular summary comparing OCR to COMPARISON for the characteristic 

In-State Cost. .......................................................................................................... 11 

Table 2.6: Tabular summary comparing OCR to COMPARISON for the characteristic 

Enrollment. .............................................................................................................. 11 

Table 2.7: Tabular summary comparing OCR to COMPARISON for the characteristic 

Percent Female. ..................................................................................................... 12 

Table 2.8:Tabular summary comparing OCR to COMPARISON for Association 

Membership. ........................................................................................................... 12 

Table 2.9: Tabular summary comparing OCR to COMPARISON for the characteristic 

Football. ................................................................................................................... 12 

Table 2.10: Tabular summary comparing OCR to COMPARISON for the characteristic 

Number of Sports................................................................................................... 13 

Table 2.11: Results of statistical tests comparing PART 3 to PARTS 1 AND 2. ............ 14 

Table 2.12: Tabular summary comparing PART 3 to NON-PART 3 for Carnegie 

Classification........................................................................................................... 14 

Table 2.13: Tabular summary comparing PART 3 to NON-PART 3 for Out-of-State 

Cost. ......................................................................................................................... 15 

Table 2.14: Tabular summary comparing PART 3 to NON-PART 3 for Enrollment. ... 15 

Table 2.15: Tabular summary comparing PART 3 to NON-PART 3 for Percent 

Female. .................................................................................................................... 16 

Table 2.16: Tabular summary comparing PART 3 to NON-PART 3 for Percent Black.

................................................................................................................................... 16 

Table 2.17:Tabular summary comparing PART 3 to NON-PART 3 for Percent Out-ofState........................................................................................................................... 17 

Table 2.18: Tabular summary comparing PART 3 to NON-PART 3 for Association 

Membership. ........................................................................................................... 17 

Table 2.19: Results of statistical tests comparing COMPLAINT to COMPLIANCE 

MONITORING......................................................................................................... 18 

Table 2.20: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Region. .................................................................................... 19 

Table 2.21: Number of OCR cases, by start year. ............................................................ 20

Table 2.22: Percentage of PART 3 cases for which surveys were conducted, 

by start year 29 

Table 2.23: End years of the OCR cases........................................................................... 21 

Table 2.24: Durations (see note) of OCR cases................................................................ 21 

Figure 3.1: Conceptual map of matrix of sports cross levels of interest........................... 24 
v

Table 3.1: Classification of the origin of the cases for INSTRUMENT institutions........ 25 

Table 3.2: Target populations for data collections conducted by the INSTRUMENT 

institutions................................................................................................................. 25

Table 3.3: Respondent selection mechanisms in data collections conducted by the 

INSTRUMENT institutions...................................................................................... 26 

Table 3.4: Level of proactivity in data collections conducted by the INSTRUMENT 

institutions................................................................................................................. 26

Table 3.5: Response rates in data collections conducted by the INSTRUMENT 

institutions................................................................................................................. 27

Table 3.6: Whether respondents asked their age in data collections conducted by 

INSTRUMENT institutions...................................................................................... 27 

Table 3.7: Whether respondents asked their class in data collections conducted by the 

INSTRUMENT institutions...................................................................................... 27 

Table 3.8: Whether respondents asked their gender in data collections conducted by the 

INSTRUMENT institutions...................................................................................... 28 

Table 3.9: Whether respondents asked their interest as athletic spectators in data 

collections conducted by the INSTRUMENT institutions. ...................................... 28 

Table 3.10: Whether respondents asked about their general attitudes toward 

(intercollegiate) athletics in data collections conducted by the INSTRUMENT 

institutions................................................................................................................. 28

Table 3.11: Whether respondents asked explicitly about their opinions of the institution’s 

athletic programs in data collections conducted by the INSTRUMENT institutions.

................................................................................................................................... 29 

Table 3.12: Whether respondents asked to provide identifying information in data 

collections conducted by the INSTRUMENT institutions. ...................................... 29 

Table 3.13: Whether respondents asked explicit questions regarding their athletic ability 

in data collections conducted by the INSTRUMENT institutions............................ 30 

Table 3.14: Whether respondents asked whether they had been recruited as athletes by 

some postsecondary institution in data collections conducted by the INSTRUMENT 

institutions................................................................................................................. 30

Table 3.15: Whether respondents were provided caveats and benefits concerning athletic 

participation in data collections conducted by the INSTRUMENT institutions. ..... 30 

Table 3.16: Whether respondents were provided any explanation of reasons for data 

collections conducted by the INSTRUMENT institutions. ...................................... 31 

Table 3.17: Whether respondents were provided any assurance of confidentiality in data 

collections conducted by the INSTRUMENT institutions. ...................................... 31 

Table 3.18: Representation of sports with respect to interest in data collections conducted 

by the INSTRUMENT institutions. .......................................................................... 32 

Table 3.19: Number of levels of interest (for each sport) in data collections conducted by 

the INSTRUMENT institutions. ............................................................................... 32 

Table 3.20: Representation of sports with respect to experience in data collections 

conducted by the INSTRUMENT institutions.......................................................... 32 

Table 3.21: Number of levels of experience (for each sport) in data collections conducted 

by the INSTRUMENT institutions. .......................................................................... 33 

Table 4.1: Level of proactivity in the additional data collections..................................... 41 

Table 4.2: Whether respondents asked their age in the additional data collections. ........ 41 
vi

Table 4.3: Whether respondents asked their class in the additional data collections. ...... 41 

Table 4.4: Whether respondents asked their interest as athletic spectators in the additional 

data collections.......................................................................................................... 42 

Table 4.5: Whether respondents asked explicitly about their opinions of the institution’s 

athletic programs in the additional data collections.................................................. 42 

Table 4.6: Whether respondents asked to provide identifying information in the additional 

data collections.......................................................................................................... 42 

Table 4.7: Whether respondents asked explicit questions regarding their athletic ability in 

the additional data collections................................................................................... 43 

Table 4.8: Whether respondents were provided any explanation of reasons for the 

additional data collections......................................................................................... 43 

Table 4.9: Whether respondents were provided any assurance of confidentiality in the 

additional data collections......................................................................................... 43 

Table 4.10: Representation of sports with respect to interest in the additional data 

collections. ................................................................................................................ 44 

Table 4.11: Number of levels of interest (for each sport) in the additional data collections.

................................................................................................................................... 44 

Table 4.12: Representation of sports with respect to experience in the additional data 

collections. ................................................................................................................ 44 

Table 4.13: Number of levels of experience (for each sport) in the additional data 

collections. ................................................................................................................ 44 

Figure 5.1: Initial screen of the prototype data collection instrument, containing the 

purpose of the data collection, a confidentiality statement and an explanation of the 

structure of the instrument. ....................................................................................... 57 

Figure 5.2: Second screen of the prototype data collection instrument, in which 

respondents provide four items of demographic and student status information. This 

example shows a respondent who is 20 years old, female, a junior and a full-time 

student....................................................................................................................... 58 

Figure 5.3: Third screen of the prototype data collection instrument, on which 

respondents with no experience, current participation or interest in future 

participation can so indicate and complete the process. ........................................... 59 

Figure 5.4: Fourth screen of the prototype data collection instrument, which is reached 

only by respondents who wish to enter information concerning athletic experience, 

interests and abilities................................................................................................. 60 

Figure 5.5: Fifth screen of the prototype data collection instrument, which is reached only 

by respondents who wish to enter information concerning athletic experience, 

interests and abilities. Here, respondents select the sports for which they wish to 

provide information. The list consists of 23 sports in which the NCAA conducts 

championships and seven “emerging sports.” The respondent illustrated here has 

chosen basketball, lacrosse and volleyball, which appear in screen 6...................... 61 

Figure 5.6: Sixth screen of the prototype data collection instrument, on which 

respondents enter information concerning experience, current participation, interest 

in future participation and ability only for those sports selected on screen 5. 

Continuing the example from screen 5, the respondent—who is female—has 

indicated high-school varsity experience, current intramural participation, interest in 

intercollegiate participation and ability for lacrosse................................................. 62 
vii

Figure 5.7: Seventh, and for most respondents final, screen of the prototype data 

collection instrument, which offers respondents the opportunity to provide 

comments or other feedback. Respondents who are members of the data analysis 

population and have expressed an interest in one or more sports (in our example, 

full-time female students who are freshmen, sophomores or juniors) are taken 

automatically to screen 8 unless they check the box “Check here if you do not wish 

to be contacted.”........................................................................................................ 63 

Figure 5.8: Eighth and final screen of the data collection instrument, reached only by 

respondents who are members of the data analysis population (in our example, fulltime female students who are freshmen, sophomores or juniors), on which they are 

asked whether they wish to be contacted by the athletic department, and if so to 

provide contact information. The illustrative values here are the same as in figures 

5.2-5.7. ...................................................................................................................... 64 

Table 5.1: Sources of values used to estimate + N ........................................................... 68 

Figure 5.9: Pictorial representation of data analysis in the absence of significant nonresponse. The proportion of “Yes” responses among non-respondents is assumed to 

be the same as among respondents. .......................................................................... 69 

Figure 5.10: Pictorial representation of non-response bias. “Yes” responses are relatively 

more frequent among respondents than among non-respondents............................. 71 

Table C.1: Tabular summary comparing OCR to COMPARISON for the characteristic 

Sector....................................................................................................................... 88 

Table C.2: Tabular summary comparing OCR to COMPARISON for the characteristic 

Region...................................................................................................................... 88 

Table C.3:Tabular summary comparing OCR to COMPARISON for the characteristic 

Urbanicity................................................................................................................. 89 

Table C.4: Tabular summary comparing OCR to COMPARISON for the characteristic 

Carnegie Classification.......................................................................................... 89 

Table C.5: Tabular summary comparing OCR to COMPARISON for the characteristic 

SELECTIVITY......................................................................................................... 90 

Table C.6: Tabular summary comparing OCR to COMPARISON for the characteristic 

In-State Cost. .......................................................................................................... 90 

Table C.7: Tabular summary comparing OCR to COMPARISON for the characteristic 

Out-of-State Cost. .................................................................................................. 91 

Table C.8: Tabular summary comparing OCR to COMPARISON for the characteristic 

Enrollment. .............................................................................................................. 91 

Table C.9: Tabular summary comparing OCR to COMPARISON for the characteristic 

Percent Female. ..................................................................................................... 92 

Table C.10: Tabular summary comparing OCR to COMPARISON for the characteristic 

Percent Black.......................................................................................................... 92 

Table C.11: Tabular summary comparing OCR to COMPARISON for the characteristic 

Percent Out-of-State.............................................................................................. 93 

Table C.12: Tabular summary comparing OCR to COMPARISON for the characteristic 

Association Membership....................................................................................... 93 

Table C.13: Tabular summary comparing OCR to COMPARISON for the characteristic 

Football. ................................................................................................................... 93 
viii

Table C.14: Tabular summary comparing OCR to COMPARISON for the characteristic 

Number of Sports................................................................................................... 94 

Table C.15: Tabular summary comparing PART 3 to NON-PART 3 for Sector. .......... 94 

Table C.16: Tabular summary comparing PART 3 to NON-PART 3 for Region. ......... 95 

Table C.17: Tabular summary comparing PART 3 to NON-PART 3 for Urbanicity. .... 95 

Table C.18: Tabular summary comparing PART 3 to NON-PART 3 for Carnegie 

Classification........................................................................................................... 96 

Table C.19: Tabular summary comparing PART 3 to NON-PART 3 for Selectivity..... 96 

Table C.20: Tabular summary comparing PART 3 to NON-PART 3 for In-State Cost.97 

Table C.21: Tabular summary comparing PART 3 to NON-PART 3 for Out-of-State 

Cost. ......................................................................................................................... 97 

Table C.22: Tabular summary comparing PART 3 to NON-PART 3 for Enrollment.... 98 

Table C.23: Tabular summary comparing PART 3 to NON-PART 3 for Percent 

Female. .................................................................................................................... 98 

Table C.24: Tabular summary comparing PART 3 to NON-PART 3 for Percent Black.

................................................................................................................................... 99 

Table C.25: Tabular summary comparing PART 3 to NON-PART 3 for Percent Out-ofState........................................................................................................................... 99 

Table C.26: Tabular summary comparing PART 3 to NON-PART 3 for Association 

Membership. ......................................................................................................... 100 

Table C.27: Tabular summary comparing PART 3 to NON-PART 3 for Football....... 100 

Table C.28: Tabular summary comparing PART 3 to NON-PART 3 for Number of 

Sports. ..................................................................................................................... 100

Table C.29: Tabular summary comparing SURVEY to NO SURVEY for Sector....... 101 

Table C.30: Tabular summary comparing SURVEY to NO SURVEY for Region...... 101 

Table C.31: Tabular summary comparing SURVEY to NO SURVEY for Urbanicity. 102 

Table C.32: Tabular summary comparing SURVEY to NO SURVEY for Carnegie 

Classification......................................................................................................... 102 

Table C.33: Tabular summary comparing SURVEY to NO SURVEY for Selectivity. 103 

Table C.34: Tabular summary comparing SURVEY to NO SURVEY for In-State Cost.

................................................................................................................................. 103 

Table C.35: Tabular summary comparing SURVEY to NO SURVEY for Out-of-State 

Cost. ....................................................................................................................... 104 

Table C.36: Tabular summary comparing SURVEY to NO SURVEY for Enrollment.

................................................................................................................................. 104 

Table C.37: Tabular summary comparing SURVEY to NO SURVEY for Percent 

Female. .................................................................................................................. 105 

Table C.38: Tabular summary comparing SURVEY to NO SURVEY for Percent Black.

................................................................................................................................. 105 

Table C.39: Tabular summary comparing SURVEY to NO SURVEY for Percent Outof-State. ................................................................................................................. 105 

Table C.40: Tabular summary comparing SURVEY to NO SURVEY for Association 

Membership........................................................................................................... 106 

Table C.41:Tabular summary comparing SURVEY to NO SURVEY for Football. .... 106 

Table C.42: Tabular summary comparing SURVEY to NO SURVEY for Number of 

Sports..................................................................................................................... 106 
ix

Table C.43: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Sector. ................................................................................... 107 

Table C.44: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Region. .................................................................................. 107 

Table C.45: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Urbanicity. ............................................................................. 108 

Table C.46: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Carnegie Classification. ...................................................... 108 

Table C.47: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Selectivity.............................................................................. 109 

Table C.48: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for In-State Cost......................................................................... 109 

Table C.49: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Out-of-State Cost................................................................. 110 

Table C.50: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Enrollment............................................................................. 110 

Table C.51: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Percent Female.................................................................... 111 

Table C.52: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Percent Black. ...................................................................... 111 

Table C.53: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Percent Out-of-State. .......................................................... 112 

Table C.54: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Association Membership. ................................................... 112 

Table C.55: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Football.................................................................................. 113 

Table C.56: Tabular summary comparing COMPLAINT to COMPLIANCE 

MONITORING for Number of Sports................................................................. 113 

Figure F.1: Pictorial representation of data and unknown values................................... 120 

Introduction 

The purpose of this report, undertaken at the behest of the Office for Civil Rights (OCR) 

of the U.S. Department of Education, is to develop a user’s guide for conducting student 

interest and ability surveys in order to satisfy Part 3 of Title IX that are based on 

scientifically accepted survey practice. 

Chapter 1 of the report provides concise background on Title IX (section 1.2), the 

“Three-Part Test” (section 1.2) for demonstrating compliance, and the responses to the 

Title IX Commission (section 1.3) that provided the impetus for producing this manual. 

Chapter 2 summarizes the data on which chapters 3 and 4 are based. It provides 

background information for ascertaining the representativeness of understanding 

institutional differences between OCR cases and a national set of COMPARISON

institutions, between OCR cases that resulted from a COMPLIANCE review and those 

that were the result of a COMPLAINT that was filed, between those OCR institutions 

that have used PART 3 as opposed to PARTS 1 AND 2, between those PART 3

institutions that have used a SURVEY and NO SURVEY institutions. The differences 

are represented by means of fourteen characteristics having to do with the nature and 

scale of each institution, the demographics of its student body, and its athletic programs. 

There are a total of 56 comparisons. This chapter highlights selected differences among 

these sets of institutions. The complete set of tables is located in appendix C. 

Chapter 3 is a review and analysis of the 52 data collection instruments contained in the 

OCR files. In section 3.1, these instruments are categorized along 21 dimensions, which 

range from the target population to the presence or absence of particular kinds of 

questions to the representation of sports and levels of interest, experience or ability. 

Section 3.2 discusses notable items from the individual data collections. Most of these are 

notable because they are problematic or simply baffling, but a few seem to be quite 

effective. The chapter concludes in section 3.3, with discussion of a number of issues that 

are generic to virtually all of the surveys. 

Chapter 4 describes a small number of data collections regarding students’ athletic 

experience, interest and ability that have been located by means of searches of the World 

Wide Web. Although most of these are web-based, none of them is dramatically better 

than the data collection instruments discussed in chapter 3. 

Finally, chapter 5 describes a recommended procedure for Part 3-stimulated data 

collection, including a web-based data collection instrument and procedures for 

principled statistical analysis of the data. The prototype instrument uses web-associated 

interactivity to avoid the pitfalls exhibited by the data collection instruments reviewed in 

chapters 3 and 4. In particular, a compact, comprehensible representation of “sports 

across levels” of experience, interest and ability is provided. Chapter 5 contains detailed 

advice as to how to conduct a scientifically valid data collection that will satisfy the 

requirement of Part 3. 
2
3

Chapter 1: Background 

This chapter contains background on Title IX, on the “Three-Part Test” (sometimes, 

“Three-Prong Test”) used by educational institutions to demonstrate compliance with the 

provisions of Title IX. 

1.1 Introduction to Title IX

Title IX of the Higher Education Act (20 U.S.C. §§ 1681-1688), enacted in 1972, 

addresses issues of gender discrimination in colleges and universities. Specifically, it 

states that: 

“...no person in the United States shall, on the basis of sex, be excluded from 

participation in, be denied the benefits of, or be subjected to discrimination under 

any education program or activity receiving Federal financial assistance...” (20 

U.S.C. § 1681 (a)). 

The initial legislation addressed a wide range of areas, from admissions to scholarships, 

but made no specific mention of intercollegiate sports. Two years later (Education 

Amendments of 1974, P.L. 93-380 § 844), Title IX was amended to include athletic 

programs. Subsequently, issues involving gender equity in intra- and inter-collegiate 

athletics have become an important part of Title IX implementation. 

Enforcement of Title IX is primarily the responsibility of the Office for Civil Rights 

(OCR) of the U.S. Department of Education. Courts, however, have resolved some cases. 

The associated body of case law (e.g., NCAA, 2004) has addressed legal issues ranging 

from the standing of plaintiffs to whether Title IX violates the equal protection clause of 

the Fourteenth Amendment to the U.S. Constitution. 

1.2 The Three-Part Test

Postsecondary educational institutions may be required to (attain and) demonstrate 

compliance with Title IX in response to either specific complaints or monitoring 

activities conducted by the OCR. As of January 2004, 132 cases involving 130 

institutions had been initiated and resolved. (This report does not address any cases 

currently in progress.) 

The 1979 Policy Interpretation of Title IX established three means by which institutions 

can demonstrate compliance. Collectively, these are known as the “Three-Part Test” (the 

term we employ) or, alternatively, as the “Three-Prong Test.” The three parts are as 

follows: an institution must (44 Fed. Reg. 71,418) 

1. Demonstrate that intercollegiate level participation opportunities for male and 

female students are provided in numbers substantially proportionate to their 

respective enrollments; or 

2. Where the members of one sex have been and are underrepresented among 

intercollegiate athletes, show a history and continuing practice of program 
4

expansion which is demonstrably responsive to the developing interests and 

abilities of the members of that sex; or

0. Where the members of one sex are underrepresented among intercollegiate 

athletes, and the institution cannot show a continuing practice of program 

expansion such as that cited above [in Part 2], demonstrate that the interests and 

abilities of the members of that sex have been fully and effectively accommodated 

by the present program. 

1.3 The Title IX Commission and the Assistant Secretary’s Letter 

On February 26, 2003, then Secretary of Education Rod Paige’s Commission on 

Opportunity in Athletics reported on the results of its investigation as to whether further 

guidance from OCR on Title IX requirements regarding student athletics was needed 

(U.S. DOE, 2003). 

In response to the Commission’s report, OCR issued a letter of clarification. The threepart test described above was retained. The letter reaffirmed that each of the three parts 

was a valid means of compliance and that “institutions have flexibility in providing 

nondiscriminatory participation opportunities to their students, and OCR does not require 

quotas.” Further, OCR “encourages schools to request individualized assistance for 

meeting the requirements of Title IX,” and promised that OCR will “share information on 

successful approaches with the broader scholastic community (Reynolds, 2003).” This 

manual is intended as a contribution to sharing that information and to providing a 

relatively easy way of creating and implementing an interest and ability survey in order to 

satisfy Part 3 of Title IX. 
5

Chapter 2: Analysis of OCR Cases 

Chapter 2 analyzes the institutional characteristics of OCR cases in order to determine 

how representative they are of the national population of postsecondary institutions. In 

addition, the Title IX Commission reported: “many institutions attempt to comply solely 

with part 1, while others seek to rely on parts 2 or 3 (2002, p. 3).” This chapter thus 

investigates whether OCR cases can be differentiated according to the approach they 

successfully used to achieve Title IX compliance. The questions considered here are: Do 

the “OCR institutions” differ from an appropriately defined universe of comparable 

institutions? Do institutions that have had a complaint filed against them differ from 

those whose efforts were the results of an OCR compliance review? Do the institutions 

employing Part 3 differ from those employing Part 1 or Part 2? Among those using Part 3 

to demonstrate compliance, do those conducting surveys differ from those that do not? Is 

there any tendency for the use of student interest surveys in Part 3 to increase over time? 

2.1 Summary of the Data

OCR provided to National Center Education Statistics (NCES) files covering 132 OCR 

cases involving 130 institutions in 43 states, over the period 1992-2002. Information 

about these 130 unique institutions constitutes the basic data that are analyzed in this 

chapter. This population is enumerated in appendix A. 

They are compared to a base population of 1,723 institutions that include the members of 

the National Collegiate Athletic Association (NCAA), the National Association of 

Intercollegiate Athletics (NAIA), and the National Junior College Athletic Association 

(NJCAA). In summary, the population of OCR institutions is decomposed into: 

• PARTS 1 AND 2, the 44 institutions that demonstrated compliance with Title IX 

using either Part 1 or Part 2. Because only eight institutions used Part 2, it was 

decided on both statistical and interpretive grounds to merge the Part 1 and Part 2 

institutions. They are referred to as NON-PART 3 institutions.

• PART 3, the 86 institutions that demonstrated compliance with Title IX using 

Part 3. This subpopulation is further divided into 

o SURVEY, the 57 institutions that, as best can be determined from the 

files, conducted a survey of student athletic interest and abilities as part of 

the compliance process.1

o NO SURVEY, the 29 institutions that, again, as best can be determined 

from the files, did not conduct surveys of student athletic interest and 

 

1

 Following completion of the NISS analysis, OCR provided documentation showing that ten of the 29 

institutions identified in this report as not having used a survey had, in fact, used one (and did not have a 

copy of the instrument). The numbers here, unlike those in the User's Guide, have not been adjusted to take 

this into account. These institutions are identified in Appendix A. 
6

abilities as part of the compliance process and achieved compliance in 

some other manner. 

Figure 2.1 summarizes these various populations, which collectively comprise 1,853 

institutions. 

Figure 2.1: The populations of institutions appearing in this report. 

SOURCE: NCES compilation of OCR files. 

Our attention focuses on four specific sets of comparisons, which we describe in order to 

decreasing scale. The first set of comparisons is of the 130 OCR institutions (NONPART 3 plus SURVEY plus NO SURVEY) to the 1,723 COMPARISON institutions. 

This addresses the question as to whether OCR is representative of the 1,853-institution 

universe. 

The second set of comparisons is between the 44 NON-PART 3 institutions and the 86 

PART 3 institutions, with OCR (n = 130) treated as the “universe.” These address 

whether there are differences between the institutions that demonstrate Title IX 

compliance using Part 1 or Part 2 and those that use Part 3. 

The third set of comparisons is between the 57 SURVEY institutions and the 29 NO 

SURVEY institutions, with PART 3 (n = 86) as the universe. These address whether 

there are differences between the PART 3 institutions that determine students’ athletic 

interests and abilities using surveys and those that do not use surveys. 
7

The final set of comparisons, not shown in the above chart, is between the 95 

COMPLAINT institutions and the 35 COMPLIANCE MONITORING institutions, 

with the 130 OCR institutions as the universe. These address whether there are 

differences between institutions against which a complaint was filed and those which 

OCR engaged in compliance monitoring in the absence of a complaint. 

The comparisons are made using 14 characteristics. These are divided into three groups. 

The first group, Institutional Characteristics, consists of seven characteristics: These 

are Sector, Geographical Region, Urbanicity, Carnegie Classification, Selectivity, 

In-State Cost, and Out-of-State Cost. The second group of characteristics, Student 

Body Demographics, contains four characteristics of the student body: These are 

Enrollment:, Percent Female:, Percent Black and: Percent Out-of-State. The third 

group, Athletic Program Characteristics, contains three characteristics: Association 

Membership, Football, and Number of Sports. Complete details describing the full set 

of characteristics are given in appendix B. 

The final tabulations report changes over time in the use of Part 3 and of a student 

interest survey in using Part 3. 

The data used in the comparisons were taken from extracts from the Integrated 

Postsecondary Education Data System (IPEDS). All IPEDS data pertain to the 2002-2003 

academic year. The decision to use this single year, even though the OCR cases span the 

period 1992-2002, was made for simplicity and consistency. 

The tables discussed in the chapter are only those involving characteristics for which 

there is a statistically significant difference between the two groups compared. The 

complete set of 56 tables is included in appendix C. 

2.2 Results 

2.2.1 Comparisons Between OCR and COMPARISON 

The tables below compare OCR to COMPARISON institutions according to 14 selected 

institutional characteristics. Only statistically significant results (p<.01) are reported here. 

For categorical characteristics, we performed standard Pearson Chi-squared tests to check 

the significance of the effect (non-independence of the characteristics). In cases where 

the cell counts were such that the Chi-squared test might be suspect, we used a Monte 

Carlo simulation based p-value computed by drawing random samples from the set of all 

contingency tables with given marginal totals. For numerical characteristics, including 

percentages, we performed standard two-sided t-tests for differences in means. 

Table 2.1 summarizes the comparisons of OCR to COMPARISON for the14 

institutional characteristics. There are nine highly significant differences: Sector, 

Region, Carnegie Classification, In-State Cost, Enrollment, Percent Female, 

Association Membership, Football and Number of Sports. There are no moderately 
8

significant differences. Finally, there are no significant differences with respect to 

Urbanicity, Selectivity, Out-of-State Cost, Percent Black or Percent Out-of-State. 
9

Table 2.1: Results of statistical tests comparing OCR to COMPARISON.

Characteristic OCR Compared to 

COMPARISON 

Full Results in 

Sector p < .001 Table 2.2 

Region p < .001 Table 2.3 

Urbanicity NS 

Carnegie p < .001 Table 2.4 

Selectivity NS 

Out-of-State Cost NS 

In-State Cost p < .001 Table 2.5 

Enrollment p < .001 Table 2.6 

Percent Female p < .001 Table 2.7 

Percent Black NS 

Percent Out-of-State NS 

Association Membership p < .001 Table 2.8 

Football p < .001 Table 2.9 

Number of Sports p < .001 Table 2.10 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated Postsecondary 

Education Data System (IPEDS), 2003. 

Institutional Characteristics 

Sector. Table 2.2 shows that “Public, 4-year or above” institutions are over- represented 

in OCR. 

Table 2.2: Tabular summary comparing OCR to COMPARISON for the characteristic Sector. 

OCR COMPARISON 

Sector Number Percent Number Percent 

Public, 4-year or above 71 54.62 453 26.29 

Private nonprofit, 4-year or above 32 24.62 801 46.49 

Private for-profit, 4-year or above 0 0.00 6 0.35 

Public, 2-year 27 20.77 436 25.30 

Private nonprofit, 2-year 0 0.00 20 1.16 

Private for-profit, 2-year 0 0.00 7 0.41 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Geographical Region. Table 2.3 shows that there is over-representation of “Southeast” 

and “Far West” in OCR.
10

Table 2.3: Tabular summary comparing OCR to COMPARISON for the characteristic Region.

OCR COMPARISON 

Region Number Percent Number Percent 

New England 8 6.15 128 7.43 

Mid East 12 9.23 342 19.85 

Great Lakes 14 10.77 265 15.38 

Plains 14 10.77 226 13.12 

Southeast 44 33.85 413 23.97 

Southwest 7 5.38 161 9.34 

Rocky Mountains 9 6.92 53 3.08 

Far West 22 16.92 125 7.25 

Other 0 0.00 10 0.58 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Carnegie Classification. Table 2.4 shows that there is over-representation of the two 

“Doctoral/Research” categories in OCR, and under-representation of “Baccalaureate.” 

Table 2.4: Tabular summary comparing OCR to COMPARISON for the characteristic Carnegie 

Classification.

OCR COMPARISON

Carnegie Classification Number Percent Number Percent 

Doctoral/Research Universities--Extensive 28 21.54 120 7.13 

Doctoral/Research Universities--Intensive 14 10.77 79 4.70 

Masters Colleges and Universities I 46 35.38 395 23.48 

Masters Colleges and Universities II 3 2.31 90 5.35 

Baccalaureate Colleges--Liberal Arts 5 3.85 190 11.30 

Baccalaureate Colleges--General 7 5.38 260 15.46 

Baccalaureate/Associates Colleges 0 0.00 23 1.37 

Associates Colleges 27 20.77 465 27.65 

Other 0 0.00 60 3.57 

Total 130 100.00 1,682 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Total number of cases is 1,682 rather than 1,723 because of missing values. 

In-State Cost. Table 2.5 shows that COMPARISON institutions are more expensive. 
11

Table 2.5: Tabular summary comparing OCR to COMPARISON for the characteristic In-State 

Cost. 

In-State Cost OCR COMPARISON 

Mean 7,194.35 10,177.50 

Standard deviation 7,412.00 8,009.88 

Minimum 432 0 

25th percentile 2,668 3,084 

Median 4,122 6,812 

75th percentile 8,154.50 16,685 

Maximum 28,965 30,330 

N/A 0 8 

n 130 1,723 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Student Body Demographics 

Enrollment. Table 2.6 shows that OCR institutions are larger than COMPARISON

institutions. 

Table 2.6: Tabular summary comparing OCR to COMPARISON for the characteristic Enrollment.

Enrollment OCR COMPARISON 

Mean 10,910.10 4,769.75 

Standard deviation 9,451.16 6,004.96 

Minimum 588 53 

25th percentile 3,203.50 1,367.50 

Median 8,996.50 2,546 

75th percentile 14,208.75 5,497.50 

Maximum 41,617 48397 

N/A 130 1,723 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Female. Table 2.7 contains the tabular summary for this characteristic. Unlike 

COMPARISON, OCR does not contain any all-female institutions. Excluding these 

would have no effect on the analyses. 
12

Table 2.7: Tabular summary comparing OCR to COMPARISON for the characteristic Percent 

Female.

Percent Female OCR COMPARISON 

Mean 56.10 58.50 

Standard deviation 6.01 11.67 

Minimum 30.82 0.00 

25th percentile 52.51 53.90 

Median 56.82 58.28 

75th percentile 60.10 63.01 

Maximum 71.96 100.00 

N/A 0 1 

n 130 1,723 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Athletic Program Characteristics 

Association Membership. Table 2.8 shows that OCR contains relatively more NCAA 

members than does COMPARISON institutions. 

Table 2.8: Tabular summary comparing OCR to COMPARISON for Association Membership. 

OCR COMPARISON 

Association Membership Number Percent Number Percent 

NCAA only 96 73.85 922 53.51 

NAIA only 6 4.62 278 16.13 

NJCAA only 12 9.23 500 29.02 

Multiple associations 1 0.77 23 1.33 

None 15 11.54 0 0.00 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Football. Table 2.9 shows that the prevalence of football is significantly higher in OCR

than in COMPARISON institutions. 

Table 2.9: Tabular summary comparing OCR to COMPARISON for the characteristic Football. 

OCR COMPARISON

Football Number Percent Number Percent 

Yes 86 66.15 622 36.10 

No 44 33.85 1,101 63.90 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
13

Number of Sports. Table 2.10 shows that relatively more OCR than COMPARISON

institutions are conference members for all four (baseball, basketball, cross country/track, 

football) sports. 

Table 2.10: Tabular summary comparing OCR to COMPARISON for the characteristic Number of 

Sports. 

OCR COMPARISON 

Number of Sports Number Percent Number Percent 

0 27 20.77 512 29.72 

1 0 0.00 58 3.37 

2 2 1.54 195 11.32 

3 34 26.15 428 24.84 

4 67 51.54 530 30.76 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Discussion. There are eight highly significant differences between OCR and 

COMPARISON institutions: Sector, Region, Carnegie Classification, In-State 

Cost, Enrollment, Association Membership, Football and Number of Sports. Three 

characteristics of the data appear to explain most of the significant differences.

First, the over-representation of “Public, 4-year or above” and institutions Carnegie 

classification “Doctoral/Research Universities--Extensive” in OCR results from the 

presence of 18 major state universities in OCR. This encompasses the differences in 

Sector, Carnegie Classification, In-State Cost Enrollment, Association 

Membership, Football and Number of Sports. More specifically, these 18 institutions 

represent 14 percent of OCR, while a conservative estimate of the prevalence of 

comparable institutions in COMPARISON is 5 percent (85 of 1,723). We do not attempt 

to explain this concentration, although the scale and visibility of such institutions may be 

a factor. 

Second, Region is strongly influenced by the “cluster” of 10 OCR cases involving 

community colleges in North Carolina. 

2.2.2 Comparisons of PART 3 to NON-PART 3 

This section addresses differences between the 86 PART 3 institutions and the 44 NONPART 3 institutions. 

Table 2.11 contains the comparisons of PART 3 to NON-PART 3 for the14 institutional 

characteristics. There are only two highly significant differences: Out-of-State Cost and 

Percent Black. There are five moderately significant differences: Carnegie 

Classification, Enrollment, Percent Female, Percent Out-of-State, and Association 

Membership. There is no statistically significant difference with respect to Sector, 

Region, Urbanicity, Selectivity, In-State Cost, Football or Number of Sports. 
14

Table 2.11: Results of statistical tests comparing PART 3 to PARTS 1 AND 2. 

Characteristic PART 3 Compared to 

NON-PART 3 

Full Results in 

Sector NS 

Region NS 

Urbanicity NS 

Carnegie .001 < p < .01 Table 2.12 

Selectivity NS 

Out-of-State Cost p < .001 Table 2.13 

In-State Cost NS 

Enrollment .001 < p < .01 Table 2.14 

Percent Female .001 < p < .01 Table 2.15 

Percent Black p < .001 Table 2.16 

Percent Out-of-State .001 < p < .01 Table 2.17 

Association Membership .001 < p < .01 Table 2.18 

Football NS 

Number of Sports NS 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Institutional Characteristics

Carnegie Classification. Table 2.12 shows that “Doctoral/Research Universities- -

Extensive” are over-represented in NON-PART 3, while “Associates Colleges” are overrepresented in PART 3. 

Table 2.12: Tabular summary comparing PART 3 to NON-PART 3 for Carnegie Classification. 

PART 3 NON-PART 3

Carnegie Classification Number Percent Number Percent 

Doctoral/Research Universities--Extensive 11 12.79 17 38.64 

Doctoral/Research Universities--Intensive 10 11.63 4 9.09 

Masters Colleges and Universities I 32 37.21 14 31.82 

Masters Colleges and Universities II 1 1.16 2 4.55 

Baccalaureate Colleges--Liberal Arts 4 4.65 1 2.27 

Baccalaureate Colleges--General 4 4.65 3 6.82 

Baccalaureate/Associates Colleges 0 0.00 0 0.00 

Associates Colleges 24 27.91 3 6.82 

Total 86 100.00 44 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

 

Out-of-State Cost. Table 2.13 shows that NON-PART 3 institutions are more 

expensive than PART 3. 
15

Table 2.13: Tabular summary comparing PART 3 to NON-PART 3 for Out-of-State Cost. 

Out-of-State Cost PART 3 NON-PART 3

Mean 10,562.21 14,940.77 

Standard deviation 5,501.64 6,551.51 

Minimum 432 3,150 

25th percentile 6,332.50 10,957 

Median 9,797 14,348 

75th percentile 13,151 18,456.50 

Maximum 28,209 28,965

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Student Body Demographics 

Enrollment. Table 2.14 shows that NON-PART 3 institutions are larger than PART 3. 

Table 2.14: Tabular summary comparing PART 3 to NON-PART 3 for Enrollment. 

Enrollment PART 3 NON-PART 3

Mean 8,764.27 15,104.23 

Standard deviation 7,444.25 11,455.11 

Minimum 592 588 

25th percentile 2,821 6,282 

Median 6,425 13,122.50 

75th percentile 12,275 21,694.50 

Maximum 35,667 41,617 

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Female. Table 2.15 shows that PART 3 institutions have a greater percentage 

female than do the others. 
16

Table 2.15: Tabular summary comparing PART 3 to NON-PART 3 for Percent Female. 

Percent Female PART 3 NON-PART 3

Mean 57.19 53.95 

Standard deviation 5.78 5.92 

Minimum 43.22 30.82 

25th percentile 53.50 50.65 

Median 57.66 53.53 

75th percentile 60.84 58.12 

Maximum 71.96 65.49 

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Black. Table 2.16 shows that the percentage of Black students is higher at 

PART 3 institutions than at NON-PART 3 institutions. 

Table 2.16: Tabular summary comparing PART 3 to NON-PART 3 for Percent Black. 

Percent Black PART 3 NON-PART 3

Mean 15.09 6.75 

Standard deviation 19.87 8.52 

Minimum 0 0 

25th percentile 4 2 

Median 8.50 4 

75th percentile 18.75 7 

Maximum 94 51 

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Out-of-State. Table 2.17 shows that the percentage of out-of-state students is 

higher at NON-PART 3 institutions than at PART 3 institutions. 
17

Table 2.17: Tabular summary comparing PART 3 to NON-PART 3 for Percent Out-of-State. 

Percent Out-of-State PART 3 NON-PART 3

Mean 17.10 31.68 

Standard deviation 18.83 26.48 

Minimum 0 0 

25th percentile 2.92 8.88 

Median 9.64 29.67 

75th percentile 25.88 47.21 

Maximum 97.05 97.20 

N/A 13 4 

n 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Athletic Program Characteristics 

Association Membership. Table 2.18 shows that unlike PART 3, NON-PART 3 is 

composed almost exclusively of “NCAA Only” institutions. 

Table 2.18: Tabular summary comparing PART 3 to NON-PART 3 for Association Membership. 

PART 3 NON-PART 3

Association Membership Number Percent Number Percent 

NCAA only 55 63.95 41 93.18 

NAIA only 6 6.98 0 0.00 

NJCAA only 12 13.95 0 0.00 

Multiple associations 1 1.16 0 0.00 

None 12 13.95 3 6.82 

Total 86 100.00 44 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Discussion. A higher proportion of PART 3 users are public, 2 year colleges, whose 

highest degree is an associates degree, are located in the Southeast, tend to have cheaper 

in-state and out-of-state tuition, and have both smaller enrollments and a smaller 

proportion of out-of-state enrollments than Parts 1 and 2 institutions. They are likely to 

have a higher proportion of females and a substantially higher proportion of Blacks. 

NON-PART 3 using institutions are more likely to belong exclusively to the NCAA, are 

slightly more likely to have football, and are less likely to have no sports, while being 

more likely to have four sports. 

2.2.3 Comparisons of SURVEY to NO SURVEY 

This section compares the 57 SURVEY institutions to the 29 NO SURVEY institutions. 

None of the differences is statistically significant. 
18

Discussion. Although none of these is statistically significant, SURVEY institutions 

average higher in and out-of-state tuition, are slightly more likely to be private 

nonprofits, 4 years or above, and somewhat less likely to be public institutions of a 

similar type than NO SURVEY institutions. Virtually all the NO SURVEY institutions 

are in the Southeast, while SURVEY institutions are scattered more evenly across the 

U.S. NO SURVEY institutions are likely to be larger than SURVEY institutions. 

2.2.4 Comparisons of COMPLAINT to COMPLIANCE MONITORING 

This section compares the 95 COMPLAINT institutions to the 35 COMPLIANCE 

MONITORING institutions. 

Table 2.19 summarizes the comparisons of COMPLAINT to COMPLIANCE 

MONITORING for the 14 institutional characteristics. Only one of the differences, 

namely Region, is statistically significant. 

Table 2.19: Results of statistical tests comparing COMPLAINT to COMPLIANCE MONITORING. 

Characteristic COMPLAINT Compared 

to COMPLIANCE 

MONITORING 

Full Results in 

Sector NS 

Region p < .001 Table 2.20 

Urbanicity NS 

Carnegie NS 

Selectivity NS 

Out-of-State Cost NS 

In-State Cost NS 

Enrollment NS 

Percent Female NS 

Percent Black NS 

Percent Out-of-State NS 

Association Membership NS 

Football NS 

Number of Sports NS 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Institutional Characteristics 

Region. Table 2.20 contains the tabular summary for this characteristic. There is a 

dramatically high number of COMPLAINT institutions in “Southeast” (42 versus 24 

percent). 
19

Table 2.20: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Region. 

COMPLAINT COMPLIANCE 

Region MONITORING

Number Percent Number Percent 

New England 4 4.21 4 11.43 

Mid East 10 10.53 2 5.71 

Great Lakes 11 11.58 3 8.57 

Plains 12 12.63 2 5.71 

Southeast 40 42.11 4 11.43 

Southwest 4 4.21 3 8.57 

Rocky Mountains 3 3.16 6 17.14 

Far West 11 11.58 11 31.43 

Total 95 100.00 35 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Discussion. While there are small differences between COMPLAINT and 

COMPLIANCE MONITORING, the former has a substantially disproportionate 

representation of institutions located in the Southeast.

2.3 Time Effects in the Use of Part 3 and Non-Part 3 

Table 2.21 shows the start dates of the 130 OCR cases, classified with respect to NONPART 3, SURVEY or NO SURVEY. There is an evident peak in 1993, with a 

secondary peak in 1999. There is also some evidence that surveys seem to be increasingly 

common over time. In particular, the fraction of PART 3 institutions belonging to 

SURVEY increases over time, as shown in Table 2.22. 
20

Table 2.21: Number of OCR cases, by start year. 

Start Year NON-PART 3 SURVEY NO SURVEY Total 

1988 1 1 

1989 2 2 4 

1990 1 1 1 3 

1991 4 3 1 8 

1992 8 5 2 15 

1993 8 9 12 29 

1994 7 3 5 15 

1995 2 3 1 6 

1996 4 3 4 11 

1997 2 3 5 

1998 2 7 9 

1999 1 11 1 13 

2000 2 4 1 7 

2001 3 3 

2002 1 1 

Source: OCR files provided to NISS. 

Table 2.22 shows the percentage of PART 3 cases belonging to SURVEY as a function 

of time. Despite fluctuations arising from small numbers, the prevalence of surveys 

among PART 3 institutions has increased over time. 

Table 2.22: Percentage of PART 3 cases for which surveys were conducted, by start year. 

Year SURVEY as a Percentage of PART 3 

1988 0 

1989 100 

1990 50 

1991 75 

1992 71 

1993 43 

1994 38 

1995 75 

1996 43 

1997 100 

1998 100 

1999 92 

2000 80 

2001 100 

2002 N/A 

Source: OCR files provided to NISS. 
21

Table 2.23 shows the end dates of the OCR cases. The 1993 peak in start dates appears to 

translate to the 1995 peak in end dates. 

Table 2.23: End years of the OCR cases. 

Year NON-PART 3 SURVEY NO SURVEY Total

1989 1 1

1990 1 1 2

1991 1 1 2

1992 2 2 4

1993 5 6 4 15

1994 10 7 3 20

1995 7 6 14 27

1996 7 2 4 13

1997 2 3 1 6

1998 3 4 7

1999 11 1 12

2000 3 7 1 11

2001 2 4 6

2002 3 3

2003 1 1

Source: OCR files provided to NISS. 

Table 2.24 shows the duration of OCR cases by type. Non-systematic study of the OCR 

files indicates that protracted cases are those in which evidence provided by the 

institution is either inadequate or unacceptable to OCR. There is mild but not statistically 

significant evidence that SURVEY cases are resolved more rapidly that NO SURVEY

cases. Otherwise, there do not seem to be any noteworthy effects. 

Table 2.24: Durations (see note) of OCR cases. 

Duration 

 (years) 

NON-PART 3 SURVEY NO SURVEY Total 

0 6 14 7 27 

1 20 28 9 57 

2 12 11 10 33 

3 3 3 3 9 

4 3 1 4 

Note: Duration is defined as the difference between the start year and the end year. 

Source: OCR files provided to NISS. 
22

Chapter 3: The Data Collection Instruments 

When an institution employs Part 3, there is the issue of precisely how it establishes that 

“interests and abilities of the members of that sex have been fully and effectively 

accommodated by the present program.” Chapter 2 showed that the most common means 

of doing so is to collect data from students regarding whether the current program 

satisfies their interests and abilities. 

Fifty-two of the OCR files contained data collection instruments.2

 These instruments 

themselves constitute the data on which chapter 3 is based. The 52 instruments and 

associated OCR cases were classified along 20 categorical dimensions. Appendix D 

contains a full list and description of these dimensions. 

The first class consists of properties of the survey itself. These are: 

• Whether the case is the result of a complaint against the institution or routine 

monitoring activities of OCR. 

• The target population that the data are intended to characterize, which may consist 

of the entire student body, only females or some other group. This is the group 

whose interests and abilities the survey purports to describe. 

• The respondent selection mechanism, which we translate into whether there is 

explicit selection of a subset of the target population or whether the data 

collection is conceptually meant to be a census. 

• The degree of proactivity in conducting the data collection, as best it could be 

determined from the files. This is the extent to which the institution exerted effort 

to secure a reasonable response rate. As discussed below, the most positive thing 

that can be said is that proactive institutions were less unsuccessful than others in 

this respect. 

The second set of dimensions consists largely of characteristics of the data collection 

instrument. Most of these are the presence or absence of specific kinds of questions: 

• Age: are respondents asked their age? 

• Class: are respondents asked which class (freshman, ...) they are a member of? 

• Gender: are respondents asked their gender? 

• Spectator interest: are respondents asked about their interest as spectators, either 

in person or via television or radio, of athletic events? 

• Attitudes about athletics: are respondents asked explicitly about their attitudes 

regarding athletics in general or intercollegiate athletics? 

• Opinion about the institution’s athletic programs: are respondents asked explicitly 

for opinions regarding the institution’s athletic programs (as opposed to implicit 

 

2

 There were five OCR case files that did not contain an instrument despite being recorded as having 

carried out a survey. 
23

questions associated with whether their personal interests and abilities are 

satisfied)? 

• Identifying information: are respondents asked for information that identifies 

them? 

• Ability: are respondents asked explicitly about their athletic ability? 

• Recruiting: are respondents asked whether they had been recruited as athletes by a 

postsecondary institution? 

Three of the dimensions are global characteristics of the instrument: 

• Caveats and benefits: are questions regarding intercollegiate athletics (especially, 

but also possibly other forms) accompanied by a statement of the potential 

disadvantages (for example, time spent in practice or missed classes) and 

advantages (for example, financial aid)? 

• Reasons for the survey: are respondents told why the survey is being conducted? 

• Statement of confidentiality: are respondents promised explicitly that their 

responses will be kept confidential? 

The final four dimensions concern how athletic interest, experience, and ability are 

represented in the data collection instrument. Because this is central to the discussion of 

how such a survey should be conducted, which is the subject of chapter 5, we elaborate at 

this point. 

Focusing for concreteness on interest (exactly the same considerations apply to 

experience and ability), the most efficient and accurate way of representing the 

information is quite clear: there is a matrix, shown in figure 3.1, of “sports across level of 

interest.” Because the nature of the question is “What is your level of interest in each 

sport?” the conceptual map, as in figure 3.1, has rows corresponding to sports and 

columns corresponding to levels of interest. Respondents are being asked to select a 

column representing their level of interest for each row. 

These selections are the heart of the data collection. The most substantive differences 

among the data collection instruments are how they operationalize such matrices. These 

differences are of two kinds. The first is how sports are represented, which occurs in the 

instruments three ways: 

• By fixed entries (e.g., archery, baseball, basketball, ...) in the “Sport” column. 

• By blank entries in the “Sport” column, in which respondents are asked to write in 

the names of sports for which they wish to provide information. 

• By blank entries in the “Sport” column, into which respondents are to place 

numerical codes for sports of interest, which are listed elsewhere in the 

instrument. 

The second difference is the number of levels, which ranges from one nonzero level of 

interest to ten levels. 
24

Therefore, the final four characteristics are: 

• For interest, representation of sports; 

• For interest, number of levels; 

• For experience, representation of sports; 

• For experience, number of levels. 

We do not do the same for ability because, despite the explicit mention of ability in Part 

3, relatively few data collection instruments ask specifically about it. Instead, most use 

experience as a surrogate for ability. 

Figure 3.1: Conceptual map of matrix of sports cross levels of interest. 

 Level of interest 

Sport None Mild Moderate Strong Overwhelming 

Sport 1 

... 

Sport n 

SOURCE: NISS 

In section 3.2, we list a number of items excerpted from the data collection instruments. 

These are classified as effective, problematic, or simply puzzling. There is a strong 

element of judgment in this classification. Because the focus is on items and not 

institutions, we do not identify the instruments to which individual items belong. 

Finally, section 3.3 discusses a number of issues that seem to span all the data collections. 

For example, none of them has response rates that satisfy NCES standards (U.S.DOE, 

2003), and, as least insofar as can be determined from the OCR files, there are no 

principled treatments of the results as statistical estimates accompanied by uncertainties. 

We note that the nature of the data collection and the items indicates that detailed 

psychometric item-level analysis is not needed (Converse and Presser, 1989 and Miller 

and Salkind, 2002 are representative references). As described in section 3.3, there are 

few examples where item wording appears to affect responses and fewer instances where 

item wording seems to be designed to engender particular responses. All items appear to 

be equally comprehensible to identifiable groups of students. Even students with 

effectively no athletic interest or experience seem, as a matter of broad culture, likely to 

understand the concepts and terms. However, there are, as noted in section 3.2.3, some 

items that might be incomprehensible to anyone. 

There is, however, one possible exception to this. Several of the instruments contain 

language and implicit assumptions that foreign students may not be able to cope with. 

These pertain to the nature of athletic programs in both secondary and postsecondary 

institutions in the U.S. While many of the OCR institutions have no or small numbers of 

foreign undergraduates, the same is not true of the “major” OCR institutions. 
25

3.1 Characteristics of the Data Collections and Instruments 

All data in the tables in this section are drawn from the OCR files provided to NISS. For 

consistency with earlier chapters, we denote by INSTRUMENT the subset of 52 

SURVEY institutions whose files contained copies of the data collection instrument. 

Case and Survey Characteristics 

Complaint: Table 3.1 shows the origins of the INSTRUMENT institutions as either 

“Complaint = Yes” or “Complaint = No”. 

Table 3.1: Classification of the origin of the cases for INSTRUMENT institutions. 

Complaint? Number Percent

Yes 37 71.2

No 15 28.8

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Among the 67 institutions using Part 3, 44 cases originated with a complaint. 

Target Population: Table 3.2 classifies the 52 data collections according to the target 

population, which may be the entire student body, female students only (the “underrepresented sex”) or another group. The most common but still infrequent example of the 

latter is a group of potential students such as applicants or campus visitors. Two findings 

of note are 1) more than one-half of the data collections targeted both female and male 

students, and 2) fewer than five of the 52 data collections made any attempt to reach 

target populations larger than the student body, usually by embedding the data collection 

in a campus tour. 

Table 3.2: Target populations for data collections conducted by the INSTRUMENT institutions. 

Target Population Number Percent

Student Body 29 55.8

Females 20 38.5

Other/Could not determine 3 5.8

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Respondent Selection: Table 3.3 shows the respondent selection mechanisms for the 

conducted by the INSTRUMENT institutions. Nearly two-thirds of the data collections 

are at some level attempts to reach the entire target population, which here we term 

“censuses.” We note, however, that “census” is used in a looser sense here than in 

chapter 5. Here, it means that any member of the target population could have responded, 

and not—as in chapter 5—that all members of this population (which in chapter 5 we 

term the data collection population) were individually contacted and requested to 

respond. 
26

Those data collections that were sample surveys usually sampled students in particular 

classes, as opposed to selecting a random sample of students. In some cases, the classes 

seem to have been chosen in some sense to over-sample students with an a priori interest 

in athletics. Examples are physical education and similar classes. The use of classes also 

may be intended to improve response rates, especially if the instrument is to be 

completed during class time with the instructor present. 

Table 3.3: Respondent selection mechanisms in data collections conducted by the INSTRUMENT 

institutions. 

Respondent Selection Number Percent

Census 34 65.4

Survey 9 17.3

Could not determine 9 17.3

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Proactivity: Table 3.4 shows the level of proactivity, as best could be determined from 

the files, of the data collections conducted by the INSTRUMENT institutions. There is 

discernible proactivity in fewer than one-fifth of the data collections. A typical data 

collection with no proactivity would be a paper form available in a central place, asking 

students to “Please complete and return this form.” Proactive approaches included 

incentives (tee shirts, for example, given to respondents) and various forms of nonresponse follow-up. 

Table 3.4: Level of proactivity in data collections conducted by the INSTRUMENT institutions. 

Proactivity Number Percent

Yes 9 17.3

No 31 59.6

Could not determine 12 23.1

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Response rate: Table 3.5 shows response rates in the data collections conducted by the 

INSTRUMENT institutions. One-half (26 of 52) of the files did not contain sufficient 

information to determine a response rate. Some of these did contain the number of 

responses, but not the number of students asked to respond. None of the response rates 

exceeded the NCES standard of 75% for censuses (U.S.DOE 2003), and only 6 (11%) 

even exceeded 50%. Despite the fact that the kind of non-response bias analysis 

described in section 5.5.3 is necessary on the basis of good statistical practice, there is no 

evidence that any of the INSTRUMENT institutions conducted or even contemplated 

performing one. On the contrary, some descriptions of the results incorrectly assert that 

response rates below 50% yield statistically valid estimates in the absence of nonresponse bias analyses. 
27

Table 3.5: Response rates in data collections conducted by the INSTRUMENT institutions. 

Response Rate Number Percent

< 10% 5 9.6

10-25% 6 11.5

26-50% 9 17.3

51-75% 6 11.5

> 75 % 0 0.0

Insufficient Information 26 50.0

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Instrument Characteristics 

We first deal with three dimensions associated with questions about respondent 

demographics. 

Age: Table 3.6 categorizes the data collections conducted by the INSTRUMENT 

institutions according to whether respondents were asked for their age. This might be 

relevant to institutions that attract many older or returning students, who may not have 

the ability to engage in varsity athletics. 

Table 3.6: Whether respondents asked their age in data collections conducted by INSTRUMENT 

institutions. 

Age Asked? Number Percent

Yes 38 73.1

No 14 26.9

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Class: Table 3.7 categorizes the data collections conducted by the INSTRUMENT 

institutions according to whether respondents were asked for their class. The form in 

which this information was asked showed some variation that reflects the nature of 

institutions. The most common forms were the “traditional” classes (freshman, 

sophomore, junior, senior) and the number of credit hours completed. 

Table 3.7: Whether respondents asked their class in data collections conducted by the 

INSTRUMENT institutions. 

Class Asked? Number Percent

Yes 36 69.2

No 16 30.8

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 
28

Gender: Table 3.8 categorizes the data collections conducted by the INSTRUMENT 

institutions according to whether respondents were asked for their gender. The “N/A” 

category represents institutions in which the target population was females and the 

question was not present. A number of institutions whose target population was females 

asked the question despite this restriction. 

Table 3.8: Whether respondents asked their gender in data collections conducted by the 

INSTRUMENT institutions. 

Gender Asked? Number Percent

Yes 36 69.2

No 7 13.5

N/A 9 17.3

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Spectator interest: Table 3.9 classifies the data collections conducted by the 

INSTRUMENT institutions according to whether respondents were asked their interest 

as athletic spectators (either in person or via television or radio). While the purpose of 

doing so is unclear, such questions were present in nearly 20 percent of the instruments. 

Possibly the institutions were simply attempting to gather additional information. It is 

also possible that the institutions were attempting to broaden the definition of “satisfied 

interest.” 

Table 3.9: Whether respondents asked their interest as athletic spectators in data collections 

conducted by the INSTRUMENT institutions. 

Spectator Interest Number Percent

Yes 10 19.2

No 42 80.8

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Attitudes about athletics: Table 3.10 shows that a small number of INSTRUMENT 

institutions asked respondents explicitly about their attitudes regarding either athletics in 

general or intercollegiate athletics. 

Table 3.10: Whether respondents asked about their general attitudes toward (intercollegiate) 

athletics in data collections conducted by the INSTRUMENT institutions. 

Attitudes Toward Athletics Number Percent 

Yes 5 9.6 

No 47 90.4 

Total 52 100.0 

SOURCE: NISS analysis of OCR data collection instruments. 

Opinion about the institution’s athletic programs: Table 3.13 shows that nearly onehalf of the data collection instruments asked respondents explicitly for opinions regarding 
29

the institution’s athletic programs. This seems to be an attempt to ask directly whether 

interests and abilities are satisfied. This is in contrast to implicit questions associated with 

whether their personal interests and abilities are satisfied, for example, that desire for 

further participation in activities not currently offered represents unsatisfied interest. 

Table 3.11: Whether respondents asked explicitly about their opinions of the institution’s athletic 

programs in data collections conducted by the INSTRUMENT institutions. 

Opinions about Institution’s 

Athletic Programs 

Number Percent 

Yes 22 42.3 

No 30 57.7 

Total 52 100.0 

SOURCE: NISS analysis of OCR data collection instruments. 

Identifying information: Table 3.12 shows that a number of institutions asked 

respondents for information that identifies them. In some cases, the intrusiveness was 

extreme: social security numbers, student ID numbers and telephone numbers. No 

information was available from the files to determine how often such requests were 

refused. In cases where the information was optional, it was usually for students who 

wished to be contacted regarding their athletic interests. 

Table 3.12: Whether respondents asked to provide identifying information in data collections 

conducted by the INSTRUMENT institutions. 

Identifying Information Number Percent

Yes 9 17.3

No 40 76.9

Optional 3 5.8

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Ability: Table 3.13 shows that, despite the explicit reference in Part 3 to “interests and 

abilities,” fewer than one-third of the data collection instruments asked respondents 

explicitly about their athletic ability. These institutions asked respondents to assess their 

own ability, which is confirmed by the item wording in the surveys that did request 

ability (see also section 3.2). The predominant practice, instead, was to take high school 

or previous (intra- or inter-) collegiate athletic experience as a surrogate for ability. 

Clearly this raises issues of its own, especially the enormous variation among high school 

programs: varsity at one high school may be the equivalent of intramural at another. 
30

Table 3.13: Whether respondents asked explicit questions regarding their athletic ability in data 

collections conducted by the INSTRUMENT institutions. 

Explicit Questions 

about Athletic Ability

Number Percent

Yes 16 30.8

No 36 69.2

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Recruiting: Table 3.14 shows that some data collections conducted by the 

INSTRUMENT institutions asked respondents whether they had been recruited as 

athletes by a postsecondary institution (not necessarily the one at which they were 

enrolled). The motivation for this is not entirely clear, although it may be another 

surrogate for ability. 

Table 3.14: Whether respondents asked whether they had been recruited as athletes by some 

postsecondary institution in data collections conducted by the INSTRUMENT institutions. 

Recruited as Athlete? Number Percent

Yes 9 17.3

No 43 82.7

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 

Caveats and benefits: Table 3.15 shows that in approximately one-quarter of the data 

collections conducted by the INSTRUMENT institutions, questions regarding 

intercollegiate athletics were accompanied by a statement of the potential disadvantages 

(for example, time spent in practice or missed classes) and advantages (for example, 

financial aid) of participation. As noted in section 3.2, there is a commonly used and 

essentially neutral statement that does not seem either intended or likely to influence 

responses. 

A small but not insignificant number of instruments asked those who indicating no 

interest in athletic participation to list the reasons(s). The most common form of doing 

this was to provide a list of possible reasons, with the respondent to check those that 

apply. While this information may be interesting, it is not clear that it is relevant to Part 3. 

Table 3.15: Whether respondents were provided caveats and benefits concerning athletic 

participation in data collections conducted by the INSTRUMENT institutions. 

Caveats and Benefits 

about Athletics? 

Number Percent

Yes 15 28.8

No 37 71.2

Total 52 100.0

SOURCE: NISS analysis of OCR data collection instruments. 
31

Reasons for the survey: Table 3.16 shows that in fewer than one-third of data 

collections conducted by the INSTRUMENT institutions were respondents given any 

information regarding why the data collection was being conducted. There are 

understandable and legitimate reasons why an institution might wish not to reveal 

(although one did) that it is the subject of an OCR case, but it is both unfair to 

respondents and detrimental to response rates to simply say “Please fill out this form.” 

Table 3.16: Whether respondents were provided any explanation of reasons for data collections 

conducted by the INSTRUMENT institutions. 

Explanation of 

Reasons for Data Collection?

Number Percent 

Yes 16 30.8 

No 36 69.2 

Total 52 100.0 

SOURCE: NISS analysis of OCR data collection instruments. 

Statement of confidentiality: Table 3.17 shows that in fewer than one-sixth of the data 

collections conducted by the INSTRUMENT institutions were respondents promised 

explicitly that their responses would be kept confidential. It is not possible to determine 

whether this is an oversight or that the responses in fact would not be kept confidential. 

In either case, the effect on response rates is likely to be negative. 

Table 3.17: Whether respondents were provided any assurance of confidentiality in data collections 

conducted by the INSTRUMENT institutions. 

Confidentiality Statement? Number Percent 

Yes 8 15.4 

No 44 84.6 

Total 52 100.0 

SOURCE: NISS analysis of OCR data collection instruments. 

To conclude this section, we address the representation of sports and number of levels for 

interest and experience. 

Interest—representation of sports: Table 3.18 shows the representation of sports with 

respect to interest in data collections conducted by the INSTRUMENT institutions. The 

clear preference, especially among smaller institutions, was “fixed entries.” 
 32

Table 3.18: Representation of sports with respect to interest in data collections conducted by the 

INSTRUMENT institutions. 

Interest—Representation of Sports Number Percent 

Fixed entries 34 65.4 

Write in entries 10 19.2 

Codes for entries 4 7.7 

Other/Could not determine 4 7.7 

Total 52 100.0 

SOURCE: NISS analysis of OCR data collection instruments. 

Interest—number of levels: Table 3.19 shows the number of levels of interest (for each 

sport) in data collections conducted by the INSTRUMENT institutions. The dominant 

practice is to offer simply one (non-zero) level of interest. 

Table 3.19: Number of levels of interest (for each sport) in data collections conducted by the 

INSTRUMENT institutions. 

Interest—Number of Levels Number Percent 

1 25 48.1 

2 3 5.8 

3 8 15.4 

4 8 15.4 

5 or more 5 9.6 

Could not determine 3 5.8 

Total 52 100.0 

SOURCE: NISS analysis of OCR data collection instruments. 

Experience—representation of sports: Table 3.20 shows the representation of sports 

with respect to experience in data collections conducted by the INSTRUMENT 

institutions. The difference from the representation of interest, with a strong shift to 

“Write-in entries,” is dramatic. 

Table 3.20: Representation of sports with respect to experience in data collections conducted by the 

INSTRUMENT institutions. 

Experience—Representation of Sports Number Percent 

Fixed entries 17 32.7 

Write in entries 19 36.5 

Codes for entries 6 11.5 

Other/Could not determine 10 19.4 

Total 52 100.0 

SOURCE: NISS analysis of OCR data collection instruments. 

Experience—number of levels: Table 3.21 shows the number of levels of experience 

(for each sport) in data collections conducted by the INSTRUMENT institutions. Three 
33

levels (usually, recreational, high school intramural and high school interscholastic) are 

most common. 

Table 3.21: Number of levels of experience (for each sport) in data collections conducted by the 

INSTRUMENT institutions. 

Experience—Number of Levels Number Percent 

1 9 17.3 

2 8 15.4 

3 11 21.2 

4 6 11.5 

5 or more 8 15.4 

Could not determine 10 19.2 

Total 52 100.0 

SOURCE: NISS analysis of OCR data collection instruments. 

3.2 Notable Items

In this section we list and comment on, without identifying specific sources, individual 

items from the data collection instruments that seem either especially effective or 

especially problematic. We also note some items that are simply baffling. 

No single instrument was free of potentially major shortcomings. 

3.2.1 Effective Items

Only a minority of data collection instruments contained direct questions about whether 

interests are being met. Examples are: 

• “Are your desires for participation in [recreational, intramural, intercollegiate, 

club] sports met at XXX?” 

• “Regardless of whether or not you are a student-athlete at XXX, are your desires 

for competition at the college level being met by the current offerings of 

intercollegiate sports at XXX?” 

The advantage of these questions is that they speak directly to the requirement in Part 3 

that institutions “demonstrate that the interests and abilities of the members of that sex 

have been fully and effectively accommodated by the present program,” as opposed to 

inferring the answer from the answers to other questions. 

Also as discussed in section 3.1, many (15 of 52), albeit a minority, of the instruments 

contained statements of caveats and benefits associated with participation in 

intercollegiate athletics. The following statement, which can be improved, because it 

mixes up advantages and disadvantages, appeared in several of the instruments: 

• “Intercollegiate athletics usually requires athletes to devote 20 hours of practice each 

week during the season. The athlete is expected to follow an individual regimen of 

training during the off-season. Many intercollegiate athletes receive financial awards 
34

that cover all or a portion of school expenses. Athletes are required to travel and 

occasionally miss classes. They are given access to academic support services, 

including tutoring, counseling and study tables.” 

This statement is informative without swaying responses in either direction. 

Some instruments were noticeably user friendly, although most were not. Examples of 

user friendliness include: 

• Skips for global “No” answers. An example of this is the question “Did you 

participate in athletic activities in high school” followed by “If no, skip to ...” 

• Guide posts. For example, one instrument states: “Students differ in how much they 

follow, and in how much they actually participate in, athletic, fitness and sports 

activities. The following questions are designed to address your interests.” 

• Definitions of responses. While terms such as “varsity letter winner” do not require 

definitions for most respondents, a small number of instruments did provide 

definitions. While the prototype instrument in chapter 5 does not do so either, 

hyperlinks in a web-based instrument can provide access to definitions for those who 

need it without impeding those who do not. 

One instrument provided an opportunity to respond without providing answers: “Thank 

you, but I do not wish to participate in this study.” While intriguing, this seems unlikely 

to be an effective means of increasing the response rate. 

3.2.2 Problematic Items 

These were numerous, but given the ultimately simple nature of the questions, there is no 

evidence that they produced serious distortion in the results. 

Many of the data collection instruments transposed or otherwise distorted the “sport 

across level of {interest, experience, ability}” matrices discussed in section 3.1. In some 

cases, sports were columns and levels of interest were rows, which is inconsistent with 

the conceptual structure of the questions. A more extreme problem was different levels of 

interest represented by separate questions rather than by columns. In some cases, the 

matrices appeared in the paper instruments without lines defining the cells, making the 

responses both difficult to fill in accurately and difficult to extract. 

There were two extreme cases. In one, which occurs in the same form in several 

instruments, respondents were asked, in effect with one sport per set of questions, to 

provide, each on a scale of 1-10 with no definition of the levels—which clearly do not 

mean the same thing in all cases: 

• “A level of interest 

• Willingness to engage in physical fitness conditioning, practice regimens and 

travel trips for the sport 

• Willingness to train during the off-season for the sport 

• Willingness to participate in the sport without a scholarship” 
35

In the second, respondents were asked, for ten sports (one sport per question), to respond 

to “I am interested in the intercollegiate sport of X 

• As a spectator /As a participant 

• Have you talked to the coach? Yes / No 

• Have you tried out? Yes / No 

There were numerous problems with scales associated with levels of interest, experience 

or ability. These include scales with no explanation, one example of which is above, and 

another example of which is to “Check sports you’re interested in,” without defining 

“interested in.” Some scales, such as the 10-level scales above, simply have too many 

levels. One instrument violated both the conceptual structure and psychometric principles 

by having skill levels that decrease from left to right, making it harder to read. 

The use of codes for sports is an extreme form of user unfriendliness. The codes may 

simplify analysis, but they do so at expense of the respondent (who may need to search 

for codes on a different page—in one case, the back—of the instrument). In addition, 

errors may occur during transcription. The most common example, which occurs in 

several instruments in exactly the same form, is a listing of 78 sports by 2-digit numerical 

codes that have no inherent or other meaning. Another example is coding of the level of 

high school participation, with the response to be indicated by filling in ‘”A” if you 

played the sport at the varsity (or elite) level”, ..., “E” if you did not play the sport’ for 

each sport. In the most extreme example, interest and ability were indicated by filling in 

letters next to sports, with the same levels used for both. 

Inexplicably, several instruments force respondents to rank their choices of sports, which 

is both unnatural and unnecessary. These forced choices unduly constrain individual 

expressions of interest and, as demonstrated in chapter 5, they are unnecessary. Examples 

of this are: 

• Asking respondents to rank sports by level of interest, separately for 

intercollegiate, club and intramural participation. 

• “Which of the following NCAA sports would be your first choice as an addition 

to the XXX women’s program (pick only one)”. This is followed by questions 

asking about past experience and interest in trying out only for that sport. The 

same is then done for a second choice. 

• “Check the one intercollegiate sport that you would participate in if offered at 

XXX.” 

• “Rank the THREE sports that you would participate in at XXX if they were 

offered as an intramural activity.” 

Although not uniformly, the instruments rely excessively on free-form responses. This 

reliance makes analysis unnecessarily complicated or even hopeless. One example of 

this is free-form entry of sport names. Others are free-form text for experience and freeform lists of reasons why the respondent is not a participant in intercollegiate athletics. 

There is even one case of free-form entry of gender: “Are you male or female?” And in 
36

cases where free-form responses may at least be defensible, there is often insufficient 

space. Examples of the latter include: 

• Following a global Yes / No response for “Have you ever participated in competitive 

sports,” the question “If so, which sports, how many years, positions? ____________ 

“ 

• 2.5 inches on one line to list previous athletic experience with 2 inches to list sports 

• “Do you plan to participate in intercollegiate athletics at XXX? 

 _____ Yes (list sport or sports) 

 _____ No” [check reasons why not]” 

Although, as discussed in section 3.2.1, statements of caveats and benefits associated 

with participation in intercollegiate athletics are necessary and desirable, several 

instruments embed caveats in individual items, which may influence the responses. 

Examples of this are: 

• “Would you be willing to spend 20 hours per week in structured practice for your 

sport?” 

• “Would you still be interested in participating even if financial aid were not 

available?” 

• Questions asking respondents their reasons for not participating in intercollegiate 

athletics. There is a “standard” set of about twenty of these that appears in several 

instruments. 

The implicit assumption that collegiate interest cannot exist without high school 

participation appears in some instruments: questions about collegiate interest are to be 

skipped if there is no high school participation. 

The instruments are rife with superfluous items. One example is “What sports, in your 

opinion, should be eliminated? ______________ “ Several instruments ask respondents 

for their race and/or ethnicity, which are often not distinguished correctly. Some ask 

about living arrangements (on campus, off campus, home, ...). 

Many instruments contain questions to which respondents may not know the answers. 

Examples of such items, most of which are also superfluous, are: 

• “Do you know other XXX women students who are interested in participating in 

intercollegiate athletics at XXX? If so, how many others (approximately) and in 

which sport(s)?” 

• In connection with recruiting, “... were you recruited by a member of the NCAA? 

the NAIA? Both? If NCAA, a member of Division I? Division II? Division III?” 

• “Do you have in interest in participating at the varsity intercollegiate level in any 

sports not currently offered by XXX” without a list of the sports currently offered. 

• “... are you eligible to participate as an intercollegiate athlete?” 
37

Finally, although there are arguments to the contrary, there is widespread forced use of 

“None” responses in cases where no response clearly means the same thing. For example, 

this happens for interest in sports, and is particularly burdensome to respondents when 

there are many sports. 

3.2.3 Baffling Items 

A number of items in the data collection instruments are simply baffling, in regard to 

what information is sought, what a meaningful response is, or both. 

One such example is the item “From the list below of NCAA sanctioned sports rank the 

top seven (7) men’s and top seven (7) women’s sports according to your preference. Rate 

your first choice with a 1 and your last choice with a 7.” Another is a question about 

multiple sports followed by “If interested, have you competed in this sport at any level?” 

One instrument contains items whose meaning and responses depend on whether the 

respondent is male or female, which become almost impossible to analyze. Another 

instrument states: “Please circle one choice for each item” followed by items with freeform responses. 

3.3 Generic Issues 

A number of generic issues apply to Part 3 data collections in general. 

Uniformly, the response rates are poor. Not all files report the response rates, but as 

shown in table 3.5, the reported rates range from less than 1 percent to 70 percent. None 

of these satisfies the NCES statistical standards. The problem posed by low response 

rates is that non-respondents may systematically differ from respondents, producing 

biased results. 

Coupled with poor response rates is an apparent problem of non-response bias, which in 

this case works to the detriment of the institutions conducting surveys because those who 

are dissatisfied with the athletic programs at an institution are probably more likely to 

respond than those that are satisfied. If this is so, then reported levels of satisfaction are 

lower than “true” levels. Yet there seems to be no awareness of possible non-response 

bias, let alone (and it is likely that the data would not support it anyway) any nonresponse bias analysis. The practice of some institutions of distributing data collection 

instruments in classes with large numbers of students with presumptive athletic interests 

may also lead to non-response bias. While this procedure is problematic, it may be 

understandable since unlike many surveys, finding absolute numbers of women with 

unsatisfied interests is what is critical to Title IX compliance. Institutions well may 

believe that such venues capture most of those who might be interested in unsatisfied 

athletic participation and they may be right, but in the absence of more systematic, easy 

to implement procedures as described in the next chapter, one cannot be certain. How to 

deal with possible non-response bias is discussed in section 5.5.3. 

While no documentation in the OCR files treats the data collection results as uncertain 

statistical estimates, this may be less problematic than it appears since, as shown above, 

many data collections are actually attempted censuses. 
38

That the data collections rely on self-reported data might be a problem, but it is no worse 

in this setting than in many others. Indeed, in most cases, there is no readily available 

meaningful corroborative data. 

To their credit, INSTRUMENT institutions seem not to have fallen victim to the 

temptation to slant item wording so as to yield a “favorable” result to the institution. This 

is not intended to impugn institutional motivation, but only to point out that if sufficient 

interest in a sport not currently offered is displayed, the institution may have to spend 

money to create a new team. 
39
40

Chapter 4: Other Data Collection Instruments 

A number of institutions other than those involved in OCR cases have conducted data 

collections regarding students’ athletic interests and abilities. Largely through Internet 

searches conducted by Education Statistics Services Institute personnel, a number of 

these data collections been identified, although in only a few cases is the actual data 

collection instrument available. In this chapter, whose structure is parallel to that of 

chapter 3, we review an additional five instruments that are available. Because of the 

small number, the level of detail is less than that in chapter 3. 

The reasons for institutions’ conducting these data collections seem to be self-assessment 

analogs of demonstrating compliance with Title IX. For instance, the report of a 1998 

survey conducted by the University of Kentucky states that: 

The survey was the fourth such joint effort conducted in support of the 

University’s efforts to evaluate the interest and accomplishments of students. 

The purpose of the survey was to gauge the levels of participation in various 

varsity sports among entering UK students and to explore possible areas of 

interest for offering additional sports at the University. The survey was to be 

distributed to all incoming freshmen during their Freshman Advising Conference. 

Of the 3,000 surveys distributed 1,642 were returned for a response rate of 54% 

[sic]. Of these responses 925 of the surveys were completed by females, and 678 

surveys were completed by males. 

One of the major goals of the survey was to assess the level of experience and 

awards earned in varsity sports in high school. Information was collected for these 

sports: basketball, across country, fast pitch softball, golf, gymnastics, rifle, 

soccer, swimming, tennis, track, and volleyball.

4.1 Characteristics of the Instruments 

Appendix E lists the five data collection instruments reviewed in this chapter. Because 

they are recent, four of these are web-based, unlike all of the instruments in the OCR 

files. 

Target Population: In all five additional data collections, the target population was the 

entire student body. 

Respondent selection: It was not possible to determine the respondent selection 

mechanism for any of the five additional data collections. It appears that most were 

meant to reach either the entire student body or the freshman class. 

Proactivity: Table 4.1 shows the level of proactivity in the five additional data 

collections. In the one proactive instance, a tee-shirt said to be worth $10 was offered to 
41

those completing the instrument, but this required providing a name and so risking 

violations of confidentiality. 

Table 4.1: Level of proactivity in the additional data collections. 

Proactivity Number Percent

Yes 1 20.0

No 4 80.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Instrument Characteristics 

Age: Table 4.2 categorizes the five additional data collections according to whether 

respondents were asked for their age. 

Table 4.2: Whether respondents asked their age in the additional data collections. 

Age Asked? Number Percent

Yes 1 20.0

No 4 80.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Class: Table 4.3 categorizes the five additional data collections according to whether 

respondents were asked for their class. 

Table 4.3: Whether respondents asked their class in the additional data collections. 

Class Asked? Number Percent

Yes 2 40.0

No 3 60.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Gender: All five additional data collections asked respondents for their gender. 

Spectator interest: Table 4.4 classifies the five additional data collections according to 

whether respondents were asked their interest as athletic spectators (either in person or 

via television or radio). 
42

Table 4.4: Whether respondents asked their interest as athletic spectators in the additional data 

collections. 

Spectator Interest Number Percent

Yes 2 40.0

No 3 60.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Attitudes about athletics: None of the five additional data collections asked respondents 

explicitly about their attitudes regarding either athletics in general or intercollegiate 

athletics. 

Opinion about the institution’s athletic programs: Table 4.5 shows that only one of 

the five additional data collection instruments asked respondents explicitly for opinions 

regarding the institution’s athletic programs. 

Table 4.5: Whether respondents asked explicitly about their opinions of the institution’s athletic 

programs in the additional data collections. 

Opinions about Institution’s 

Athletic Programs 

Number Percent 

Yes 1 20.0 

No 4 80.0 

Total 5 100.0 

SOURCE: NISS analysis of other data collection instruments. 

Identifying information: Table 4.6 shows that two of the five additional data collections 

asked respondents for information that identifies them. This seems to result from web 

administration: the information was requested in order to spare respondents future emails asking them to complete the survey. In one case, the information was said to be 

necessary in order to receive a tee-shirt promised to respondents. 

Table 4.6: Whether respondents asked to provide identifying information in the additional data 

collections. 

Identifying Information Number Percent

Yes 2 40.0

No 3 60.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Ability: Table 4.7 shows that only one of the five additional surveys asked respondents 

explicitly about their athletic ability. 
43

Table 4.7: Whether respondents asked explicit questions regarding their athletic ability in the 

additional data collections. 

Explicit Questions 

about Athletic Ability

Number Percent

Yes 1 20.0

No 4 80.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Recruiting: None of the 5 additional data collections asked respondents whether they 

had been recruited as athletes by a postsecondary institution. 

Caveats and benefits: None of the 5 additional data collections contained a statement of 

the potential disadvantages and advantages (for example, financial aid) of participation. 

Several of the instruments asked those who indicated no interest in athletic participation 

to list the reasons(s). 

Reasons for the survey: Table 4.8 shows two of the five additional data collections give 

respondents information regarding why the survey was being conducted. One of these 

actually mentioned Title IX. 

Table 4.8: Whether respondents were provided any explanation of reasons for the additional data 

collections. 

Explanation of 

Reasons for Data Collection?

Number Percent 

Yes 2 40.0 

No 3 60.0 

Total 5 100.0 

SOURCE: NISS analysis of other data collection instruments. 

Statement of confidentiality: Table 4.9 shows that only one of the five additional data 

collections promised confidentiality to respondents. In fact, as noted above in connection 

with Identifying Information, use of the web seems to have created more ways of 

threatening confidentiality. 

Table 4.9: Whether respondents were provided any assurance of confidentiality in the additional 

data collections. 

Confidentiality Statement? Number Percent 

Yes 1 20.0 

No 4 80.0 

Total 5 100.0 

SOURCE: NISS analysis of other data collection instruments. 
44

Interest—representation of sports: Table 4.10 shows the representation of sports with 

respect to interest in the five additional data collections. 

Table 4.10: Representation of sports with respect to interest in the additional data collections. 

Representation of Sports Number Percent

Fixed entries 4 80.0

Write in entries 1 20.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Interest—number of levels: Table 4.11 shows the number of levels of interest (for each 

sport) in the five additional data collections. 

Table 4.11: Number of levels of interest (for each sport) in the additional data collections. 

Number of Levels Number Percent

1 2 40.0

2 1 20.0

3 2 40.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Experience—representation of sports: Table 4.12 shows the representation of sports 

with respect to experience in the five additional data collections. 

Table 4.12: Representation of sports with respect to experience in the additional data collections. 

Representation of Sports Number Percent

Fixed entries 3 60.0

Write in entries 1 20.0

Other/Could not determine 1 20.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 

Experience—number of levels: Table 4.13 shows the number of levels of interest (for 

each sport) in the five additional data collections. 

Table 4.13: Number of levels of experience (for each sport) in the additional data collections. 

Number of Levels Number Percent

1 4 80.0

2 1 20.0

Total 5 100.0

SOURCE: NISS analysis of other data collection instruments. 
45

4.2 Notable Items

There are few examples of notable items in the five additional data collections that differ 

from the notable items discussed in section 4.2. One, but only one, of the data collections 

provides careful definitions of terms. Another, which is replete with HTML form 

elements, in effect assigns multiple sports of interest to different questions. As noted in 

section 4.2, this happens as well in the INSTRUMENT data collection instruments, 

some of which (as does not occur here) actually force respondents to rank sports by level 

of interest or experience. The impact is that analysis of effectively parallel questions 

labeled “Activity 1” and “Activity 2” (and “Intercollegiate Activity 1,” “Intercollegiate 

Activity 2”, “Intercollegiate Activity 3”) is needlessly complicated. 

That four of the five additional data collections are web-based reflects evolving 

information technologies. However, in general the instruments fail to capitalize on their 

being web-based, and in fact use of the web introduces some additional issues. 

Perhaps the most striking aspect is the lack of exploitation of interactivity. For example, 

three of the surveys, including the one paper-based data collection, incorporate skips that 

allow recipients to skip questions that become inapplicable as the result of answers to 

others, but those questions are displayed nevertheless. By contrast, the data collection 

instrument proposed in chapter 5 does not even display superfluous questions, and makes 

the selections adaptively, in response to earlier answers. 

In general, the web-based data collections use devices such as check boxes, radio buttons 

and drop-down lists sensibly, in order to constrain responses to some allowable set, 

although one lengthy web-based data collection has a number of free-form text entries. 

Although doing so is straightforward, none of the web-based data collections makes use 

of hyperlinks, or similar devices such as mouse-overs, to provide access to “metadata” 

such as definitions of terms. 

Two of the web-based data collections introduce gratuitous confidentiality problems by 

requesting identifying information in order to prevent respondents from being contacted 

again. (One example is “Your name is requested so that we will not send you another 

email about this survey.”) Evidently, respondents (the basis of selection is not clear.) 

were notified by e-mail that they had been selected, and the data collection administration 

process seems unable to prevent follow-up without identifying information. There are 

readily available alternatives (such as one-way hashed keys) that make it possible to track 

who has responded without a link to their responses. 

 
46

Chapter 5: Implementation of Part 3 Data Collections 

In this chapter we describe a process for implementation of data collection when Part 3 of 

the “Three-Part Test” is employed. 

For clarity, we focus on one very specific problem. Attention is restricted to a single sport 

not currently offered at the intercollegiate (varsity) level by the institution, and we 

assume—solely for concreteness—that women are the underrepresented sex. The 

problem is to determine (see section 1.2) “whether the interests and abilities of [women] 

have been fully and effectively accommodated by the present program [for that sport],” 

on the basis of data collected from women students. 

We formulate the problem conceptually and mathematically in section 5.1. Sections 5.2 

through 5.5 address key steps in the procedure of solving it: specifying the process 

(section 5.2), data collection process (section 5.3), web-based data collection (section 5.4) 

and data analysis (section 5.5). A number of precautions imposed by the entire process 

are discussed in section 5.6, and section 5.7 discusses issues arising prior to and 

following the data collection process. Each principal component of the chapter contains 

three particular items: 

Practice among SURVEY Institutions, a summary of how that component was addressed 

by the 57 SURVEY institutions. Most of this information is also in chapter 3, but 

including it here makes this chapter more self-contained. 

Recommendation for Improvement, which would improve SURVEY institution practice 

without imposing large barriers in terms of information technology or statistical 

sophistication. 

High-Quality Recommendation, describing an approach that satisfies the NCES statistical 

standards and other important criteria. 

5.1 Problem Formulation 

As stated in the introduction to this chapter, we restrict attention to a single sport not 

currently offered at the intercollegiate level, and assume that women are the 

underrepresented sex. We focus on an institution employing Part 3 that is attempting to 

determine, using data collected from women students, whether the interests and abilities 

of women have been “fully and effectively accommodated by the present program.” The 

alternative is that the interests and abilities of women can be accommodated only by 

offering the sport at the intercollegiate level. 

We now describe an operational formulation of the problem. Let M (for minimum) be the 

minimal number of team members necessary to “field” a team in the given sport. This 

number must be specified by the institution. It depends on the sport and possibly 

contextual factors. For instance, a basketball team cannot play with fewer than five 

players, but this is not the value of M for basketball, which is presumably in the range 10-

15. NCAA or other association rules may provide information about how large M might 
47

be (M cannot exceed maximum allowable team sizes), but prevailing values in the 

conference to which the institution belongs are also relevant. 

There is, conceptually, some number + N of women students who, given the current 

offerings of the institution, possess the interest and ability to compete in the sport at the 

intercollegiate level. If + N were known with certainty, then determination of compliance 

by OCR would be straightforward: 

• If N ≥ M + , and if other OCR-determined criteria are fulfilled, then the 

institution must offer the sport at the intercollegiate level. These criteria are 

complex, and discussed in section 5.7. 

• If N < M + , then the interests and abilities of women have been accommodated 

by the present program. 

It is the “conceptually” and “known with certainty” qualifications in this initial 

formulation that create several significant difficulties. 

The first difficulty is that the definitions of interest and ability are subjective. How to 

assess them for the purposes of Part 3 is discussed in section 5.4. 

The second difficulty is that exact determination of + N may be impossible. As discussed 

in chapter 3, most institutions that have demonstrated Title IX compliance using Part 3 

have done so using data collections that conceptually are censuses but yield very low 

response rates. In the samples, retaining our one-sport/women-underrepresented setting, 

only a—randomly or otherwise—selected subset of women were asked whether their 

interests and abilities are accommodated by the present program. Even absent other 

problems that we discuss momentarily, rather than obtain exact knowledge of + N , the 

institution may be forced either to construct a statistical estimator + Nˆ or to regard + N as 

random and calculate its distribution given the observed data. Then, the simple decision 

criterion stated above is no longer valid; alternatives are discussed in section 5.5. 

But, there are further complications. By far the most difficult-to-deal-with of these is that 

in almost all cases, the number of respondents—those who are asked to complete the data 

collection instrument who actually do so—is only a fraction, and possibly a rather small 

fraction. Non-response is a problem in its own right, because it changes both the form 

and the properties of the estimator + Nˆ . 

Another problem is non-response bias: the prevalence of interested and able students in 

the sample who do not respond may differ from the prevalence of interested and able 

students in the sample who do respond. Indeed, in the setting of Part 3, non-response bias 

seems almost inevitable: uninterested students are less likely to respond than interested 

students. Dealing with non-response bias is discussed in section 5.5.3. 
48

There are additional issues when only a sample of the target population is asked to 

provide information, because the sampling process introduces additional randomness and 

uncertainty. Sampling is avoidable, however, while non-response bias is not. 

A third difficulty is that the data collection process itself may influence the results. This 

issue was alluded to in chapter 3, in connection with whether data collection instruments 

accurately or inaccurately depict the benefits and obligations associated with 

intercollegiate sports. The prototype instrument in section 5.4.1 is specifically intended 

not to influence the data it generates. 

Given these difficulties, an ideal Part 3 implementation would proceed as follows: 

1. The target population for purposes of Title IX compliance consists of full-time 

women undergraduate students. 

2. The data collection protocol is a census: all members of the target population are 

asked for information. In fact, as discussed in section 5.2, we recommend that all 

students, both female and male, be part of the data collection. 

3. Data collection is web-based. 

4. Because non-response may be inevitable, telephone-based follow-up of nonrespondents may be conducted at a level that supports necessary non-response 

bias analyses. 

5. Data analysis is restricted to responses from the data analysis population, and 

consists of combining the responses and the non-response bias analysis in a 

principled statistical manner to produce an estimator + Nˆ and to calculate the 

distribution of this estimator. 

6. The decision criterion employed by OCR is to calculate the conditional 

probability that + Nˆ (or, in Bayesian formulations, + N itself) exceeds the 

minimum team size M given the observed data from both the census and the nonresponse bias sample. If this probability exceeds a pre-determined threshold, and 

if other criteria are satisfied, then OCR would determine that the institution must 

offer the sport at the intercollegiate level. Otherwise, the determination would be 

that the interests and abilities of women are accommodated by the present 

program. 

Sections 5.2-5.5 describe various parts of this process in more detail. 

5.2 Process Specification 

Here we discuss selection of the data collection population, the data collection protocol 

and the data analysis population. 

Data Collection Population. As also mentioned in section 3.1, selection of the data 

collection population presents both conceptual and logistical issues. For a number of 

reasons, collecting data only from members of the underrepresented sex is not feasible. 

Consequently, there are only two defensible choices: 
49

• The entire student population; 

• A “catchment” population consisting of both the entire student population and a 

set of actual or potential applicants. 

If the data collection population is the entire student population, then while there are 

response rate and other issues, at least the population is well-defined and in principle 

accessible. 

Legal cases, however, have implied that the data collection should be larger—for 

example, a set of potential applicants. This would happen if applicants believed that the 

institution was not able to satisfy their athletic interests and abilities, and therefore chose 

either not to apply or to decline offers of admission. The “potential applicant/catchment” 

population is very problematic. It depends on the nature of the institution, and ranges 

from quite local to the entire nation or beyond, for institutions with international students. 

Even if definable, such a large data collection population is almost surely unreachable in 

any meaningful manner. Attempts to do so described in the OCR files are weak at best. 

They include requesting information from participants in campus tours or (local) high 

school administrators. Either of these approaches raises more problems than it solves. 

Moreover, the statistical implications of employing a catchment population are 

effectively impossible to characterize. Even the population size is not known, which 

prevents calculation of response rates, and non-response bias analyses are simply 

infeasible. 

A data collection population of applicants is better defined but not easier to sample. The 

same is true of surrogates, such as campus tour participants. 

If the data collection population is entire student population, then some issues remain. 

For example, does “student” mean “full-time student?” Must students be enrolled 

currently? Given that the purpose of the data collection is to determine whether a sport 

must be offered to women at the intercollegiate level, perhaps the most sensible definition 

would be students who are eligible for intercollegiate athletic participation as determined 

by the athletic association (for example, the NCAA) to which the institution belongs and 

the institution itself. However, this is not likely to be feasible if criteria such as academic 

standing are involved. 

Practice among SURVEY Institutions: As discussed in section 3.1 (see table 3.4), most 

of the 52 INSTRUMENT institutions treated the entire student body as the data 

collection population. 

Recommendation for Improvement: The data collection population should be the entire 

population of full-time students. 

High Quality Recommendation: The data collection population should be the entire 

student population eligible for intercollegiate athletic participation. 
50

Data Collection Protocol. Again, there are two choices: 

• A census: all members of the data collection population are asked to provide 

information regarding whether their interests and abilities are accommodated by 

the present program. 

• A sample survey: only a subset of members of the data collection population are 

asked to provide information regarding whether their interests and abilities are 

accommodated by the present program. 

Our use of these terms is consistent with that of the federal statistical agencies. While a 

census is a larger scale undertaking than a sample survey, it is superior in almost every 

respect. If response could be mandated, then + N would be known with certainty, and the 

“simple” decision criterion described above is applicable. Moreover, none of the 

complicated methodology for dealing with non-response bias outlined in section 5.5.3 is 

necessary. In reality, however, mandating a response may not be feasible. 

Perhaps most important, employing a census avoids two difficult issues associated with 

sample surveys: selection of the sampling mechanism and selection of the sample size. In 

the OCR surveys described in chapter 3, two classes of sampling mechanisms are 

employed widely: 

• Simple random samples, in which those asked to provide data are selected 

randomly from the data collection population, in such a way that all members of 

the population have equal probability of being in the sample. This method has the 

advantage of simplifying analysis of the data, but has other shortcomings 

discussed below. 

• Targeted non-random samples. The “target” in this case is generally a subpopulation of what we have termed “the target population” thought to be likely to 

contain students with athletic ability and interest. The principal example is 

enrollees in health or physical education courses. 

Intermediate methods, such as randomly selected courses, are present as well in the 

surveys discussed in chapter 3. These surveys also, however, include some simply 

indefensible sampling methods, whose statistical properties cannot be adequately 

described. The most egregious example is placing survey forms in a place where students 

might simply pick them up. 

The statistical implications of such samples are virtually impossible to characterize. In 

particular, non-response bias analyses are not possible. For this reason we recommend 

that such samples not be employed. 

Given the ready availability of electronic means of data collection (see section 5.4), 

simple random samples seem unnecessary. Another reason to avoid them is that small 

sample sizes may not be feasible. To illustrate why this is true, let P be the size of the 

target population, and suppose that the sample size is S. Then roughly, each student in the 

sample represents P/S students in the target population. Let + NS be the number of positive 
51

responses (interested and able to participate at the intercollegiate level in the given sport) 

in the sample. Recalling that the goal is to estimate the number + N of students in the 

target population interested and able to participate at the intercollegiate level in the given 

sport and ignoring non-response, the estimated value of + N is 

( ) ˆ P S

S

N N N S = S + −

+

+ + . 

To explain briefly (more complicated expressions appear in section 5.5), the + NS sampled 

students who responded positively are known with certainty to be interested and able. 

The fraction of students sampled who responded positively, NS / S + , is assumed to be the 

same as the fraction of the P-S unsampled students who are interested and able, so the 

estimated number of such students is just the product of these two terms. 

If S is small, then every positive response in the survey has a large effect on + Nˆ . At the 

extreme, if P/S exceeds the minimum team size M, then a single positive response in the 

sample produces an estimated value of + N that exceeds M! Of course, there is also high 

variability in the estimate, but still this is a clear problem. But unless P/S is large, the 

advantages of a sample are negated. Therefore, arguments in favor of a sample are not 

convincing. 

One issue, discussed briefly in chapters 3 and 4, is the use of incentives to promote 

responses to the census. At an extreme, the institution might require response, for 

example, linking completion to registration status or refusing to provide grade reports to 

those who had not responded. Even though measures this extreme may be seen by some 

to have negative consequences such as heavy handedness, decreased data quality in the 

form of frivolous or inaccurate responses is a more important issue, in part because there 

are no models or tools to characterize effects on data quality. This problem would be 

particularly problematic with a data collection instrument of the type described in section 

5.4.1. Checking the global “no athletic experience, current participation or interest in 

future participation” box on screen 3 there (see figure 5.3) becomes the simplest way to 

fulfill the requirement. The clear consequence of this behavior is an underestimate of + N . 

Practice among SURVEY Institutions: As shown in table 3.5, most of the 

INSTRUMENT conceptualized the data collection as a census. However, the term 

“census” is used in chapter 3 in a much looser sense than in this chapter, to mean the 

absence of a discernible, explicit sampling mechanism. 

Recommendation for Improvement: The data collection protocol should be a census, in 

the strict sense that every member of the data collection population is contacted 

individually (for example, by e-mail) and requested to respond. 
52

High Quality Recommendation: The data collection protocol should be a census in the 

strict sense that every member of the data collection population is contacted individually 

and requested to respond. 

Data Analysis Population. By “data analysis population” we mean that part of the data 

collection population whose data will be used for purposes of determining compliance 

with Title IX. For the specific setting of this chapter, only females—the assumed 

underrepresented sex—belong to the data analysis population. OCR has stated that 

fourth-year and beyond students (assuming a traditional four-year curriculum) should 

also be excluded from the data analysis population. The data collection instrument in 

section 5.4.1 requests both gender and year, and so supports either choice of data analysis 

population. The analysis procedures in section 5.5 are valid in either of these cases. 

Practice among SURVEY Institutions: The OCR files contain few to no details regarding 

statistical analyses of the data. Moreover, there are cases in the files where the data 

collection population contained both females and males, but respondents were not asked 

to provide their gender. 

Recommendation for Improvement: The data analysis population should consist of all 

full-time first-, second- and third-year students of the underrepresented sex. 

High Quality Recommendation: The data analysis population should consist of all fulltime first-, second- and third-year students of the underrepresented sex who meet all 

criteria for participation in intercollegiate athletics. 

5.3 Data Collection Process 

Here we discuss principal steps in an electronic data collection process centered on a 

web-based collection instrument of the type described in section 5.4. We assume that 

recommendations in section 5.2 are implemented: the data collection population is the 

entire student population, and the data collection protocol is a census. 

Prior to any of the steps described below, the data collection instrument and software 

must be in place; the former is described in section 5.4.1 and the latter in section 5.4.2. 

The initial step is to compile a database of e-mail addresses and telephone numbers (for 

non-response follow-up; see section 5.5.3) of all members of the data collection 

population. We assume that all students are required to have e-mail addresses at the 

institution. 

The second step is to send an e-mail message to each member of the data collection 

population, requesting that she or he complete the data collection instrument. In addition 

to the request to provide the data, this message should contain: 

• A link to the URL of the data collection instrument; 

• Embedded within the link, a unique, encrypted ID that allows the software to 

record that a person has responded without being able to link to that person’s 
53

response (see below). This strategy also precludes multiple responses by one 

individual. 

Non-response follow-up is likely to be necessary. In principle, using the encrypted ID, 

the institution could monitor responses in such a way that it can determine who has 

responded without being able to link responses to respondents, and could send e-mail 

messages to non-respondents. This may not be desirable, however, and a non-response 

bias analysis may be the only feasible form of follow-up. The NCES standard of a 

response rate of 85 percent for a census not used as a sampling frame (U.S.DOE, 2003) 

does not seem likely to be met in the absence of follow-up. 

If the response rate falls below 85 percent, then the NCES statistical standards (U.S. 

DOE, 2003) call for a non-response bias analysis, which is described in section 5.5.2. 

The purpose of the non-response bias analysis is to determine whether the likelihood of 

response depends on the value of the response. In this case, there is reason to suspect that 

it does: those with interest and ability are more likely to respond. 

The non-response bias analysis consists of a data collection phase and an analysis phase. 

In the former, a random sample of non-respondents to the census is contacted and their 

responses ascertained. The data collection phase of the non-response bias analysis would 

ordinarily be done by telephone. 

As discussed in section 5.2, “requiring” a response in a strong sense (that failure to 

respond places a student somehow “not in good standing”) may not be possible. 

Nevertheless, there may be methods beyond follow-up that increase response. These 

include: 

 

• Embedding the survey in a process—registration is the clear and perhaps only 

example—that every student must perform. This was somewhat successful in 

some of the OCR cases. The web-based instrument in section 5.4.1 could be 

linked in a natural way to web-based registration, and could inherit the 

confidentiality protections and mechanisms for preventing multiple responses 

associated with the registration process. 

• Positive mechanisms: for instance, one survey in chapter 4 offers a tee shirt said 

to be worth $10 to respondents. Alone these seem unlikely to be effective at a 

scale at which they are feasible economically. Moreover, their effects on nonresponse bias are difficult to characterize. 

• Public relations activities, including statements by institution leaders about the 

importance of responding. 

 

Practice among SURVEY Institutions: The proactivity characteristic reported in table 3.6 

measures, albeit qualitatively, the extent to which SURVEY instruments were proactive 

in attempting to increase response rates. This table shows little evidence of proactivity. 

Recommendation for Improvement: The data collection process should be web-based, 

using e-mail and, if necessary to follow up on non-response, telephone as a means of 
54

contacting members of the data collection population. Linking the data collection to 

registration should be considered. 

High-Quality Recommendation: The data collection should be web-based, using e-mail to 

initiate the process. Linking the data collection to registration should be considered. 

Means such as those described in section 5.5 should be in place to deal with low response 

rates. 

5.4 Web-Based Data Collection 

In this section, drawing on chapters 3 and 4, we sketch the structure of a data collection 

instrument that would accomplish the necessary objectives (collection of information 

regarding whether “interests and abilities of the members of that [underrepresented] sex 

have been fully and effectively accommodated by the present program.”) with minimal 

effort. The instrument is web-based, allowing effective implementation of skips and other 

selection devices. For example, a respondent can choose from a larger list of sports the 

subset for which she wishes to respond in detail. 

Web-based instruments are absent entirely from the SURVEY institution instruments 

discussed in chapter 3, which is understandable because most of the OCR cases preceded 

widespread use of the web and e-mail. By contrast, they dominate in the more recent 

instruments in chapter 4. Alternative modes of data collection appearing in the 

instruments in chapters 3 and 4 include paper forms (mailed, distributed in classes, or 

“left for the taking”) and one telephone survey. 

Section 5.4.1 describes the instrument itself; the software necessary to create it and store 

data is discussed in section 5.4.1. 

5.4.1 Data Collection Instrument 

The proposed data collection instrument consists of seven screens, which are shown in 

figures 5.1-5.7, together with one additional screen, shown in figure 5.8, to which only 

respondents in the data analysis population who indicated interest and ability are taken. 

This instrument requests little more information than absolutely necessary to determine 

compliance with Title IX. Requests for large amounts of superfluous information are a 

concern because they lower response rates. However, compared to some of the 

instruments in chapter 3, this instrument is quite minimal. 

We now describe the screens one-by-one. Not all respondents need to proceed through all 

seven (or eight) screens. 

Screen 1, shown in figure 5.1, is an introductory screen, on which respondents are 

informed of the purposes of the data collection and provided an explicit confidentiality 

statement as well as an explanation of the structure of the instrument. 
55

Screen 2, shown in figure 5.2, requests four items of demographic information—age, 

gender, year in school and student status. The dropdown boxes and radio buttons 

constrain responses to those allowed by the institution conducting the census. 

Screen 3, shown in figure 5.3, explains the next set of questions—on athletic experience, 

participation and ability. It allows respondents with no experience, current participation 

or interest in future participation to so indicate and complete the instrument without 

having to view any of the other screens. 

Screen 4 of the proposed instrument, shown in figure 5.4, is reached only by respondents 

who wish to enter information concerning athletic experience, interests and abilities. It 

lists the responses that will be allowed when the information is requested (on screen 6), 

and contains a neutral statement of the burdens and benefits associated with participation 

in intercollegiate athletics. A more sophisticated version of the instrument might contain 

hyperlinks to definitions of various terms. 

Screen 5, shown in figure 5.5, allows respondents who wish to enter information 

concerning athletic experience, interests and abilities to select the sports for which they 

wish to provide information. The purpose of this is to reduce the size and complexity of 

screen 6, on which the information is actually entered. Only those sports selected on 

screen 5 are listed on screen 6. The list of sports in figure 5.5 is illustrative, consisting of 

twenty-three sports in which the NCAA conducts championships and seven NCAAidentified “emerging sports.”3

 Reflecting the considerations noted in section 5.7, sports 

for which Title IX non-compliance is not feasible because of the absence of competitive 

opportunities would not need to be included on this screen. 

Screen 6, shown in figure 5.6, is where actual information regarding experience, current 

participation, interest in future participation and ability is entered. These four categories 

appear side-by-side, which is sensible conceptually and psychometrically, but was not 

done in any of the 57 data collection instruments reviewed in chapters 3 and 4. The 

allowable responses, which are constrained by radio buttons that also prevent multiple 

responses, are as follows: 

• For experience at the high school level, “Recreational,” “Intramural,” “Club,” 

“Junior Varsity” and “Varsity”; 

• For current participation, “Recreational,” “Intramural,” “Club” and 

“Intercollegiate”; 

• For interest in future participation at the institution: “Recreational,” “Intramural,” 

“Club” and “Intercollegiate”; 

• For ability: “Yes, I have the ability” and “No, I would need to develop the 

ability”. 

The reason for inclusion of four separate categories is that, as discussed further in section 

5.5.1, a positive response is defined in terms of at least three and possibly all four. 

 

3

 See www2.ncaa.org/sports/general_information/emerging_sports.html. 
56

For some purposes, the number of allowable responses might be reduced. If the sole 

concern were interest in intercollegiate participation, and assuming that an intercollegiate 

team does not exist currently, “Intercollegiate” could be eliminated from current 

participation (as could any others than do not apply), and all but “Intercollegiate” could 

be eliminated from interest in future participation. Because this screen would be 

generated dynamically, using information from screen 5, the sets of allowable responses 

can be sport-dependent. 

Although not shown in figure 5.6, hyperlinks could be used to provide access to 

definitions of these terms (or any other terms, for example, sports with which not all 

respondents may be familiar). Placing the definitions in a separate window avoids 

impeding the flow of the data collection instrument. 

Screen 6 does not implement default responses, but it could. It does state clearly how 

item non-responses will be treated. 

Screen 7, shown in figure 5.7, is for most respondents the final screen of the instrument. 

It offers the opportunity for comments or other feedback, asks them to click a button to 

record their responses, and thanks them for participating. It also informs those who have 

responded positively about interest and ability that unless they check the “Check here if 

you do not wish to be contacted” box, they will be taken to one more screen (screen 8), 

on which they will be asked for contact information. 

Screen 8, shown in figure 5.8, is a screen reached only by respondents who are members 

of the data analysis population (in the example in this chapter, full-time female students 

who are freshmen, sophomores or juniors) who indicate interest and ability in one or 

more sports not currently offered at the intercollegiate level. (The language on both this 

screen and screen 7 is less precise, because no clear purpose is served by complete 

details.) On this screen, such respondents may request to be contacted by the athletic 

department regarding their interests. It summarizes their responses and asks for contact 

information—name, e-mail address and telephone number. This information could either 

be stored in a CSV file separate from the main data, or forwarded by e-mail to the 

appropriate office in the institution. 

Practice among SURVEY Institutions: As noted above, no SURVEY institution 

employed web-based data collection. 

Recommendation for Improvement: For extremely narrow surveys (one sport, for 

example) respondents might be asked simply to respond to an e-mail message. Simplified 

web-instruments (for instance, with less protection for confidentiality) are possible, but 

should state explicitly their shortcomings relative to the high-quality recommendation. 

Although, as stated in section 5.6, we believe that if failure to respond (at the item level) 

is treated as a “no interest” or “no ability” response, the instrument should state so in 

some difficult-to-ignore manner. 
57

High-Quality Recommendation: The web-based data collection instrument should have 

the same properties as the prototype described here: 

• Simplicity; 

• Explicit explanation of reasons for the data collection; 

• Explicit confidentiality statement; 

• Opportunity for global “no interest or ability” response; 

• Opportunity to filter sports for which detail is provided; 

• Non-prejudicial wording of items; 

• Inclusion of all of experience, current participation, interest in future participation 

and ability; 

• Fixed-form responses. 

Figure 5.1: Initial screen of the prototype data collection instrument, containing the purpose of the 

data collection, a confidentiality statement and an explanation of the structure of the instrument. 

SOURCE: NISS-produced prototype. 
58

Figure 5.2: Second screen of the prototype data collection instrument, in which respondents provide 

four items of demographic and student status information. This example shows a respondent who is 

20 years old, female, a junior and a full-time student. 

SOURCE: NISS-produced prototype. 
59

Figure 5.3: Third screen of the prototype data collection instrument, on which respondents with no 

experience, current participation or interest in future participation can so indicate and complete the 

process. 

SOURCE: NISS-produced prototype. 
60

Figure 5.4: Fourth screen of the prototype data collection instrument, which is reached only by 

respondents who wish to enter information concerning athletic experience, interests and abilities. 

SOURCE: NISS-produced prototype. 
61

Figure 5.5: Fifth screen of the prototype data collection instrument, which is reached only by 

respondents who wish to enter information concerning athletic experience, interests and abilities. 

Here, respondents select the sports for which they wish to provide information. The list consists of 23 

sports in which the NCAA conducts championships and seven “emerging sports.” The respondent 

illustrated here has chosen basketball, lacrosse and volleyball, which appear in screen 6. 

SOURCE: NISS-produced prototype. 
62

Figure 5.6: Sixth screen of the prototype data collection instrument, on which respondents enter 

information concerning experience, current participation, interest in future participation and ability 

only for those sports selected on screen 5. Continuing the example from screen 5, the respondent—

who is female—has indicated high-school varsity experience, current intramural participation, 

interest in intercollegiate participation and ability for lacrosse. 

SOURCE: NISS-produced prototype. 
63

Figure 5.7: Seventh, and for most respondents final, screen of the prototype data collection 

instrument, which offers respondents the opportunity to provide comments or other feedback. 

Respondents who are members of the data analysis population and have expressed an interest in one 

or more sports (in our example, full-time female students who are freshmen, sophomores or juniors) 

are taken automatically to screen 8 unless they check the box “Check here if you do not wish to be 

contacted.” 

SOURCE: NISS-produced prototype. 
64

Figure 5.8: Eighth and final screen of the data collection instrument, reached only by respondents 

who are members of the data analysis population (in our example, full-time female students who are 

freshmen, sophomores or juniors), on which they are asked whether they wish to be contacted by the 

athletic department, and if so to provide contact information. The illustrative values here are the 

same as in figures 5.2-5.7. 

SOURCE: NISS-produced prototype. 
65

5.4.2 Software 

The screens shown in figures 5.1-5.8 are static HTML prototypes. There are at least two 

paths to create the software for a full-blown implementation that would generate most of 

these screens dynamically. 

The first of these is commercial tools for web surveys, such as Survey Solutions (Perseus 

Corporation, 2005). The extent to which commercial tools support functionality such as 

confidentiality-preserving non-response follow-up is not clear. It is also likely that they 

involve significant hardware and software overheads that are really not necessary in the 

setting of this chapter. 

The second path is to create CGI, Java, or Visual Basic scripts that 

• Allow movement from each screen to the next, including dynamic generation of 

all screens other than the initial one in figure 5.1; 

• Record responses (see additional discussion below). 

Implementation of these scripts, together with appropriate security and support, would be 

a moderate-sized but straightforward programming task. 

However, full automation of the process would require one more layer of scripts that 

would customize such items as 

• The institution name; 

• Details of wording, with defaults provided that can be edited as necessary; 

• The list of sports on screen 5; 

• The possibly sport-dependent responses on screen 6. 

Storage of responses is straightforward. Other than the free-form text response on screen 

7, the instrument described in section 5.4.1 generates only a small number of data items 

for each respondent: 

• Four items of demographic information from screen 2; 

• One Yes/No global “no athletic interest, current participation or interest in future 

participation” from screen 3; 

• K Yes/No responses for each sport from screen 5, where K is the number of sports 

listed there; 

• At most 4K categorical responses from screen 6, one each for experience, current 

participation, interest in future participation and ability. 

The total number of items is 5K + 5. To illustrate, for the 30 sports shown in the example 

in figure 5.5, the survey generates 155 items. 

There are, of course, constraints on the item values. First, the instrument itself permits 

only pre-defined responses to all items (other than on screen 7). This guarantees that 

responses are interpretable and analyzable. Second, if the value of the global Yes/No 
66

response on screen 3 is “No,” then the remaining 5K responses are empty. Similarly if the 

response on screen 5 for a sport is “No,” meaning that it is not listed on screen 6, then its 

four items associated with screen 6 are empty. 

Finally, the software could store the data in a comma-separated-variable (CSV) file, with 

one (5K + 5) item line for each response. Since no identifying information is stored, 

confidentiality of responses is guaranteed. The CSV data file can then be read and 

manipulated by tools ranging from Microsoft Excel to more powerful statistical packages 

such as SAS, SPSS, STATA and S-Plus. 

The encrypted ID provided in the original e-mail would be severed from the response 

itself, and used in the database file containing respondents’ e-mail addresses to mark that 

a response had been received. 

Practice among SURVEY Institutions: No SURVEY institution employed web-based 

data collection. 

Recommendation for Improvement: OCR, NCES or another agency should investigate 

use of commercial software, and either recommend particular products or support 

development of custom tools such as those described here. 

High-Quality Recommendation: OCR, NCES or another agency should investigate use of 

commercial software, and either recommend particular products or support development 

of custom tools such as those described here. In either case, data should be stored in 

“long form” (allowing empty responses) in CSV files, which maximizes flexibility and 

portability. 

5.5 Data Analysis 

Recall that goal analysis of the data generated by the census is to estimate + N , the 

number of students in the data analysis population interested and able to participate at the 

intercollegiate level in the given sport. If data concerning multiple sports is collected in 

the same instrument, then each sport requires an analysis of the type described in this 

section. For technical reasons, it is necessary to calculate the distribution of the 

estimator + Nˆ . 

We assume throughout this section that respondents answer truthfully. While there is 

good reason to expect untruthful responses in some settings, there seems to be none in 

this case. 

As discussed in section 5.2, the data analysis population consists of the subset of 

members of the data collection population whose data are relevant to determination of 

compliance with Title IX. Although the techniques in this section are applicable to any 

choice of the data analysis population, for concreteness, we take that population to be 

full-time female students not in their final year of study. 
67

Practice among SURVEY Institutions: There is little evidence in the OCR files that any 

of the 57 SURVEY institutions or the five “additional survey” institutions in chapter 4 

has conducted principled statistical analyses of their data that account for possible nonresponse bias. 

5.5.1 Preprocessing 

The “raw data” generated by the data collection instrument in section 5.4.1 consist, in a 

form different from that in section 5.4.1, of six items for each sport: 

• The global Yes/No response from screen 3, in which case the remaining five 

items are empty; 

• The Yes/No selection response for that sport from screen 5; if this response is 

“No,” then the remaining four items are empty; 

• The four categorical responses from screen 6. 

Preprocessing of the data reduces these six items to a single Yes/No response, with “Yes” 

signifying that the respondent is interested in and able to participate in the sport at the 

intercollegiate level, and “No” signifying all other cases. While there is some 

flexibility—and obviously changes would be needed if a different instrument were 

employed—we recommend that “Yes” require all of the following conditions: 

 

• On, screen 3, global response = “Yes”; 

• On screen 5, selection response for that sport = “Yes”; 

• On screen 6, for that sport 

o Experience in high school for = “Club,” “Junior Varsity” or “Varsity” 

o Current participation at any level (i.e., the response is not empty) 

o Interest in future participation = “Intercollegiate”. 

This approach provides justification for three of the categories on screen 6. 

Another approach, keeping in mind that ability is self-characterized, would also require 

• On screen 6, Ability = “Yes, I have the ability” 

We do not specifically recommend for or against this stronger criterion. 

Effecting this preprocessing, either within a statistical package or with dedicated 

software, is straightforward. The result, for each sport and respondent, is a single Yes/No 

item. 

Recommendation for Improvement: Data should be pre-processed to produce a single 

“Yes = Interested and able” or “No = either not interested or not able” response for each 

respondent and each sport. 
68

High-Quality Recommendation: Data should be pre-processed to produce a single “Yes = 

Interested and able” or “No = either not interested or not able” response for each 

respondent and each sport. 

5.5.2 Data Analysis in the Absence of Non-Response 

In this section and the next, “non-response” means only subject-level non-response—

failure of a member of the data collection population to respond at all. The data collection 

instrument in section 5.4 contains two mechanisms designed to minimize item nonresponse. The first is screen 3, which permits a global “no experience, participation or 

interest” response that concludes the data collection. The second is the statement on 

screen 6 that “... missing responses will be treated as (depending on category) “No 

Experience,” “No Current Participation,” “No Interest in Future Participation” and for 

Ability, “Not applicable.”.” 

Here we describe analysis of the data for a single sport when there is not a problem with 

response rate. According to NCES statistical standards for censuses (U.S. DOE, 2003), 

this requires a subject-level response rate of at least 85 percent. 

In this case, the analysis is straightforward. Recall the notation: 

• P = size of target population, all of whom have been requested to provide 

information; 

• + N = number of women in the target population who are interested in and able to 

participate in the sport at the intercollegiate level; 

• M = minimum team size. 

Also, let 

• R = number of respondents; 

• + NR = number of “Yes” responses. 

Table 5.1 summarizes the definitions and sources of these values. 

Table 5.1: Sources of values used to estimate + N

Value Definition Source 

P Target population size Institution + N Number of “Yes” responses in full target population To be estimated

M Minimum team size Institution 

R Number of responses Data + N R Number of “Yes” responses Data 

Source: This document. 
69

It is important to emphasize that we do not assume that R = P, which corresponds to a 

100 percent response rate. Were this to happen, then + + N = NR , and the simple decision 

rule in section 5.1 can be applied by OCR. 

Instead, we assume only that R ≥ .85P . This obviates the need for the more complex 

analysis described in section 5.5.3, but still requires estimation of the number of “Yes” 

responses among non-respondents. The estimated value of + N is then 

( ) ˆ P R

R

N N N R = R + −

+

+ +

The principal assumption underlying this equation is that the rate of “Yes” responses 

among the P-R non-respondents has the same value, namely NR / R + , as among 

respondents. This is shown pictorially in Figure 5.9. 

Figure 5.9: Pictorial representation of data analysis in the absence of significant non-response. The 

proportion of “Yes” responses among non-respondents is assumed to be the same as among 

respondents. 

Source: NISS 

The value + Nˆ is not exact. Rather, it is a statistical estimator derived from the census 

data. As such, it has an associated probability distribution. As noted in section 5.1, we 

recommend that OCR employ a decision criterion of the form 

• If ≥ > α + } ˆ P{N M , where α is a threshold set in advance by agreement between 

OCR and the institution—in words, if the data indicate that there is sufficiently 

high probability that + Nˆ exceeds M, then the data collection demonstrates that 

the interests and abilities of women are not accommodated by the present 

program. What action would be necessary, and under what additional conditions 

(see section 5.1), must be determined by OCR. 

• If ≥ ≤ α + } ˆ P{N M (in words, if the data indicate that there is not sufficiently high 

probability that + Nˆ exceeds M), then the interests and abilities of women are 

accommodated by the present program, and no action by the institution is 

necessary. 
70

Computation of } ˆ P{N ≥ M + is a technical issue, because in models such as that 

described in appendix F, this probability depends on the unknown value + N . These 

difficulties can be avoided by treating the data values as known rather than unknown 

values, a simplification that is acceptable for high response rates but not when, as in 

section 5.5.2, a non-response bias analysis is conducted. Conditional on the data, the 

second component of + Nˆ , that is, the estimated number of “Yes” responses among nonrespondents, has a binomial distribution with parameters P – R and NR / R + , and tables, 

approximations or simulation may be used to calculate } ˆ P{N ≥ M + . 

There is one case in which this entire analysis is unnecessary. If NR ≥ M + then among the 

census respondents alone there are sufficiently many interested and able students, and it 

is certain that N ≥ M + . 

Practice among SURVEY Institutions: Only one-half of the OCR files containing data 

collection instruments report response rates or contain enough information to calculate 

response rates; reported values range from less than 1 percent to approximately 70 

percent. There is no description in the files of any principled statistical analyses that were 

performed on the data. Those files that do contain results have nothing beyond 

tabulations of responses to items on the data collection instrument. Not one file contains 

any evidence that results were viewed as uncertain, or that uncertainties were calculated. 

Recommendation for Improvement: An institution that does follow up on nonrespondents should include in both e-mails and the web-based data collection instrument 

explicit, difficult-to-ignore statements that non-responses will be recorded as “no 

interest.” In this case, the estimated value of + N is 

+ + N = NR ˆ

In this case, + N is certain to exceed + Nˆ , so a decision criterion on the form 

• If N < M + ˆ , then the institution is in compliance with Title IX 

• If N ≥ M + ˆ , then the institution may not be in compliance with Title IX, 

depending on additional considerations discussed above and in section 5.7 

is lenient in favor of the institution. 

As an intermediate step, non-responses should not be treated as “no interest” and + N should be estimated as 

( ) ˆ P R

R

N N N R = R + −

+

+ +

The decision criterion stated above is then unbiased, neither advantaging nor 

disadvantaging the institution. 
71

High-Quality Recommendation: In the absence of significant non-response (that is, if the 

response rate exceeds 85 percent), + N should be estimated using methods described here, 

and the compliance decision should be based on } ˆ P{N ≥ M + . 

5.5.3 Data Analysis in the Presence of Non-Response 

When there is significant non-response, the central assumption in section 5.2.1 and figure 

5.9 is called into question: that the probability of a “Yes” response is the same among 

non-respondents as among respondents. This failure is known as non-response bias and is 

shown pictorially in Figure 5.10, where the relative frequency of “Yes” is higher for 

respondents than for non-respondents. 

Figure 5.10: Pictorial representation of non-response bias. “Yes” responses are relatively more 

frequent among respondents than among non-respondents. 

Source: NISS. 

There is, in fact, reason to suspect that the phenomenon in figure 5.8 arises in the setting 

of this chapter. Students who are interested and consider themselves able to participate in 

the sport at the intercollegiate level clearly have reason to respond, while those with no 

athletic interests have much less motivation to respond. 

NCES statistical standards (U.S.DOE, 2003) prescribe that when the response rate (R/P

in the notation of section 5.2.1) is less than 85 percent a non-response bias analysis be 

conducted. This process, which can be very resource-intensive, consists of 

• Selecting (in most cases) a random sample of non-respondents; 

• Contacting them (almost always by phone, to maximize the chances of 

reaching them); 

• Ascertaining their response. 

In principle, every selected non-respondent should be contacted, but often of course this 

is not possible. 

Before proceeding, we emphasize that if NR ≥ M + , then regardless of the response rate, it 

is certain that N ≥ M + , because among the census respondents alone there are 
72

sufficiently many interested and able students. If this happens, there is no need to collect 

any additional data, and in particular no need for the non-response bias analysis. 

In the most simplistic case, the products of the non-response bias analysis are: 

• A sample size S NR ; 

• A number + NNRBS of positive responses in the non-response bias analysis sample, 

from students in the non-response bias analysis sample who are interested and 

able to participate in the sport at the intercollegiate level; 

• An estimate qNR ˆ of the frequency of “Yes” responses among the remaining nonrespondents (i.e., those not in the non-response bias analysis sample). Assuming 

that the non-response bias analysis sample is a simple random sample, then in 

almost all cases, qNR NNRBS S NR ˆ / + = . 

Using the same notation as in section 5.2.1, the estimated value of + N is then 

ˆ ( ) ˆ R NRBS NR NR N = N + N + q P − R − S + + +

The distribution of + Nˆ , even conditional on the data, is more complex than in section 

5.2.2, because qNR ˆ is based on a sample of non-respondents. An approach that parallels 

the approach in section 5.5.2 is to condition on all data values. In this case, the first two 

terms in the expression above are known, and the third term—corresponding to the 

number of students who neither responded nor are in the non-response bias analysis 

sample but who are interested and able—has a binomial distribution with parameters 

P − R − S NR and qNR NNRBS S NR ˆ / + = , which allows calculation of } ˆ P{N ≥ M + . Then, the 

decision criteria described in section 5.5.2 can be applied in the same manner. 

The approach in the preceding paragraph overlooks randomness associated with the nonresponse bias sample. At a deeper level, it is also inconsistent with the underlying 

purpose of the non-response bias analysis, which is to determine if the probability of 

response depends on whether the response is positive or negative. A Bayesian modeling 

strategy is outlined in appendix F. In principle, it should be preferred to the approach 

described in the preceding paragraph, but it may be beyond the capabilities of some 

institutions, although the services of those who could implement it are readily available. 

Recommendation for Improvement: In the presence of significant non-response (that is, if 

the response rate is less than 85 percent and consequent possible non-response bias, OCR 

should require, as NCES does, a non-response bias analysis. The decision criterion can be 

based solely on the estimator ˆ ( ) ˆ R NRBS NR NR N = N + N + q P − R − S + + + , without 

consideration of associated uncertainties. 

High-Quality Recommendation: In the presence of significant non-response (that is, if the 

response rate is less than 85 percent) and consequent possible non-response bias, OCR 
73

should require, as NCES does, a non-response bias analysis. The estimated probability 

that + N exceeds M should be calculated using methods described in appendix F. 

5.6 Precautions 

The process outlined in sections 5.1-5.6 contains a number of safeguards against what 

would ordinarily be considered an “unsafe” situation: data collection conducted by an 

organization with a definable vested interest in the outcome. Specifically, if the 

institution did not want to offer the sport at the intercollegiate level, then one can assume 

that the institution would like the data collection to demonstrate + N is less than M, so that 

it will not have to offer the sport. 

The census recommendation in section 5.2 avoids a potentially biased sample—a subset 

of the target population where interest and ability are thought to be low. Interestingly, 

many of the INSTRUMENT institution surveys summarized in chapter 3 actually focus 

on sub-populations where interest and ability would be expected to be high, such as 

students enrolled in physical education courses. This recommendation also avoids 

excessively small samples meant to produce no interested and able respondents. As 

discussed in section 5.2, such a strategy is risky, because every positive response carries 

high weight. 

The data collection instrument in section 5.4.1 contains no prejudicial items or wording 

designed to induce negative responses. Some, but relatively few, of the instruments 

summarized in chapter 3 exhibit this shortcoming. 

The data analysis procedures in section 5.5 specifically do not ever equate failure to 

respond at all with either lack of interest or ability. The OCR files do not contain 

information sufficient to determine whether this was done by any of the INSTRUMENT

institutions. 

5.7 Pre- and Post-Data Collection Procedures 

The issues noted here lie outside the data collection process itself, but are important. 

When an institution should (or must) conduct data collections such as those described in 

this chapter is influenced by both external and internal factors. OCR complaints or 

monitoring activities (see section 2.2.4) are the principal external stimuli. As noted in 

section 2.3, in the OCR files analyzed by NISS, use of Part 3, and within PART 3

institutions, use of data collections, seem to increase over time. An institution may also, 

however, wish to conduct periodic assessments on its own, or in response to petitions 

from students. 

An OCR determination of non-compliance requires that the region in which the 

institution is located offer competitive opportunities in the sport for the under-represented 

sex. Presumably this can be determined in advance of any data collection. Whether the 

conference to which the institution belongs offers competitive opportunities is not part of 

the decision process. 
74

Following completion of the data collection, if the estimated value of + N exceeds the 

minimum team size M, then additional steps take place before OCR would determine that 

the institution must offer the sport at the intercollegiate level: 

1. Especially if the data collection is confidential as in section 5.4, the institution 

must identify those students who stated themselves to be interested, and ascertain 

that they remain interested. Holding one or more widely announced meetings 

appears to be the most common way of doing this. Students who self-identified 

using the process pictured in figure 5.8 can be contacted directly. If the number of 

students identified at this stage is less than M, the remaining steps do not occur. 

There is no logical necessity, however, that this number be less than + Nˆ : the 

meeting itself may generate additional students—in particular, non-respondents to 

the data collection—who consider themselves interested and able. 

2. If sufficiently many students are identified as interested, the institution 

(ordinarily, the athletic department) must determine if those students who state 

they are able to compete at the intercollegiate level are actually able do to so. In 

most cases, this would be by means of tryouts. 

3. Finally, if the number of students who are interested and able, as determined by 

steps 1 and 2, exceeds M, then OCR would declare the institution not to be in 

compliance with Title IX. 

Considerations of cost (that is, whether the institution would need to build or otherwise 

access) new facilities are not part of the OCR criteria for determination of Title IX 

compliance. 
75
76

Acknowledgments 

The report was written by Alan F. Karr, Director of NISS, and Ashish P. Sanil, Research 

Statistician at NISS. Katherine Kantner of NISS performed an inventory of material in 

the files. We thank John Clement and Susan Lapham of the Education Statistics Services 

Institute (ESSI) for extensive preparatory efforts on the data. Dennis Carroll, Associate 

Commissioner of NCES, has provided extremely valuable advice and guidance. 
77
78

References 

Converse, J. M. and Presser, S. (1989). Survey Questions: Handcrafting the Standardized 

Questionnaire. Newbury Park, CA: Sage. 

Dillman, D. A. (2000). Mail and Internet Surveys: The Tailored Design Method. New 

York: Wiley. 

Gelman, A. Carlin, J. B., Stern, H. S., and Rubin, D. B. (1995). Bayesian Data Analysis. 

London: Chapman and Hall. 

Miller, D. C., and Salkind, N. J. (2002). Handbook of Research Design and Social 

Measurement, Sixth Edition. Thousand Oaks, CA: Sage. 

U.S. Department of Education, Institute of Education Sciences, National Center for 

Education Statistics (2003). NCES Statistical Standards, NCES 2003-601. 

National Collegiate Athletic Association (2004). Gender Equity/Title IX. Information 

available on-line at 

www1.ncaa.org/membership/ed_outreach/gender_equity/resource_materials/ind

ex.html

Perseus Corporation (2005). Survey Solutions. Information available on-line at 

www.perseus.com. 

Wentland, E. J., and Smith, K. W. (1993). Survey Responses: An Evaluation of their 

Validity. New York: Academic Press. 
79
80

Appendix A: List of OCR Populations 

This appendix lists the 130 OCR institutions, and categorizes each as belonging to 

SURVEY, NO SURVEY or PARTS 1 AND 2 (also referred to in chapter 2 as NONPART 3). PART 3 is the union of SURVEY and NO SURVEY. Each institution is also 

characterized as to whether it belongs to COMPLAINT or COMPLIANCE 

MONITORING. See figure 1.1 for details. 

ID Name Population 

COMPLAINT / 

COMPLIANCE 

MONITORING 

100724 Alabama State University NO SURVEY COMPLAINT 

164447 American International College SURVEY COMPLIANCE 

MONITORING 

109350 Antelope Valley Community College SURVEY COMPLAINT 

104151 Arizona State University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

106458 Arkansas State University NO SURVEY COMPLAINT 

100858 Auburn University NO SURVEY COMPLAINT 

223232 Baylor University SURVEY COMPLAINT 

175421 Belhaven College SURVEY COMPLAINT 

219709 Belmont University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

142115 Boise State University SURVEY COMPLIANCE 

MONITORING 

128744 University of Bridgeport PARTS 1 AND 2 COMPLAINT 

230038 Brigham Young University PARTS 1 AND 2 COMPLAINT 

198084 Brunswick Community College SURVEY COMPLAINT 

110662 University of California Los Angeles PARTS 1 AND 2 COMPLAINT 

110705 University of California Santa Barbara PARTS 1 AND 2 COMPLIANCE 

MONITORING 

110565 California State University Fullerton PARTS 1 AND 2 COMPLIANCE 

MONITORING 

198215 Catawba College PARTS 1 AND 2 COMPLAINT 

198233 Catawba Valley Community College SURVEY COMPLAINT 

128771 Central Connecticut State University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

132903 University of Central Florida NO SURVEY* COMPLAINT 

206941 University of Central Oklahoma SURVEY COMPLIANCE 

MONITORING 

111887 Cerritos College SURVEY COMPLIANCE 

MONITORING 

217688 Charleston Southern University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

166124 College of the Holy Cross SURVEY COMPLIANCE 

MONITORING 

126614 University of Colorado PARTS 1 AND 2 COMPLAINT 

126818 Colorado State University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

162210 Columbia Union College PARTS 1 AND 2 COMPLAINT 
81

198367 Craven Community College NO SURVEY* COMPLAINT 

113193 Cuesta College PARTS 1 AND 2 COMPLAINT 

182670 Dartmouth College PARTS 1 AND 2 COMPLAINT 

113634 Diablo Valley Community College NO SURVEY COMPLIANCE 

MONITORING 

144892 Eastern Illinois University SURVEY COMPLIANCE 

MONITORING 

156620 Eastern Kentucky University SURVEY COMPLAINT 

113980 El Camino Community College SURVEY COMPLIANCE 

MONITORING 

155025 Emporia State University SURVEY COMPLAINT 

101189 Faulkner University NO SURVEY* COMPLAINT 

169910 Ferris State University PARTS 1 AND 2 COMPLAINT 

198552 Forsyth Tech Community College SURVEY COMPLAINT 

110556 Fresno State University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

131496 Georgetown University SURVEY COMPLIANCE 

MONITORING 

237385 Glenville State College SURVEY COMPLIANCE 

MONITORING 

159009 Grambling University SURVEY COMPLAINT 

104717 Grand Canyon University SURVEY COMPLAINT 

191649 Hofstra University SURVEY COMPLAINT 

145619 Illinois Benedictine College NO SURVEY* COMPLAINT 

151351 Indiana University NO SURVEY COMPLAINT 

151324 Indiana State University SURVEY COMPLIANCE 

MONITORING 

153603 Iowa State University NO SURVEY COMPLIANCE 

MONITORING 

175856 Jackson State University NO SURVEY COMPLAINT 

101480 Jacksonville State University NO SURVEY COMPLAINT 

134945 Jacksonville University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

156921 Jefferson Community College SURVEY COMPLAINT 

162928 Johns Hopkins University PARTS 1 AND 2 COMPLAINT 

155317 University of Kansas NO SURVEY COMPLAINT 

157076 Kentucky Wesleyan College PARTS 1 AND 2 COMPLAINT 

117788 Los Angeles City College PARTS 1 AND 2 COMPLAINT 

117247 Laney College SURVEY COMPLAINT 

198817 Lenoir Community College SURVEY COMPLAINT 

142328 Lewis-Clark State College NO SURVEY COMPLIANCE 

MONITORING 

173920 Mankato State University PARTS 1 AND 2 COMPLAINT 

163268 University of Maryland Baltimore County SURVEY COMPLAINT 

163286 University of Maryland College Park PARTS 1 AND 2 COMPLAINT 

166629 University of Massachusetts Amherst PARTS 1 AND 2 COMPLAINT 

135726 University of Miami PARTS 1 AND 2 COMPLAINT 

140483 Middle Georgia College NO SURVEY* COMPLAINT 

220978 Middle Tennessee State University NO SURVEY COMPLAINT 

174233 University of Minnesota Duluth PARTS 1 AND 2 COMPLAINT 
82

174066 University of Minnesota Twin Cities PARTS 1 AND 2 COMPLAINT 

180489 University of Montana SURVEY COMPLAINT 

185590 Montclair State University SURVEY COMPLIANCE 

MONITORING 

174358 Moorhead State University SURVEY COMPLIANCE 

MONITORING 

157401 Murray State University PARTS 1 AND 2 COMPLAINT 

227146 Navarro College SURVEY COMPLAINT 

181464 University of Nebraska NO SURVEY COMPLAINT 

187985 University of New Mexico SURVEY COMPLAINT 

188030 New Mexico State University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

199111 University of North Carolina Asheville NO SURVEY COMPLAINT 

199139 University of North Carolina Charlotte NO SURVEY COMPLAINT 

200332 North Dakota State University SURVEY COMPLAINT 

167358 Northeastern University PARTS 1 AND 2 COMPLAINT 

127741 University of Northern Colorado SURVEY COMPLAINT 

147703 Northern Illinois University SURVEY COMPLAINT 

171456 Northern Michigan University SURVEY COMPLIANCE 

MONITORING 

147767 Northwestern University PARTS 1 AND 2 COMPLAINT 

171599 Olivet College SURVEY COMPLAINT 

120342 Orange Coast Community College NO SURVEY* COMPLIANCE 

MONITORING 

209542 Oregon State University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

199333 Pitt Community College SURVEY COMPLAINT 

155681 Pittsburg State University SURVEY COMPLIANCE 

MONITORING 

215293 University of Pittsburgh SURVEY COMPLAINT 

121363 Porterville College PARTS 1 AND 2 COMPLAINT 

218539 Presbyterian College PARTS 1 AND 2 COMPLAINT 

148131 Quincy College SURVEY COMPLAINT 

233277 Radford College PARTS 1 AND 2 COMPLAINT 

195003 Rochester Institute of Technology PARTS 1 AND 2 COMPLAINT 

199485 Rockingham Community College SURVEY COMPLAINT 

122180 Sacramento City College NO SURVEY COMPLAINT 

102049 Samford University NO SURVEY COMPLAINT 

122409 San Diego State University PARTS 1 AND 2 COMPLAINT 

122755 San Jose State University PARTS 1 AND 2 COMPLIANCE 

MONITORING 

102067 Shelton State Community College NO SURVEY COMPLAINT 

123563 Solano Community College NO SURVEY* COMPLIANCE 

MONITORING 

218733 South Carolina State University NO SURVEY* COMPLAINT 

199722 Southeastern Community College SURVEY COMPLAINT 

160612 Southeastern Louisiana University PARTS 1 AND 2 COMPLAINT 

130493 Southern Connecticut State University SURVEY COMPLIANCE 

MONITORING 

179566 Southwest Missouri State University SURVEY COMPLAINT 
83

155812 St Mary College SURVEY COMPLAINT 

123554 St Mary’s College SURVEY COMPLAINT 

186432 St Peters College SURVEY COMPLAINT 

196194 State University of New York Oswego SURVEY COMPLAINT 

199768 Surry Community College SURVEY COMPLAINT 

221740 University of Tennessee Chattanooga NO SURVEY COMPLAINT 

221768 University of Tennessee Martin NO SURVEY* COMPLAINT 

206084 University of Toledo SURVEY COMPLAINT 

164076 Towson State University SURVEY COMPLAINT 

221892 Trevecca Nararene University SURVEY COMPLAINT 

102368 Troy State University NO SURVEY COMPLAINT 

178615 Truman State University SURVEY COMPLAINT 

236887 Walla Walla Community College NO SURVEY* COMPLIANCE 

MONITORING 

230782 Weber State University PARTS 1 AND 2 COMPLAINT 

157951 Western Kentucky University SURVEY COMPLIANCE 

MONITORING 

125727 Westmont College SURVEY COMPLAINT 

156125 Wichita State University PARTS 1 AND 2 COMPLAINT 

199926 Wilkes Community College SURVEY COMPLAINT 

240444 University of Wisconsin Madison PARTS 1 AND 2 COMPLAINT 

240471 University of Wisconsin River Falls PARTS 1 AND 2 COMPLAINT 

240189 University of Wisconsin Whitewater SURVEY COMPLAINT 

240727 University of Wyoming PARTS 1 AND 2 COMPLIANCE 

MONITORING 

237109 Yakima Valley Community College SURVEY COMPLAINT 

 

 

PARTS 1 AND 2 

= 44 

COMPLAINT 

= 95 

 

SURVEY 

= 57 

COMPLIANCE 

MONITORING 

= 35 

 

NO SURVEY 

= 29 

 

 Total = 130 Total = 130 

*Following the completion of our analysis, OCR provided documentation showing that 10 of the 29 

institutions identified as not having surveys in the analysis presented in this report had, in fact, used a 

survey. However, copies of the survey instruments used were not available for analysis and shortness of 

time precluded reanalysis of the affected data. 
84

Appendix B: The Institutional Characteristics

Here we list the 14 characteristics on which institutions are compared in chapter 2. For 

each, we give its name, its definition (which also appears in chapter 2), its nature 

(categorical or numerical), the Integrated Postsecondary Education Data System data 

elements from which it is derived, the IPEDS files containing those data elements and the 

formula defining the characteristic. 

• Sector 

o Definition: The control and level of the institution 

o Nature: Categorical 

o IPEDS data element: SECTOR-19 

o IPEDS source: dctfile_9-30-2004_6774.csv 

o Formula: Sector = SECTOR-19 

• Geographical Region 

o Definition: The geographical region of the U.S. in which the institution is 

located 

o Nature: Categorical 

o IPEDS data element: OBEREG-8 

o IPEDS source: dctfile_9-30-2004_6774.csv 

o Formula: Region = OBEREG-8 

• Urbanicity 

o Definition: The degree of urbanization of the institution's locale 

o Nature: Categorical 

o IPEDS data element: LOCALE-38 

o IPEDS source: dctfile_9-30-2004_6774.csv 

o Formula: Urbanicity = LOCALE-38 

• Carnegie Classification 

o Definition: The 2000 Carnegie Classification of institutions based on their 

degree-granting activities from 1995-96 through 1997-98 

o Nature: Categorical 

o IPEDS data element: CARNEGIE-37 

o IPEDS source: dctfile_9-30-2004_6774.csv 

o Carnegie = CARNEGIE-37 

• Selectivity 

o Definition: The ratio of the total number of admissions to the total number 

of applicants 

o Nature: Numerical percentage 

o IPEDS data elements: APPLCN-3016, ADMSSN-3018 

o IPEDS source: dctfile_9-30-2004_6774.csv 

o Formula: Selectivity = ADMSSN-3018 / APPLCN-3016
85

• In-State Cost

o Definition: The price of attendance for full-time, first-time in-state

undergraduate students for the full academic year, including tuition and 

fees, books and supplies, room and board, and other expenses

o Nature: Numerical

o IPEDS data element: CHG2AY3-551

o IPEDS source: dctfile_9-30-2004_6774.csv

o Formula: In-State Cost = CHG2AY3-551

• Out-of-State Cost

o Definition: The price of attendance for full-time, first-time out-of-state

undergraduate students for the full academic year, including tuition and 

fees, books and supplies, room and board, and other expenses

o Nature: Numerical

o IPEDS data element: CHG3AY3-557

o IPEDS source: dctfile_9-30-2004_6774.csv

o Formula: Out-of-State Cost = CHG3AY3-557

• Enrollment

o Definition: The number of first-time, degree/certificate-seeking 

undergraduate students enrolled (full or part time) at the institution

o Nature: Numerical

o IPEDS data element: ENRLT-3024

o IPEDS source: dctfile_9-30-2004_6774.csv

o Formula: Enrollment = ENRLT-3024

• Percent Female

o Definition: The percentage of the student body that is female

o Nature: Numerical percentage

o IPEDS data elements: EFRACE16-1186, EFRACE16-1188

o IPEDS source: ef2003a_9-30-2004_6253.csv 

o Formula: Percent Female = EFRACE16-1188 / (EFRACE16-1186 + 

EFRACE16-1188)

• Percent Black

o Definition: The percentage of the student body that is Black non-Hispanic

o Nature: Numerical percentage

o IPEDS data element: PCTMIN1-29

o IPEDS source: dctfile_9-30-2004_6774.csv

o Formula: Percent Black = PCTMIN1-29
86

• Percent Out-of-State

o Definition: The percentage of undergraduate students whose state of 

residence is other than the state in which the institution is located

o Nature: Numerical percentage

o IPEDS data elements: FIPS-7, EFCSTATE-1198

o IPEDS sources: dctfile_9-30-2004_6774.csv for FIPS-7, ef2003c_9-30-

2004_6668.csv for EFCSTATE-1198 

o Formula: Percent Out-of-State =∑ ≠ i s s Ns , where Ns is the number of 

students whose state of residence is s and sI is the state in which the 

institution is located.

• Association Membership

o Definition: To which of the NCAA, NAIA, and NJCAA the institution 

belongs, including none and possibly more than one

o Nature: Categorical

o IPEDS data elements: ASSOC1-601, ASSOC1-602, ASSOC1-603:

o IPEDS source: dctfile_9-30-2004_6774.csv

o Association is defined by the following table:

ASSOC1-601 ASSOC1-602 ASSOC1-603 Value of Association 

No No No None 

Yes No No NCAA Only 

No Yes No NAIA Only 

No No No NJCAA Only 

More than one “Yes” Multiple Associations 

• Football

o Definition: Whether the institution is a conference member for football

o Nature: Categorical

o IPEDS data element: SPORT1-607

o IPEDS source: dctfile_9-30-2004_6774.csv

o Formula: Football = SPORT1-607

• Number of Sports

o Definition: The number of sports among football, baseball, basketball and 

cross-country/track for which the institution is a conference member

o Nature: Categorical

o IPEDS data elements: SPORT1-607, SPORT1-609, SPORT1-611, 

SPORT1-613

o IPEDS source: dctfile_9-30-2004_6774.csv 

o Formula: Number of Sports = number of “Yes” responses among 

SPORT1-607, SPORT1-609, SPORT1-611, SPORT1-613
87
88

Appendix C: Complete Set of Institutional Comparisons 

C.1 Comparisons of OCR to COMPARISON 

This section contains the 14 comparisons of the 130 OCR institutions to the 1,723 

COMPARISON institutions. For each comparison there is a tabular summary. For 

categorical characteristics, this table contains complete information. The tables for 

numerical characteristics contain summary statistics of the distributions—the mean, 

standard deviation, minimum value, 25th percentile, median (50 percentile), 75th 

percentile and maximum value. 

Institutional Characteristics

Sector. Table C.1 contains the tabular summary for this characteristic. 

Table C.1: Tabular summary comparing OCR to COMPARISON for the characteristic Sector. 

OCR COMPARISON 

Sector Number Percent Number Percent 

Public, 4-year or above 71 54.62 453 26.29 

Private nonprofit, 4-year or above 32 24.62 801 46.49 

Private for-profit, 4-year or above 0 0.00 6 0.35 

Public, 2-year 27 20.77 436 25.30 

Private nonprofit, 2-year 0 0.00 20 1.16 

Private for-profit, 2-year 0 0.00 7 0.41 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Geographical Region. Table C.2 contains the tabular summary for this characteristic. 

Table C.2: Tabular summary comparing OCR to COMPARISON for the characteristic Region.

OCR COMPARISON 

Region Number Percent Number Percent 

New England 8 6.15 128 7.43 

Mid East 12 9.23 342 19.85 

Great Lakes 14 10.77 265 15.38 

Plains 14 10.77 226 13.12 

Southeast 44 33.85 413 23.97 

Southwest 7 5.38 161 9.34 

Rocky Mountains 9 6.92 53 3.08 

Far West 22 16.92 125 7.25 

Other 0 0.00 10 0.58 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
89

Urbanicity. Table C.3 contains the tabular summary for this characteristic.

Table C.3: Tabular summary comparing OCR to COMPARISON for the characteristic Urbanicity. 

OCR COMPARISON 

Urbanicity Number Percent Number Percent 

Large city 22 16.92 266 15.53 

Mid-size city 42 32.31 446 26.04 

Urban fringe of large city 19 14.62 314 18.33 

Urban fringe of mid-size city 9 6.92 132 7.71 

Large town 11 8.46 65 3.79 

Small town 20 15.38 370 21.60 

Rural 6 4.62 111 6.48 

Other 1 0.77 9 0.53 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Carnegie Classification. Table C.4 contains the tabular summary for this characteristic. 

Table C.4: Tabular summary comparing OCR to COMPARISON for the characteristic Carnegie 

Classification.

OCR COMPARISON

Carnegie Classification Number Percent Number Percent 

Doctoral/Research Universities--Extensive 28 21.54 120 7.13 

Doctoral/Research Universities--Intensive 14 10.77 79 4.70 

Masters Colleges and Universities I 46 35.38 395 23.48 

Masters Colleges and Universities II 3 2.31 90 5.35 

Baccalaureate Colleges--Liberal Arts 5 3.85 190 11.30 

Baccalaureate Colleges--General 7 5.38 260 15.46 

Baccalaureate/Associates Colleges 0 0.00 23 1.37 

Associates Colleges 27 20.77 465 27.65 

Other 0 0.00 60 3.57 

Total 130 100.00 1,682 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Total number of cases is less than 1,723 because of missing responses. 
90

Selectivity. Recall that this is the percentage of applicants who are admitted and 

enrolled. Table C.5 contains the tabular summary for this characteristic. 

Table C.5: Tabular summary comparing OCR to COMPARISON for the characteristic 

SELECTIVITY. 

Selectivity OCR COMPARISON

Mean 69.85 69.56 

Standard Deviation 17.93 18.78 

Minimum 12.55 3.48 

25th Percentile 59.65 59.99 

Median 74.06 73.46 

75th Percentile 82.16 82.10 

Maximum 97.16 100.00 

N/A 30 521 

n 130 1,723

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

In-State Cost. Table C.6 contains the tabular summary for this characteristic. 

Table C.6: Tabular summary comparing OCR to COMPARISON for the characteristic In-State 

Cost. 

In-State Cost OCR COMPARISON 

Mean 7,194.35 10,177.50 

Standard deviation 7,412.00 8,009.88 

Minimum 432 0 

25th percentile 2,668 3,084 

Median 4,122 6,812 

75th percentile 8,154.50 16,685 

Maximum 28,965 30,330 

N/A 0 8 

n 130 1,723 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
91

Out-of-State Cost. Table C.7 contains the tabular summary for this characteristic. 

Table C.7: Tabular summary comparing OCR to COMPARISON for the characteristic Out-ofState Cost. 

Out-of-State Cost OCR COMPARISON 

Mean 12,044.18 12,496.45 

Standard deviation 6,211.07 6,790.17 

Minimum 432 0 

25th percentile 7,322.50 7,076 

Median 11,477.50 11,720 

75th percentile 15,231.50 17,050 

Maximum 28,965 30,330 

N/A 0 8 

n 130 1,723 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Student Body Demographics 

Enrollment. Table C.8 contains the tabular summary for this characteristic. 

Table C.8: Tabular summary comparing OCR to COMPARISON for the characteristic Enrollment.

Enrollment OCR COMPARISON 

Mean 10,910.10 4,769.75 

Standard deviation 9,451.16 6,004.96 

Minimum 588 53 

25th percentile 3,203.50 1,367.50 

Median 8,996.50 2,546 

75th percentile 14,208.75 5,497.50 

Maximum 41,617 48,397 

N/A 130 1,723 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
92

Percent Female. Table C.9 contains the tabular summary for this characteristic. 

Table C.9: Tabular summary comparing OCR to COMPARISON for the characteristic Percent 

Female.

Percent Female OCR COMPARISON 

Mean 56.10 58.50 

Standard deviation 6.01 11.67 

Minimum 30.82 0.00 

25th percentile 52.51 53.90 

Median 56.82 58.28 

75th percentile 60.10 63.01 

Maximum 71.96 100.00 

N/A 0 1 

n 130 1,723 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Black. Table C.10 contains the tabular summary for this characteristic. 

Table C.10: Tabular summary comparing OCR to COMPARISON for the characteristic Percent 

Black.

Percent Black OCR COMPARISON 

Mean 12.27 12.97 

Standard deviation 17.33 19.92 

Minimum 0 0 

25th percentile 3 3 

Median 6 6 

75th percentile 15 13 

Maximum 94 100 

N/A 130 1,723 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
93

Percent Out-of-State. Table C.11 contains the tabular summary for this characteristic. 

Table C.11: Tabular summary comparing OCR to COMPARISON for the characteristic Percent 

Out-of-State.

Percent Out-of-State OCR COMPARISON

Mean 22.26 24.13 

Standard deviation 22.83 24.25 

Minimum 0 0 

25th percentile 3.75 4.56 

Median 16.32 15.10 

75th percentile 36.36 37.06 

Maximum 97.20 100 

N/A 17 351 

n 130 1,723

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Athletic Program Characteristics 

Association Membership. Table C.12 contains the tabular summary for this 

characteristic. 

Table C.12: Tabular summary comparing OCR to COMPARISON for the characteristic 

Association Membership. 

OCR COMPARISON 

Association Membership Number Percent Number Percent 

NCAA only 96 73.85 922 53.51 

NAIA only 6 4.62 278 16.13 

NJCAA only 12 9.23 500 29.02 

Multiple associations 1 0.77 23 1.33 

None 15 11.54 0 0.00 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Football. Table C.13 contains the tabular summary for this characteristic. 

Table C.13: Tabular summary comparing OCR to COMPARISON for the characteristic Football. 

OCR COMPARISON

Football Number Percent Number Percent 

Yes 86 66.15 622 36.10 

No 44 33.85 1,101 63.90 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
94

Number of Sports. Table C.14 contains the tabular summary for this characteristic. 

Table C.14: Tabular summary comparing OCR to COMPARISON for the characteristic Number of 

Sports. 

OCR COMPARISON 

Number of Sports Number Percent Number Percent 

0 27 20.77 512 29.72 

1 0 0.00 58 3.37 

2 2 1.54 195 11.32 

3 34 26.15 428 24.84 

4 67 51.54 530 30.76 

Total 130 100.00 1,723 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

C.2 Comparisons of PART 3 to NON-PART 3 

This section contains the 14 comparisons of the 86 PART 3 institutions to the 44 NONPART 3 institutions. 

 

Institutional Characteristics

Sector. Table C.15 contains the tabular summary for this characteristic. 

Table C.15: Tabular summary comparing PART 3 to NON-PART 3 for Sector. 

PART 3 NON-PART 3

Sector Number Percent Number Percent 

Public, 4-year or above 45 52.33 26 59.09 

Private nonprofit, 4-year or above 17 19.77 15 34.09 

Private for-profit, 4-year or above 0 0.00 0 0.00 

Public, 2-year 24 27.91 3 6.82 

Private nonprofit, 2-year 0 0.00 0 0.00 

Private for-profit, 2-year 0 0.00 0 0.00 

Total 86 100.00 44 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 
95

Geographical Region. Table C.16 contains the tabular summary for this characteristic. 

Table C.16: Tabular summary comparing PART 3 to NON-PART 3 for Region. 

PART 3 NON-PART 3

Region Number Percent Number Percent 

New England 3 3.49 5 11.36 

Mid East 8 9.30 4 9.09 

Great Lakes 10 11.63 4 9.09 

Plains 10 11.63 4 9.09 

Southeast 34 39.53 10 22.73 

Southwest 5 5.81 2 4.55 

Rocky Mountains 4 4.65 5 11.36 

Far West 12 13.95 10 22.73 

Total 86 100.00 44 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Urbanicity. Table C.17 contains the tabular summary for this characteristic. 

Table C.17: Tabular summary comparing PART 3 to NON-PART 3 for Urbanicity. 

PART 3 NON-PART 3

Urbanicity Number Percent Number Percent 

Large city 11 12.79 11 25.00 

Mid-size city 27 31.40 15 34.09 

Urban fringe of large city 13 15.12 6 13.64 

Urban fringe of mid-size city 7 8.14 2 4.55 

Large town 8 9.30 3 6.82 

Small town 14 16.28 6 13.64 

Rural 5 5.81 1 2.27 

Other 1 1.16 0 0.00 

Total 86 100.00 44 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
96

Carnegie Classification. Table C.18 contains the tabular summary for this 

characteristic. 

Table C.18: Tabular summary comparing PART 3 to NON-PART 3 for Carnegie Classification. 

PART 3 NON-PART 3

Carnegie Classification Number Percent Number Percent 

Doctoral/Research Universities--Extensive 11 12.79 17 38.64 

Doctoral/Research Universities--Intensive 10 11.63 4 9.09 

Masters Colleges and Universities I 32 37.21 14 31.82 

Masters Colleges and Universities II 1 1.16 2 4.55 

Baccalaureate Colleges--Liberal Arts 4 4.65 1 2.27 

Baccalaureate Colleges--General 4 4.65 3 6.82 

Baccalaureate/Associates Colleges 0 0.00 0 0.00 

Associates Colleges 24 27.91 3 6.82 

Total 86 100.00 44 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 

Selectivity. Table C.19 contains the tabular summary for this characteristic. 

Table C.19: Tabular summary comparing PART 3 to NON-PART 3 for Selectivity. 

Selectivity PART 3 NON-PART 3

Mean 70.98 68.14 

Standard deviation 17.35 18.86 

Minimum 12.55 18.18 

25th percentile 59.65 59.76 

Median 75.01 73.63 

75th percentile 84.38 80.48 

Maximum 97.16 95.73 

N/A 26 4 

n 86 44

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
97

In-State Cost. Table C.20 contains the tabular summary for this characteristic. 

Table C.20: Tabular summary comparing PART 3 to NON-PART 3 for In-State Cost. 

In-State Cost PART 3 NON-PART 3

Mean 6,132.05 9,270.68 

Standard deviation 6,525.05 8,601.29 

Minimum 432 468 

25th percentile 2,250.75 3,316.50 

Median 3,847 5,125.50 

75th percentile 6,367.75 15,424.50 

Maximum 28,209 28,965 

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Out-of-State Cost. Table C.21 contains the tabular summary for this characteristic. 

Table C.21: Tabular summary comparing PART 3 to NON-PART 3 for Out-of-State Cost. 

Out-of-State Cost PART 3 NON-PART 3

Mean 10,562.21 14,940.77 

Standard deviation 5,501.64 6,551.51 

Minimum 432 3,150 

25th percentile 6,332.50 10,957 

Median 9,797 14,348 

75th percentile 13,151 18,456.50 

Maximum 28,209 28,965

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
98

Student Body Demographics 

Enrollment. Table C.22 contains the tabular summary for this characteristic. 

Table C.22: Tabular summary comparing PART 3 to NON-PART 3 for Enrollment. 

Enrollment PART 3 NON-PART 3

Mean 8,764.27 15,104.23 

Standard deviation 7,444.25 11,455.11 

Minimum 592 588 

25th percentile 2,821 6,282 

Median 6,425 13,122.50 

75th percentile 12,275 21,694.50 

Maximum 35,667 41,617 

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Female. Table C.23 contains the tabular summary for this characteristic.

Table C.23: Tabular summary comparing PART 3 to NON-PART 3 for Percent Female. 

Percent Female PART 3 NON-PART 3

Mean 57.19 53.95 

Standard deviation 5.78 5.92 

Minimum 43.22 30.82 

25th percentile 53.50 50.65 

Median 57.66 53.53 

75th percentile 60.84 58.12 

Maximum 71.96 65.49 

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
99

Percent Black. Table C.24 contains the tabular summary for this characteristic. 

Table C.24: Tabular summary comparing PART 3 to NON-PART 3 for Percent Black. 

Percent Black PART 3 NON-PART 3

Mean 15.09 6.75 

Standard deviation 19.87 8.52 

Minimum 0 0 

25th percentile 4 2 

Median 8.50 4 

75th percentile 18.75 7 

Maximum 94 51 

N/A 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Out-of-State. Table C.25 contains the tabular summary for this characteristic. 

Table C.25: Tabular summary comparing PART 3 to NON-PART 3 for Percent Out-of-State. 

Percent Out-of-State PART 3 NON-PART 3

Mean 17.10 31.68 

Standard deviation 18.83 26.48 

Minimum 0 0 

25th percentile 2.92 8.88 

Median 9.64 29.67 

75th percentile 25.88 47.21 

Maximum 97.05 97.20 

N/A 13 4 

n 86 44 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
100

Athletic Program Characteristics 

Association Membership. Table C.26 contains the tabular summary for this 

characteristic. 

Table C.26: Tabular summary comparing PART 3 to NON-PART 3 for Association Membership. 

PART 3 NON-PART 3

Association Membership Number Percent Number Percent 

NCAA only 55 63.95 41 93.18 

NAIA only 6 6.98 0 0.00 

NJCAA only 12 13.95 0 0.00 

Multiple associations 1 1.16 0 0.00 

None 12 13.95 3 6.82 

Total 86 100.00 44 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Football. Table C.27 contains the tabular summary for this characteristic. 

Table C.27: Tabular summary comparing PART 3 to NON-PART 3 for Football. 

PART 3 NON-PART 3

Football Number Percent Number Percent 

Yes 53 61.63 33 75.00 

No 33 38.37 11 25.00 

Total 86 100.00 44 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Number of Sports. Table C.28 contains the tabular summary for this characteristic. 

Table C.28: Tabular summary comparing PART 3 to NON-PART 3 for Number of Sports. 

PART 3 NON-PART 3

Number of Sports Number Percent Number Percent 

0 24 27.91 3 6.82 

1 0 0.00 0 0.00 

2 2 2.33 0 0.00 

3 18 20.93 16 36.36 

4 42 48.84 25 56.82 

Total 86 100.00 44 100.00 

SOURCE. U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 
101

C.3 Comparisons of SURVEY to NO SURVEY 

This section summarizes the 14 comparisons of the 57 SURVEY institutions to the 29 

NO SURVEY institutions. For each comparison there is a tabular summary. Notable 

effects in the summaries are described. 

Institutional Characteristics 

Sector. Table C.29 contains the tabular summary for this characteristic. SURVEY has 

relatively more “Private nonprofit, 4-year or above” institutions than NO SURVEY, 

which in turn has relatively more “Public, 4-year or above” institutions than SURVEY. 

Table C.29: Tabular summary comparing SURVEY to NO SURVEY for Sector. 

SURVEY NO SURVEY

Sector Number Percent Number Percent 

Public, 4-year or above 27 47.37 18 62.07 

Private nonprofit, 4-year or above 14 24.56 3 10.34 

Private for-profit, 4-year or above 0 0.00 0 0.00 

Public, 2-year 16 28.07 8 27.59 

Private nonprofit, 2-year 0 0.00 0 0.00 

Private for-profit, 2-year 0 0.00 0 0.00 

Other 0 0.00 0 0.00 

Total 57 100.00 29 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 

Geographical Region. Table C.30 contains the tabular summary for this characteristic. 

Table C.30: Tabular summary comparing SURVEY to NO SURVEY for Region. 

SURVEY NO SURVEY

Region Number Percent Number Percent 

New England 3 5.26 0 0.00 

Mid East 8 14.04 0 0.00 

Great Lakes 8 14.04 2 6.90 

Plains 7 12.28 3 10.34 

Southeast 16 28.07 18 62.07 

Southwest 5 8.77 0 0.00 

Rocky Mountains 3 5.26 1 3.45 

Far West 7 12.28 5 17.24 

Total 57 100.00 29 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Urbanicity. Table C.31 contains the tabular summary for this characteristic. 
102

Table C.31: Tabular summary comparing SURVEY to NO SURVEY for Urbanicity. 

SURVEY NO SURVEY

Urbanicity Number Percent Number Percent 

Large city 9 15.79 2 6.90 

Mid-size city 18 31.58 9 31.03 

Urban fringe of large city 8 14.04 5 17.24 

Urban fringe of mid-size city 4 7.02 3 10.34 

Large town 5 8.77 3 10.34 

Small town 9 15.79 5 17.24 

Rural 4 7.02 1 3.45 

Other 0 0.00 1 3.45 

Total 57 100.00 29 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Carnegie Classification. Table C.32 contains the tabular summary for this 

characteristic. 

Table C.32: Tabular summary comparing SURVEY to NO SURVEY for Carnegie Classification. 

SURVEY NO SURVEY

Carnegie Classification Number Percent Number Percent 

Doctoral/Research Universities--Extensive 6 10.53 5 17.24 

Doctoral/Research Universities--Intensive 6 10.53 4 13.79 

Masters Colleges and Universities I 23 40.35 9 31.03 

Masters Colleges and Universities II 1 1.75 0 0.00 

Baccalaureate Colleges--Liberal Arts 3 5.26 1 3.45 

Baccalaureate Colleges--General 2 3.51 2 6.90 

Baccalaureate/Associates Colleges 0 0.00 0 0.00 

Associates Colleges 16 28.07 8 27.59 

Total 57 100.00 29 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 
103

Selectivity. Table C.33 contains the tabular summary for this characteristic.

Table C.33: Tabular summary comparing SURVEY to NO SURVEY for Selectivity. 

Selectivity SURVEY NO SURVEY

Mean 72.51 68.35 

Standard deviation 17.05 17.96 

Minimum 22.73 12.55 

25th percentile 60.17 60.86 

Median 76.11 71.86 

75th percentile 84.59 78.82 

Maximum 97.16 89.83 

N/A 19 7 

n 57 29 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

In-State Cost. Table C.34 contains the tabular summary for this characteristic. 

Table C.34: Tabular summary comparing SURVEY to NO SURVEY for In-State Cost. 

In-State Cost SURVEY NO SURVEY

Mean 7,100.19 4,229.14 

Standard deviation 7,430.36 3,629.14 

Minimum 432 432 

25th percentile 1,896 2,280 

Median 4,254 3,612 

75th percentile 9,274 4,426 

Maximum 28,209 17,110 

N/A 57 29 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
104

Out-of-State Cost. Table C.35 contains the tabular summary for this characteristic. 

Table C.35: Tabular summary comparing SURVEY to NO SURVEY for Out-of-State Cost. 

Out-of-State Cost SURVEY NO SURVEY

Mean 11,084.54 9,535.55 

Standard deviation 6,024.46 4,201.16 

Minimum 2,026 432 

25th percentile 6,327 6,374 

Median 9,784 10,720 

75th percentile 14,298 12,353 

Maximum 28,209 17,552 

N/A 57 29 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Student Body Demographics 

Enrollment. Table C.36 contains the tabular summary for this characteristic. 

Table C.36: Tabular summary comparing SURVEY to NO SURVEY for Enrollment. 

Enrollment SURVEY NO SURVEY

Mean 7,641.05 10,971.97 

Standard deviation 5,866.34 9,580.47 

Minimum 592 1,692 

25th percentile 2,285 3,868 

Median 5,941 7,258 

75th percentile 11,900 15,672 

Maximum 23,552 35,667 

N/A 57 29 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
105

Percent Female. Table C.37 contains the tabular summary for this characteristic. 

Table C.37: Tabular summary comparing SURVEY to NO SURVEY for Percent Female. 

Percent Female SURVEY NO SURVEY

Mean 57.88 55.85 

Standard deviation 5.88 5.43 

Minimum 43.22 43.87 

25th percentile 53.67 52.49 

Median 57.77 57.60 

75th percentile 61.70 59.56 

Maximum 71.96 64.50 

N/A 57 29 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Black. Table C.38 contains the tabular summary for this characteristic. 

Table C.38: Tabular summary comparing SURVEY to NO SURVEY for Percent Black. 

Percent Black SURVEY NO SURVEY

Mean 11.96 21.24 

Standard deviation 14.55 26.76 

Minimum 0 0 

25th percentile 3 5 

Median 8 12 

75th percentile 17 23 

Maximum 94 94 

N/A 57 29 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Percent Out-of-State. Table C.39 contains the tabular summary for this characteristic. 

Table C.39: Tabular summary comparing SURVEY to NO SURVEY for Percent Out-of-State. 

Percent Out-of-State SURVEY NO SURVEY

Mean 17.43 16.47 

Standard deviation 21.02 14.06 

Minimum 0 0

25th percentile 2.75 5.96 

Median 8.47 13.07 

75th percentile 24.89 26.37 

Maximum 97.05 46.09 

N/A 9 4 

n 57 29

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
106

Athletic Program Characteristics 

Association Membership. Table C.40 contains the tabular summary for this 

characteristic. 

Table C.40: Tabular summary comparing SURVEY to NO SURVEY for Association 

Membership.

SURVEY NO SURVEY

Association Membership Number Percent Number Percent 

NCAA only 36 63.16 19 65.52 

NAIA only 4 7.02 2 6.90 

NJCAA only 10 17.54 2 6.90 

Multiple associations 1 1.75 0 0.00 

None 6 10.53 6 20.69 

Total 57 100.00 29 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Football. Table C.41 contains the tabular summary for this characteristic. 

Table C.41:Tabular summary comparing SURVEY to NO SURVEY for Football. 

SURVEY NO SURVEY

Football Number Percent Number Percent 

Yes 36 63.16 17 58.62 

No 21 36.84 12 41.38 

Total 57 100.00 29 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Number of Sports. Table C.42 contains the tabular summary for this characteristic. 

Table C.42: Tabular summary comparing SURVEY to NO SURVEY for Number of Sports. 

SURVEY NO SURVEY

Number of Sports Number Percent Number Percent 

0 16 28.07 8 27.59 

1 0 0.00 0 0.00 

2 2 3.51 0 0.00 

3 11 19.30 7 24.14 

4 28 49.12 14 48.28 

Total 57 100.00 29 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 
107

C.4 Comparisons of COMPLAINT to COMPLIANCE MONITORING 

This section contains the 14 comparisons of the 95 COMPLAINT institutions to the 35 

COMPLIANCE MONITORING institutions. 

Institutional Characteristics 

Sector. Table C.43 contains the tabular summary for this characteristic.

Table C.43: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Sector. 

COMPLAINT COMPLIANCE 

Sector MONITORING

Number Percent Number Percent 

Public, 4-year or above 48 50.53 23 65.71 

Private nonprofit, 4-year or above 26 27.37 6 17.14 

Private for-profit, 4-year or above 0 0.00 0 0.00 

Public, 2-year 21 22.11 6 17.14 

Private nonprofit, 2-year 0 0.00 0 0.00 

Private for-profit, 2-year 0 0.00 0 0.00 

Other 0 0.00 0 0.00 

Total 95 100.00 35 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 

Geographical Region. Table C.44 contains the tabular summary for this characteristic. 

Table C.44: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Region. 

COMPLAINT COMPLIANCE 

Region MONITORING

Number Percent Number Percent 

New England 4 4.21 4 11.43 

Mid East 10 10.53 2 5.71 

Great Lakes 11 11.58 3 8.57 

Plains 12 12.63 2 5.71 

Southeast 40 42.11 4 11.43 

Southwest 4 4.21 3 8.57 

Rocky Mountains 3 3.16 6 17.14 

Far West 11 11.58 11 31.43 

Total 95 100.00 35 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
108

Urbanicity. Table C.45 contains the tabular summary for this characteristic. 

Table C.45: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Urbanicity. 

COMPLAINT COMPLIANCE 

Urbanicity MONITORING

Number Percent Number Percent 

Large city 18 18.95 4 11.43 

Mid-size city 31 32.63 11 31.43 

Urban fringe of large city 11 11.58 8 22.86 

Urban fringe of mid-size city 8 8.42 1 2.86 

Large town 4 4.21 7 20.00 

Small town 17 17.89 3 8.57 

Rural 5 5.26 1 2.86 

Other 1 1.05 0 0.00 

Total 95 100.00 35 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Carnegie Classification. Table C.46 contains the tabular summary for this 

characteristic. 

Table C.46: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Carnegie Classification. 

COMPLAINT COMPLIANCE

Carnegie Classification MONITORING

Number Percent Number Percent 

Doctoral/Research Universities--Extensive 19 20.00 9 25.71 

Doctoral/Research Universities--Intensive 12 12.63 2 5.71 

Masters Colleges and Universities I 31 32.63 15 42.86 

Masters Colleges and Universities II 3 3.16 0 0.00 

Baccalaureate Colleges--Liberal Arts 4 4.21 1 2.86 

Baccalaureate Colleges--General 5 5.26 2 5.71 

Baccalaureate/Associates Colleges 0 0.00 0 0.00 

Associates Colleges 21 22.11 6 17.14 

Total 95 100.00 35 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 
109

Selectivity. Table C.47 contains the tabular summary for this characteristic. 

Table C.47: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Selectivity. 

Selectivity COMPLAINT COMPLIANCE 

MONITORING

Mean 68.21 74.05 

Standard deviation 18.03 17.27 

Minimum 12.55 22.73 

25th percentile 57.37 65.62 

Median 73.63 77.81 

75th percentile 81.26 87.53 

Maximum 97.16 95.09 

N/A 23 7 

n 95 35 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

In-State Cost. Table C.48 contains the tabular summary for this characteristic. 

Table C.48: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for InState Cost. 

In-State Cost COMPLAINT COMPLIANCE 

MONITORING

Mean 7,688.76 5,852.40 

Standard deviation 7,573.85 6,878.30 

Minimum 468 432

25th percentile 2,863 2,556 

Median 4,279 3,595 

75th percentile 11,830 5,162 

Maximum 28,965 28,209 

n 95 35 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
110

Out-of-State Cost. Table C.49 contains the tabular summary for this characteristic. 

Table C.49: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Out-of-State Cost. 

Out-of-State Cost COMPLAINT COMPLIANCE 

MONITORING

Mean 12,388.43 11,109.80 

Standard deviation 6,208.69 6,209.99 

Minimum 2,026 432 

25th percentile 7,286 7,842 

Median 11,700 10,976 

75th percentile 16,625 12,960 

Maximum 28,965 28,209 

n 95 35 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Student Body Demographics 

Enrollment. Table C.50 contains the tabular summary for this characteristic. 

Table C.50: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Enrollment. 

Enrollment COMPLAINT COMPLIANCE 

MONITORING

Mean 10,150.21 1,2972.66 

Standard deviation 9,554.30 8,974.40 

Minimum 588 1,276 

25th percentile 2,746.50 7,247 

Median 6,903 11,901 

75th percentile 13,811 16,489 

Maximum 40,567 41,617 

Total 95 35 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
111

Percent Female. Table C.51 contains the tabular summary for this characteristic. 

Table C.51: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Percent Female. 

Percent Female COMPLAINT COMPLIANCE 

MONITORING

Mean 56.79 54.20 

Standard deviation 6.39 4.36 

Minimum 30.82 43.87 

25th percentile 52.75 51.33 

Median 57.60 53.78 

75th percentile 60.85 57.74 

Maximum 71.96 61.74 

n 95 35 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

 

Percent Black. Table C.52 contains the tabular summary for this characteristic. 

Table C.52: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Percent Black. 

Percent Black COMPLAINT COMPLIANCE 

MONITORING 

Mean 14.46 6.31 

Standard deviation 19.49 6.22 

Minimum 1 0 

25th percentile 4 2 

Median 8 4 

75th percentile 17 8 

Maximum 94 25 

n 95 35 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
112

Percent Out-of-State. Table C.53 contains the tabular summary for this characteristic. 

Table C.53: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Percent Out-of-State. 

Percent Out-of-State COMPLAINT COMPLIANCE 

MONITORING

Mean 22.71 21.19 

Standard deviation 22.73 23.39 

Minimum 0 0 

25th percentile 5.29 1.74 

Median 15.33 17.51 

75th percentile 37.47 26.20 

Maximum 97.20 97.05 

N/A 15 2 

n 95 35

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Athletic Program Characteristics 

Association Membership. Table C.54 contains the tabular summary for this 

characteristic. 

Table C.54: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Association Membership.

COMPLAINT COMPLIANCE 

Association Membership MONITORING

Number Percent Number Percent 

NCAA only 68 71.58 28 80.00 

NAIA only 5 5.26 1 2.86 

NJCAA only 12 12.63 0 0.00 

Multiple associations 1 1.05 0 0.00 

None 9 9.47 6 17.14 

Total 95 100.00 35 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 
113

Football. Table C.55 contains the tabular summary for this characteristic. 

Table C.55: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for 

Football. 

COMPLAINT COMPLIANCE

MONITORING Football Number Percent Number Percent 

Yes 60 63.16 26 74.29 

No 35 36.84 9 25.71 

Total 95 100.00 35 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

Number of Sports. Table C.56 contains the tabular summary for this characteristic. 

Table C.56: Tabular summary comparing COMPLAINT to COMPLIANCE MONITORING for

Number of Sports. 

COMPLAINT COMPLIANCE 

Number of Sports MONITORING

Number Percent Number Percent 

0 21 22.11 6 17.14 

1 0 0.00 0 0.00 

2 2 2.11 0 0.00 

3 22 23.16 12 34.29 

4 50 52.63 17 48.57 

Total 95 100.00 35 100.00 

SOURCE: U. S. Department of Education, National Center for Education Statistics, Integrated 

Postsecondary Education Data System (IPEDS), 2003. 

NOTE: Rows with counts of zero are preserved for comparability with other tables. 
114

Appendix D: Data Collection Classification Characteristics 

Here we list the 20 characteristics according to which the 52 INSTRUMENT data 

collection instruments are classified in section 3.1. Table 3.5, in that same section, 

contains additional information concerning response rates, even though these are not in 

the list of characteristics. 

Characteristic Values 

CASE AND SURVEY (4 characteristics) 

Complaint? Y = Yes 

N = No 

Target Population S = Student Body 

F = Females Only 

O = Other 

Sampling C = Census 

S = True Survey 

Proactivity (exhortations, incentives, follow-up, ...) Y = Yes 

N = No 

INSTRUMENT (16 characteristics) 

Age Asked? Y = Yes 

N = No 

Class Asked? Y = Yes 

N = No 

Gender Asked? Y = Yes 

N = No 

Spectator Interest Y = Yes 

N = No 

Attitudes toward Athletics Y = Yes 

N = No 

Opinion about Institution’s Athletic Programs Y = Yes 

N = No 

Identifying Information Y = Yes 

N = No 

Explicit Questions about Athletic Ability Y = Yes 

N = No 

Whether Recruited as an Athlete Y = Yes 

N = No 

Statement of Caveats and Benefits? Y = Yes 

N = No 

Explanation of Reasons for Survey? Y = Yes 

N = No 

Statement of Confidentiality of Responses? Y = Yes 

N = No 

Interest—Representation of Sports F = Fixed Rows 

W = Write In 

C = Codes 

Interest—Number of Levels N 

Experience—Representation of Sports F = Fixed Rows 

W = Write In 

C = Codes 

Experience—Number of Levels N 
115
116

Appendix E: Data Collection Instruments in Chapter 4 

The table below lists the five additional data collections reviewed in chapter 4, showing 

the institution, the year during which the data collection was conducted, and the source of 

information about it. 

Institution Year Source of Information

James Madison University 2000 www.jmu.edu/instresrch/resrchstud/Athletics/Report 2000.pdf

North Arkansas College 2004 www.northark.edu/Departments/IR_Web/title9survey_2004.htm

Nova Southeastern University 2004 www.nova.edu/athletics/

Radford University 2002 www.radford.edu/~irpa/Surveys/SportsParticipation2002.pdf

Western Carolina University 2004 www.wcu.edu/assess/surveys/sports/sport_interest.asp
117
118

Appendix F: Complete Analysis in the Presence of NonResponse 

This appendix lays out technical details of analysis of the data when a non-response bias 

analysis must be conducted (see section 5.5.3). It is also less than strictly prescriptive, 

with some discussion of alternative models and assumptions. 

We suppose that the recommendations in chapter 5 are implemented: that the target 

population is all female students and the data collection is a census. It is likely that many 

members of the target population will either not respond at all, or will not respond in a 

timely manner. This appendix addresses the case of section 5.5.3, where the response 

rate, for whatever reason, fails to meet the NCES standard of 85%, so that a non-response 

bias analysis is necessary. 

In this appendix we lay out a full Bayesian analysis (Gelman, et al., 1995) of the data. 

This formulation differs slightly from that in section 5.5.3. Here we assume that + N , the 

“true” number of interested and able students in the target population, is an unobservable 

random variable, so that the decision criterion depends on the conditional probability that 

N ≥ M + given the observed data. More specifically, with + NR the number of interested 

and able students among the respondents to the census, then the decision process is as 

follows: 

• If NR ≥ M + , that is, if the number of interested and able respondents exceeds the 

minimum team size M, then clearly 1 { ≥ | Data} ≥ { ≥ | Data} = + + P N M P NR M , 

and there is no need for a non-response bias analysis. 

• If NR < M + then the non-response bias analysis generates additional data. The 

analysis described below produces the conditional probability } P{N ≥ M | Data + . 

o If ≥ ≥α + P{N M | Data} , where α is the pre-set threshold, then—as 

described in section 5.5—the institution may not be in compliance with 

Title IX. (As described in chapter 5, non-compliance also requires at least 

that appropriate facilities and competition exist.) 

o If, on the other hand, ≥ <α + P{N M | Data} , then the data do not provide 

sufficient evidence that N ≥ M + , and the institution is in compliance with 

Title IX. 

In order to allow for possible non-response bias, the model contains three parameters: 

• + N , the number of interested and able students in the target population 

• q = P{Response to census| Positive response} +

• q = P{Response to census| Negative response} −

The non-response bias analysis is necessary since there is reason to suspect that + − q ≥ q , 

i.e., that interested and able students are more likely to respond to the census. 
119

In the Bayesian formulation, all three parameters are treated as random, and assigned 

prior distributions representing a priori knowledge about them, which are discussed 

below. The Bayesian paradigm is then to 

• Calculate the conditional distribution of the data given the parameters 

( , , ) + + − N q q , a process that we describe below. 

• Use Bayes theorem to calculate the posterior distribution of the parameters 

( , , ) + + − N q q given the observed values of the data: 

( , , )

(Data | , , ) ( , , | Data) + + −

+ + −

+ + − = p N q q

p N q q p N q q . 

Finally, to obtain the conditional distribution of + N given the data, it is necessary to 

integrate out + q and − q in this expression: 

∫∫ + + + − + − p( . N | Data) = p(N ,q ,q | Data)dq dq

The data are generated by both the census and the non-response bias analysis, and consist 

of: 

• + NR , the number of positive responses to the original census 

• − NR , the number of negative responses to the original census 

• + NNRBS , the number of positive responses in the non-response bias sample 

• − NNRBS , the number of negative responses in the non-response bias sample. 

A more convenient but equivalent representation of the data consists of: 

• + NR , the number of positive responses to the original census 

• + − NNR = P − NR − NR , the number of non-responses to the original census 

(recall that P is the size of the target population) 

• + NNRBS , the number of positive responses in the non-response bias sample 

• NNRBS , the number of non-respondents in the non-response bias sample. 

Figure F.1 shows the data pictorially. 
120

Figure F.1: Pictorial representation of data and unknown values. 

Source: NISS 

For the prior distributions of + N , + q and − q , we recommend: 

• For + N , a discrete uniform distribution on {0,K,K}, where K is set on the basis 

of external knowledge and the minimum team size M. A reasonable choice is K = 

5M. If there is external knowledge about + N , arising for example from an earlier 

survey, then it also can be incorporated into the prior distribution. 

• For + q and − q , beta distributions chosen so that the prior mean of + q exceeds that 

of − q . 

Independence of the three priors can be assumed. However, it is also possible to choose a 

joint prior distribution for ) ( , + − q q that imposes the restriction . + − q ≥ q

The remaining step is to calculate the conditional distribution of the data given the 

parameters ) ( , , + + − N q q . By properties of conditional probabilities, 

( , | , , , , ) ( , | , , )

( , , , | , , )

+ + + − + + + + −

+ + + + −

= p N N N q q N N p N N N q q

p N N N N N q q

NRBS NRBS R NR R NR

R NR NRBS NRBS

Concerning the second term in this expression, assuming (there is no meaningful 

alternative) that response is independent across students, 

• The conditional distribution of + NR given ) ( , , + + − N q q is binomial with 

parameters + N and + q
121

• Similarly, the conditional distribution of − NR given ) ( , , + + − N q q is binomial with 

parameters + P − N and − q , where P is the size of the target population 

• + NR and − NR are conditionally independent given ) ( , , + + − N q q . 

Consequently, 

+ + − − − − + −

+

+ + −

+

+ + + − − + + −

+ − + + −

+ + + −

− ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛

− −

− − ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛

=

= = = − −

= = = − −

= =

j N j P k j k j N

R R

R R

R NR

q q P k j

P N

q q j

N

P N j N q q P N p k j N q q

P N j N p k j N q q

P N j N k N q q

( ) (1 ) ( ) (1 )

{ | , , } { | , , }

{ , | , , }

{ , | , , }

Suppose that the non-response bias analysis consists of selecting a simple random sample 

of p% of the NNR non-respondents: each non-respondent is selected with probability p

independent of what others are selected. This is not the only way to perform the sampling 

for the non-response bias analysis, but it is one for which the computations are least 

complicated. Assume that all students in the non-response bias sample are contacted and 

do respond. Then: 

• The conditional distribution of + NNRBS given ( , , , , ) N q q NR NNR

+ + − + is binomial 

distribution with parameters + + N − NR and p

• Similarly, the conditional distribution of − NNRBS given ) ( , , , , N q q NR NNR

+ + − + is 

binomial distribution with parameters + + NNR + NR − N and p

• + NNRBS and − NNRBS are conditionally independent given ( , , , , ) N q q NR NNR

+ + − + . 

This implies that 

R NR R l N l

R i N N i NR R l i N N N l i

NRBS R NR NRBS R NR

NRBS NRBS R NR

NRBS NRBS R NR

NR

R NR R

p p l i

N N N

i

N N

p p l i

N N N

p p i

N N

P N i N q q N N P N l i N q q N N

P N i N l i N q q N N

P N i N l N q q N N

−

+ + + +

− + − − +

+ +

− −

+ +

+ + + − + − + + − +

+ − + + − +

+ + + − +

− ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛

−

+ −

⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛ − =

− ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛

−

+ − − ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛ − =

= = = −

= = = −

= =

+ + + +

(1 )

(1 ) (1 )

{ | , , , , } { | , , , , }

{ , | , , , , }

{ , | , , , , }

This completes calculation of the components of ) ( , , , | , , + + + + − p NR NNR NNRBS NNRBS N q q .