# 2012 **/1 4**  **B EGINNING**  

# P **OSTSECONDARY** 

# S **TUDENTS** 

# L **ONGITUDINAL** 

# S **TUDY** 

# : **(B PS**  **:12 /14**  **)**

**S upporting Statement Part**  **s B**  **and C**

**Request for OMB Review OMB #**  **1850 -**  **0 631**  **v. 8** 

Submitted by National Center for Education Statistics

U.S. Department of Education

**October 24**  **, 201**  **3**II 

## Table of Contents List of Tables ................................ ................................ ................................ ................................ ................. ii B. Collection of Information Employing Statistical Metho ds ................................ ................................ ...... 3 1. Respondent Universe and Sampling ................................ ................................ ................................ .. 3 a. NPSAS:12 Full - scale Institution Universe and Sample ................................ ............................... 3 b. NPSAS:12 Full - scale Student Universe and Sample ................................ ................................ ... 5 c. Id entification of FTBs in NPSAS:12 ................................ ................................ ........................... 6 d. BPS:12/14 Full - scale Sample ................................ ................................ ................................ ....... 7 2. Procedures for the Collection of Information ................................ ................................ .................... 8 a. Statistical methodology for stratification an d sample selection ................................ ................... 8 b. Estimation procedure ................................ ................................ ................................ ................... 8 c. Degree of accuracy

needed for the purpose described in the justification ................................ ... 8 d. Unusual problems requiring specialized sample procedures ................................ ....................... 8 e. Any use of periodic (less frequent than annual) data collection cycles to reduce the burden ..... 8 3. Methods for Maximizing Response Rates ................................ ................................ ......................... 8 a. Locating ................................ ................................ ................................ ................................ ........ 8 b. Prompting Cases with Mail, E - mail, and SMS Contacts ................................ ............................. 9 c. Telephone Interviewing and Help Desk Support ................................ ................................ ....... 10 d. Quality

Control ................................ ................................ ................................ .......................... 11 4. Tests of Procedures and Methods ................................ ................................ ................................ .... 11 a. Previous Studies ................................ ................................ ................................ ......................... 11 b. BPS:12/14 Full Scale ................................ ................................ ................................ ................. 12 5. Reviewing Statisticians and Individuals Responsible for Designing and Conducting the Study ................................ ................................ ................................ ................................ ................ 21 6. Other Contractors’ Staff Responsible for Conducting the Study ................................ ..................... 21 C. Overview of Anal ys is Topics and Survey Items ................................ ................................ .................... 22 References ................................ ................................ ................................ ................................ .................... 22 

## List of Tables Table 1. BPS:12/14 Sample Size by institution characteristics: 2012 ................................ ................. 7 Table 2. Summary

of start dates and activities for each data collection phase, b y s ample ................ 17 Table 3. Estimated Power, Within Decile, Using 10 percent and 25 percent Underlying

Response Rates ................................ ................................ ................................ .................... 20 Table 4. Estimated Power, Within Decile, Using 45 percent and 60 percent Underlying

Response Rates ................................ ................................ ................................ .................... 203 

### B. **Collection of Information Employ**  **ing Statistical Methods 1.**  **Respondent Universe**  **and Sampling** The target population for the 2012/14 Beginnin g Postsecondary Students Longitudinal Study (BPS:12/14) full - scale consist s of all students who began their postsecondary education fo r the first time during the 2011

– 12 academic year at any Title IV - eligible postsecondary institution in the United States. The sample students were the first - time beginners (FTBs) from the 20 11 - 12 National Postsecondary Student Aid Study (NPSAS:12 ) full - scale . Because the students in the BPS:12/14 full - scale sample come from the NPSAS:12 full - scale sample, this section also describes the NPSAS:12 full - scale sample design, which was a two - stage sample consisting of a sample of institutions at the first stage, and a sample of students from within sampled institutions at the second stage. The BPS:12/14 full - scale sample is comprised of students from the NPSAS:12 full - scale sample who were determined to be FTBs, or were potential FTBs, as indicated by the NPSAS institution. **a. NPSA** **S:12 Full**  **- scale**  **Institution Universe and Sample** To be eligible for NPSAS:12, students must have been enrolled in a NPSAS - eligible institution in any term or course of instruction at any time during the 2011

– 12 academic year. Institutions must have also met the following requirements:

 offer an educational program designed for persons who have completed sec ondary education;  offer at least one academic, occupational, or vocational program of study lasting at least 3 months or 300 clock hours;  offer courses that were open to more than the employees or members of the company or group (e.g. union) that administers the institution;  be located in the 50 states or the District of Columbia;  not be a U.S. service academy institution; and  have signed the Title IV participation agreement with ED. 1 NPSAS excluded institutions providing only avocational, recreational, or remedial courses, or only in - house courses for their own employees or members. U.S. service academies were also excluded because of the academies’ unique funding/tuition base. The initial institution sample s for the NPSAS field test and full - scale studies were selected simultaneously , prior to the full - scale study , using sequential probability minimum replacement (PMR) sampling (Chromy 1979), which resembles stratified systematic sampling with probabilities proportional to a composite measure of size (Folso m, Potter, and Williams 1987). This is the same methodology that has been used since NPSAS:96. Institution measure of size was determined using annual enrollment data from the most recent IPEDS 12 - Month Enrollment Component and first time beginner (FTB) full - time enrollment data from the most recent IPEDS Fall Enrollment Component. Composite measure of size sampling was used to ensure that target sample sizes were achieved within institution and student sampling strata, while also achieving approximately eq ual student weights across institutions . The

1 A Title IV eligible institution is an institution that has a written agreement (program participation agreement) with the U.S. Secretary of Education that allows the institution to participate in any of the Title IV federal student financial assistance programs oth er than the State Student Incentive Grant (SSIG) and the National Early Intervention Scholarship and Partnership (NEISP) programs.4

institution sampling frame for NPSAS:12 full - scale was constructed using the 2009 Integrated Postsecondary Education Data Sy stem (IPEDS) header, Institution Characteristics (I C), Fall and 12 - Month Enrollment, an d Completions files. All eligible students from sampled institutions comprised the student sampling frame . From the stratified frame, a total of 1,970 institutions were selected to participate in either the field test or full - scale study . U sing simple ran dom sampling within institution strata , a subsample of 300 institutions was selected for the field test sample , with the remaining 1,670 institutions comprising the sample for the full - scale study . This sampling process eliminated the possibility that an institution would be burdened with participation in both the field test and full - scale samples , and ensured representativeness of the full - scale sample. The institution strata used for the sampling design were based on institution level, control, and highe st level of offering , including :

1. public less - than - 2 - ye ar, 2. public 2 - ye ar, 3. public 4 - ye ar non - doctorate - granting, 4. public 4 - ye ar doctorate - granting, 5. private nonprofit less - than - 4 - year, 6. private nonprofit 4 - ye ar non - doctorate - granting, 7. private nonprofit 4 - ye ar doctorate - granting, 8. private for - profit less - than - 2 - year, 9. private for - profit 2 - ye ar, and 10. private for - profit 4 - ye ar. Due to the growth of the for- profit sector, private for- profit 4 - ye ar and private for- profit 2 - year institutions were separated into their own strata unlike in previous administrations of NPSAS. In order to approximate proportional representation of institutions w ithin each institution stratum, additional implicit stratification for the full - scale was accomplished by sorting the sampling frame by the following classifications: (1) historicall y Black colleges and universities indicator; (2) Hispanic - serving institutions indicator ; (3) Carnegie classifications of degree - granting postsecondary institutions; (4) 2 - digit Classification of Instruction al Programs (CIP) code of the largest program for less - than - 2 - ye ar institutions; (5) the Office of Business Economics Region from the IPEDS header file (Bureau of Economic Anal ys is of the U.S. Department of Commerce Region); (6) state and s ys tem for states with large s ys tems, e.g., the SUNY and CUNY s ys tems in New York, the state and technical colleges in Georgia, and the California State University and University

of California s ystems in California; and (7) the institution measure of size. The institution sample was freshened in order to add newly eligible institutions to the sample. The newly- available 2009

– 10 IPEDS IC header, 12 - month and fall enrollment, and completions files were used to create an updated sampling frame of current NPSAS - eligible insti tutions. This frame was compared to the original frame and 387 new or newly eligible institutions were identified for the freshened sampling frame. A sample size of 20 was selected in order to produce similar probabilities of5

selection to the originally selected institutions within sector (stratum) in order to minimize unequal weights and subsequently variances. From the 1,690 institutions selected for the NPSAS:12 full - scale data collection , 100 percent met eligibility requirements; of those, approximate ly 88 percent (or 1,480 institutions) provided enrollment lists. **b. NPSA** **S:12 Full**  **- scale**  **Student Universe and Sample**  Student s eligible for the NPSAS:12 full - scale were those who attended a NPSAS eligible institution during the 2011

– 12 academic year and who wer e

 enrolled in either : (a) an academic program; (b) at least one course for credit that could be applied toward fulfilling the requirements for an academic degree; (c) exclusively non credit remedial coursework but determined b y the institution to be eligib le for Title IV aid; or (d) an occupational or vocational program that required at least 3 months or 300 clock hours of instruction to receive a degree, certificate, or other formal award;  not currentl y enrolled in high school; and  not solely

enrolled in a General Educational Development (GED) or other high school completion program. The full - scale student sampling frame was created from lists of students enrolled at NPSAS sampled institutions between July, 1 201 1 and April 30, 201 2 . While the NPSAS:12 stu dy year covers the time period between July 1, 2011 and June 30, 2012 to coincide with the federal financial aid award ye ar , the abbreviated sampling period of July 1 to April 30 facilitate d timely completion of data collection and data file preparation. P revious cycles of NPSAS have shown that the terms beginning in May and June add little to enrollment and aid totals . To create the student sampling frame, each participating institution was asked to submit a list of eligible students with the following data provided for each listed student :

 name;  student ID (if different than Social Security number);  Social Security

number;  date of birth;  date of high school graduation (month and year);  degree level during the last term of enrollment (undergraduate, ma sters, doctoral research/scholarship/other, doctoral - professional practice, or other graduate);  class level if undergraduate (first, second, third, fourth, or fifth year or higher);  undergraduate degree program;  Classification of Instructional Program code or major;  FTB status; and  contact information . Requesting contact information for eligible students prior to sampling allowed for student interviewing to begin shortly after sample selection , which facilitated management of the schedule for data collectio n, data processing, and file development.6

The NPSAS:12 full - scale sample was randomly selected from the frame with s tudents sampled at fixed rates according to student education level and institution sampling strata. Sample yield was monitored and sampling rates were adjusted when necessary . The full - scale sample achieved a size of about 12 8,120 students , of which approximately 5 9,74 0 were potential FTBs, 5 1 , 0 50 were other undergraduate students, and 17,3 3 0 were graduate students. The achieved sample size was higher than originall y targeted because institution participation rates were higher than estimated, sampling continued longer than scheduled, and a higher sample size was desired to help meet interview yi eld targets.

**c. Identification of FTBs**  **in NPSA** **S :12**  Close attention was paid to accuratel y identify FTBs in NPSAS to avoid unacceptabl y high rates of misclassification (e.g., false positives) 2 which, in prior BPS administrations , have resulted in (1) excessive cohort loss, (2) excessive cost to “replenish” the sample, and (3) an inefficient sample design (excessive oversampling of “potential” FTBs) to compensate for anticipated misclassification errors . To address this concern, participating institutions were asked to provide additional information for all eligible students and matching to administrative databases was utilized to further reduce false positives prior to sample selection. Participating institutions were asked to provide the FTB status and high school graduation date for every listed student. High school graduation date was used to remove students from the frame who were co - enrolled in high school. FTB status, along with class and student level s , were used to exclude misclassified FTB students in their third year or higher and/or those who were not an undergraduate student. FTB status, along with date of birth, were used to identify students older than 18 to send for pre sampling matching to administrative databases. If the FTB indicator was not provided for a student on the lists, but th e student was 18 ye ars of age or younger and did not appear to be dually enrolled in high school, the student was sampled as an FTB. Otherwise, if the FTB indicator was not provided for a student on the list and the student was over the age of 18, then the student was sampled as an “other undergraduate” (but such students would be included in the BPS cohort if identified during the student interview as an FTB ) . Prior to sampling, students over the age of 18 listed as potential FTBs were matched to NSLDS rec ords to determine if any had a federal financial aid history pre - dating the NPSAS year (earlier than July 1, 2011). Since NSLDS maintains current records of all Title IV federal grant and loan funding, any student with disbursements from the prior year or earlier could be reliably excluded from the sampling frame of FTBs. Given that about 60 percent of FTBs receive some form of Title IV aid in their first year, this matching process could not exclude all listed FTBs with prior enrollment, but significantl y improved the accuracy

of the list prior to sampling, yielding fewer false positives. After undergoing NSLDS matching, students over the age of 18 still listed as potential FTBs were matched to the National Student Clearinghouse (NSC) for further narrowing of potential FTBs based on evidence of earlier enrollment. Matching to NSLDS identified about 20 percent of cases as false positives and matching to NSC identified about 7 percent of cases as false positives. In addition to NSLDS and NSC, a subset of potential FTBs on the student sampling frame was sent to CPS for matching to evaluate the benefit of the CPS match for the full - scale study . Of the 2,103,620 students sent, CPS identified about 17 percent as false

2 A student identified by the institution on the enrollment list as an FTB who turns out to not be an FTB is a false positive.7

positives. Overall, matching to all sources identified about 27 percent of potential FTB students over the age of 18 as false positives, with many of the false positives identified by CPS also identified by

NSLDS or NSC. The matching appeared most effective among pub lic 2 - ye ar and private for - profit institutions. While public less - than 2 - ye ar and private nonprofit less - than - 4 - ye ar institutions have a high percent of false positives, they

represent a small percentage of the total sample. Since this pre - sampling matching was new to NPSAS:12, the FTB sample size was set high to ensure that a sufficient number of true FTBs would be interviewed. In addition, FTB selection rates were set taking into account the error rates observed in NPSAS:04 and BPS:04/06 within each sector . A dditional information on NPSAS:04 methodology is available in the study methodology report, publication number NCES 2006180 , or BPS:04/06 methodology report, publication number NCES 2008184 . These rates were adjusted to reflect the improvement in th e accuracy of the frame from the NSLDS and NSC record matching. Sector- level FTB error rates from the field test were used to help determine the rates necessary for full - scale student sampling. **d. BPS:12/14**  **Full -** **scale Sample**  At the conclusion of the NPSAS:12 full - scale , 30,076 students had been interviewed and confirmed to be FTBs, and all will be included in the BPS:12 / 14 full - scale . In addition, the full - scale sample will include the 7,090 students who did not respond to the NPSAS:12 full - scale , but were po tential FTBs according to student records or institution lists. The distribution of the BPS:12/14 full scale sample is shown in table 1, by institution sector. **Table 1. BPS:12/14 Sample Size by**  **inst itution characteristics: 201**  **2**

**Institution characteristics**  **Confirmed and Potential FTBs from the NPSA S:12 Full**  **- scale**  **Total NPSA** **S Interview**

**Respondent NPSA** **S Study**

**Me mber Interview**  **Nonrespondents NPSA** **S Non -**  **Study**

**Me mber Interview**  **Nonrespondents** To tal 37,170 30,080 4,610 2,480 Institution ty pe Public Less - than - 2 - ye ar 250 190 30 30 2 - ye ar 11,430 9,080 1,290 1,050 4 - ye ar non - doctorate granting 1,930 1,700 150 90 4 - ye ar doctorate - granting 3,510 3,220 220 80 Private nonprofit Less - than - 4 - ye ar 380 310 40 30 4 - ye ar non - doctorate granting 2,430 2,160 120 140 4 - ye ar doctorate - granting 2,720 2,470 130 120 Private for- profit Less - than - 2 - ye ar 1,630 1,200 380 50 2 - ye ar 3,530 2,620 710 200 4 - ye ar 9,370 7,110 1,550 700 NOTE: Detail may not sum to totals because of rounding. Potential FTB’s are those NPSAS:12 non - respondents who appeared to be FTB’s in NPSAS:12 student records data. Non study members are those NPSAS:12 sample members who, across all data sources, did not have sufficient data to support the analytic objectives of the study. SOURCE: U.S. Department of Education, National Center for Education Statistics, 2011 – 12 National Postsecondary Student Aid Study (NPSAS:12) Full - scale8

**2. Procedures for the Collection of Information**  **a. Statistical methodology**  **for stratification and sample selection**  The BPS:12/14 full - scale sample will consist of all interviewed and confirmed FTBs from the NPSAS:12 full - scale and a subsample of potential FTBs who did not respond to the NPSAS:12 full scale . The sample size of 7,090 for the subsample was determined b y estimating the impact on the unequal weighting effect due to subsampling and identifying the sample size that minimized this effect subject to available resources. The institution and student sampling strata used in th e NPSAS:12 full scale will be retained for use in anal ys is. The subsample of potential FTBs will be selected using stratified random sampling where the strata are defined b y cross - classifying institution ty pe and nonrespondent classification (study member or non - study

member). **b. Estimation procedure**  The students in the BPS:12/14 full - scale were also in the NPSAS:12 full - scale . Because the NPSAS:12 full - scale was a probability sample, sampling weights (which are the inverse of the selection probability) are available for each of the students in the NPSAS:12 full - scale and BPS:12/14 full - scale . These base weights , suitably adjusted for subsampling of NPSAS non - interview respondents, can be used in analy se s of the results of the full - scale experiment. In additio n, non - response weight adjustments and post - stratification adjustments will be applied to these base weights, and the fully adjusted weights used in the anal ys is of BPS:12/14 full - scale interview data.

**c. Degree of accuracy**  **needed for the purpose described in the justification**  The sample will include interviewed FTBs from the NPSAS:12 full - scale and interview nonrespondents who were potential FTBs. Estimates of the overall unequal weighting effect due to subsampling indicate that the variance of overall estimates will increase b y a factor of four under the proposed subsampling plan relative to the situation where no subsampling is performed and all interview nonrespondents are included in the BPS full - scale . The proposed subsample minimizes the overall impact on variance subject to resource considerations.

**d. Unusual problems requiring specialized sample procedures**  No special problems requiring speciali zed sample procedures have been identified. **e. An**  **y use of periodic (less frequent than annual) data collection cy** **cles to reduce the burden**  BPS interviews are conducted no more frequentl y than every two ye ars. **3. Methods for Maximizing Response Rates**  **a. Locating**  Achieving a high response rate for BPS:12/14 depends on successfully locating sample members and gaining their cooperation. The availability , completeness, and accuracy of the locating data collected in the NPSAS:12 interview and institution records will impact the success of locating efforts. For BPS:12/14, all contact information previousl y collected for sample members, including current and prior address information, telephone numbers, and e - mail addresses, will be stored in a centralized locator9

datab ase. This database provides telephone interviewers and tracers with immediate access to all the contact information available for BPS:12/14 sample members and to new leads developed through locating efforts. RTI ’s locating procedures use a multi - tiered tracing approach, in which the most cost - effective steps will be taken first to minimize the number of cases that require more expensive tracing efforts. RTI ’ s proposed locating approach includes five basic stages : 1. **Advance Tracing** primarily consists of n ational database batc h searches conducted prior to the start of data collection. This step capitalizes on the locating data collected for the BPS sample during NPSAS:12 . 2. **Telephone Locating and Interviewing** includes calling all telephone leads, prompting s ample members to complete the online or telephone interview, attempting to get updated contact information from parents or other contacts, and following - up on leads generated by these contact attempts. 3. **Pre -**  **Intensive Batch Tracing** is conducted between tel ephone locating efforts and intensive tracing, and is intended to locate cases as inexpensively as possible before moving on to more costly intensive tracing efforts. 4. **Intensive Tracing** consists of tracers checking all telephone numbers and conducting credit bureau database searches after all current telephone numbers have been exhausted . 5. **Other Locating Activities** will take place as needed and may include use of social networking sites and use of new tracing resources. **Initial Contact Update.** BPS:12/14 wil l also conduct an initial contact address update to encourage sample members to update their contact information prior to the start of data collection. The initial contact mailing and email will introduce the study and ask sample members to update their co ntact information via the study website. The address update will be conducted several weeks prior to the start of data collection. **Panel Maintenance.** Each year following the BPS:12/14 data collection, we contact sample members b y mail and email to encourage them to update their contact information using a Web address update form . Conducting periodic panel maintenance activities allows us to maintain up - to - date contact information, increasing the likelihoo d of locating sample members in the future. Reducing the time elapsed between contacts maintains sample members’ connection to the study , maximizing response rates for the subsequent follow - up. Receiving approval to conduct the panel maintenance activiti es as part of this OMB package will allow the activities to take place well in advance of the BPS:12/17 data collections. The initial contact letter and email, the panel maintenance postcard and email, and the Web address update form may be found in Appendix F.

**b. Prompting Cases with Mail, E**  **- mail, and SMS Contacts**  Past experience on recent postsecondary studies, including the BPS:12/14 field test, has shown that maintaining frequent contact with sample members with mailings , e - mails , and text messages sent at regular intervals will maximize response rates. The reminders to be sent regularl y throughout the study include the following:10

 Initial contact letter and e - mail sent to introduce the study and request updated contact information for the sample member.  Data collection announcement letter and e - mail sent to announce the start of the data collection period and encourage sample members to complete the Web interview.  Reminder letters and postcards sent to nonresponding sample members throughout data co llection and customized to address to concerns of nonresponding sample members.  Brief reminder e - mails to remind nonresponding sample members to complete the interview.  Brief text (SMS) message reminders sent to those sample members who have provided consent to receive text message s . Example contacting materials are also included in Appendix F .

**c. Telephone Interviewing and Help Desk Support**  **Telephone interviewer training.** Well - trained interviewer s play a critical role in gaining sample member cooperation. The BPS:12/14 training program will include an iLearning module, which will provide staff with an introduction to the study prior to in - person training; 12 hours of in - person project training that will include training exercises, mock interviews, and hands - on practice; and ongoing training throughout data collection. The BPS:12/14 Telephone Interviewer manual will cover the background and purpose of BPS, instructions for providing Help Desk support, and procedures for administering the telephone interview. **Interviewing and Prompting.** Telephone interviewing will be conducted using the same survey instrument as self

- administered Web interviews. D ata collection for the calibration study will begin in February 2013, and sample members selected for the calibration study will be encouraged to complete the survey on the Web during a 4 - week earl y completion period. After four weeks, outbound calling will begin for sample members who did not participate during the earl y completion period. The remaining BPS:12/14 s ample will begin data collection approximately 7 weeks after the calibration sample. Telephone interviewing and prompting will be managed using the CATI Case Management Sy stem (CMS). The CATI - CMS is equipped to provide interviewers with access to all loc ating information and case histories, track scheduled appointments, and record notes about previous contacts with the sample member. Once cases are available for outbound calling, the CATI - CMS call scheduler will determine the order in which cases are to be called. The scheduler algorithm weighs several factors to determine the most appropriate number and time to call and automaticall y delivers the case to the interviewer for dialing. The CATI - CMS facilitates efficient case management by reducing su

pervisory and clerical time with integrated reports, monitoring appointments and callbacks automaticall y, and ensuring consistency

of approach across interviewers. For most cases, outbound prompting will begin 4 weeks after the start of data collection. A s with NPSAS:12 and the BPS:12/14 field test, RTIplans to initiate telephone prompting efforts earlier for some challenging cases to maximize response rates. These may include NPSAS:12 nonrespondents and sample members from institutional sectors with his torically low participation rates. For these cases, outbound prompting will begin as early as one week after the start of data collection.11

**Refusal Conversion.** Avoiding refusals is important to maximizing the response rate for BPS:12/14. All interviewers will be train ed in techniques that are designed to gain cooperation and avoid refusals whenever possible. Our training program stresses the importance of learning the most frequently asked question s, and when a sample member has concerns about participation, our interviewers are expected to respond to these questions knowledgably and confidentl y. When a refusal does occur, interviewers will enter comments into CATIto record all pertinent information about the case, including unusual circumstances or reas ons given by the sample member for refusing. Through our monitoring efforts, we will identify interviewers who are especiall y adept at addressing sample members’ concerns and will place them on a team of refusal conversion specialists. When a refusal is encountered, these specialists will review the call notes to determine the reason for the refusal and any concerns or questions posed by the sample member. The refusal specialists will determine whether a follow - up call is appropriate and will then tail o r their response to resolve the issue.

**d. Quality** 

**Control** RTIhas developed quality control procedures based on “best practices” identified over the course of conducting thousands of survey research projects. Interviewer monitoring will be conducted using RT I’s Quality Evaluation Sy stem (QUEST), a proprietary s ys tem that includes standardized monitoring protocols, performance measures, evaluation criteria, and reports to analyze performance data across interviewers. As in previous studies, RTIwill use QUES T to monitor approximately 7 percent of all recorded completed interviews, approximately 10 percent of recorded refusals, representing about 3 percent of overall budgeted interviewer hours. Calls will be reviewed b y call center supervisors for key elements such as professionalism and presentation; case management and refusal conversion; and reading, probing, and keying skills. Feedback will be provided to interviewers and patterns of poor performance will be carefully documented, and if necessary , addressed with additional training. Sample members will be notified that the interview may be monitored by

supervisory

staff. Regular Quality Circle (QC) meetings will provide another opportunity to ensure quality and consistency

in interview administration. The se meetings give interviewers a forum to ask questions about best practices and share their experiences with other interviewers. They provide opportunities for brainstorming strategies for avoiding or converting refusals and optimizing interview quality, which ensures a positive experience for sample members. **4. Tests of**  **Procedures and Methods**  The design of the BPS:12/14 full - scale data collection

—in particular, its use of responsive design principles to reduce bias associated with nonresponse

—expands on data collection experiments designed for a series of preceding NCES studies. Although final results are not available for the most recent of those experiments, the design proposed for BPS:12/14 builds on what is known to date.

**a. Previous Studies**  NCES began efforts to reduce nonresponse bias through the use of responsive design principles in 2011 , as it fielded the NPSAS:12 and B&B:08/12 f ield t ests. In this earl y phase of work, the effort focused on the reduction of nonresponse bias through the identification of those cases least likely to12

respond. As a result, two strands of work emerged: (1) th e development of models that accuratel y predicted response propensity, and (2) the identification of incentive amounts that would convert potential non - respondents. Although both the NPSAS:12 and B&B:08/12 f ield t ests demonstrated that it was possible to develop models that were reasonabl y predictive of response propensity, results related to incentives were mixed. Taken together, results of experimentation in these studies suggested that relativel y high incentive amounts ( e.g., $70 for B&B ) are unlikel y to convert cases with the lowest propensity scores, but that they can be useful in incentiviz ing response among cases with low - moderate to moderate scores. Additionally , differential response to reduced incentive amounts among high propensity cases in NPSA S:12 and B&B:08/12 suggest that the undergraduate population as a whole may be more sensitive to lower incentive amounts than students who have already completed their baccalaureate degree. However , neither study

yielded evidence of bias reduction. Result s from the EL S:2002/12 f ield t est, which also focused on simple response propensity, produced similar , equivocal results. As a result of its experience with the studies identified above, NCES sought to develop methods to better identify

cases that were pot entiall y bias - inducing. This approach involved: (1) identifying a set of substantive variables believed to be related to key study outcomes (typically frame variables for cross sectional studies, but including prior - wave data for longitudinal studies), (2) using those variables to calculate the Mahalanobis distance of each case from either the full sample (for models estimated prior to data collection) or respondents (for models estimated during data collection), and (3) identifying ranges of distance score s for additional incentives or contractor effort. Generall y, this “second generation” of responsive design efforts sought to replace targeting based on response propensity

— which did not seem to reduce bias

—with targeting based on a measure of multivariate difference. The conversion of so called “high - distance” cases, it was hy pothesized, would reduce nonresponse bias. To date, distance - based approaches have been employed in the B&B:08/12 full - s cale, the EL S:2002/12 f

ull s cale, the BPS:12/14 f ield t est, and the HSL S:09 201 3 Update field test . While analy sis of the B&B design and data collection for HSL S full - scale continue, the results from EL S and BPS FT are mixed. Although there was some evidence of a reduction in bias in the ELS and B&B design s , which invo lved a mix of increased contractor effort and higher incentives, no bias reduction was found in BPS, despite increasing response among high - distance cases through higher incentives.

**b. BPS:12/14 Full Scale**  After reviewing almost two years of related data, NCES experts are not convinced that they have identified an optimal model for characterizing a given case’s likelihood of contributing to nonresponse bias or identified an optimal procedure for delivering a given case’s ideal conversion intervention (ty pically an incentive). As a result, NCES proposes an experiment al effort within the BPS:12/14 f ull s cale that seeks to address the latter concern through an incentive calibration study and the former through the use of a two - component measure of “import ance” that jointly considers a case’s likelihood of inducing bias and its propensity

to respond. We begin by describing the proposed importance measure, proceed to a discussion of the calibration experiment, and then outline how the two will be combined in the BPS:12/14 f ull s cale data collection.13

**The importance measure.** In order to reduce nonresponse bias in survey variables b y directing effort and resources during data collection, and minimize the cost associated with achieving this goal, three related co nditions have to be met: (1) the targeted cases must be drawn from groups that are underrepresented on key

survey variable values among those who already responded, (2) their likelihood of participation should not be excessively low or high (i.e., targete d cases who do not respond cannot decrease bias; targeting high propensity cases can further bias estimates), and (3) they should be numerous enough to impact survey estimates within domains of interest. For example, targeting cases based on response prope nsities may reduce nonresponse bias, but it may fail if the targeted cases are extremely difficult to convert and do not respond to the intervention as desired. One approach to meeting these conditions is to target cases based on two dimensions: the likel ihood of a case to contribute to nonresponse bias if not interviewed, and the likelihood that the case could be converted to a respondent. These dimensions form an importance score, such that: ∝ ∙ ( ) Where I is the calculated importanc e score, is a measure of under - representativeness on key variables that reflects their likelihood to induce bias if not converted, and ( ) is the predicted final response propensity . Let index sample members and the data collection phases with responsive design interventions. The importance score will be determined b y the combination of two models: a bias - likelihood model and a response propensity model, which are described in detail below. The calculation needs to balance two distinct scenarios: (1) low propensity cases that will likely

never respond, irrespective of their underrepresentation, and (2) high propensity cases that, because they are not underrepresented in the data, are unlikel y to reduce bias. There are a numb er of potential methods for combining these two models. Once in production, NCES will have more information about the distribution of both propensity and representation from the calibration study , which will allow it to explore linear and nonlinear functio ns that optimize the potential for nonresponse bias and available incentive resources. **Bias -**  **likelihood (U) model .**  A desirable model to identify cases to be targeted for intervention would use covariates (Z) that are strongl y related to the survey variables of interest (Y), to identify sample members who are under- represented (using a response indicator, R) with regard to these covariates. We then have the following relationships, using a single Z and Y for illustration: Z R Y Nonresponse bias arises when there is a relationship between R and Y. Just as in adjustment for nonresponse bias (see Little and Vartivarian, 2005 ), a Z - variable cannot be eff ective in nonresponse bias reduction if corr(Z,Y) is weak or nonexistent, even if corr(Z,R) is substantial. That is, selection of Z variables based only

on their correlation with R may not help to identify

cases that contribute to nonresponse bias. The goal is to identify sample cases that have Y - variable values that are associated with14

lower response rates, as this is the most direct way to reduce nonresponse bias in an estimate of a mean, for example. The key Z - variable selection criterion should then be association with Y. Good candidate Z variables would be the Y - variables or their proxies measured in a prior wave and any correlates of change in estimates over time. A second set of useful Z - variables would be those used in weighting and those used to def

ine subdomains for anal ys is

– such as demographic variables. This should help to reduce the variance inflation due to weighting and nonresponse bias in comparisons across groups. Key , however, is the exclusion of variables that are highl y predictive of R , but quite unrelated to Y. These variables, such as the number of prior contact attempts and prior refusal, can dominate in a model predicting the likelihood of participation and mask the relationship of Z variables that are associated with Y. As needed in Phases 3 - 5 , described below , we will conduct a logistic regression in order to predict the survey

outcome (R) through the current phase using onl y substantive and demographic variables and their correlates from NPSAS:12 and the sampling frame (Z), and select two - way interactions. The goal of this model is not to maximize the ability to predict survey

response ( p ), but to obtain a predicted likelihood of a completed interview reducing nonresponse bias if successfully interviewed. Because of this key diff erence, we use (1

– p ) to calculate a case - level prediction representing bias - likelihood, rather than response propensity . Variables to be used in the bias - likelihood model will come from base - year survey responses, school characteristics, and sampling frame information 3

. Itis important to note that paradata, particularly those variables that are highl y predictive of response but quite unrelated to the survey variables of interest, will be excluded from the bias - likelihood model. Potential variables for the model include, but are not limited to:  highest degree expected,  p arents’ level of education ,  age,  gender,  number of dependent children,  income percentile,  h ours worked per week while enrolled,  school sector,  undergraduate degree program,  e xpected wage , and  h igh school graduation year . **Response propensity (P(R)) model.** Prior to the start of data collection, a response propensity model will be developed to predict likelihood to respond to BPS:12/14 based on NPSAS:12 data and response behavior. It will inc lude variables from the base ye ar that have been shown to predict survey response, including, but not limited to:  responded during earl y completion period,

3 Key variables will use imputed data to account for nonresponse in the base year data.15

 interview mode (web/telephone),  ever refused,  call count, and  tracing/locating status (located/required intensive tracing) . In addition to estimating this initial model, we will evaluate the feasibility and potential benefit of adjusting the predicted response propensity scores at the start of each phase based on response behavior in the current wave. **The incentive calibration experiment.**  Prior to the start of full - scale data collection, approximately

10 percent of the main sample will be identified to participate in the incentive calibration study and will be randomly

assigned incentive offers of $0 to $50, in $5 increments. The results from this randomized calibration study will then be used to determin e the incentive amounts for the remaining 90 percent of the sample. NCES will calculate each calibration sample member’s response propensity and, seven weeks prior to the main study , move the sample to production. During production, response rates by incentive level within propensity decile will be monitored, with the goal of identifying the optimal incentive amount for each. Other than the experimental manipulation above, colle ction activities for the calibration sample will be identical to those used for the main study. These are described in more detail be low, and summarized in table 2. Both main study and calibration samples will be exposed to a six phase data collection pro tocol : Phase 1: A four- week web only collection period . As their first phase of data collection begins, students in both BPS:12/14 subsamples

– calibration and main study

– will receive a letter asking them to log onto the web to complete the questionnaire. The calibration subsample will begin data collection on or around 2/18/2014 and the main subsample will begin data collection on or around 4/8/2014. Baseline incentive amounts (ranging from $0 to $50 in $5 increments) for the main sample will be deter mined by each case’s predicted response propensity and the results of the calibration study . Results of the NPSAS:04 field test experiment showed that some students will respond when no incentive is offered ($0). Although no a priori assessment of respons e propensity was estimated in the NPSAS:04 field test, cases that responded without being offered an incentive would have been considered high propensity

cases. Given those results, and the findings from B&B:08/12 that showed there was not a significant difference in response rates between $20 and $35 dollar incentive amounts for the highest propensity

cases, we expect that about 30 percent of sample members with the highest propensity to respond will participate when offered the lowest incentive amounts (between $0 and $20). For the middle 40 percent, or about 13,000 cases, we expect that initial incentive offers at or above $35 will be required to improve likelihood of response. In the B&B:08/12 field test this group was shown to be the most influenced by monetary

incentives, but no comparable study has yet explored the potentially

diminishing returns of larger monetary incentives. One goal of the calibration study is to better understand the relationship between response rates and monetary incentives, u

p to the proposed total of $95, for the middle propensity group. For the 30 percent of cases with the lowest propensity to respond, B&B:08/12 results also showed that offering an additional $15 over the base incentive did not significantl y increase respon se rates, suggesting that the highest incentive offered to the B&B:08/12 low propensity

cases ($55) is an estimate for the maximum cumulative incentive that will be required for the BPS low propensity

cases.16

Phase 2: Web and CATI data collection period . After the four- week, Phase 1 data collection period, in addition to the web interview option, outbound calling to sample members will begin and continue through the end of the survey . Promised incentives will be offered at the amount set at the start of data collection. This Phase 2 portion of data collection will continue for approximately 3 weeks. Phase 3: First stage of a targeted nonresponse follow - up period. Approximately 7 weeks into data collection for each sample, predicted bias - likelihood values will be calculated for all interview nonrespondents. The product of the bias - likelihood and response propensity will be used to calculate the target importance score described above . Prior to creating the importance score, bias - likelihood estimates will be centered to the median predicted bias - likelihood score to ensure that no overrepresented cases are targeted. Additionall y, propensity scores above high and low cutoffs, determined by a review of the predicted distribution, will be excluded as potential targets 4

. We expect that about 58 percent of the initial sample still will be nonrespondents at this point in data collection. Among them, c ases with the highest importance scores will be identified for special intervention during this phase. Targeted cases in the calibration subsample will be randomly selected to receive a $15, $30 , or $45 incentive increase. We anticipate that about 30 percent of cases (about 6, 5 00) will be t argeted in the main subsample to receive either a $15, $30, or $45 incentive increase based on the results of the calibration study . All other cases will continue to be offered the baseline, promised incentive offered in Phase 1. Phase 4: Second stage of the targeted nonresponse follow - up period. Approximately four weeks after th e start of Phase 4 for each sample, predicted - likelihood values and importance scores will be recalculated for all remaining, nonresponding cases. Phase 4 targeted cases will be assigned to special field or call center staff who will use local exchanges to make outbound calls. Both targeted and non targeted cases will be offered the same monetary incentives they

were offered at the beginning of Phase 4. Phase 5: Third stage of a targeted nonresponse follow - up period. Approximately 21 weeks into data collec tion for each sample, predicted - likelihood values and importance scores will be recalculated for any remaining nonresponding cases. Targeted cases in both the calibration and main subsamples will be offered an abbreviated questionnaire that can be complete d in approximately 10 minutes and on a mobile device. The incentive amounts determined in previous phases will continue to apply ; nonrespondents newly targeted in Phase 5 will continue to be offered their original promised monetary incentive. Phase 6: Abbreviated phase. Approximately 25 weeks into data collection for each sample, all remaining nonrespondent cases will be offered an abbreviated questionnaire that can be completed in approximately

10 minutes and on a mobile device. The offered incentives, as determined in previous phase(s), will continue to apply .

4 These adjustments wil l help ensure that currently over - represented groups, high propensity/low importance cases , and very- difficult - to convert nonrespondents are not included in the target set of nonrespondents. The number of targeted cases will be determined in conjunction wi th the COR and by the overall and within sector distributions of importance scores.17

**Ta ble 2.**  **Summary of start dates and activities for each data collection phase, by sample Phase**  **Start date Activity** 

**Calibration subsample**  **Ma in** 

**subsample Calibration subsample**  **Ma in subsample**  1 2/18/2014 4/8/2014 Begin web collection; first calibration Begin web collection; baseline incentives determined by results of first calibration 2 3/18/2014 5/6/2014 Begin CATI collection Begin CATI collection 3 4/8/2014 5/27/2014 Target cases; second calibration Target cases; incentive increase determined by results of second calibration 4 5/6/2014 6/24/2014 Field/local exchange calling for targeted cases Field/local exchange calling for targeted cases 5 7/15/2014 9/2/2014 Abbreviated interview with mo bile access for targeted cases Abbreviated interview with mo bile access for targeted cases 6 8/12/2014 9/30/2014 Abbreviated interview for all remaining nonrespondents Abbreviated interview for all remaining nonrespondents **Analysis of the BPS:12/14 Full Scale Responsive Design Effort**  **.** Our analysis plan is based upon three premises: (1) sample cases that would contribute to nonresponse bias can be identified at the beginning of the third and subsequent data collection phases, (2) the interventions during the third through fifth data collection phases are effective at increasing participation, and (3) increasing response rates among the targeted cases will reduce nonresponse bias by converting cases that would otherwise induce bias if they

failed to respond . We intend to examine these three aspects of the BPS:12/14 responsive design and its implementation as follows: 1. Evaluate targeted cases for under
- representativeness and potential impact To maximize the effectiveness of the BPS:12/14 responsive design approach, targeted cases need to be associated with survey responses that are underrepresented among the respondents , and the targeted groups need to be large enough to change observed estimates. Inaddition to assessing model fit metrics and the effective identification of cases contributing to nonresponse bias for each of the models used in the importance score calculation, the distributions of the targeted cases will be reviewed b y ke y variable s within sector prior to identifying final targeted cases. During data collection, these reviews will help ensure that the cases most likely to decrease bias are targeted and that project resources are used efficientl y. After data collection, similar summaries will be used to describe the composition of the targeted cases along dimensions of interest. 2. Evaluate the effectiveness of each phase of data collection in increasing participation. The second key

component of the BPS:12/14 responsive design is the ef fectiveness of the changes in survey protocol for increasing participation. Each phase introduces a feature

– additional promised incentives, special field work, and an abbreviated mobile instrument. A calibration study will be used to determine the optimal baseline and Phase 3 intervention incentives.18

Evaluation of the calibration study will occur during data collection so that findings can be implemented in the main subsample data collection. Approximately four weeks after the start of data collection fo r the calibration subsample, a logistic regression model will be estimated within each propensity

decile . After estimating predicted response propensities, the minimum dollar amount associated with a predicted propensity that is not significantl y different than the highest predicted propensity will be used as the baseline incentive for that propensity decile in the main subsample. Approximately

four weeks after the start of the calibration study , a similar anal ys is will be conducted to determine the optimal incentive increase for the first phase of targeted nonrespondents. In addition to analyzing data for the calibration study , we will evaluate response rates during the course of data collection, expecting an increase for targeted cases over the course of p hases that include effective interventions. Furthermore, the cases that are not targeted with additional interventions will serve as a baseline for the pattern of responding over the course of the survey . Targeted nonrespondent completion rates are expecte d to increase with each phase, relative to the group that receives onl y standard recruitment efforts. 3. Evaluate the ability to reduce nonresponse bias. The rich frame, administrative, and prior wave data used in determining which cases to target for nonre sponse bias reduction can, in turn, be used to evaluate (1) nonresponse bias in the final estimates and (2) changes in nonresponse bias over the course of data collection. Unweighted and weighted (using design weights) estimates of absolute relative nonresponse bias will be computed for each variable used in the models:

where is the respondent mean and is the full sample mean. The mean of these bias estimates can be tracked during the course of the survey and particular attention will be devoted to changes in the mean bias over each phase of data. BPS:12/14 Responsive Design Research Questions . With the assumption that increasing the rate of response among targeted cases will reduce nonresponse bias, the BPS:12/14 responsive design experiment will e xplore the following research questions which build to our ultimate goal: 1. Are targeted respondents different from non - targeted respondents on key variables? 2. Did targeted cases respond at higher rates than non - targeted cases? 3. Did conversion of targeted ca ses reduce nonresponse bias? Each of these questions may be stated in terms of a null hy pothesis as follows: Research question 1 : Are targeted respondents different from non - targeted respondents on key variables? H 0 : At the end of Phases 3, 4, and 5, and the end of data collection, there will be no difference in weighted or unweighted estimates of key variables between targeted respondents and non - targeted and never - targeted respondents. Research question 2 : Did t argeted cases respond at higher rates than non - targeted cases? r s s y y y 

r y s y19

H 0 : At the end of Phases 3, 4, and 5, and at the end of data collection, there will be no difference in weighted or unweighted response rates between targeted cases and non - targeted and nevert argeted cases. Research question 3 : Did conversion of targeted cases reduce nonresponse bias? H 0 : At the end of Phases 3, 4, and 5, and at the end of data collection, there will be no difference in weighted or unweighted changes in bias between targeted cases and non - targeted and nevertargeted cases. Although not directl y related to bias reduction, but relevant to the number of nonrespondents that ultimately

require intervention and the nature of the Phase 3 intervention , are the questions: 1. What is the optimal baseline incentive amount within propensity deciles? 2. What is the optimal intervention amount for Phase 3 targeted nonrespondents? These questions will be explored by estimating regression models and simulating marginal outcomes as outlined in King et al. (2000). The calibration sample will be a random sample of approximately 10 percent of the 37,170 full scale sample members. An estimated probability of response will be produced for each of the approximately

3,717 calibration sample members , and the calibration sample members will then be grouped into deciles constructed from the estimate response probabilities. The approximately 371 calibration sample members in each decile will be randomly assigned to one of eleven incentive levels; $0, $5, $10, $ 15, $20, $25, $30, $35, $40, $45, and $50 so that approximately 33 calibration sample members will be assigned to each incentive level within each decile. Calibration sample response rates will be examined, within each response probability decile, prior to the start of full - scale data collection in order to identify the incentive levels for use in the full - scale data collection. Formall y, within each decile, ten cutpoints will be examined in order to determine the minimum incentive level that maximizes re sponse rates for that decile. The first cutpoint will be set between the $0 and $5 incentive levels and the response rate for the $0 incentive level will be compared with the response rate among indivi duals who received $5 to $50 as an incentive. The seco nd cutpoint will be set between the $5 and $10 incentive levels , and the response rate for individuals who received $5 or less will be compared with the response rate for individuals who received $10 or more. The third through ninth cutpoints will be defin ed in a similar fashion , with the tenth cutpo int set between $45 and $50; the response rate for individuals offered the $50 incentive will be compared with the response rate for individuals offered $45 or less. A power anal ys is was conducted in order to assess the ability to detect differences in response rates when c onsidering any given cutpoint. F or the purposes of estimating power, the total number of individuals within a given decile was assumed to be 363 (33 individuals for each of eleven incentive l evels), an alpha level of 0.05 was used, and a one - sided test was specified. Given these assumptions, the power to detect any given difference still varies significantly depending upon the values of the proportions being estimated. In the tables below, s ample power estimates are provided under different assumptions about the underlying response rates.20

Using a two - group chi - square test of equal proportions with unequal sample sizes in each group, the power to detect a difference of 15 percent is 76 percent if the underl ying true response rate is 25 percent for one group comprised of 33 individuals and the underl ying true response rate is 10 percent for the other group comprised of 330 individuals. This power analy sis corresponds to the comparison involving the first and tenth cutpoints because in those comparisons there will be two groups; one group comprised of 33 individuals and the other group comprised of the remaining 330 individuals. Additional power estimates were calculated for all cutpoints under th e assumption that the underl ying true response rates were 10 percent and 25 percent in the two groups. Table 3 shows the power estimates for the ten cutpoints with the lowest power of 65 percent occurring for the first cutpoint and the highest power of 98 percent occurring with the fifth and sixth cutpoints. **Table 3**  **. Estimated p**  **ower, w**  **ithin d**  **ecile, u**  **sing 10 percent**  **and 25 percent**  **u nderlying**  **r esponse**  **r ates**  **Cutpoint Group 1** 

**Sample Size Group 2** 

**Sample Size Total**  **True Response Rate Group 1**  **True Response Rate Group 2**  **Power** 1 33 330 363 0.1 0.25 65 2 66 297 363 0.1 0.25 89 3 99 264 363 0.1 0.25 96 4 132 231 363 0.1 0.25 97 5 165 198 363 0.1 0.25 98 6 198 165 363 0.1 0.25 98 7 231 132 363 0.1 0.25 97 8 264 99 363 0.1 0.25 95 9 297 66 363 0.1 0.25 91 10 330 33 363 0.1 0.25 76 Because the power estimates vary depending upon the assumed values of the underl ying response rates, table 4 shows the power estimates at the lowest possible power , when the underl ying true response rates approach 50 percent . **Table 4.**  **Estimated p ower,**  **w ithin**  **d ecile,**  **u sing 45**  **percent and 60**  **percent u**  **nderlying r** **esponse r** **ates Cutpoint**  **Group 1**

**Sample Size Group 2** 

**Sample Size Total**  **True Response Rate Group 1**  **True Response Rate Group 2**  **Power** 1 33 330 363 0.45 0.60 50 2 66 297 363 0.45 0.60 71 3 99 264 363 0.45 0.60 82 4 132 231 363 0.45 0.60 86 5 165 198 363 0.45 0.60 88 6 198 165 363 0.45 0.60 88 7 231 132 363 0.45 0.60 86 8 264 99 363 0.45 0.60 82 9 297 66 363 0.45 0.60 71 10 330 33 363 0.45 0.60 50 Given the sensitivity

of power estimates to assumptions regarding the underlying true response rates, there appears to be sufficient power to support the proposed calibration experiment across a wide range of possible true underlying response rates.21

**5. Rev**  **iewing Statisticians and Individuals Responsible for Designing and**  **Conducting the Study** Names of individuals consulted on statistical aspects of study design, along with their affiliation and telephone numbers are:

Name Affiliation Telephone Dr. Jennifer Wi ne RTI (919) 541 - 6870 Dr. James Chrom y RTI (919) 541 - 7019 Dr. Natasha Janson RTI (919) 316 - 3394 Mr. Peter Siegel RTI (919) 541 - 6348 Dr. David W ilson RTI (919) 541 - 6990 Dr. Bryan Shepherd RTI (919) 316 - 3482 Dr. Paul Biemer RTI (919) 541 - 6056 Dr. Em ilia Peytcheva RTI (919) 541 - 7217 Dr. Andy Peytchev RTI (919) 485 - 5604 Dr. Alexandria Radford RTI (202) 600 - 4296 Dr. John Riccobono RTI (919) 541 - 7006 In addition to these statisticians and survey design experts, the following statisticians at NCES have also reviewed and approved the statistical aspects of the study: Dr. Tracy Hunt - White, Ted Socha, Dr. Matt Soldner, Dr. Sean Simone, and Dr. Sarah Crisse y . **6. Other Contractors’ Staff Responsible for Conducting the Study**  The study is being conducted b y the Sample Survey s D ivision of the National Center for Education Statistics (NCES), U.S. Department of Education . NCES’s prime contractor is RTI. Principal p rofessional staff of the contractors, not listed above, who are assigned to the study are provided below: Name Affiliation Telephone Ms. Nicole Ifill RTI (202) 600 - 4295 Mr. Jason Hill RTI (919) 541 - 6425 Ms. Donna Anderson RTI (919) 990 - 8399 Mr. Jeff Franklin RTI (919) 485 - 2614 M s. Chris Rasmussen RTI (919) 541 - 6775 Ms. Tiffany Mattox RTI (919) 485 - 7791 Mr. Michael Brya n RTI ( 919 ) 541 - 7498 Dr. Jennie W oo RTI (510) 665 - 827622 

### C. **Overview of A**  **nal ysis Topics**  **and Surve**  **y Items**  The BPS:12/14 full - scale data collection instrument is presented in Appendix G . Many of the data elements to be used in BPS:12/14 appeared in the previously approved NPSAS:12 and BPS:04/09 interviews . New items, as well as items that are to be included in the abbreviated intervi ew , are identified in Appendix G .



### References Chromy , J.R. (1979). Sequential Sample Selection Methods. In Pr oceedings of the Section on Survey Resear ch Methods, American Statistical Association (pp. 401

–406). Alexandria, VA : American Statistical Association. Folsom, R.E., Potter, F.J., & Wi lliams, S.R. (1987). Notes on a Composite Size Measure for SelfWe ighting Samples in Multiple Domains. Pr oceedings of the Section on Survey Resear ch Methods o

f the American Statistical Association , 792 - 796. King, Gary , Michael Tomz, and Jason Wittenberg. 2000. Making the Most of Statistical Analyses: Improving Interpretation and Presentation. American Journal of Political Science 44: 341

–355. Little, R. J. and Vartivarian, S. (2005). Does Weighting for Nonresponse In crease the Variance of Survey Means?. Statistics Canada, 31: 161 - 168.