1 **SUPPORTING STA**

**TEMENT PA RT B**  **FOR PA PERWORK REDUCTION A**  **CT SUBMISSION Gainful Employ**  **ment Recent Graduates Employ ment and Earnings Survey Pilot**  **Test B. Collection of Information**  **Employ ing Statistical Methods**  The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. The following documentation should be provided with the Supporting Statement Part A to the extent that it applies to the methods proposed. For further information, please consult OMB’s Stan dards and Guidelines for Statistical Surveys

. 1. Describe the potential respondent universe (including a numerical estimate) and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, state and lo cal government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected res ponse rates for the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection. The sample frame for the RGEES pi lot test is NSLDS, a transactional data base containing information about those who have been awarded federal student aid. The respondent universe consists of graduates from gainful emplo ym ent programs who received federal student aid and who completed the ir program between July 1, 2009 and June 30, 2011. The target population is approximately 2,500,000. 1
**S ampling and Expected Yield**  We will conduct a pilot study with a sample of 3,400 federal aid recipients selected from NSL DS. The pilot will be used to compare median earnings collected through the appeals survey to median earnings for graduates from comparable programs based on a match to the Social Security

Administration as part of the 2012 gainful employ ment informational rates. The results of the pil ot will also be compared to earnings estimates in the CPS and the ACS. The anticipated distribution of the sample b y program area is provided in Table 1. The expected y ield is based upon an assumed 60% response rate. The specific programs, listed by the co de assigned in the NCES Classification of Instructional Programs (CIP), were selected based on the assumed likelihood of their graduates being survey ed under the gainful employ ment regulations.

1 "Program Integrity: Gainful Employment; Final Rule." Federal Register 79 (31 October 2014): 65061.2

**Table 1: Pilot test starting sample and yield, by program Program**  **CIP Code*** **Sample size Expected**  **Yield Total**  **3,400 2,040**  **Cosmetology and related personal grooming services**  **12.04 700**  **420 Somatic bodywork and related therapeutic**  **services 51.35**  **700 420**  **Practical nursing, vocational nursing, and nursing**  **assistants 51.39**  **700 420** 
**All others All others**  **1,300 780** 
*Includes all 6

- digit CI P codes within each general category

. **Nonresponse Bias Analysis** To the extent that those who respond to survey s and those who do not differ in important way s, there is a potential for nonresponse bias in estimates from survey data. Overall unit response rates for the RGEES are expected to be at least 50 percent of an i dentified cohort and, in keeping with the draft standards for its administration, a nonresponse bias anal ys is (NRBA) will be required when unit response rates are less than 80 percent. At the completion of the pilot study , the NRBA will compare respondents and nonrespondents within the program areas targeted (see Table 1). Demographic and other characteristics of both the respondents and nonrespondents needed for the NRBA will be obtained from NSL DS from which the sample will have been selected. To be a respondent, a graduate must answer at least one of the earnings questions. Nonresponse may

be due to refusal, noncontact, noninterview due to incapacity , language difficulties, etc. Nonresponse bias will be computed for the pilot study similarly to how it will be computed by

institutions that conduct the survey . To measure nonresponse bias, we will compare respondents and nonrespondents. within program areas, on information available from graduates’ student records. Anal ys is of recent program level data identified four variables as those most correlated with earnings

— students with Pell grants, students with mothers with a bachelor’s degree or above, students who are independent, and gender. The NRBA will examine these attributes of the program graduates to determine whether response rates vary

on the attributes and whether the characteristics of respondents and nonrespondents differ on these characteristics. The measure of nonresponse bias will be calculated as the ratio of the difference between resp ondents and nonrespondents to the reported survey

mean for a particular attribute. For a mean, an estimate of the bias of the respondent mean is given by :

_ _ _ _ _ B(yr) = y r – y t = (n nr /n) (y r – y nr )3 Where: y t = the mean based on all cases; y r = the mean based onl y on respondent cases; y nr = the mean based only on the nonrespondent cases; n = the total number of cases; and n nr = the number of nonrespondent cases.

2. Describe the procedu res for the collection of information, including:

 Statistical methodology for stratification and sample selection.

 Estimation procedure.

 Degree of accuracy needed for the purpose described in the justification.

 Unusual problems requiring specialized sampling procedures, and

 Any use of periodic (less frequent than annual) data collection cycles to reduce burden. This section describes the data collection procedures to be used in the RGEES pilot test. Once selected for participation in the pilot survey , sample members will receive several mailings, depending on the speediness of their reply and the stage of data collection. In earl y September, sample members will receive an initial mailing containing introductory letters from RTI and NCES and a paper co py of the survey

with a business repl y envelope. Nonrespondents to the initial mailing will receive a series of follow

- up mailings, including a postcard and a second letter, sent from RTI b y Priority Mail, with another copy

of the survey and a business re pl y envelope. In addition to hard copy mailings, sample members with email addresses on the NSLDS data base will receive an email reminder, approximately every 7 to 10 day s, throughout the course of data collection. The survey instrument is provided in App endix 2 and respondent contact materials are provided in Appendix 3. **Survey Monitoring** Mail survey

returns will be processed upon receipt, and reports from the survey management sy stem will be prepared at least weekl y. The reports will be used to continually assess the progress of data collection. 3. Describe methods to maximize response and to deal with issues of non

- response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections base d on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied. The RGEES incorporates a number of features to maximize response rates. This section discusses those features. **Total Design Method/Respondent**
**- Friendly Design.**  Survey s that take advantage of respondent

- friendl y design have demonstrated increases in survey response (Dillman, Smyth,4

and Christian 2008; Dillman, Sinclair, and Clark 1993). As noted prev iously , we will include a respondent incentive in the initial mailing. The data collection contractor will maintain an email address and a toll

- free questionnaire assistance (TQA) line to answer respondent questions or concerns. If a respondent chooses to provide their information to the TQA staff, staff will be able to collect the respondent’s information over the phone. **Engaging Respondent Interest and Cooperation.** The content of respondent letters and postcards is focused on communicating the legitimacy and importance of the study , how pilot data will be used, and that there is minimal burden required in completing the survey (5 minutes). To further encourage parti cipation within the timeline of the pilot test, all participants will be offered a $25 incentive for returning a completed survey. Respondents will be able to choose between receiving the incentive pay ment either by

check or b y Pa yP al, sent to an email ad dress provided in the survey , once receipt of the survey is confirmed. Choice of pay

ment method will be included at the end of the survey where participants will be asked to provide an email address if they choose PayPal. **Nonresponse Follow**
**- up. As liste**  **d above, t** he data collection protocol includes several nonresponse follow

- up activities throughout the 2

- month data collection. In addition to the number of contacts, changes in method of the follow

- up contact (e.g., Priority Mail) are designed to capture the attention of sample members. 4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they cal l for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information. Not applicable.
5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other persons who will actually collect and/or analyze the information for the agency. The persons listed below participated in th e stud y design and are responsible for the collection and anal ys is of the data: Sharon, Boivin, NCES Chris Chapman, NCES Marily n Seastrom, NCES Sean Simone, NCES T. Austin L acy , RTI International Peter Siegel, RTI International Jennifer Wine, RTII nterna tional Erin Dunlop Velez, RTII nternational