Part A: Supporting 

Statement for 

Paperwork Reduction 

Act Submission

Effectiveness of 

Promising Strategies in 

Federal College Access 

Programs: Study of 

Student Messaging in 

GEAR UP 

Prepared for:

Marsha Silverberg

U.S. Department of Education

555 New Jersey Ave, NW

Room 502I

Washington, DC 20208-5500

Submitted by:

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Table of Contents ▌pg. i

Part A: Supporting Statement for Paperwork Reduction Act 

Submission

Table of Contents

Part A: Supporting Statement for Paperwork Reduction Act Submission......................................i

A. Justification.............................................................................................................................A-1

Introduction ..............................................................................................................................A-1

A.1 Circumstances Making the Collection of Information Necessary ..................................A-6

A.2 Purposes and Use of the Information Collection ............................................................A-7

A.2.1 Data to Be Collected.......................................................................................A-7

A.3 Use of Information Technology and Burden Reduction.................................................A-9

A.4 Efforts to Identify Duplication........................................................................................A-9

A.5 Efforts to Minimize Burden in Small Businesses...........................................................A-9

A.6 Consequences of Not Collecting the Information.........................................................A-10

A.7 Special Circumstances Justifying Inconsistencies with Guidelines in 5 CFR 1320.6 ..A-10

A.8 Consultation Outside the Agency .................................................................................A-10

A.8.1 Federal Registrar Announcement................................................................A-10

A.8.2 Consultations Outside the Agency ...............................................................A-10

A.8.3 Unresolved Issues.........................................................................................A-10

A.9 Payments or Gifts to Respondents................................................................................A-10

A.10 Assurance of Confidentiality ........................................................................................A-10

A.11 Questions of a Sensitive Nature....................................................................................A-11

A.12 Estimate of Response Burden .......................................................................................A-11

A.13 Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to 

Respondents or Record-Keepers...................................................................................A-12

A.14 Estimates of Costs to the Federal Government.............................................................A-12

A.15 Changes in Burden........................................................................................................A-12

A.16 Plans for Analysis, Publication and Schedule...............................................................A-13

A.16.1 Analysis Plans ..............................................................................................A-13

A.16.2 Timeline and Publication plans ....................................................................A-15

A.17 Approval to Not Display Expiration Date ....................................................................A-16

A.18 Exceptions to Item 19 of OMB Form 83-1...................................................................A-16

References..............................................................................................................................A-17
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-1

A. Justification

Introduction

The U.S. Department of Education (ED) will conduct a demonstration to test the effectiveness of a 

promising strategy to improve college-related outcomes in the federal college access program 

Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP). The 

demonstration will use a randomized controlled trial (RCT) design to test the effectiveness of sending 

customized messaging to students, first during the summer after high school graduation, and then in

the fall and spring of their expected first year of college. 

This study is being sponsored by ED’s Institute of Education Sciences (IES) , in collaboration with 

the Office of Postsecondary Education, and implemented by Abt Associates Inc. and its partners, 

Survey Research Management (SRM), Program and Policy Insight (PPI), Digital Design Group 

(DDG), and consultant Ben Castleman (together, the “study team”).

Overview of the GEAR UP Program 

Section 404H of the Higher Education Act of 1965, amended by the 2008 Higher Education 

Opportunities Act (HEOA) (20 USC 1070A-27) authorizes the GEAR UP program. The program 

provides funding – over $300 million in FY 2014 – to state agencies and local partnerships to support 

disadvantaged students in obtaining a high school diploma and to prepare them for success in 

postsecondary education. Grantees are expected to provide a comprehensive set of intervention 

services such as academic support, counseling, mentoring, and outreach to disadvantaged middle and 

high school students, as well as information about college financial aid options and the benefits of 

postsecondary education to students and their families. State grantees may, and partnership grantees 

must, focus on a cohort of students (typically all those in 7th or 8th grade) and provide services to them 

through high school, though funds can be used to serve other students as well. State GEAR UP 

grantees are required to, and local partnership can, also provide scholarships to help participating 

students pay for college attendance. 

Importantly, starting with the 2008 Higher Education Opportunity Act, GEAR UP grantees may now 

apply for a seventh year of funding to support students into their first year of college in order to 

improve college enrollment and persistence. The first group of grantees with this funding, those 

awarded grants in FY 2011, will begin providing extended services to 2016-2017 seniors transitioning 

to postsecondary education the following year.

Overview of the Student Messaging in GEAR UP Demonstration

This demonstration is designed to address a new need among GEAR UP grantees for strategies to 

serve students during the 7th year. According to recent focus groups conducted by Abt with GEAR 

UP grantees, grantees find it difficult to reach every GEAR UP student on a one-on-one basis while in 

high school, or to track and support students as they leave school and transition to college, 

particularly since students may be scattered across a variety of postsecondary institutions both in and 

out of state. Nationally, as many as 40 percent of first-generation, low-income high school seniors 

intending to go to college fail to enroll the following fall, with information gaps, logistical 

complexities, and concerns about in fitting identified as primary explanations of this “summer melt” 

(Castleman and Paige, 2014). Of those who matriculate, many students drop out during their first 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-2

year or fail to re-enroll the second year (NSC, 2014b), and persistence is lower among students from 

low-income and high minority high schools (NSC, 2014a).

The demonstration will investigate a particular promising approach emerging from recent rigorous 

research of approaches to improve college matriculation (Castleman & Page 2013) and college 

persistence (Castleman & Page, 2014) through low-cost student messaging, which may enhance the 

existing GEAR UP services. Customized messages will be sent to students identified as seniors, who 

intend to go to college, in high schools that are part of the GEAR UP program. The messages, 

delivered via a digital format (phone texting), will remind students about tasks related to enrolling in 

and persisting in college. All students will continue to be eligible for any regular GEAR UP services

provided. Outcomes for this study will include college enrollment, FAFSA completion, and 

persistence, which will be drawn from administrative data.

Student messaging will proceed in two phases and comprise two independent treatments. Phase 1 

messages (Treatment 1) will be sent during the summer that follows students’ senior year in high 

school. Phase 2 messages (Treatment 2) will be sent during what would be students’ first year of 

college if they enroll in college directly after high school. 

Students randomly assigned to Treatment 1 will receive a minimum of 10 messages designed to relay 

information that students may not be aware of or may not have easy access to. The messages will 

inform students about key enrollment-related tasks they need to complete such as submitting a 

deposit, filling out the housing form, submitting their proof of health insurance form, registering for 

classes, etc. Students randomly assigned to Treatment 2 will receive a minimum of 8 messages 

designed to support students to complete their first semester and persist into the second semester. 

These messages will be focused on key tasks students must complete such as paying tuition and 

registering for the next semester as well as common challenges faced in the first semester such as 

academic unpreparedness and lack of knowledge about campus supports.

All messages will be grounded in both social psychological and behavioral economic research 

suggesting that the messages be aligned to key time points such as housing deadlines, registration 

deadlines, tuition payment due dates, the add/drop period, and mid-term exams and that the messages 

be action oriented such as submit your housing form and go to this webpage to register for classes. 

To the extent possible, messages will be customized for each student based on the information they 

provide on the baseline survey about college intentions. For example, deadlines for submitting 

particular forms and links to find more information will be specific to the college a student indicated 

as the planned college of enrollment on the baseline survey. 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-3

Exhibit A-1 contains a preliminary outline of key prompts that are timed to key milestones or 

pressure points in enrolling and persisting in college. 

Exhibit A-1. Draft Outline of Student Messages

Topic Personalization

Enrollment Messages (Treatment 1)

Log into college's web portal to access important

paperwork and deadlines

Link to specific website

Register for orientation and placement tests Institution-specific registration deadlines

Complete housing forms Link to specific forms, include deadlines

Sign up or waive health insurance (if relevant) Link to specific forms, include deadlines

Submit deposit for fall enrollment Institution-specific deposit deadlines

Register for fall classes Institution-specific registration deadlines

Reminder of orientation and/or welcome 

activities dates

Institution-specific dates/activities

Persistence Messages (Treatment 2)

Provide links to campus resources for student 

support

Links to specific resources

Reminder about upcoming midterms Link to campus specific academic support center

Mindset booster on overcoming challenges in 

college

Link to campus counseling center

Check financial aid eligibility for second year Link to requirements

Register for spring classes Institution-specific registration deadlines

Overview of the Evaluation

The study will use a rigorous random assignment design to test the effectiveness of sending 

customized messages to GEAR UP students. GEAR UP projects that were awarded awards in Federal 

Fiscal Year 2011 will be invited to volunteer for the demonstration (see Appendix A for invitation), 

because they are the first to have funds to support a 7th year of services. To test the effectiveness of 

the customized messaging, the study will assign students within participating high schools selected by 

volunteer GEAR UP projects to either a control condition (eligible for all GEAR UP services), or a 

treatment condition (eligible for all GEAR UP services, plus receiving the customized messages). 

Students will be randomly assigned to conditions at two points in time (1) in the spring of their senior 

year of high school and (2) the end of summer after senior of high school. This two-phase design 

maximizes our ability to investigate both the effects of messaging during the summer before college 

and the effects of messaging during freshman year of college.

 At the first phase of random assignment (end of 2016-2017 school year), students in high 

schools served by GEAR UP projects who declare their intention to enroll in college will be 

randomly assigned to receive messages in the summer after high school (Treatment 1) or not 

(Control 1). The comparison of Treatment 1 to Control 1 will test the impacts of messaging 

on college enrollment – i.e., to see if it reduces summer melt among GEAR UP students. 

 At the second phase of random assignment (end of summer 2017), students will be randomly 

assigned to receive messages (Treatment 2) or not (Control 2) in what would be their first 

year of college if they go to college right after high school. Random assignment in this 

second phase will occur within both the Treatment 1 and Control 1 conditions, so that within 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-4

the Treatment 1 and Control 1 groups, half of the students will be assigned to Treatment 2 

and half assigned to Control 2. Exhibit A-2 displays the phases of random assignment and 

their relationship to each other. Comparison of Treatment 2 to Control 2 will test the impacts

of different messaging on college persistence. 

Exhibit A-2: Two-Phase Randomization Process and Treatment Received

Phase 1

Timing of 

Randomization

College Intending Seniors

(summer 

messaging)

End of 2016-17 

school year

T 1 C1

(summer messaging) (no summer messaging)

Phase 2

End of summer 

2017

 T2 C2 T2 C2

(frosh 

messaging)

(summer and 

frosh 

messaging)

(only summer 

messaging)

(only frosh 

messaging)

(no 

messaging)

Baseline data will be collected from all students via a survey, and college-related outcome data will 

be extracted from national datasets (National Student Clearinghouse Data (NSC) and the Federal 

Student Aid (FSA) database). 

The study will address the three research questions:

1. To what extent do the messages—above and beyond the services GEAR UP may already 

provide—improve student college-related outcomes? 

2. What types of college advising are typically received by GEAR UP students? 

3. Is there variation in the impact of the messages, and to what extent is the variation associated 

with student characteristics or features of the GEAR UP project or school? 

To answer these questions we will conduct both impact and descriptive analyses. The first report, 

which will address each research question with a focus on college enrollment-related outcomes, will 

be published in 2018. The final report, which will also include college persistence, will be available in 

2020.

Exhibit A-3 presents the research questions along with the data sources for each question, the analytic 

approach and outcomes of interest. 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-5

Exhibit A-3. Evaluation Questions, Data Sources, Analytic Approach, and Outcomes of Interest

Research Question Data Sources Analytic Approach

Outcomes of 

Interest

1) To what extent do the 

messages—above and 

beyond the services 

GEAR UP may already 

provide—improve 

student college-related 

outcomes?

 Student rosters

 Student baseline 

survey data

 National Student 

Clearinghouse Data

 Federal Student Aid 

(FSA) database

 Impact analysis of RCT 

(HLM)

 Baseline survey sample: 

16,000 students in 80 

schools.

 Analytic sample size: 

approximately 5,600 

students in 80 high 

schools that intend to 

enroll in college in 

Phase 1; approximately 

4,000 students across a 

large number of colleges 

in Phase 2, equally 

divided between 

treatment and control 

groups. 

 College 

enrollment 

(Phase I)

 College 

persistence 

into sophomore 

year (Phase II)

 FAFSA 

completion

2) What types of college 

advising are typically 

received by GEAR UP 

students?

 Student baseline 

survey data

 Descriptive analysis

 Survey sample: 16,000 

students in 80 schools.

 Description of 

GEAR UP 

program 

components 

provided and 

received 

3) Is there variation in the 

impact of the 

messages, and to what 

extent is the variation 

associated with student 

characteristics or 

features of the GEAR 

UP project or school?

 Student baseline 

survey data

 National Student 

Clearinghouse Data

 Federal Student Aid 

(FSA) database

 Descriptive (subgroup) 

analysis

 Sample size: 5,600 

students in 80 high 

schools that intend to 

enroll in college in 

Phase 1; approximately 

4,000 students across a 

large number of colleges 

in Phase 2.

 College 

enrollment 

(Phase I)

 College 

persistence 

into sophomore 

year (Phase II)

 FAFSA 

completion

Altogether, we expect that 80 GEAR UP high schools will participate in the first phase of the research 

project, with an estimated total of 16,000 seniors.

1 All seniors will be surveyed, with the consent 

process giving them (or the parents) an option to opt of completing the baseline survey or other data 

collection activities. The survey will gather information from students for participation in the student 

messaging intervention (see Appendix C for the consent letter). 

 

1 An estimate of the number of seniors in each school is 200, given that the average secondary school 

enrollment in 2011-2012 was 788 students (NCES, 2013).
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-6

It is expected that 70 percent of seniors will express intent to attend college2 and 50 percent of these

seniors would provide assent and information for texting, resulting in 2,800 students in the Treatment 

1 group, and 2,800 students in the Control 1 group. Assuming a matriculation rate for seniors of 50 

percent3the phase 2 sample will include 4,000 students entering college, 2,000 students in Treatment 

2 group and 2,000 students in the Control 2 group.

A.1 Circumstances Making the Collection of Information Necessary

The study’s data collection and design will allow for a rigorous assessment of a promising practice 

within GEAR UP that focuses on program improvement targeting college-related outcomes for 

participating disadvantaged youth. Nationally, children from disadvantaged families hold lower 

educational expectations (Goyette, 2008; Kao & Tienda, 1998) and are less likely to attend, persist, 

and complete college than their more economically advantaged peers (e.g. Haskins, 2008; Bailey & 

Dynarski, 2011). For those who do enroll, the transition to and first year in college can bring 

particular challenges (Liu, Sharkness, & Pryor, 2008; Darling & Smith (2008)). While factors such as 

students’ academic preparation and financial circumstances are cause for concern, a substantial 

number of low-income students fall off track in the complex processes of obtaining financial aid, and 

applying to, enrolling in and staying in college (Bowen, Chingos, & McPherson, 2009; Roderick, 

Nagaoka, Coca, & Moeller, 2008). 

Experimental evidence suggests that relatively low-cost interventions may help low-income students 

remain on track. For example, an intervention that involved ten text messages with customized 

messages to college-intending high school graduates increased college enrollment among students 

with less access to college-planning supports and who were not as far along with their college 

planning at the completion of high school (Castleman & Page 2013). Also, two to three hours of 

transition counseling during the summer after high school slightly increased college enrollment 

among students in some sites and improved persistence into the sophomore year even more 

(Castleman, Page, & Schooley, 2014). Finally, messaging during students’ freshman year providing 

information about financial aid deadlines and FAFA completion improved college persistence into the 

sophomore year among community college students (Castelman & Page, 2014). 

These emerging practices can be tested in the context of GEAR UP, and fill a need to identify 

effective (or ineffective) strategies for providing 7th year program services in low-cost ways. The first 

cohort of GEAR UP students eligible for those services will be high school seniors in school year 

2016-2017. The demonstration is being introduced at a key time, before grantees have solidified their 

plans for the 7th year and when they are most open to implementing strategies that will be assessed 

using random assignment.

 

2 Among 2010 seniors, 82 percent of graduating students planned to attend a postsecondary institution and 

the percent among parents with no college education was 71 percent (Aud et al. 2012).

3 This is the immediate college enrollment rate in the first fall after high school graduation for students 

graduating from low income schools according to the national student clearinghouse research center 

(National Student Clearinghouse 2014a). 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-7

A.2 Purposes and Use of the Information Collection

This ICR requests approval for data collection that involves burden to the public, which will occur 

primarily during the first phase of the study when study participants are seniors in high school. Much 

of the data collection will draw upon administrative data, so the study’s burden on projects and 

students will mainly be limited to the student roster collection and baseline survey collection.

All information will be collected by the Abt study team. Exhibit A-4 outlines the combination of 

administrative and survey data that will be collected to examine the implementation and impacts of 

the customized messaging intervention. 

Exhibit A-4. Data Collection Plan

Schedule Data Collection Activity Respondent Mode

Phase I – College Matriculation Study

Winter 2015-2016 List of participating 

schools

GEAR UP grantees will 

identify schools that serve 

seniors and are willing to 

participate in the study

GEAR UP 

project 

directors/ site

coordinators

E-mail

Fall 2016 List of GEAR UP 

seniors

Gather student rosters of high 

school seniors with access to 

GEAR UP services from 

participating schools 

GEAR UP 

project 

directors/ site 

coordinators

Electronic

Spring 2017 Administer baseline 

survey

Administer a brief baseline 

survey to capture student 

assent, intention to enroll in 

college, and a small number 

of covariates for the analysis 

(e.g., demographics)

Students (high 

school)

Paper/Electro

nic

Fall 2017 Obtain data from 

National Student 

Clearinghouse 

(NSC) and Federal 

Student Aid Center 

(FSA)

NSC will provide matriculation 

and persistence outcome

data; FSA data will indicate 

whether student completed 

the FAFSA for the first and 

second years of college

None 

(administrative 

record)

Electronic

Phase II – College Persistence Study

Fall 2018 Obtain data from 

National Student 

Clearinghouse 

(NSC) and Federal 

Student Aid Center 

(FSA)

NSC will provide matriculation 

and persistence outcome

data; FSA data will indicate 

whether student renewed the 

FAFSA

None 

(administrative 

record)

Electronic

A.2.1 Data to Be Collected

This ICR requests clearance for all collection activities of the demonstration. 

Student Rosters. The study team will collect rosters of high school seniors who have access to 

GEAR UP services in the high schools in the demonstration. In order to obtain this information, the 

study team will request student rosters from the GEAR UP site coordinator from participating schools 

(see Appendix B for roster request). In requesting student rosters, the study team will request only 

directory information as designated under the Family Educational Rights and Privacy Act (FERPA) 

and its implementing regulations (20 U.S.C 1232g and 34 CFR Part 99) such as the student’s name,

email, date of birth, telephone number, zip code, parent contact information, and district or school 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-8

identification numbers.4 These data will help in locating students and parents for the study’s consent 

processes and baseline data collection effort.

The study team will reach out to each site coordinator via a telephone call during the middle of the 

2016-2017 academic year to request student rosters of 12th grade GEAR UP students. Site 

coordinators will be asked to submit student rosters through a password protected secure file transfer 

portal (SFTP). 

Baseline Student Survey. The baseline student survey will be administered in the spring of the 

2016/2017 school year to high school seniors who have not opted-out or whose parents have not 

opted out (see Appendix C for consent letter). The survey is voluntary and student assent will be 

collected at the beginning of the baseline student survey. The baseline survey will collect data that 

will: be used to determine eligibility for the intervention (students must be intending to enroll in 

college); be used as covariates to improve the precision of impact estimates; and allow for checks on 

the equivalence of the treatment and control group individuals that are included in the final analytic 

sample (see Appendix D for student survey). The baseline survey will gather information about 

student demographics and other characteristics that are likely correlated with college-going outcomes. 

These characteristics will include college-going expectations and plans,5such as the number of 

colleges to which a student plans to apply; the type of degree the student plans to pursue first after 

high school graduation (Associate’s or Bachelor’s); knowledge of the college application process; and 

understanding of financial aid options and college costs. As such the student survey does not contain 

any questions of a sensitive nature. In developing the survey we will consider scales with established 

validity and reliability whenever possible. 

The baseline student surveys will be administered in paper or electronic format during . To facilitate 

data collection, the site liaison will work with each GEAR UP school to: facilitate the logistics for the 

baseline student survey administration; track response rates and communicate with project 

directors/site coordinators, as needed; remind absent students about the survey (see Appendix E for 

survey reminder); and distribute individualized thank you or reminder letters to students near the end 

of the survey fielding period.

Postsecondary Enrollment and Financial Aid Data. In November 2017, postsecondary enrollment 

data will be extracted from the National Student Clearinghouse (NSC) for students who were seniors 

in 2016-17. The NSC data cover all 50 states and both public and private institutions. The NSC 

provides student-level data on college enrollment and completion for 9,800 member institutions; 

together, these institutions represent 91 percent of enrolment in higher education in the U.S. Although 

the NSC has lower coverage of two-year than four-year institutions, fewer private than public 

 

4 Only districts and schools that have directory information policies in place prior to July 1, 2016 that allow for 

student information designated as “directory information” to be shared in compliance with Federal law with 

the evaluator will be included in the participating GEAR UP high schools. In accordance with FERPA, if 

any parents or guardians of a student opt out of the disclosure of “directory information,” the school or 

district will not provide the “directory information” for that student to the evaluator unless the parent or 

guardian of that student provides consent under 34 CFR 99.30.

5

Students who indicate some intent to attend college will be eligible for the intervention and will comprise the 

pool of students who are randomly assigned to the treatment and control conditions.
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-9

institutions, and gaps in data in some states, no other national-level datasets contain individual 

student-level data on postsecondary enrollment, full-time versus part-time status and completion. 

In November 2017, data on FAFSA completion and receipt of Federal student aid (such as Pell 

grants, Federal Work-Study, etc.) will be obtained from the FSA data center. To obtain these data, we 

will prepare a data file containing identifying information for students in the study sample for 

submission to the FSA. 

NSC and FSA data will be obtained again in November 2018 to assess persistence into a second year 

of college for the treatment and control groups. 

The collection of all NSC and FSA data will be completely non-intrusive to students since their 

directory information from rosters and baseline surveys will be used for all subsequent follow-ups. 

A.3 Use of Information Technology and Burden Reduction

The data collection plan reflects sensitivity to issues of efficiency, accuracy, and respondent burden. 

The study will use a combination of mechanical and electronic technology to collect data. For each 

data collection task, the study team has selected the form of technology that enables the collection of 

valid and reliable information in an efficient way while minimizing respondent burden. 

The Abt study team will collect student rosters from site coordinators. To minimize burden, the 

evaluator’s electronic mail address and toll-free telephone number will be included on the request for 

student rosters should site coordinators have questions, and site coordinators will be provided with an 

SFTP link to upload rosters with minimal effort and time. Taken together, these procedures are all 

designed to minimize the burden on respondents.

Additionally, we will explore with individual schools whether the student baseline survey can be 

administered electronically, or with a paper-and-pencil option if computer access on a classroomlevel scale is not feasible. 

Further, the study will rely on existing sources of data for much of the data, which will add no 

respondent burden. 

A.4 Efforts to Identify Duplication

To date, there has been no rigorous study of strategies that could improve the effectiveness of

enhancements to GEAR UP related to students’ matriculation and persistence in college. Our survey 

instruments will be designed to reflect the efforts of a consortium of state GEAR UP grantees to 

develop common program service definitions. To the extent possible, the study team will use existing 

data for the study rather than duplicate data collection efforts. The study team will utilize all 

publically available data on high school characteristics and about GEAR UP programming. The 

information collected in the student rosters and in the student survey is not available elsewhere.

A.5 Efforts to Minimize Burden in Small Businesses

No small businesses will be involved as respondents. The primary small entities for this study are 

GEAR UP high schools. The study team will minimize burden by training data collection staff to 

make their contacts with GEAR UP site coordinators as straightforward and concise as possible. 

Further, the study team will administer the student survey, minimizing the burden placed on site 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-10

coordinators. All notification mailings, conversations, and presentations are designed to be clear, 

brief, and informative. 

A.6 Consequences of Not Collecting the Information

The consequences of not collecting specific data are described below:

 Without collecting the student rosters, the study team could not identify the survey sample or 

define treatment and control groups, rendering it impossible to conduct the study.

 Without collecting data through the student baseline survey, the study team would not be able 

to identify which GEAR UP students intend to matriculate in college, and, consequently, 

which students are eligible to receive the messages. In addition, not collecting the student 

baseline survey data would limit the study’s ability to make conclusions about the 

intervention impact on matriculation and persistence. 

 Without collecting the NSC and FSA data the study team could not measure the key 

outcomes and complete the impact analyses.

A.7 Special Circumstances Justifying Inconsistencies with Guidelines in 5 

CFR 1320.6

There are no special circumstances concerning the collection of information in this study.

A.8 Consultation Outside the Agency

A.8.1 Federal Registrar Announcement

A 60-day notice to solicit public comments was published in the Federal Register on DATE. XXX

comments were received.

A.8.2 Consultations Outside the Agency

Over the course of the study, the study team will assemble a Technical Working Group (in 

consultation with ED) composed of consultants with various types of expertise in the areas relevant to 

this study. The Technical Working Group will convene in the spring of 2015 to discuss the study 

design, instrumentation, and data collection plans. 

A.8.3 Unresolved Issues

There are no unresolved issues.

A.9 Payments or Gifts to Respondents

There will be no payments or gifts to respondents. 

A.10 Assurance of Confidentiality

The study team will conduct all activities in accordance with all relevant regulations and 

requirements. These include the Education Sciences Institute Reform Act of 2002, Title I, Part E, 

Section 183, that requires “[all] collection, maintenance, use, and wide dissemination of data by the 

Institute ... to conform with the requirements of section 552 of Title 5, United States Code, the 

confidentiality standards of subsections (c) of this section, and sections 444 and 445 of the General 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-11

Education Provisions Act (20 U.S.C. 1232 g, 1232h).” These citations refer to the Privacy Act, the 

Family Education Rights and Privacy Act, and the Protection of Pupil Rights Amendment.

In addition, all data collected for the study shall remain confidential in accordance with Section 552a 

of Title 5, United States Code, the confidentiality standards subsection (c) and sections 444 and 445 

of the General Educations Provision Act. Subsection (c) of Section 183, referenced above, requires 

the director of IES to “develop and enforce standards designed to protect the confidentiality of 

persons in the collection, reporting, and publication of data.” The study will also adhere to 

requirements of subsection (d) of Section 183 prohibiting disclosure of individually identifiable 

information as well as making the publishing or inappropriate communication of individually 

identifiable information by employees or staff a felony.

In addition, the following verbatim language will appear on all letters, fact sheets, and other study 

materials:

Per the policies and procedures required by the Education Sciences Reform Act of 2002, Title 

I, Part E, Section 183, responses to this data collection will be used only for statistical 

purposes. The reports prepared for this study will summarize findings across the sample and 

will not associate responses with a specific program, district, or individual. Any willful 

disclosure of such information for nonstatistical purposes, except as required by law, is a 

class E felony.

Data will be presented in aggregate statistical form only. All study staff involved in collecting, 

reviewing, or analyzing individual-level data will be knowledgeable about data security procedures 

and will sign nondisclosure agreements (see Appendix F). Respondents will be assured that all 

information identifying them or their school will be kept private to the extent allowed by law. The 

confidentiality procedures adopted for this study during all rounds of data collection, data processing, 

and analysis consist of the following:

 All paper files will be converted to an electronic format and the paper files will be shredded 

immediately after they have been converted. 

 Electronic data files with sensitive data will be removed from computers and working servers 

in a manner that ensures that the information cannot be recovered. 

 At end of contract with the ED, the Abt study team will prepare a restricted use data file of 

de-identified data that will be submitted to ED. The Abt study team will destroy all student 

identifiers but will retain de-identified student data for three years.

 All electronic copies of de-identified student data maintained by the Abt study team will be 

destroyed within three years of the final contract payment from ED, unless otherwise directed 

by ED at the request of the Abt study team or by ED’s request.

A.11 Questions of a Sensitive Nature

There are no questions of a sensitive nature included in the information requested. 

A.12 Estimate of Response Burden

The total respondent burden for the data collection effort covered by this clearance request is 4,160

hours, for a total estimated cost to respondents of $4,123.20. Exhibit A-5 presents time estimates of 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-12

respondent burden for the data collection activities requested for approval in this submission. The 

burden estimates are based on the following assumptions:

 One site coordinator is expected at each of the anticipated 80 high schools. It is assumed that 

collecting and submitting the GEAR UP student rosters in each school will take 

approximately two hours.

 The total cost to site coordinators to collect student rosters is $4,123.20. The salary estimate 

for School and Career Counselors is based on an average hourly wage of $25.77 in 2012.

6

.

 An average of 200 students in the 80 schools. It is estimated that the survey will take 15 

minutes to complete. 

Exhibit A-5. Estimate of Respondent Burden

Informant/Data Collection 

Activity #

of Resp

ondents

#

of Resp

onses

H

ours/ Resp

onse

T

otal Burden H

ours/

resp

ondent

T

otal Burden

H

ours

C

ost per Resp

ondent

(hourly

wage)

T

otal C

osts per

Resp

ondent

T

otal C

osts

GEAR UP Site Coordinators

Collect and Submit Student 

Rosters 80 1 2.0 2.0 160 $25.77 $51.54 $4,123.20

Students

Student Baseline Data Collection 16,000 1 0.25 0.25 4,000 NA NA NA

Total 4,160 $4,123.20

A.13 Estimate of Total Capital and Startup Costs/Operation and Maintenance 

Costs to Respondents or Record-Keepers

There are no annualized capital/startup or ongoing operation and maintenance costs involved in 

random assignment or the collection of student rosters.

A.14 Estimates of Costs to the Federal Government

The estimated cost to the federal government of the data collection activities for the study as 

described above is $1,769,371. The data collection activities will be carried out from fall 2015 to fall 

2018. Thus, the average annual cost to the federal government is $442,342.

A.15 Changes in Burden

This is a request for a new collection of information.

 

6 Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook, 2014-15 Edition, 

School and Career Counselors, on the Internet at http://www.bls.gov/ooh/community-and-socialservice/school-and-career-counselors.htm (January 8, 2015).
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-13

A.16 Plans for Analysis, Publication and Schedule

A.16.1 Analysis Plans

Impact and descriptive analyses will be conducted to answer the study research questions, as 

described below. Part B of this Supporting Statement provides additional methodological detail.

1. To what extent do the messages—above and beyond the services GEAR UP may already 

provide—improve student outcomes? 

Given that participating schools are volunteering to participate in the study (rather than being selected 

at random) and inferences will be focused on students in these or similar schools, the study will use a 

model with fixed treatment effects to assess the effect of messaging on student outcomes (Schochet, 

2008).

7The study team will control for student demographic characteristics, including gender, 

race/ethnicity, and first generation college student status. Reflecting the randomization of students 

within schools, the study team will include a set of dummy variables for schools and a treatment 

indicator to provide an estimate of the average impact of the messaging demonstration on students 

across all schools.

The study team will conduct a test of the null hypothesis of no treatment impact for each outcome 

variable. The study team will interpret a positive and statistically significant impact at the 5-percent 

level (using a two-tailed test) as compelling scientific evidence that the intervention improves the 

targeted outcome (e.g., college enrollment, college persistence, FAFSA completion). For continuous 

outcome variables, the study team will estimate multilevel linear models; for dichotomous outcomes 

(e.g., college enrollment, FAFSA completion), the study team will estimate multilevel logistic 

regression models.

Multilevel Logistic Regression

For dichotomous outcomes (e.g., enrollment, persistence, and FAFSA completion), a logistic 

regression will be used. This model is:

(1) [ = 1] =





(  0 +  1 (Tij) +  2 (Y∗ij) + 





1

1

J

j

 2+j Schj + 

M

m 1

 2+(J−1)+m X mij )

Where  is the covariate-adjusted log-odds of the outcome occurring (e.g., enrolled in college, 

completed FAFSA) versus not occurring (e.g., did not enroll in college, did not complete FAFSA) for 

students in school j; is a regression coefficient indicating the difference in log-odds of the 

outcome between the treatment and control group, controlling for all other variables in the model; and 

through  parameter estimate for the average effect (in log-odds) of each student-level 

covariate on the outcome , controlling for all other variables in the model.

To convert estimates produced by logistic regression to an impact measured in percentage point 

increases in probability of the binary outcome, we calculate the individual level probability of the 

outcome given treatment implied by individual level covariates and model estimates, we repeat this 

calculation if the individual were assigned to control, and find the mean difference between these two 

probabilities.

 

7

If we suspect that treatment effects differ by school, we will interact schools with the treatment indicator. 
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-14

Hierarchical Linear Model

The following model will be used to estimate program impacts on continuous outcome variables. 

Given that our core outcome measures (i.e., confirmatory contrasts) are all dichotomous (i.e., college 

enrollment, FAFSA completion, and persistence into the second year of college), this HLM describes 

our methodology for assessing exploratory contrasts.

Yij =  0 +  1 (Tij) +  2 (Y*

ij) + 





1

1

J

j

 2+j Schj

+ 

M

m 1

 2+(J-1)+m X mij +

ij 

where Yij is the value of the outcome (e.g., amount of Federal financial aid received) for the i

th student 

in the j

th GEAR UP school; Tij is 1 if student i is randomized to the treatment condition in schools j

and 0 if assigned to the control condition in school j; Y*

ij is the baseline score for the outcome of 

interest for student i in school j; Schj = 1 if student is in school j, =0 else; X mij are additional 

covariates representing demographic characteristics of student i at school j (e.g., race/ethnicity, 

English language learner status);  0 is the conditional mean follow-up score for the outcome of 

interest for control students; 1is the treatment effect i.e. the mean difference of the outcome 

between intervention and control students. The overall treatment estimate (  1) and will be a 

precision-weighted estimate, where the weights are inversely proportional to the variance of the 

treatment effect in each school. ij  is the residual error term assumed to be normally distributed with 

mean 0 and variance 

2

.

2. What types of college advising are typically received by GEAR UP students? 

Descriptive analyses will be used to provide information on the types of college advising GEAR UP 

students typically receive. It is important to understand the existing college advising to which the 

messaging will be added. Further, this collection and analysis has the added benefit of filling a gap in 

program information about the intensity of and emphasis on college advising. The study team will 

describe the types and intensity of college advising regularly received by GEAR UP students at the 

high school level as reported on the student survey. One example of the type of college advising that 

might be received by students is one-on-one counseling or advising sessions about applying to

college; the study team would describe the percent of students that attending such sessions. These 

results will provide another important source of contextual information for interpreting impact 

findings. 

3. Is there variation in the impact of the messages, and to what extent is the variation 

associated with student characteristics or features of the GEAR UP project or school? 

To examine variation in program impacts by student characteristics or features of the GEAR UP 

project or school, the study team will include an interaction of the treatment indicator with the

subgroup variable of interest in the impact models described above.8 The study team will include 

interaction terms in the model for our core dichotomous outcomes of college enrollment, FAFSA 

completion, and college persistence (i.e., equation 1 above), allowing the treatment impact to vary by 

student characteristics, such as knowledge of the college application process, understanding of 

financial aid options and college costs, or amount of college advising received (e.g.one-on-one 

 

8 This approach requires the assumption that gender is not correlated with the marginal effect of the other 

model covariates. We will evaluate whether this assumption is tenable during preliminary data exploration.
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-15

counseling or advising sessions about applying to college ) – or by program characteristics such as 

whether the GEAR UP program was funded under a state grant or a partnerships grant. If significant 

interaction effects are found, the sample will be broken into subgroups, the treatment effect will be 

estimated for each subgroup, and the means and variances of the subgroup will be compared

A.16.2 Timeline and Publication plans

Timeline 

The study is expected to be conducted across five years. Exhibit A-6 (below) displays the full study 

timeline. The student baseline survey will be administered in the spring of 2017, the random 

assignment for Phase I will take place in the late spring of 2017, and the randomization for Phase II 

will take place at the end of the summer of 2017. NSC and FSA data will be collected in the fall of 

2017 and 2018. 

Exhibit A-6 Study Timeline

Timeframe Activity

Planning

Fall 2014 Refine the study design and develop data collection forms

Winter 2015 Revise data collection forms; obtain approval from Abt’s 

Institutional Review Board (IRB); intervention design

Spring 2015 Submit Office of Management and Budget (OMB) package (for 

approval by October); refine intervention design

Summer 2015 Refine intervention design; revise OMB package as needed

Recruitment

Recruitment

Fall 2015 Develop recruitment materials; Finalize intervention design

Winter 2016 Recruit GEAR UP grantees and schools; Setup intervention 

message delivery system.

Spring 2016 Ensure schools and districts have necessary approval from their 

review boards

Summer 2016 Finalize commitments from grantees and schools

Roster and 

Baseline Data 

Collection

Fall 2016 Collect 12th grade rosters from participating schools

Winter 2016 Obtain student and parent consent

Spring 2017 Administer baseline surveys to all seniors with consent.

Conduct first random assignment of students intending to enroll 

in college.

Intervention 

Implementation

Summer 2017 Deliver college enrollment reminders

Conduct second random assignment.

Fall & Spring 2017 Send college persistence messages through students’ first year

of college

Outcomes Data 

Collection

Fall 2017 Obtain extant data from National Student Clearinghouse (NSC) 

and Federal Student Aid data center (FSA)

Fall 2018 Obtain extant NSC and FSA data

Reporting

Spring 2018 Draft Report #1

Summer 2018 Final Report #1

Summer 2019 Draft Persistence Report

Winter 2020 Final Persistence Report

Publication plans

The evaluation plans call for two reports. The first, published in summer 2018, will be based on data 

collected through 2017that addresses Phase I of the study (i.e., college enrollment and FAFSA

completion). The second report will be available in early 2020, and will cover the Phase II of the 

study (i.e., college persistence).
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-16

A.17 Approval to Not Display Expiration Date

No exemption is requested. The data collection instruments will display the expiration date.

A.18 Exceptions to Item 19 of OMB Form 83-1

The submission describing data collection requires no exemptions to the Certificate for Paperwork 

Reduction Act (5 CFR 1320.9).
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-17

References

Aud, S., Hussar, W., Johnson, F., Kena, G., Roth, E., Manning, E., Wang, X., & Zhang, J. (2012). 

The condition of education 2012 (NCES 2012-045). U.S. Department of Education, National 

Center for Education Statistics. Washington, DC. Retrieved June 27, 2012 from 

http://nces.ed.gov/pubsearch.

Bailey, M. J., & Dynarski, S. M. (2011). Inequality in postsecondary education. In G. J. Duncan and 

R. J. Murnane (Eds.). Whither opportunity? Rising inequality and the uncertain life chances of 

disadvantaged children. New York, NY: Russell Sage Foundation.

Bowen, W. G., Chingos, M.M., & McPherson, M.S. 2009. Crossing the Finish Line: Completing

College at America’s Public Universities. Princeton, NJ: Princeton University Press.

Castleman, B. L. & Page, L. C. (2013). The not-so-lazy days of summer: Experimental interventions 

to increase college entry among low-income high school students. New Directions for Youth 

Development, 140, 77-97.

Castleman, B.L., & Page, L.C. (2014). Freshman year financial aid nudges: An experiment to 

increase FAFSA renewal and college persistence. EdPolicy Works Working Paper. Available 

http://curry.virginia.edu/uploads/resourceLibrary/29_Freshman_Year_Financial_Aid_Nudges

.pdf

Castleman, B.L., Page, L.C., & Schooley, K. (2014). The forgotten summer: Mitigating summer 

attrition among college-intending low-income high school graduates. The Journal of Policy

Analysis and Management. DOI: 10.1002/pam.21743

Darling, R. & Smith, M. (2008). First-generation college students: First-year challenges. In M. S. 

Hunter, B McCalla-Wriggins, & E. White (Eds) Academic Advising: New Insights for Teaching 

and Learning in the First Year. National Resource Center for the First-Year Experience and 

Students in Transition, Columbia SC.

Goyette, K. A. (2008). College for some to college for all: Social background, occupational 

expectations, and educational expectations over time. Social Science Research, 37(2), 461-484.

Haskins, R. (2008). Education and economic mobility. In J. B. Isaacs, I. V. Sawhill, and R. Haskins 

(Eds.). Getting ahead or losing ground: Economic mobility in America. Washington, DC: The 

Brookings Institution. Retrieved June 27, 2012 from 

http://www.brookings.edu/research/reports/2008/02/economic-mobility-sawhill.

Kao, G., & Tienda, M. (1998). Educational aspirations of minority youth. American Journal of 

Education, 106(3), 349-384.

Liu, A., Sharkness, J., & Pryor, J. (2008). Findings from the 2007 Administration of Your First 

College Year (YFCY): National Aggregates. Higher Education Research Institute, University of 

California, Los Angeles. Available http://www.heri.ucla.edu/PDFS/YFCY_2007_Report05-07-

08.pdf

National Center for Education Statistics (2012). Digest of Education Statistics, 2011. Washington, 

DC: US Department of Education.
Revised OMB Supporting Statement Promising Strategies in Federal College Access Programs

Abt Associates Inc. Part A: Justification ▌pg. A-18

National Center for Education Statistics (2013). Common Core of Data (CCD), "Public 

Elementary/Secondary School Universe Survey," 2011-12. 

http://nces.ed.gov/programs/digest/d13/tables/dt13_216.80.asp

National Student Clearinghouse (2014a). Report: High School Benchmarks 2014, National College 

Progression Rates for high schools participating in the National Student Clearinghouse Student

Tracker Service. Available http://nscresearchcenter.org/hsbenchmarks2014

National Student Clearinghouse (2014b). Snapshot Report: First-Year Persistence and Retention 

Rates by Starting Enrollment Intensity: 2009-2012. Available: 

http://nscresearchcenter.org/snapshotreport-persistenceretention14/

Roderick, M., Nagaoka, J., Coca, V., & Moeller, E. (2009). From High School to the Future: Making 

Hard Work Pay Off. Chicago: Consortium on Chicago School Research.

Schochet, P. (2008). Statistical Power for Random Assignment Evaluations of Education Programs. 

Journal of Educational and Behavioral Statistics, 33(1), 62-87.

U.S. Department of Education. (2014). Gaining Early Awareness and Readiness for Undergraduate 

Programs (GEAR UP) Funding Status. Retrieved on January 13, 2015 from: 

http://www2.ed.gov/programs/gearup/funding.html