Study of Pell Grant Recipients Who Transfer Among Eligible Institutions

Request for OMB Clearance

Supporting Statement B

**August 19, 2008**

Contents

Section B. Methodology. 1

B1. Sample Design and Estimation. 1

B2. Survey Procedures. 2

B3. Methods to Maximize Response Rates. 5

B4. Tests of Procedures. 6

B5. Individuals Responsible for Study Design and Performance 6

Appendix A Description of Survey Items 7

Appendix B Letter to Pell Grant Recipient 11

Appendix C Westat Pledge of Confidentiality 12

**List of tables**

**Table Page**

Table 1 Standard errors for percentage of students who could not
transfer all academic courses 4

Table 2 Standard errors for percentage of students with costs over some
threshold 4

Table 3 Power for comparisons between small and large institutions 4

Table 4 Power for comparisons between racial/ethnic groups 5

## Section B. Methodology. 

### B1. Sample Design and Estimation.

Westat will select a sample of 668 students from the Pell Grant
recipient file, provided by ED, which contains all Pell grantee
recipients who have transferred from 2-year institutions to another
institution in the 2005-06 school year. We will select 167 students each
from the following categories of receiving institutions: 2-year public,
2-year private, 4-year public, and 4-year private.

Some variables will be used to form student strata, while other
variables will be used to sort students within each stratum. A
systematic sample will be selected within each stratum from the sorted
listing of students. The allocation of sample across strata will be
proportional to the number of students in each stratum. Stratification
ensures that the proportion of the sample in categories of variables
that are used to form the strata will be close to the proportion of all
students in the categories. To illustrate the methodology for sampling
with strata, suppose a stratum consists of 50 students and we wish to
sample at a rate of 1 in 10. A random number between 1 and 10 is first
selected. Suppose the random number is 6. The sample would then consist
of the 6^th^, 16^th^, 26^th^, 36^th^, and 46^th^ students on the sorted
list. If the primary sort variable within the stratum was the geographic
region, then this methodology ensures that the proportion of the sample
from each region in the stratum is approximately equal to the proportion
of students in the stratum that is from the region. Both the formation
of strata and systematic sampling from sorted students within strata
reduces sampling error and ensures representativeness for the selected
variables. The Westat proprietary sampling software, WESSAMP, will be
used to select the sample.

We wish to obtain an interviewed sample size of 200 students. To account
for an expected 70 percent nonresponse rate, we plan to select 668
students. If the field period were longer, it would be possible to
select substitutes for each sampled student who would only be
interviewed when an initially sampled student was determined to be a
nonrespondent.

### B2. Survey Procedures.

#### Overview 

The survey will be conducted primarily by telephone, with initial
notification of the survey by mail and with the option of instead
responding through the web. Westat's strategy is to call respondents by
phone within 1 week of the expected arrival of the introductory letter.
We will then attempt to collect the responses by telephone, using CATI.
The cover letter also will offer the option of responding through the
web.

The transcript component of the study will be conducted by collecting
transcripts either from students (who download and email the
transcripts, or who fax printed copies of the transcripts) or from the
sending and receiving institutions (for those students who give
permission to collect transcripts but do not directly send the
transcripts to Westat). Westat will reimburse institutions at their
standard rate for sending the transcripts, and will reimburse students
\$10 for sending the transcripts directly. College catalogs will be
obtained online to facilitate coding of the transcripts.

#### 1. Weighting and estimation

This study is intended to be a preliminary study, and will not produce
national estimates. Therefore the data will not be weighted.

#### 2. Oversampling of Small Institutions

The Pell Grant recipients are unequally distributed, with many of the
recipients located at a relatively small number of very large
institutions in 2005-06. If all students are given the same probability
of selection, then we will sample multiple students per institution from
the larger institutions and relatively few from the smaller
institutions. In some ways this is beneficial. For general statistics,
such as the general types of courses that were accepted for credit, the
sampling error will be minimized if students are sampled with equal
probability. On the other hand, if one is interested in looking at
particular classes of institutions, such as those institutions that have
fewer than 1,000 students, then the sample size will be very small for
such a class. Thus, if this is important, it may be desirable to sample
students transferring to small institutions at a higher rate. For
example, it could be that it is the smaller colleges that are more
restrictive in accepting transfer credits. If we are looking for
students who had problems (in order to understand how/why they had
problems), then oversampling the small institutions would be useful.

We thus face two competing priorities. For some types of analysis,
sampling error will be lower if small institutions are oversampled, but
for other types of analysis such oversampling will increase sampling
error. For ED, the highest priority is to have accurate overall
estimates, and developing separate estimates for small institutions is
not a key priority. Therefore, we will not oversample small
institutions.

#### 3. Stratification and Sort Variables

There are several variables for use in stratifying and sorting students
prior to sample selection. The three variables that we consider to be
most important will be used to form strata. These variables are:

Institution control (public, private non-profit, private profit-making)

Institution type

Enrollment size in the institution being transferred to

We expect that institutional control will be collapsed to two categories
(public/private) for at least some strata, and that institution type
will also be collapsed to fewer categories for at least some strata. For
the enrollment size in the institution, there will be six categories
with about equal numbers of cases in the central four categories, while
the tail categories would likely have fewer than the others. This will
depend on the distribution of this variable.

A number of strata will be determined, each of which is homogeneous with
respect to these three variables. There will be no more than 72 strata.
Some of these might be small and these would then be collapsed. For
example, the 2-year transfer institution type might not be further
stratified by the enrollment categories. As another example, the two
private categories could be combined into one private category.

Within each stratum students will be sorted by the following variables,
listed in priority order:

National Assessment of Educational Progress (NAEP) region

Student year

Institution

ID number

### B3. Methods to Maximize Response Rates. 

To help maximize the response rate, even given a short data collection
time period, we will perform the following. (1) All sampled Pell Grant
recipients will be sent a cover letter using ED stationery that
describes the purpose of the survey. (2) The address and telephone
number will be taken from the most recently available Pell Grant files,
in order to ensure accuracy, and updated through a national change of
address service. (3) The sample members will be contacted by telephone,
helping to quickly confirm that the locating information is correct and
allow tracing to begin if it is not. (4) Survey responses will be
collected over the telephone, making maximum use of the telephone
contact and avoiding the possibility of delay or of the subject
forgetting to follow through. (5) Respondents will be given the option
of responding through the Web or of calling a toll-free number if they
wish to respond immediately. (6) The response status of each sampled
Pell Grant will be monitored through an automated receipt control
system, so that failures to reach a subject or to get a response can be
followed up quickly. (7) The study members will be offered a financial
incentive of \$10 that will be included with the original letter
notifying the student of the survey. (8) To facilitate collection of
transcripts, respondents will be asked to sign and mail permission forms
to Westat; however, we will also collect data on the telephone and web
surveys on whether students gave permission to collect their transcript
so that we can tell institutions that we have permission even if we do
not received the signed permission forms. We will pay students \$10 to
send both transcripts directly to Westat in lieu of the reduced data
collection costs for Westat.

All questionnaires that are received will be reviewed for consistency
and completeness; if a questionnaire has too few items completed to be
counted as a response (or if it has missing or conflicting data on key
items), telephone interviewers will seek to obtain more complete
responses. Interviewers who have received training in telephone
interview techniques and specific training on the survey will conduct
all of the telephone interviews.

### B4. Tests of Procedures.

The questionnaire has been reviewed extensively by both Westat and ED.
Westat will perform a test of the computer software for both the CATI
and the web version of the questionnaire in order to verify that
respondents progress appropriately through the questionnaire, and that
data are accurately recorded.

**This study itself is a test of the procedures needed to accurately
measure students' experiences when transferring from one institution to
another: e.g., whether students are able to report on their experiences
accurately, or whether transcripts are needed in order to have accurate
information.**

### B5. Individuals Responsible for Study Design and Performance.

Several key staff are responsible for the study design and performance.
These individuals are as follows:

> Archie Cubarrubia, Project Officer, Office of the Under Secretary,
> U.S. Department of Education (202-260-5548)
>
> Vickie Schray, Office of the Under Secretary, U.S. Department of
> Education
>
> (202-401-1021)
>
> David Bergeron, Office of Postsecondary Education (202-502-7815)
>
> Dennis Berry, Office of Vocational and Adult Education (202-245-7814)
>
> Greg Henschel, Office of Vocational and Adult Education (202-245-7661)
>
> Tom Weko, National Center for Education Statistics (202-502-7643)
>
> Michelle Coon, National Center for Education Statistics (202-502-7357)
>
> Brad Chaney, Task Leader, Westat (301-294-3946)
>
> Denise Glover, Project Director, Westat (301-251-2269)
>
> Diane Ward, Operations Manager, Westat (301-251-4309)
>
> Eric Jodts, Telephone Center Operations Manager, Westat (301-610-8844)
>
> David Morganstein, Statistician, Westat (301-251-8215)
>
> Gary Shapiro, Statistician, Westat (301-517-8028)

#  {#section .TT-Table-Title}

Appendix A -- Description and Justification of Survey Items

### Research Question 1. Types of Courses that were Accepted and Not Accepted (questionnaire items 1-6)

It is important to collect information on all attempts to transfer
courses (whether successful or unsuccessful) in order to develop a
complete picture of the ease or difficulty of transferring courses.
However, there may be some students who decide themselves to opt out of
transferring credits (e.g., because of a switch to an entirely new area,
so that the student's past studies do not seem relevant); in that case,
their new institutions never face the choice of accepting or rejecting
transfer credits. Data on such opting out (and the reasons for it) are
important because opting out also results in increased costs and delayed
entry into the workforce (see questionnaire item 1).

Past research indicates people are much more accurate when providing
general descriptions of their educational experience than when providing
specific information such as the number of courses in a particular area.
For this reason, the questionnaire asks for the number of courses using
a few broad categories rather than asking for specific numbers. It asks
students to respond in terms of either the number of courses rather than
the number of credits because we expect the number of courses to be
easier to remember. Focusing on the number of courses also avoids the
complication of whether the number of credits may have changed simply
through the conversion of quarters to semesters, rather than through a
decision to deny credit (see questionnaire item 2). In case some
respondents can remember only the number of credits, the questionnaire
includes instructions to help respondents to convert that number into
the number of courses.

To measure students' success in transferring courses, the questionnaire
asks for two kinds of responses: a general response that does not depend
on course counts (i.e., all of the courses were accepted, some were
accepted, or none were accepted), along with counts using broad
categories. The first question helps to introduce the topic to
respondents, simplifies the questionnaire for those students whose
courses were all treated in the same way, and is less subject to memory
errors. The use of broad categories of counts allows for the fact that
students may have difficulty giving precise answers (see questionnaire
items 3 and 4).

ED has a particular interest in knowing whether CTE courses are treated
differently than other courses. The questionnaire therefore examines
this topic by asking for students' experience in transferring both CTE
and general academic courses (see questionnaire item 5). It is possible
that institutions may not accept certain introductory courses for
transfer because they are not considered to be at the college level.
This category may be worthy of special attention, though it also may be
difficult to define. Instead, we chose to address this issue when asking
for reasons the transfer courses were not accepted (see questionnaire
item 6f).

The questionnaire asks students for the reasons courses were not
accepted for transfer, using a series of yes/no items. Potential reasons
include a limit on the total number of courses/credits that could be
transferred, low course grades, general problems relating to the
previous institution, problems relating to the course subject area, and
courses that are designed to reflect institutional emphases and that
therefore must be taken at the same institution (see questionnaire item
6).

### Research Question 2. Costs Associated With Transferring Credits (questionnaire items 7--10)

Institutions may charge some type of fee for processing/accepting
transfer courses (see questionnaire item 7). Students also face
additional costs due to the need to repeat courses, take extra courses,
and possibly delay graduation (see questionnaire items 12 to 14.) The
questionnaire asks both about the frequency of such costs and the level
of the costs. Question 8 asks about the nature of any repeated or extra
courses (8a), how much the student learned form repeating any courses
(8b), and how the student's grades compared when courses were repeated
(8c). Question 9 asks about extra expenses, and question 10 asks about
delays to the student's graduation.

### Personal Information and Willingness to Participate in the Transcript Study (questionnaire items 11-14)

Personal information is typically collected last, so that respondents
will already feel involved in the questionnaire. Many types of personal
information will presumably be available through the FAFSA, so there is
no need to collect the same information again on this questionnaire
(especially since the questionnaire will be limited in length). For
example, information on gender and age will be available. However, the
FAFSA does not include race/ethnicity, and there may be important
differences in course-taking patterns or choices of institutions that
lead to differences in transferring credits by race/ethnicity (see
questionnaire item 12). FAFSA also does not include information on
students' course of study. Often students at 2-year institutions have
not chosen a course of study, and the questionnaire is already designed
to collect information on whether the students were successful in
transferring CTE and general academic. What may be important, however,
is whether students' transfer reflects a change in their course of
study. That is, the greater the change that students made in their
education/career plans, the less applicable their earlier preparation
might be. Students who make the greatest changes may be the least likely
to have their transfer credits accepted (see questionnaire item 11).

Finally, question 13 is designed to support the transcript data
collection: by directly obtaining the data from the student, or by
obtaining a signed permission form (or at least the student's verbal
assent) to collect the data.

Appendix B -- Letter to Pell Grant Recipient

Dear Pell Grant recipient:

The U.S. Department of Education wishes to know more about the
experiences of students who transfer from one postsecondary institution
to another. We want to know if they have trouble in transferring credits
they have earned to their new institutions, and if so, what kind of
problems they have encountered. Additionally, we would like to review
transcripts from the sending and receiving institutions attended by
transfer students to provide additional details about the courses and
credits requested for transfer.

As a past Pell Grant recipient who transferred, you have been randomly
selected for a short survey on this topic. The survey will take
approximately 15 minutes to complete. We also are enclosing a transcript
permission form for you to sign and return to our contractor, Westat.
Westat will pay the institutions for the transcripts. You can respond to
the survey in three ways. (1) In a few days, we plan to call you to get
your answers over the telephone. Alternatively, if you wish, you can
respond immediately (2) by calling the following toll-free number
(1-888-487-0511) or (3) by going to the following web site
(PellGrantStudy.com) and using the ID shown below:

Website: PellGrantStudy.com

ID: «PIN»

The survey and transcript study are voluntary, and will have no effect
on your receipt of Pell Grants. As a way of thanking you in advance for
your time, we are enclosing a check for \$10. You may complete the
survey using any one of the three methods above. We would greatly
appreciate your response by **September 15, 2008.**

There is nothing in the survey that should present any risk to you, but
in any case please be assured that your answers will be used for
statistical purposes and will not be disclosed, or used, in identifiable
form for any other purpose except as provided by law.

Thank you for your cooperation with this survey. Your responses will
help us to better serve you and other students in the future.

Sincerely,

Archie Cubarrubia

Office of the Under Secretary

U.S. Department of Education

Appendix C

**EMPLOYEE OR CONTRACTOR\'S ASSURANCE OF CONFIDENTIALITY OF SURVEY
DATA**

**Statement of Policy**

Westat is firmly committed to the principle that the confidentiality of
individual data obtained through Westat surveys must be protected. This
principle holds whether or not any specific guarantee of confidentiality
was given at time of interview (or self-response), or whether or not
there are specific contractual obligations to the client. When
guarantees have been given or contractual obligations regarding
confidentiality have been entered into, they may impose additional
requirements which are to be adhered to strictly.

**Procedures for Maintaining Confidentiality**

-   1\. All Westat employees and field workers shall sign this assurance
    of confidentiality. This assurance may be superseded by another
    assurance for a particular project.

-   2\. Field workers shall keep completely confidential the names of
    respondents, all information or opinions collected in the course of
    interviews, and any information about respondents learned
    incidentally during field work. Field workers shall exercise
    reasonable caution to prevent access by others to survey data in
    their possession.

-   3\. Unless specifically instructed otherwise for a particular
    project, an employee or field worker, upon encountering a respondent
    or information pertaining to a respondent that s/he knows
    personally, shall immediately terminate the activity and contact
    her/his supervisor for instructions.

-   4\. Survey data containing personal identifiers in Westat offices
    shall be kept in a locked container or a locked room when not being
    used each working day in routine survey activities. Reasonable
    caution shall be exercised in limiting access to survey data to only
    those persons who are working on the specific project and who have
    been instructed in the applicable confidentiality requirements for
    that project.

-   Where survey data have been determined to be particularly sensitive
    by the Corporate Officer in charge of the project or the President
    of Westat, such survey data shall be kept in locked containers or in
    a locked room except when actually being used and attended by a
    staff member who has signed this pledge.

-   5\. Ordinarily, serial numbers shall be assigned to respondents
    prior to creating a machine-processible record and identifiers such
    as name, address, and Social Security number shall not, ordinarily,
    be a part of the machine record. When identifiers are part of the
    machine data record, Westat\'s Manager of Data Processing shall be
    responsible for determining adequate confidentiality measures in
    consultation with the project director. When a separate file is set
    up containing identifiers or linkage information which could be used
    to identify data records, this separate file shall be kept locked up
    when not actually being used each day in routine survey activities.

-   6\. When records with identifiers are to be transmitted to another
    party, such as for keypunching or key taping, the other party shall
    be informed of these procedures and shall sign an Assurance of
    Confidentiality form.

-   7\. Each project director shall be responsible for ensuring that all
    personnel and contractors involved in handling survey data on a
    project are instructed in these procedures throughout the period of
    survey performance. When there are specific contractual obligations
    to the client regarding confidentiality, the project director shall
    develop additional procedures to comply with these obligations and
    shall instruct field staff, clerical staff, consultants, and any
    other persons who work on the project in these additional
    procedures. At the end of the period of survey performance, the
    project director shall arrange for proper storage or disposition of
    survey data including any particular contractual requirements for
    storage or disposition. When required to turn over survey data to
    our clients, we must provide proper safeguards to ensure
    confidentiality up to the time of delivery.

-   8\. Project directors shall ensure that survey practices adhere to
    the provisions of the U.S. Privacy Act of 1974 with regard to
    surveys of individuals for the Federal Government. Project directors
    must ensure that procedures are established in each survey to inform
    each respondent of the authority for the survey, the purpose and use
    of the survey, the voluntary nature of the survey (where applicable)
    and the effects on the respondents, if any, of not responding.

[PLEDGE]{.underline}

I hereby certify that I have carefully read and will cooperate fully
with the above procedures. I will keep completely confidential all
information arising from surveys concerning individual respondents to
which I gain access. I will not discuss, disclose, disseminate, or
provide access to survey data and identifiers except as authorized by
Westat. In addition, I will comply with any additional procedures
established by Westat for a particular contract. I will devote my best
efforts to ensure that there is compliance with the required procedures
by personnel whom I supervise. I understand that violation of this
pledge is sufficient grounds for disciplinary action, including
dismissal. I also understand that violation of the privacy rights of
individuals through such unauthorized discussion, disclosure,
dissemination, or access may make me subject to criminal or civil
penalties. I give my personal pledge that I shall abide by this
assurance of confidentiality.

Signature