Comments to Department of Education (Experimental Sites)

From: University of Kansas (Lawrence Campus and Medical Center Campus)

May 28, 2008

1.  **Is the collection (of information) necessary to the proper
    functions of the Department?**

-   Since one of the main purposes of the Experimental Sites program is
    to provide data to support the assertion that some regulations are
    burdensome, the collection of the data is critical. We advocate for
    a more thorough statistical analysis of the data, including stronger
    recommendations. The current Analysis Reports compiled by
    Experimental Sites staff are more anecdotal in nature and do not
    provide a thorough statistical analysis of the data.

-   In theory, the existence of the Experimental Sites program should
    save the Department money. If the data is analyzed and used to
    advocate for relief from the burden of certain regulations, this
    should result in lower administrative and operational costs for
    schools and for the Department.

Response: The data collection effort remains necessary to monitor the
school and student behavior as well as potential detrimental outcomes
under the experimental conditions. The type of (comparative) statistical
analysis called for in the comment would require the collection of
comparable data from schools not participating in the experiment. Only
the current ability to benefit (ATB) experiment has a "built-in"
comparison group -- the students who establish eligibility by passing a
conventional ATB exam can be compared to those made eligible under the
experimental criteria of demonstrated success in a program without the
benefit of federal aid. We will consider expanding our collection of
such comparative data in future experiments -- either from other
non-participating schools or from students not affected by the
experimental condition.

2.  **Will this information be processed and used in a timely manner?**

-   The Experimental Sites staff collects the data from the school
    reports and compiles the annual Analysis Reports which are
    distributed to the schools. Although this information is
    interesting, we believe it is vital that the Department distribute
    this report, as well as policy recommendations based on the
    available data, to those who can affect change in policy. We are
    hopeful that the Experimental Sites staff is using these reports in
    such a manner.

-   It might be worthwhile to offer an open session (possibly a
    conference call) to all Experimental Sites schools to discuss
    participation in the program. This would be an opportunity for
    participating schools to collaborate on ways to increase the
    visibility and the strength of the Experimental Sites program.

-   We recommend that the Department take advantage of the existence of
    student policy groups who advocate for change. They may have the
    resources to provide more thorough analysis of the data collected.
    In addition, the Department could seek data analysis assistance from
    graduate students who are looking for research projects for their
    Master's or PhD programs. This could benefit both the students and
    the Department.

Response: The Experimental Site staff produces their reports in a timely
manner and shares them with Department of Education staff who are
responsible for policy and who work with congressional staff. The
commenter offers suggestions for expanding the use of the report, i.e.,
changing broader policy. Results from at least 2 experiments have
already contributed to permanent legislative change regarding the 30-day
delay for first year, first time borrowers, and the multiple
disbursement for single term loans. One barrier faced by advocates for
the experiments is the tendency of policy makers to believe that the
schools that participate in the experiments are not representative of
all institutions which leads them to discount our positive findings as
being unlikely to be realized at every school. The limitations in
analysis are not a result of lack of analytical expertise in the
Department, but by the nature of the data itself.

3.  **Is the estimate of burden accurate?**

-   In most, if not all, cases, we believe the time and effort for
    schools to collect and report the data is less burdensome than the
    regulations impacted by Experimental Sites. In addition, we believe
    the broader, more global goal of the Experimental Sites program,
    which is to examine effectiveness and possibly affect policy,
    supports the need to collect and report data in order to enhance the
    institutional and governmental administration of the Federal Student
    Aid programs.

Response: Comment does not question our estimate of burden. In fact we
feel it captures the willingness of all participating schools to supply
the information in support of efforts they feel will improve efficiency
in the delivery of financial aid.

4.  **How might the Department enhance the quality, utility, and clarity
    of the information to be collected?**

-   Currently, the Experimental Sites Annual Reporting Tool asks schools
    to report the estimated savings in work hours and administrative
    costs. It is clear from the Analysis Reports that few schools report
    this information since it is optional. This could be very useful
    information for the Department and for those who can affect policy
    change. We would recommend that the Department collect this
    information from all schools rather than making it optional.

Response: We agree with the sentiment behind this comment, but do not
feel it would be fair to add a new reporting requirement to existing
experiments -- especially as they wind down. We would like to add such a
requirement for new experiments, but this would be contingent on finding
a sufficient number of schools willing to participate in any future
experiment with such a requisite reporting requirement.

5.  **How might the Department minimize the burden of this collection on
    the respondents, including through the use of information
    technology?**

-   The Annual Reporting Tool has been improved greatly over the years.
    The format of the report is very user-friendly and the availability
    of the report through the web is convenient for schools. The Tool
    could be improved by adding language that makes it clear why the
    Department is asking for the data and what will be done with the
    data collected.

Response: We agree with the point of this comment. However, the concern
is addressed in other documents or communications with participating
schools, rather than in the reporting templates. Each participating
school signed an Experimental Sites Amendment to its Title IV "Program
Participation Agreement". This amendment explained why the Department
would collect data and how it might be used. The legislative authority
for the Initiative speaks to this as well. The reporting templates are
quite detailed and we are concerned that attempting to address the issue
on the templates could prove to be a distraction.