27990 Federal Register / Vol. 61, No. 107 / Monday, June 3, 1996 / Proposed Rules

DEPARTMENT OF EDUCATION

34 CFR Part 701

RIN 1850–AA52

Standards for Conduct and Evaluation

of Activities Carried Out by the Office

of Educational Research and

Improvement (OERI)—Designation of

Exemplary and Promising Programs

AGENCY: Department of Education.

ACTION: Notice of proposed rulemaking

(NPRM).

SUMMARY: The Assistant Secretary is

developing these standards pursuant to

the Office of Educational Research and

Improvement’s authorizing legislation,

the ‘‘Educational Research,

Development, Dissemination, and

Improvement Act of 1994.’’ The major

purpose of these standards is to provide

quality assurance that programs

designated by the Department of

Education as either exemplary or

promising have met criteria that will

allow educators, professional

organizations, and others to use these

programs with confidence.

DATES: Comments must be received on

or before August 2, 1996.

ADDRESSES: All comments concerning

these proposed regulations should be

addressed to Eve M. Bither, U.S.

Department of Education, 555 New

Jersey Avenue, N.W., Room 500,

Washington, D.C. 20208–5530.

Comments may also be sent through the

Internet to: (EvelBither@ed.gov).

Comments that concern information

collection requirements should be sent

to the Office of Management and Budget

at the address listed in the Paperwork

Reduction Act section of this preamble.

FOR FURTHER INFORMATION CONTACT:

Sharon Bobbitt, U.S. Department of

Education, 555 New Jersey Avenue,

N.W., Room 508, Washington, D.C.

20208–5643. Telephone: (202) 219–

2126. Internet:

(SharonlBobbitt@ed.gov).

Individuals who use a

telecommunications device for the deaf

(TDD) may call the Federal Information

Relay Service (FIRS) at 1–800–877–8339

between 8 a.m. and 8 p.m., Eastern time,

Monday through Friday.

SUPPLEMENTARY INFORMATION:

Background

On March 31, 1994, President Clinton

signed Pub. L. 103–227, which includes

Title IX, the ‘‘Educational Research,

Development, Dissemination, and

Improvement Act of 1994’’ (the ‘‘Act’’).

The Act restructured the Office of

Educational Research and Improvement

(OERI) and provided it with a broad

mandate to conduct an array of research,

development, dissemination, and

improvement activities aimed at

strengthening the education of all

students.

Statutory Requirements

The legislation directed the Assistant

Secretary to develop, in consultation

with the National Educational Research

Policy and Priorities Board, such

standards as may be necessary to govern

the conduct and evaluation of all

research, development, and

dissemination activities carried out by

the Office to ensure that those activities

meet the highest standards of

professional excellence. The legislation

requires that the standards be developed

in three phases.

In the first phase, standards were

promulgated to establish the peer

review process and evaluation criteria to

be used for the review of applications

for grants and cooperative agreements

and proposals for contracts. The final

regulations setting out these standards

were published in the Federal Register

on September 14, 1995 (60 FR 47808).

These proposed regulations address the

second phase of development by

establishing the criteria to be used in

reviewing potentially exemplary and

promising educational programs. The

Assistant Secretary will publish at a

later date additional proposed

regulations for phase three of the

standards, which are to govern

evaluation of the performance of

recipients of grants and contracts and

cooperative agreements with OERI.

The OERI legislation requires that

expert panels be established to review

educational programs and recommend

to the Secretary those programs that

should be designated as exemplary or

promising and disseminated through the

Department’s National Education

Dissemination System. The legislation

further requires the Assistant Secretary

to develop standards that describe the

procedures the panels will use in

reviewing the educational programs.

Section 941(a)(3) of the legislation

broadly defines ‘‘educational program’’

to include ‘‘educational polices,

research findings, practices and

products.’’ Educational programs may

range in size and complexity from an

individual instructional program—such

as an elementary school science

program—to a comprehensive reform

initiative involving multiple goals and

participants. Programs at all levels of

education—preschool, elementary,

secondary, and postsecondary—are

eligible for consideration.

The Act also requires that the

Assistant Secretary review the

procedures utilized by the National

Institutes of Health (NIH), the National

Science Foundation (NSF), and other

Federal departments or agencies

engaged in research and development

and actively solicit recommendations

from research organizations and

members of the general public.

In developing the review and

evaluation procedures for the proposed

standards, OERI has reviewed and

considered dissemination practices and

procedures used for identifying

promising and exemplary programs by

various foundations, research

organizations, associations, and Federal

agencies including NIH, NSF, the

Federal Emergency Management

Agency, the Department of Health and

Human Services, and the National

Endowment for the Arts. OERI adapted

these review and evaluation procedures

as appropriate.

Proposed Standards

The proposed standards have been

developed by the Assistant Secretary in

consultation with the Board. The

standards proposed in this NPRM—

• Require that expert panels be

established to review educational

programs and recommend to the

Secretary those programs that should be

designated as exemplary or promising

and disseminated through the

Department’s National Education

Dissemination System; and

• Establish a process that panels will

use to review and evaluate educational

programs and determine which

programs to recommend to the Secretary

for designation as exemplary or

promising.

Educational programs may be

submitted at any time for consideration

for designation as exemplary or

promising. In addition, the Assistant

Secretary will periodically establish and

announce in the Federal Register

specific topic areas of high priority for

which programs will be invited or

sought out. The legislation also provides

that the Secretary may identify

educational programs for the panels to

review.

Educational program submissions

may include, as evidence of the

effectiveness of the program, a range of

assessments, evaluative information

from users, and other objective

performance indicators that are

appropriate to the program. The

legislation ensures that a panel may not

eliminate any program from

consideration based on the lack of one

type of supporting data such as test

scores.
Federal Register / Vol. 61, No. 107 / Monday, June 3, 1996 / Proposed Rules 27991

A standing group of experts, which

will include teachers and others, will be

appointed by the Assistant Secretary as

appropriate. From that group, the

Assistant Secretary will select members

who have relevant knowledge and

experience in specific topic areas to

form expert panels to review programs

in accordance with the criteria in these

proposed regulations.

In determining whether an

educational program should be

recommended as exemplary or

promising, the panel is required by the

legislation to consider (a) whether,

based on empirical data, the program is

effective and should be designated as

exemplary, or (b) whether there is

sufficient evidence to demonstrate that

the program shows promise for

improving student achievement and

should be designated as promising.

These proposed regulations require a

panel to evaluate whether a program has

met all of the criteria of educational

effectiveness set forth in Subpart C of

these proposed regulations. A panel

may determine that a program shows

promise for improving student

achievement and recommend that the

program be designated as promising if

the program has met all of the criteria

with respect to one context, or with one

population. A panel may determine that

a program is effective and recommend

that the program be designated as

exemplary if the program has met all of

the criteria with respect to multiple

contexts, or with multiple populations.

Use of these criteria for evaluating

programs will ensure that programs

disseminated by the Department are

high-quality, research-based programs

that have provided evidence indicating

they have improved teaching or learning

or both. The Department’s

dissemination system is designed to

make programs available to the public

as quickly as possible. The system will

enable the Department to respond to all

forms of requests for information and

assistance, and to support the

applications of research and best

practice. The system will use electronic

networking and the capabilities of:

National Research Institutes;

Educational Resources Information

Center (ERIC);

Regional Educational Laboratories;

Department-supported technical

assistance providers;

National Library of Education; and

Other public and private nonprofit

entities, including education

associations and networks.

Prior to the adoption of these

standards, exemplary programs were

validated by the Department’s Program

Effectiveness Panel (PEP) and

disseminated through the National

Diffusion Network (NDN). With the

adoption of these standards, the

Department will recognize and

disseminate promising educational

programs in addition to exemplary

programs.

Regulatory Flexibility Act Certification

The Secretary certifies that these

proposed regulations would not have a

significant economic impact on a

substantial number of small entities.

The small entities that would be

affected by these proposed regulations

are small local educational agencies

(LEAs) and private schools receiving

Federal funds under this program.

However, the regulations would not

have a significant economic impact on

the small LEAs and private schools

affected, because the regulations would

not impose excessive regulatory burdens

or require unnecessary Federal

supervision. The regulations would

impose minimal requirements to ensure

the proper expenditure of program

funds.

Paperwork Reduction Act of 1995

Section 701.4 contains information

collection requirements. As required by

the Paperwork Reduction Act of 1995,

the Department of Education will

submit a copy of this section to the

Office of Management and Budget

(OMB) for its review. (44 U.S.C. 3504(h))

These regulations affect the following

types of entities eligible to submit a

program for review: Any public or

private agency, organization or

institution, or individual.

The public reporting burden is

estimated to range from 2 to 6 hours for

each program submitted for review. The

actual burden will be determined by

how much descriptive information

about their program each entity wishes

to provide.

Organizations and individuals

desiring to submit comments on the

information collection requirements

should direct them to the Office of

Information and Regulatory Affairs,

OMB, Room 10235, New Executive

Office Building, Washington, D.C.

20503; Attention: Wendy Taylor.

The Department considers comments

by the public on this proposed

collection of information in—

• Evaluating whether the proposed

collection of information is necessary

for the proper performance of the

functions of the Department, including

whether the information will have

practical utility;

• Evaluating the accuracy of the

Department’s estimate of the burden of

the proposed collection of information,

including the validity of the

methodology and assumptions used;

• Enhancing the quality, usefulness,

and clarity of the information to be

collected; and

• Minimizing the burden of the

collection of information on those who

are to respond, including through the

use of appropriate automated,

electronic, mechanical, or other

technological collection techniques or

other forms of information technology

(e.g., permitting electronic submission

of responses).

OMB is required to make a decision

concerning the collection of information

contained in these proposed regulations

between 30 and 60 days after

publication of this document in the

Federal Register. Therefore, a comment

to OMB is best assured of having its full

effect if OMB receives it within 30 days

of publication. This does not affect the

deadline for the public to comment to

the Department on the proposed

regulations.

Invitation To Comment

Interested persons are invited to

submit comments and recommendations

regarding these proposed regulations.

All comments submitted in response

to these proposed regulations will be

available for public inspection, during

and after the comment period, in Room

600, 555 New Jersey Avenue, NW.,

Washington, DC, between the hours of

8:30 a.m. and 4 p.m., Monday through

Friday of each week except Federal

holidays.

Assessment of Educational Impact

The Secretary particularly requests

comments on whether the proposed

regulations in this document would

require transmission of information that

is being gathered by or is available from

any other agency or authority of the

United States.

List of Subjects in 34 CFR Part 701

Education, Educational research,

Reporting and recordkeeping

requirements.

Dated: May 22, 1996.

(Catalog of Federal Domestic Assistance

Number does not apply)

Sharon P. Robinson,

Assistant Secretary for Educational Research

and Improvement.

The Secretary proposes to amend

Chapter VII of Title 34 of the Code of

Federal Regulations by adding a new

Part 701 to read as follows:
27992 Federal Register / Vol. 61, No. 107 / Monday, June 3, 1996 / Proposed Rules

PART 701—STANDARDS FOR

CONDUCT AND EVALUATION OF

ACTIVITIES CARRIED OUT BY THE

OFFICE OF EDUCATIONAL

RESEARCH AND IMPROVEMENT

(OERI)—DESIGNATION OF

EXEMPLARY AND PROMISING

PROGRAMS

Subpart A—General

Sec.

701.1 What is the purpose of these

standards?

701.2 What definitions apply?

701.3 What entity is eligible to submit a

program for review?

701.4 What must an entity submit for

review?

Subpart B—Selection of Panel Members

701.10 How are panels established?

701.11 Who may serve as a member of the

standing group?

701.12 How is the membership of expert

panels determined?

Subpart C—The Expert Panel Review

Process

701.20 How does an expert panel evaluate

programs?

701.21 What is the difference between an

exemplary and a promising program?

701.22 What criteria are used to evaluate

programs for exemplary or promising

designation?

Authority: 20 U.S.C. 6011(i)

Subpart A—General

§ 701.1 What is the purpose of these

standards?

(a) The standards in this part

implement section 941(d) of the

Educational Research, Development,

Dissemination, and Improvement Act of

1994.

(b) These standards are intended to

provide quality assurance that programs

designated by the Department of

Education as either exemplary or

promising have met criteria that will

allow educators, professional

organizations, and others to use these

programs with confidence.

(Authority: 20 U.S.C. 6011(i)(2)(E))

§ 701.2 What definitions apply?

Definition in the Educational

Research, Development, Dissemination,

and Improvement Act of 1994. The

following term used in this part is

defined in 20 U.S.C. 6041(a)(3):

Educational program

(Authority: 20 U.S.C. 6041(a)(3))

§ 701.3 What entity is eligible to submit a

program for review?

Any public or private agency,

organization, or institution, or an

individual, may submit an educational

program for review.

(Authority: 20 U.S.C. 6011(i)(2)(E))

§ 701.4 What must an entity submit for

review?

(a) To have its educational program

considered for designation as exemplary

or promising, the eligible entity must

submit to the Secretary a description of

the program and a discussion of the

program’s educational effectiveness,

responsive to the criteria in Subpart C,

§ 701.22.

(b) Information submitted must

include, to the extent relevant to the

particular program—

(1) A program abstract of 250 words

or less;

(2) A description of the salient

features of the program;

(3) A description of the program’s

philosophy and history;

(4) Site information, including

demographics;

(5) A description of evaluation results;

(6) Funding and staffing information;

and

(7) Organization name, address,

telephone and fax numbers, e-mail

address (if available), and contact

person.

(Authority: 20 U.S.C. 6011(i)(2)(E))

Subpart B—Selection of Panel

Members

§ 701.10 How are panels established?

(a) The Assistant Secretary, in

consultation with the National

Educational Research Policy and

Priorities Board established under 20

U.S.C. 6021, establishes a standing

group of educational experts. The

Assistant Secretary may expand the

membership of the standing group as

necessary.

(b) The Assistant Secretary selects

members from the standing group, based

on their areas of expertise, to serve on

expert panels in specific topic areas for

the purpose of reviewing and evaluating

educational programs and

recommending, to the Secretary, those

programs that should be designated as

exemplary or promising.

(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))

§ 701.11 Who may serve as a member of

the standing group?

An individual may serve as a member

of the standing group for the purpose of

reviewing and evaluating educational

programs for exemplary or promising

status if that individual possesses two or

more of the following qualifications:

(a) Demonstrated expertise and

experience in one or more specific

educational areas.

(b) Demonstrated expertise and

experience across a broad range of

educational policies and practices.

(c) Experience in evaluating

educational programs.

(d) Experience or expertise in

developing educational products.

(e) Current employment as a teacher,

principal or other school-based or

community-based professional (such as

a guidance counselor, school media

specialist, or health professional).

(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))

§ 701.12 How is the membership of expert

panels determined?

(a) For the review of each program, or

group of programs, the Assistant

Secretary establishes an expert panel

comprised of individuals who are

members of the standing group.

(b) In establishing the membership of

each expert panel, the Assistant

Secretary—

(1) Selects individuals who have indepth knowledge of the subject area or

content of the program or group of

programs to be evaluated;

(2) Selects at least one current teacher,

principal, or other school-based or

community-based professional;

(3) Ensures that no more than onethird of the panel members are

employees of the Federal Government;

and

(4) Ensures that each panel member

does not have a conflict of interest, as

determined in accordance with

paragraph (c) of this section, with

respect to any educational program the

panel member is asked to review.

(c) Panel members are considered

employees of the Department for the

purposes of conflicts of interest analysis

and are subject to the provisions of 18

U.S.C. 208, 5 CFR 2635.502, and the

Department’s policies used to

implement those provisions.

(Authority: 20 U.S.C. 6011(i)(2)(E), 6041(d))

Subpart C—The Expert Panel Review

Process

§ 701.20 How does an expert panel

evaluate programs?

(a) Each panel member shall—

(1) Independently review each

program based on the criteria in

§ 701.22;

(2) Provide written comments based

on an analysis of the strengths and

weaknesses of the program according to

the criteria;

(3) Participate in site visits if

appropriate; and

(4) Participate in a meeting of the

expert panel, if appropriate, to discuss

the reviews.

(b) A panel may not eliminate an

educational program from consideration

based solely on the fact that the program

does not have one specific type of

supporting data, such as test scores.
Federal Register / Vol. 61, No. 107 / Monday, June 3, 1996 / Proposed Rules 27993

(c) Each expert panel shall make a

recommendation to the Secretary as to

whether the program is exemplary,

promising, or neither.

(Authority: 20 U.S.C.6011(i)(2)(E), 6041(d))

§ 701.21 What is the difference between an

exemplary and a promising program?

(a) A panel may recommend to the

Secretary that a program be designated

as promising if the panel determines

that the program has met each of the

criteria of educational effectiveness in

§ 701.22 with respect to one context or

one population.

(b) A panel may recommend to the

Secretary that a program be designated

as exemplary if the panel determines

that the program has met each of the

criteria of educational effectiveness in

§ 701.22 with respect to multiple

contexts or multiple populations.

(Authority: 20 U.S.C.6011(i)(2)(E), 6041(d)(2))

§ 701.22 What criteria are used to evaluate

programs for exemplary or promising

designation?

In determining whether an

educational program (‘‘program’’

includes educational polices, research

findings, practices and products) should

be recommended as exemplary,

promising, or neither, each expert panel

shall consider the following criteria of

educational effectiveness:

(a) Evidence of success. The expert

panel considers—

(1) Whether, based on a range of

assessments, information from users, or

other indicators as appropriate, the

program contributes to solving

substantial or important problems in

teaching or learning; and

(2) The extent to which—

(i) Program effects are beneficial to the

populations for whom the program was

designed; or

(ii) The product performs as expected

for the educational consumers it was

said to benefit.

(b) Quality of the program. The expert

panel considers—

(1) Whether the program has clear

goals, is based on sound research and

practice, and incorporates accurate and

up-to-date content;

(2) Whether the program represents a

substantially improved alternative to

existing options;

(3) The extent to which the program

promotes equity and is free of bias based

on race, gender, age, culture, ethnic

origin, disability, or limited English

proficiency status;

(4) Whether the program is based on

high expectations for the success of all

participants;

(5) Whether the program is

appropriate to the target audiences; and

(6) The extent to which any materials

associated with the program conform to

accepted standards of technical quality.

(c) Educational significance. The

expert panel considers—

(1) The extent to which the program

has the potential to increase knowledge

or understanding of educational

problems, and issues, or effective

strategies for teaching or learning; and

(2) Whether the program is described

clearly enough so that it can be adapted

or adopted in new sites.

(d) Usefulness to others. The expert

panel considers—

(1) Whether the cost of the program

(including money, staff time, and other

required resources) is reasonable in light

of expected benefits and compared to

other alternatives; and

(2) Whether the program is available

for use by others.

(Authority: 20 U.S.C. 6011(i)(2)(E),

6041(d)(2))

[FR Doc. 96–13801 Filed 5–31–96; 8:45 am]

BILLING CODE 4000–01–P