Source: https://www.federalregister.gov/documents/2014/10/22/2014-24929/direct-grant-programs-and-definitions-that-apply-to-department-regulations
Timestamp: 2017-11-20 00:53:50
Document Index: 573011520

Matched Legal Cases: ['arts 75', 'art 77', 'arts 75', 'art 77', 'arts 75', 'arts 75', 'arts 75', 'art 77', 'arts 75', 'arts 75', 'art 77', 'arts 75', 'art 75', '§\u200977', 'art 79', 'arts 75', '§\u200975', '§\u200977']

A Proposed Rule by the Education Department on 10/22/2014
63062-63066 (5 pages)
2014-24929
I. WWC Evidence Standards (34 CFR Parts 75 and 77)
II. Special Consideration of Applications Supported by “Evidence of Promise” and Clarification of That Definition (34 CFR 77.1(c))
III. Definition of “Large Sample” (34 CFR 77.1(c))
https://www.federalregister.gov/d/2014-24929 https://www.federalregister.gov/d/2014-24929
On August 13, 2013, the Department of Education (the Department) published a notice of final regulations in the Federal Register to amend our Education Department General Administrative Regulations (EDGAR).
In this document, the Department proposes to further amend EDGAR to add a definition of “What Works Clearinghouse Evidence Standards” (WWC Evidence Standards) in our regulations to standardize references to this term. In addition, the Department proposes to amend the definition of “large sample” in our regulation. We also propose technical edits to our regulations to improve the consistency and clarity of the regulations. Finally, we propose to redesignate our regulations and to include in that redesignated section an additional provision that would allow the Secretary to give special consideration to projects supported by evidence of promise.
Postal Mail, Commercial Delivery, or Hand Delivery: If you mail or deliver your comments about these proposed regulations, address them to Alli Moss, U.S. Department of Education, 400 Maryland Avenue SW., Room 4W319, Washington, DC 20202.
The Department published a notice of final regulations in the Federal Register (78 FR 49338) on August 13, 2013 to amend EDGAR. In this document, we propose further amendments to EDGAR to standardize a term and make other amendments to improve the consistency and clarity of these regulations.
During and after the comment period, you may inspect all public comments about these proposed regulations by accessing Regulations.gov. You may also inspect the comments in person in Room 4W335, 400 Maryland Avenue SW., Washington, DC, between 8:30 a.m. and 4:00 p.m., Washington, DC Start Printed Page 63063time, Monday through Friday of each week except Federal holidays. Please contact the person listed under FOR FURTHER INFORMATION CONTACT.
The Department proposes to add a definition of “What Works Clearinghouse Evidence Standards” to 34 CFR part 77. This definition would incorporate the most recent version of the What Works Clearinghouse (WWC) Procedures and Standards Handbook (WWC Handbook), Version 3.0, which was made public in March 2014. Instead of continuing to separately cite the WWC Handbook in various provisions of parts 75 and 77, we propose to add, to part 77, a single definition of the WWC Evidence Standards that incorporates the current version of the WWC Handbook, and then to use that defined term, as applicable, throughout parts 75 and 77.
By adding a definition of “WWC Evidence Standards” and updating the applicable references throughout 34 CFR parts 75 and 77 to incorporate the most recent version of the WWC Handbook, the Department will provide more effective guidance to applicants and grantees as they design and implement rigorous evaluations of their projects. Because Version 3.0 of the WWC Handbook provides further clarification, and does not introduce new requirements, on evaluation- and evidence-related concepts, updating the citations does not substantively change the regulations in 34 CFR parts 75 or 77.
Section 75.266 currently provides that the Secretary may give special consideration, through establishing a separate competition or awarding competitive preference, to discretionary grant applications supported by strong evidence of effectiveness or moderate evidence of effectiveness. In our experience using evidence in discretionary grant competitions, we think it may be beneficial to also include in 34 CFR 75.266 (which we propose to redesignate as 34 CFR 75.226) a provision for giving special consideration to applications supported by evidence of promise, which is a less rigorous standard, because evidence of effectiveness in the education field continues to develop. By including evidence of promise in newly redesignated 34 CFR 75.226, we would allow more flexibility to discretionary grant programs oriented towards supporting evidence-based projects.
The Department proposes to modify the definition of “large sample” in 34 CFR part 77.1 to remove the requirement that analysis units be randomly assigned to treatment or control groups. In implementing our discretionary grant programs, we discovered a discrepancy between the existing definition, specifically its references to random assignment of students, teachers, classrooms, schools, or other single analysis units to treatment or control groups, and the definition of “moderate evidence of effectiveness” in 34 CFR 77.1. Under the definition of “moderate evidence of effectiveness,” a quasi-experimental design study (as defined in 34 CFR 77.1) that includes a large sample could meet the standard, but many such studies do not randomly assign units of analysis to treatment or control groups. We propose to revise the definition of “large sample” to eliminate the random assignment of analysis units into treatment or control groups as a mandatory element. Therefore, for instance, a quasi-experimental design study with a sample of 350 or more students (or other single analysis units), or 50 or more groups (such as classrooms or schools) that contains 10 or more students, could meet the definition of “moderate evidence of effectiveness” in 34 CFR 77.1.
We group major issues according to subject, with appropriate sections of the proposed regulations referenced in parentheses. We discuss other substantive issues under the sections of the proposed regulations to which they pertain.
Generally, we do not address proposed regulatory changes that are technical or otherwise minor in effect.
Current Regulations: The current regulations include multiple references to the WWC Evidence Standards, in each case accompanied by a footnote citing the WWC Handbook, throughout 34 CFR parts 75 and 77, as follows:
1. Factors (viii) and (ix) of the selection criterion “Quality of the project evaluation” in 34 CFR 75.210(h); and
2. Definitions in 34 CFR 77.1(c) of “evidence of promise,” “moderate evidence of effectiveness,” “quasi-experimental design study,” “randomized controlled trial,” and “strong evidence of effectiveness.”
Proposed Regulations: In each provision of 34 CFR parts 75 and 77 that references the WWC Evidence Standards, we propose to update the reference to use a common term, and to define that term in part 77 with reference to Version 3.0 of the WWC Handbook.
Reasons: By updating all references to WWC Evidence Standards in 34 CFR parts 75 and 77, and adding a common definition that references Version 3.0 of the WWC Handbook, we would: (1) Help ensure that applicants and grantees are aware of the most accurate and appropriate resources that are available relating to the WWC Evidence Standards; (2) no longer need the multiple footnotes that reference the current version of the WWC Handbook; and (3) streamline the process for updating our regulations to reflect future versions of the WWC Handbook.
Current Regulations: Under 34 CFR 75.266, the Secretary may give special consideration to applications supported by strong or moderate evidence of effectiveness, by establishing a separate competition or awarding competitive preference. In 34 CFR 77.1(c), the definition of “evidence of promise” references “quasi-experimental study” instead of “quasi-experimental design Start Printed Page 63064study,” a term defined later in the section.
Proposed Regulations: We propose to amend 34 CFR 75.266 to provide that the Secretary may give special consideration to applications supported by evidence of promise, and to redesignate that section as 34 CFR 75.226. We also propose to amend the definition of “evidence of promise” to replace the reference to “quasi-experimental study” with “quasi-experimental design study,” to clarify that the term used in the definition of “evidence of promise” is “quasi-experimental design study,” which is defined in 34 CFR 77.1(c). We also propose to change the paragraph designations in this definition for consistency.
Reasons: We propose these changes in order to provide greater flexibility to discretionary grant programs that reward evidence-based projects in their competitions, to correct the definition of “evidence of promise,” and to provide applicants and grantees consistent and clear information when referencing that definition. We propose to redesignate 34 CFR 75.266 as 34 CFR 75.226 so that the section will be included under the subheading “Selection Procedures” in subpart D of part 75 instead of under the subheading “Miscellaneous.”
Current Regulations: In 34 CFR 77.1(c), the definition of “large sample” currently refers to students, classrooms, schools, groups, or other single analysis units that “were randomly assigned to a treatment or control group.”
Proposed Regulations: We propose to remove the reference to random assignment to treatment or control groups in the definition of “large sample.”
Reasons: We propose this change to eliminate inconsistencies between the definition of “large sample” and the definition of “moderate evidence of effectiveness.” We do not believe that random assignment to a treatment or control group is necessary because the concept of random assignment is embedded within the definition of randomized controlled trial (as defined in 34 CFR 77.1(c)). In order for the “large sample” definition to align fully with the “moderate evidence of effectiveness” definition, the “large sample” definition must not require that units of analysis be randomly assigned into treatment or control groups.
In accordance with both Executive orders, the Department has assessed the potential costs and benefits, both quantitative and qualitative, of this regulatory action. The potential costs associated with this regulatory action are those resulting from statutory requirements and those we have determined to be necessary for administering the Department's programs and activities.
Would the proposed regulations be easier to understand if we divided them into more (but shorter) sections? (A Start Printed Page 63065“section” is preceded by the symbol “§ ” and a numbered heading; for example, § 77.1 Definitions that apply to Department regulations.)
The Secretary certifies that these proposed regulations would not have a significant economic impact on a substantial number of small entities because the proposed regulations primarily clarify and update regulations previously published in the Federal Register.
These proposed regulations do not contain any changes to the Department's current information collection requirements.
These proposed regulations affect direct grant programs of the Department that are subject to Executive Order 12372 and the regulations in 34 CFR part 79. One of the objectives of the Executive order is to foster an intergovernmental partnership and a strengthened federalism. The Executive order relies on processes developed by State and local governments for coordination and review of proposed Federal financial assistance.
For the reasons discussed in the preamble, the Secretary proposes to amend parts 75 and 77 of title 34 of the Code of Federal Regulations as follows:
2. Section 75.210 is amended by revising paragraphs (h)(2)(viii) and (ix) to read as follows.
(viii) The extent to which the methods of evaluation will, if well-implemented, produce evidence about the project's effectiveness that would meet the What Works Clearinghouse Evidence Standards without reservations. * * *
3. Section 75.266 is redesignated as § 75.226 and the newly redesignated section is revised to read as follows:
5. In § 77.1 paragraph(c) is amended by:
A. Revising the definitions of Evidence of promise, Large sample, Moderate evidence of effectiveness, Quasi-experimental design study, Randomized controlled trial, and Strong evidence of effectiveness. Start Printed Page 63066
(a) There is at least one study that is a—
(2) Quasi-experimental design study that meets the What Works Clearinghouse Evidence Standards with reservations; or
(3) Randomized controlled trial that meets the What Works Clearinghouse Evidence Standards with or without reservations.
(b) The study referenced in paragraph (a) found a statistically significant or substantively important (defined as a difference of 0.25 standard deviations or larger) favorable association between at least one critical component and one relevant outcome presented in the logic model for the proposed process, product, strategy, or practice.
(a) There is at least one study of the effectiveness of the process, product, strategy, or practice being proposed that meets the What Works Clearinghouse Evidence Standards without reservations, found a statistically significant favorable impact on a relevant outcome (with no statistically significant and overriding unfavorable impacts on that outcome for relevant populations in the study or in other studies of the intervention reviewed by and reported on by the What Works Clearinghouse), and includes a sample that overlaps with the populations or settings proposed to receive the process, product, strategy, or practice.
(b) There is at least one study of the effectiveness of the process, product, strategy, or practice being proposed that meets the What Works Clearinghouse Evidence Standards with reservations, found a statistically significant favorable impact on a relevant outcome (with no statistically significant and overriding unfavorable impacts on that outcome for relevant populations in the study or in other studies of the intervention reviewed by and reported on by the What Works Clearinghouse), includes a sample that overlaps with the populations or settings proposed to receive the process, product, strategy, or practice, and includes a large sample and a multi-site sample. (Note: multiple studies can cumulatively meet the large and multi-site sample requirements as long as each study meets the other requirements in this paragraph.)
(a) There is at least one study of the effectiveness of the process, product, strategy, or practice being proposed that meets the What Works Clearinghouse Evidence Standards without reservations, found a statistically significant favorable impact on a relevant outcome (with no statistically significant and overriding unfavorable impacts on that outcome for relevant populations in the study or in other studies of the intervention reviewed by and reported on by the What Works Clearinghouse), includes a sample that overlaps with the populations and settings proposed to receive the process, product, strategy, or practice, and includes a large sample and a multi-site sample. (Note: multiple studies can cumulatively meet the large and multi-site sample requirements as long as each study meets the other requirements in this paragraph.)
(b) There are at least two studies of the effectiveness of the process, product, strategy, or practice being proposed, each of which: Meets the What Works Clearinghouse Evidence Standards with reservations, found a statistically significant favorable impact on a relevant outcome (with no statistically significant and overriding unfavorable impacts on that outcome for relevant populations in the studies or in other studies of the intervention reviewed by and reported on by the What Works Clearinghouse), includes a sample that overlaps with the populations and settings proposed to receive the process, product, strategy, or practice, and includes a large sample and a multi-site sample.
[FR Doc. 2014-24929 Filed 10-21-14; 8:45 am]