1 **Evaluation Report Instructions ESEA Title II, Part D**  **Public Burden Statement:** According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 30 hours per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. The obligation to respond to this collection is required to obtain or retain benefit (Title II, Part D, Subpart 1 of the Elementary and Secondary Education Act of 1965, as amended . Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the U.S. Department of Education, 400 Maryland Ave., SW, W ashington, DC 20210 - 4537 or email ICDocketMgr@ed.gov and reference the OMB Control Number 1810 - 0702 . Note: Please do not return th e completed evaluation report to this address. **Report Due Date:** Annually beginning on September 30, 201 X **Report Period Covered:** Fiscal ye ar ( FY ) 20 1X/ school year ( SY ) 20 1X - 201 X sub recipient awards . The State Evaluation Report should cover Title II, Part D (Ed Tech) activities 1 funded with FY 20 XX regular allocations and/or FY 20 1X allocations awarded under the American Recovery

and Investment Act of 2009 (ARRA). The report should include evaluation of Ed Tech - funded activities that were initiated during SY 201X using FY 20 1X funds and activities that were continued from prior awards using FY 20 1X funds. **Qualification:** The Department recognizes that for the evaluation report due by September 30, 201 X , evaluations of FY 20 1X Ed - Tech - funded activities may not have produced sufficient evaluative data to report findings, conclusions and recommendations. Nonetheless, States are expected to have planned and initiated evaluations of FY 20 1X Ed Tech - funded activities well in advance of the report due date. S tates should report on the details and status of their evaluations as of the report due date. In addition, evaluations of activities begun in a prior ye ar and continued with FY 2009 funds should be at a stage in their implementation to, at minimum, produce preliminary findings, conclusions and recommendations. **Purpose:** Se ctions 2402(a)(7) and 2413(b)(4) of the Elementary and Secondary Education Act, as amended ( ESEA ) require States and local educational agencies (LEAs) 1 The term “Activities” refers to the projects, programs, activities, services, strategies, hardware and software, etc. for which LEAs received Title II, D formula or compe titive grant funds. States need only report on those funded activities that are of a sufficient size to be effective, consistent with the purposes of this part.2

that receive Ed Tech funds to condu ct rigorous evaluation of the effectiveness of Ed Tech formula and competitive grant - funded projects, activities and strategies in integrating technology

into curricula and instruction and improving student achievement. The purpose of this reporting requirement is to identify from the results of those evaluations innovative projects, activities and strategies that effectivel y infuse technology with curriculum and instruction, show evidence of positive impacts for student learning, and can be widely

replicat ed by

State educational agencies and LEAs in the State and in other States . **Report Template:** The Evaluation Report Template that follows provides instructions to States for preparing their annual report on the evaluation of ESEA Title II, Part D - funded activities. 2 Ingeneral, the evaluation report will:  Detail the activities being evaluated ,  Explain the process and measures used to evaluate the effectiveness of the activity ,  Present findings and conclusions supported by evidence about the activities’ effectiveness , and  Propose concrete and usable recommendations for disseminating, replicating and scaling of effective projects, activities and practice, as appropriate, based on the evidence and lessons learned. The detailed instructions are designed to support the Department’s plans for (a) compiling and anal yz ing the information extracted from the 52 State evaluation reports and (b) presenting the information in a way that is meaningful and useful to the Department, States, school districts and other u sers. **Page Limitations:** The Department is not imposing restrictions on the number of pages of the report. The numbers of pages indicated for each section are suggestive. Font for body text is to be 12 point, Times New Roman for readability. **Report Sub mission:**  Reports are to be submitted to the Department via e - mail no later than 5:00 PM ET on September 30, 201 X . Each State must submit its report in two electronic forms: one in Microsoft Word version and one as a PDF containing the authentic signature o f the submitting official. E - mail reports to the Ed Tech program mail box which can be accessed at: www.eett.ed.gov . Questions concerning the reports should be addressed to : Dr. Edward Smith Edward.smith@ed.gov

2 The reports submitted September 30, 201X and September 30, 201X should include activities funded with FY 2009 ARRA Ed Tech funds.3

**Report Template and Specific Instructions: Title Page**  The title and opening pages of the report should include the following information.  Title of the r eport  Date ( d ate s ubmitted)  Name and l ocation of State d epartment of e ducation  Key State officials responsible for the report (Chief State School Officers, Technology

Director, others as appropriate)  Names and organizations of evaluators (if external)  Name s and organizations of report prepare r s  Acknowledgements  Signature line for State Official s ubmitting the report **Table of contents** The Table of Contents lists the “ Titled Sections ” (and sometimes sub - sections) of the report in the order of their location in the report and indicates the beginning page number for each section. The Table of Contents also should include a section with page references for t ables, graphs, f igures and a

ppendi c es. **Executive Summary** (

suggested 2 - 3 pages) The Executive Summary

is a stand - alone section that summarizes the evaluation. It should contain enough information to familiarize the reader with the main points of the evaluation without having to read through all of the report details. It should:  Id entify

the activities and ini tiative being evaluated  Explain the purpose and objectives of the evaluation  Describe the key

aspects of the evaluation approach and methods  Summarize the main findings, conclusions, and recommendations. **I. Introduction**  (

suggested 2 - 4 pages) This section introduces the reader to the evaluation and the report contents.  Briefl y describe the activity that is being evaluated, who m the activity is intended to benefit and how it is intended to benefit them. Explain the importance of the activity in meeting the instructional improvement and student achievement goals and purposes of Ed Tech .  Discuss the purpose 3 and objectives of the evaluation (i .e., why the evaluation was conducted and what it was and was not intended to accomplish) . 3 Consistent with the purposes and requirements in sections 2402(a)(7), 2413(b)(4) and other goals and purposes of ESEA Title II, Part D, the purposes of this evaluation are (1) to conduct rigorous evaluation of the effectiveness of Title II, Part D formula and competitive grant - funded projects, activities and strategies in integrating technology into curricula and instruction, and (2) to identify effective practices that can be widely replicated by State educational agencies and local educational agencies in the State and in other States.4

 Explain the evaluation approach that the State used to meet this reporting requirement, the rationale for selecting that approach, and how the State expects that the approach selected will contribute to the evaluation purposes.

**Examples of Possible Approaches**  **Individual sub -**  **grantees conducted their own evaluations; the report presents a compilation of the individual evaluation reports on the**  **effectiveness of their individual programs and strategies.**  **The State conducted a statewide evaluation of all funded activities measured against a common set of performance criteria or indicators.**   **The State conducted an evaluation of clusters of related activities across the funded activities.**   **The State conducted a meta**  **- evaluation of individual project**  **evaluations.**

 Id entify

the evaluation team and whether the evaluation was conducted wholly or in part by internal (district or State officials) or b y external sources; identify who prepared the report.  Ev aluation timetable  The total cost allocation for the evaluation and amount expended at the time of the report.  Introduce the reader to the structure of the report. **II. State Ed Tech Program Context** ( suggested 3 - 5 pages) This s ection of the report should provide the reader with an overall picture of the Ed Tech grant program as context for understanding what is being evaluated and why. **A. Summary: State**  **Ed Tech Allocations and Awards**  1. Total amount of FY 20 1X f unding for the State. **FY 2**

**____ ARRA**  **Allocation FY 2** 

**___ Regular**  **Allocation Total FY**  **2**

**___ Allocation**  2. The number, percent and amounts of FY 20 1X grants awarded competitively and by formula based on the total FY 201X appropriation (the combined total of ARRA and regular funds). **Type of Award**  **Number**

**of Awards**  **Percent**

**of Awards**  **Range of Award**

**Am ounts**  **(Lowest –** **Highest) Average** 

**(Median) Award Am**  **ount Formula**  **Competitive**5

**B. Competitive Grant Program**  **Description** 1. Describe the overall goal(s) of the State’s competitive grant program (i.e. what the program sought to achieve). 2. Describe how the goals and objectives of the State’s Ed Tech competitive program are aligned with the State’s strategic goals for education reform and improvement. 3. Briefl y describe how the goals, objectives and activities funded under the State’s Ed Tech competitive program are expected to achieve the objectives of the Ed Tech program :  Improve student achievement  Improve teaching  Integrate technology

with curriculum, instruction, and professional development. 4. Briefl y describe how the goals and objectives of the State’s Ed Tech competitive program support(s) the U.S. Department of Education’s strategic priorities for education reform and improvement:  Improving the quality of teachers and school - leaders  Adopting career and college - ready standards for students  Using data to support and improve instruction and assess school and teacher effectiveness  Implementing high quality , high intensity effort to reform struggling schools  Implementing School and classroom innovations **A. Formula Grant Program Description**  1. Briefl y describe how the goals, objectives and activities funded under the State’s Ed Tech formula grant program are expected to achieve the objectives of the Ed Tech program to:  Improve student achievement  Improve teaching  Integrate technology

with curriculum, instruction, and professional development. 2. Briefl y describe how the goals, objectives and activities funded under the State’s Ed Tech formula grant program support(s) the U.S. Department of Education’s strategic priorities for education reform and improvement:  Improving the quality of teachers and school - leaders  Adopting career and college - ready standards for students  Using data to support and improve instruction and assess school and teacher effectiveness  Implementin g high quality , high intensity effort to reform struggling schools  Implementing School and classroom innovations . **III. The Activity**  (

suggested 5 - 7 pages each for formula and competitive) This section of the report describes in detail the activity that is being evaluated. Whether the evaluation is of a single activity , a cluster of activities, or whether the activity is being6

implemented in different way s in different settings, the activity must be described in sufficient detail so that the average reader will understand what the activity is, its scale and complexity , what it seeks to achieve for whom, and the “Theory of Action” that explains why

the activity is expected to achieve the intended results. A full description of what is being evaluated i s not only critical to understanding the evaluation and its results, but provides the context for assessing the extent to which the activity can be replicated and under what conditions. Include a detailed discussion of each of the following aspects of the funded activity : **A. Features**   Nature of the activity and the educational/instructional issue it seeks to address.  The specific programs, services, technology , strategies, etc. involved in the activity .  The specific outcomes that the activity seeks to achieve and for whom.  “Theory

of Action,” i.e. h ow the activity is expected to produce the desired outcomes. Explain the key assumptions (including the expected “results chain”) underl yi ng the relationship between activities and expected outcomes.  The pl anned duration of the activity and the status of its implementation at the time of the report. **B. Resources Allocated**   The total Ed Tech funded amount for the activity per annum  Who provides the services/activities (e.g., an y partners, outside vendors, district, school or State officials)  The total resources for the activity from all sources, including human resources and budget(s) (i.e., per pupil expenditures from State, federal, local funds)  The percent/proportion of funds accounted for by Ed Tech funding. **C. Scale and Complexity**   The number of districts, schools, classrooms, teachers, and students involved in the activity

(i.e., either participate directly or are expected to benefit).  The demographics of the participating districts, schools and classrooms, including: the locale, s ize, percent age of families in poverty , number/percent age of districts/schools in “improvement” status, number/percent age of the lowest performing schools in the district/State identified by

the State for Title I , Part A School Improvement Grant funding.  Other contextual variables that have or could have an impact on the program and its results. **IV. The Evaluation** (

suggested 3 - 5 pages each for formula and competitive ) **A. Scope**  Make clear which aspect(s) of the activity and which intended outcomes of activity are covered b y this evaluation report and which are not.7  Define the unit of anal ys is covered b y the evaluation (e.g., evaluation of a single project, a series of individua l project evaluations, a cluster of related activities across projects, a cluster of projects, a subcomponent, process or strategy within a project, etc.)  Explain the evaluation parameters for this report (i.e., which of the expected outcomes will be addre ssed in the report and which will not be address ed).  Explain the rationale for selecting the specific focus for this report . **B. Objectives and Questions**  Provide a clear explanation of the evaluation’s primary objectives, evaluation criteria and key

questions.  Specify

the primary

objective(s) of the evaluation and how that ( those ) objective ( s ) contribute s (contribute) to the evaluation purpose.  Define the evaluation criteria (the performance measures or success standards ) that will be used to assess the extent to which the intended results of the activity were achieved.  Explain the rationale for selecting the particular criteria.  Detail the key evaluation questions the evaluation addresses and explain how the answers to those questions will satisfy the evaluation objectives. Also, point out any

key questions that the evaluation does not address and why . **C. Evaluation Methods**  Describe in detail th e data collection methods and anal ys e s selected for the evaluation , and the rationale for their selection. Discuss each of the following:  The sources of information (e.g. State assessments, classroom observations, interviews, student records) and the rati onale for their selection.  Explain how the data will be used to answer the evaluation questions .  Describe the criteria and process for selecting the sample for the evaluation (e.g., random, purposive) , the extent to which the sample is representative of the entire target population , and the rationale for using the sample selection criteria and process . If applicable, explain how comparison and treatment groups were assigned.  The procedures (i.e., who c ollected the data, how, the timeframe and circumstances) and the instruments (e.g. i nterview protocols, observation tools) used to collect the data . Discuss their appropriateness for the data source and evidence of their reliability and validity.  Methods used for data analy sis.  Any major limitations of the methodology and the resulting implications for the evaluation and how to interpret findings. Complete the evaluation matrix provided below.8



### V. **The Results** (

suggested 3 - 5 pages each for formula and competitive) **A. Findings**  **4**  Specify

the actual results of the data collection, including rates of return and missing data ;  Describe the procedures for anal yz ing the data, including the steps evaluators took to confirm the accuracy of the data ;  Discuss the statistical analy sis, if appropriate ;  First present fin dings as objective facts free of interpretations; then interpret findings to give meaning to the objective data ;  Structure findings and interpretations around the evaluation questions so that it is easy to make the connection between the questions the evaluation sought to answer and what was learned ;  Present positive and negative findings, unanticipated findings, as well as identify questionable findings ;  Id entify

potential weaknesses in the data and the possible influence on the way findings should b e interpreted and conclusions drawn ; and  Use graphics ( e.g.: charts or tables) to illustrate the information, as appropriate. **B. Conclusions**  Present the conclusions based on the interpretation of findings around each of the key questions. Conclusions report ed should be well substantiated by the evidence and logically

connected to evaluation findings. 

### VI. **Recommendations and Lessons Learned** ( suggested 3 - 5 pages each for formula and competitive )

4 The Department recognizes that for the evaluation report due by September 30, 201 X , evaluations of FY 20 1X Ed Tech - funded activities may not have produced sufficient evaluative d ata to report findings, conclusions and recommendations. However, evaluations of activities begun in a prior year and continue d with FY 201X funds should be at a stage in their implementation to, at minimum, produce preliminary findings, conclusions and r ecommendations. **Evaluation Matrix**

**Key Questions**  **Data**

**Sources Data** 

**collection Methods /**  **Instruments Performance** 

**Indicators/ Success**  **Standards Methods for** 

**Data Analysis** 9  Based on the results of evaluation, propose concrete and usable recommendations for dissemination, replication and scaling of the activity , as appropriate.  Discuss what lessons were learned and wh y those lessons are meaningful and applicable for scaling the activity in other contexts. **Appendi c**  **es** The following should be included as Appendi c es , as appropriate:  List of Acron ym s and Abbreviations  Evaluation timetable  Bibliograph y of references  Instruments used to collect data: questionnaires, survey s, student assessments, observation protocols, sample student and/or teacher portfolio, interview protocols, etc. Include evidence or attestation as to their reliability and validity .  Detailed results chain associated with the “Theory of Action”  Tables of findings