New York Codes, Rules and Regulations › Title 8 - EDUCATION DEPARTMENT › Chapter XXI - Annual Program Plans › Part 2302 - Fiscal Year 1978 Annual Program Plan For Libraries And Learning Resources And Educational Innovation And Support › Section 2302.6 - State advisory council evaluation

Current through Register Vol. 46, No. 28, July 10, 2024
(a) In conducting its advisory, evaluation
				and reporting activities, the New York State advisory council is supported by
				the Office for Program Analysis and Evaluation in the State Education
				Department. The Office for Program Analysis and Evaluation serves as support
				staff for the Executive Deputy Commissioner of Education by conducting
				evaluation activities in the areas of policy planning, policy analysis, and
				program evaluation. The office's program evaluation function is executed by the
				Bureau for Program Planning and Evaluation. Major efforts have been made over
				the past two years by that bureau to develop and implement a model for the
				evaluation of department programs. Results of program evaluations conducted by
				the bureau are reported directly to the executive deputy commissioner. It is in
				the context of this program evaluation function that the Bureau for Program
				Planning and Evaluation works with the State advisory council in conducting an
				evaluation of the title IV of the Elementary and Secondary Education Act
				program.
(b) The organizational
				pattern described above places the title IV of the Elementary and Secondary
				Education Act evaluators from the Office for Program Analysis and Evaluation
				(Bureau for Program Planning and Evaluation) in a unique position to assist the
				council in monitoring and evaluating implementation of the title IV of the
				Elementary and Secondary Education Act program. While individual advisory
				council members are encouraged to conduct assessments of the title IV of the
				Elementary and Secondary Education Act program on their own, it is recognized
				that they cannot commit the amount of time and effort necessary to obtain a
				comprehensive understanding of the title IV of the Elementary and Secondary
				Education Act program. In assisting the council in their evaluation, the title
				IV of the Elementary and Secondary Education Act evaluators will be able to
				observe the program on a day-to-day, week-to-week basis without having direct
				responsibility for the administration of the program. This allows the
				evaluators to retain their objectivity while still being knowledgeable of the
				intricacies of the program. In addition, the credibility of the evaluation
				findings is increased because the evaluators are viewed by both council members
				and program administrators as knowledgeable about program operations. This
				increased level of credibility should result in greater utilization of
				evaluation data by the advisory council in making decisions on program policy
				and by program administrators in modifying program operations.
(c) During the past two Federal fiscal years
				(fiscal year 1976 and fiscal year 1977) the Office for Program Analysis and
				Evaluation in the State Education Department has assisted the New York State
				advisory council in conducting its advisory, evaluation and reporting
				activities. In both of these years, the State advisory council's evaluation of
				title IV of the Elementary and Secondary Education Act has focused on the
				management processes employed in the program. For example, under part B,
				evaluators have assessed the extent to which LEA part B applications reflect
				contemporary standards of planning and evaluation for the purpose of improving
				State guidelines, forms, and instructions as well as technical assistance given
				to LEA's. In addition, the evaluators worked with part B program administrators
				in developing instruments for the collection of monitoring and evaluation
				information on projects. Under part C, evaluators collected and analyzed data
				on technical assistance given to local school districts. In addition,
				evaluators worked with program administrators in devising a system for
				reviewing competitive project proposals submitted under part C. Joint efforts
				between the evaluators and program administrators have also resulted in the
				development of a series of instruments to be used for monitoring the
				implementation of part C projects.
(d) The emphasis on evaluating management
				processes under title IV of the Elementary and Secondary Education Act was
				necessitated by the fact that local district projects were only in the early
				stages of implementation during fiscal year 1976 (i.e. preparing project plans
				and beginning implementation of the plans). During fiscal year 1977 most local
				projects under parts B and C have been implemented. Procedures for monitoring
				both B and C projects have been established and implemented. Therefore,
				evaluation work in fiscal year 1978 will focus on the results of project
				implementation. Emphasis will be placed on evaluating the outcomes of projects
				and the progress being made in following New York State strategies for part B
				and part C. The next two sections of this evaluation plan will present the
				implications of this outcome orientation for evaluation of part B and part
				C.
(e) Part B--Evaluation strategy.
(1) As indicated in the fiscal year 1977
				program plan, the State advisory council has viewed its part B evaluation
				strategy as a cumulative strategy; i.e. generalizations on the impact of part B
				projects in the State can be made only after reviewing data from a number of
				project years. Since part B is an acquisitions program, the first project year
				in a district is often spent obtaining materials and infusing these materials
				into the library, guidance program or the curriculum in a subject area. At the
				end of the initial project year, librarians, counselors or teachers may have
				actually used materials with students for only a few months. It is extremely
				difficult to attribute changes in student behavior to materials which have been
				used for such a short period of time. Therefore, a cumulative strategy in which
				data is gathered on a project over time will begin to give the council greater
				insights into the impact of IV-B programs on target populations. Implementation
				of the part B evaluation strategy for fiscal year 1978 will complete the
				four-phase cumulative approach for the fiscal year 1976 and fiscal year 1977
				projects.
(2) The part B evaluation
				strategy for fiscal year 1978 is designed to complete phases three and four of
				the strategy outlined in the fiscal year 1977 plan for projects funded with
				fiscal year 1976 funds and fiscal year 1977 funds (see diagram below). For
				projects supported with fiscal year 1976 funds, phases one and two have been
				completed for a sample of districts. The data collection instruments for phases
				three and four have been developed for use on-site by representatives of the
				IV-B program office. Evaluation work in fiscal year 1978 will concentrate on
				conducting analyses with data gathered on project implementation and impact for
				fiscal year 1976 projects.
(3) In
				terms of projects funded with fiscal year 1977 monies, phases one and two are
				being conducted in spring 1977 (see fiscal year 1977 program plan). Phases
				three and four will be completed in spring 1978 after projects have been
				implemented. Prior to collecting data on fiscal year 1977 projects, revisions
				will be made in the data collection instruments based on experience with the
				fiscal year 1976 projects. Analysis of the data on project implementation and
				impact for fiscal year 1977 projects will also be conducted in spring
				1978.
(4) The emphasis on
				evaluating the outcomes of part B projects will be continued for the projects
				funded with fiscal year 1978 money. The four-phase strategy will be modified
				for these projects by dropping phase 2--planning quality study--from the
				evaluation design. Work on the financial impact study will be conducted for
				fiscal year 1978 projects in spring 1978. Phases three and four which deal with
				project monitoring and evaluation will be implemented in spring 1979. The
				specific objectives and activities of phases three and four for fiscal year
				1978 projects will be included in the fiscal year 1979 program plan since these
				activities will be conducted during that fiscal year.