- 24 -

Used by itself, the qualitative approach has significant limitations. Most importantly, generalization beyond the particular process and context studied is difficult. Second, it requires well- trained and skillful evaluators, able both to participate closely in a process and to maintain the degree of objectivity necessary for analysis and interpretation. Finally, as a method it is open to partial and careless use (although positivistic methodologies are prey to the same problem).

Because of these problems, and because of its recent emergence, qualitative evaluation has not become widespread practice in U.S. educational evaluation. Its influence has been felt, however, as evaluators have moved towards hybrid models which seek to incorporate the best of both positivistic and qualitative methods in an attempt to marry valid measurement of outcomes, and assessment of process and context factors, to the need for localized evaluations which meet the utilization needs of educational decision-makers.

Changes in Legislation and Funding: The Rise of Local Accountability As evaluators struggled with issues of utilization and methodology, changes also took place in the political and financing context of evaluation. Recent years have seen a lessening of support for the idea that the federal government is the appropriate agency for the solution of social problems, a perception brought about, at least in part, by the widely held view that many large-scale social programs substantially failed. This conservative national agenda has become evident as the federal government has increasingly shifted responsibility for both funding, and educational direction and evaluation management to state (SEA) and local (LEA) education agencies. As one evaluator noted "the golden age of evaluation, from a funding standpoint, is over" (Webster, 1987; p. 49). The evaluation capability that was developed on both the national and local levels during this "golden age" has, however, continued to nurture the continued growth within the profession. The new and smaller role of the federal government in education, and in evaluation, helps to explain the changes that have occurred within the past few years.

A central event in the transition towards increased local control of monitoring and evaluation was marked by the failure of the Vocational Education Data System (VEDS) (Hoachlander, et. al., 1985). Created by federal legislation in 1976, the system was to be built on State Occupational Information Coordinating Committees to improve the quality of labor market information; VEDS would provide a ccmplex array of data on vocational education programs. The system imposed uniform requirements on the states for defining program types, student characteristics and financial information.

The system met with massive resistance and a relatively high level of non-compliance. The data requirements conflicted with established state and local practice, and placed an insupportable burden on state and local agencies. In the end, the system was withdrawn, and replaced in subsequent legislation with simpler requirements that reduced the data collection burden. At the same time, state-level systems for monitoring of vocational education remained comparatively undeveloped.