Pseudoissues in Practice Evaluation: Impediments to Responsible Practice

In recent years a body of literature in relation to practice evaluation has developed, and part of this literature pertains to tension and conflicts between practice and evaluation research (Kagle, 1982; Kagle & Cowger, 1984; Ruckdeschel & Farris, 1981; Saleebey, 1979; Thomas, 1978, 1983) and ethical issues that arise in evaluation (Bloom & Orme, 1993; Nelsen, 1994). Ivanoff, Blythe, and Briar (1987) noted that the main criticisms of practice evaluation are related to the difficulty of single-system design, the belief that the complexity of practice is not conducive to evaluation, and the impact of negative findings on the profession. Other criticisms are that evaluation is time consuming (Penka & Kirk, 1991) and prone to gender and ethnic bias (Nelsen, 1994). Additionally, there is debate about whether the incorporation of evaluation activities is compatible with the "art" of helping (Saleebey, 1979; Witkin, 1991). Blythe and Rodgers (1993) stated, "Amazingly, the debate about whether practitioners should evaluate their practice with single-system design continues in the professional literature and in discussions among colleagues" (p. 107). The debate about evaluation will not abate, and evaluation will not be truly integrated as part of practice until some of the criticisms of empirical practice and evaluation are recognized for what they are - pseudoissues. This article reviews the often-stated barriers to practice evaluation and explains why they can be seen as pseudoissues. Evaluation activities help practitioners become more aware of and sensitive to legitimate barriers to effective practice and pave the way for overcoming these barriers. Overview of Practice Evaluation Strategies Some of the criticisms about empirical practice arise in relation to single-system design as a practice method (Ivanoff et al., 1987). Although single-system design is often discussed as though it is the only available evaluation strategy, other evaluation methods exist. The methods reviewed here are single-system designs, goal attainment scaling, and systematic planned practice (SPP). Single-System Designs In single-system designs repeated measures of the target problem or desired outcome are collected on one client, or "system." Grinnell (1993) classified single-system designs as either evaluative or experimental. Designs for the purpose of evaluation focus on whether the client improved during treatment. Evaluation designs include the B and AB designs. In the AB design, the baseline phase (A) is followed by an intervention (B). Although client change can be tracked, it is not possible to say that change is a result of the intervention. Experimental designs are concerned with the causal link between client change and treatment. Experimental designs include multiple-baseline designs and designs in which the intervention is withdrawn, such as the ABA design. In the ABA design the intervention is withdrawn, but measurement of the target behavior continues in the second baseline phase, thus allowing an assessment of the relationship between intervention and client change. Debate about the use of single-system designs often centers on how aspects of the design - for example, withdrawal of the intervention - are incompatible with practice. Goal Attainment Scaling Goal attainment scaling, initially developed for use in mental health (Kiresuk & Sherman, 1968), has been used in family agencies, hospitals, and schools (Fiester, 1979; Harris, 1981; Rock, 1987; Woodward, Santa-Barbara, Levin, & Epstein, 1978). Goal attainment scaling involves selecting target behaviors and scaling these behaviors along five attainment levels from "much more than expected level of outcome" (+2) to "much less than expected level of outcome" (-2). The midpoint (0) is the "expected" level of outcome and is formulated by the worker and client as the outcome that can be realistically expected given the available resources. …