Evaluating Intervention Effects in Single‐Case Research Designs

Single-case research design (SCRD) is a rigorous method for evaluating treatment effects (Horner et al., 2005; Shadish, Cook, & Campbell, 2002) that is promoted by counselors across settings (Lenz, 2015; Lundervold & Belwood, 2000). Although the use of SCRDs is not new, interest in the analysis and interpretation of SCRD results has increased (Gast, 2010; Parker, Vannest, & Davis, 2011; Smith, 2012). This momentum is due, in part, to the emphasis on higher-quality research and evidence-based practice (Lenz, 2013; Parker & Vannest, 2009; Parker, Vannest, & Davis, 2014; Sharpley, 2007), with the increased use of effect size (ES) calculations for quantifying results receiving notable attention (Parker & Brossart, 2003; Parker, Vannest, & Davis, 2011). In this article, we describe the application and interpretation of ESs within SCRDs for increasing rigor in research, informing practice, and demonstrating counseling outcomes. An ES is a quantitative index of practical significance that estimates the meaningfulness of change associated with an intervention. Although visual analysis remains the most widely used analysis for SCRD (Smith, 2012), visual analysis can be supplemented with an ES to provide standardized and reliable results that contribute to evidence-based practices. Therefore, when counselors provide credible ES estimations in addition to visually analyzed data, they improve the credibility, reliability, and defensibility of their findings. This article illustrates general guidelines for four techniques that counselors can use when evaluating the outcomes of SCRDs. The article is organized by the steps involved in using ES: determining functional relationships, internal validity, and experimental control; calculating ES; and interpreting ES. Steps Involved in Using ES Step 1: Determining Functional Relationships, Internal Validity, and Experimental Control American Statesman Henry Clay (1777-1852) stated, "Statistics are no substitute for judgment." This statement is particularly appreciated in SCRD analysis, in which the first step is not to calculate, but to evaluate. The clinical judgment of the counselor is required to evaluate the design and the data to decide if a functional relationship is demonstrated. A functional relationship is considered present when a believable and consistent pattern of change is demonstrated in target outcomes by a systematic manipulation of an independent variable (Kennedy, 2005). Some have suggested that three or more replications are required to establish this functional relationship (Horner et al., 2005; Kratochwill et al., 2013). However, this determination is ultimately based on a synthesis decision about changes in the mean or level between phases, the immediacy of the change, the nature of the change, the presence and direction of trend in the baseline and intervention, and the degree of variability. Determining that a functional relationship exists between the counseling treatment and the outcome variable is recommended prior to calculating an ES. Experimental control is critical in empirical SCRD research, but there are no set rules for replicating effects, especially in clinical practice. If replication of effect is not established, the ES may indicate the size of a change but not the reason for the change. For example, if a counselor investigates the effect of cognitive behavior therapy on reported client anxiety, the counselor may begin by taking repeated measures during a baseline period. The counselor will then introduce the treatment and continue data collection. These time-series data permit the calculation of an ES on the data collected. However, the resulting number cannot tell the counselor if the change was a result of the cognitive behavior therapy or something else, such as an undisclosed change in medication. The "something else" is any alternative explanation for the behavior change, and these alternative explanations are typically related to an uncontrolled threat to the internal validity of the study. …