Plowing forward: The incremental development of knowledge

As all researchers know, knowledge advancement is methodical and incremental. For the practice world, the research enterprise often seems piecemeal, halting, and frustratingly slow: Each individual study seems to address only a small piece of the big picture; findings from one study often contradict those of earlier, seemingly similar, studies; and conclusions often are tentative. When a body of research yields inconsistent findings, readers are left to wonder, "Is the treatment not effective, or are the studies unable to detect real effects?" The professional literature needs to be very clear about the extent to which recommended treatments have evidentiary support. Established knowledge needs to be distinguished from reasonable hypotheses and mere assertions (Shonkoff, 2000). Three articles in this issue add important knowledge about widely used interventions: intensive family preservation services (IFPS), mediation, and treatment for alcohol dependence. These articles demonstrate how details of research methodology may affect findings and confidence in conclusions we draw. And they offer important lessons in the incremental development of knowledge. Equivocal findings from earlier studies challenged the hope and assumption that IFPS was effective in preventing out-of-home placements of high-risk children. Kirk and Griffith re-examined the ability of IFPS to prevent out-of-home placements of children in abusive or neglectful families, and the study yielded findings that are inconsistent with those of earlier research. The reason may be due to several aspects of the methodology. First, this study examined only cases that fit the intended clients--families at highest risk--for the specific treatment. Subjects comprised a statewide, six-year, archival population of high-risk child protective services children. In many other studies, lower-risk families received the treatment. Indeed, Kirk and Griffith found that treatment effects increased as risk increased. Second, the study ensured a high degree of treatment fidelity among service providers. Conclusions about a treatment's effects cannot be drawn unless that treatment received a fair test--that is, the treatment was actually implemented, and it was implemented as designed. Variation among providers with respect to actual services delivered is a vexing concern to service delivery and research. Third, this study used a retrospective, population-based design that selected cases for comparison on the basis of standardized instruments. And finally, this study carefully controlled for other risk factors that affect the outcome and placement rates and that may not have been sufficiently controlled in earlier studies, thereby adversely affecting findings. Findings led the authors to conclude that IFPS outperformed traditional child welfare services, with success defined as placement prevention: IFPS significantly reduced placement rates or delayed placements of children compared with children of the same risk level but who received traditional child welfare services. Once again, a study yields results at odds with earlier research--results that support the effectiveness of IFPS. And again, we ask, "What do we make of the findings of an individual study?" For researchers, confidence in conclusions rises with methodological rigor, particularly internal and external validity. The authors sought to minimize methodological problems evident in earlier research and provided detailed, explicit rationales for their design decisions. Such details heighten the confidence readers can place in conclusions and provide a standard for studies that seek to replicate and extend their findings. Although widely used as an intervention for interpersonal conflict, mediation and its underlying principles of justice and trust have been understudied. Ashford and Faith examined the effects of dispute resolution methods on the attitudes of parents in child-dependency disputes. The study also examined theoretical predictions from the psychology of justice and trust literature in relation to parents' attitudes regarding dissatisfaction with the juvenile court system, the fairness of the third parties (social work mediators or judges), and the degree of settlement achieved. …