What Is Missing in Counseling Research? Reporting Missing Data

Counseling researchers have increasingly demonstrated their ability to incorporate advanced statistical procedures into empirical research (e.g., Utsey, Giesbrecht, Hook, & Stanard, 2008). Despite evidence of statistical competencies, many counseling researchers fail to report missing data as part of their data analysis process, possibly indicating difficulty in understanding and interpreting this methodological procedure. Professional journals presume authors take appropriate steps to properly handle missing data, but most articles do not address this issue. Even though statisticians have made significant advances in applying missing data mechanisms since Rubin (1976) published his seminal work on the topic, there have been limited attempts to bring these concepts into the mainstream. In many instances, social scientists have struggled to make sense of these complex and often esoteric concepts (McKnight, McKnight, Sidani, & Figueredo, 2007). McKnight et al. (2007) postulated several reasons why the statistical literature has not been infused into social science research practices, including a lack of specific training with this particular statistical procedure and limited understanding of statistical software applications for handling missing data. Schafer and Graham (2002) added that procedures used to manage missing values were often not a component of the data analysis process. Whatever the reasons, implications for not addressing missing values are significant because of the serious threats to internal (e.g., statistical power) and external (e.g., generalizability of findings) validity (Acock, 2005; Croninger & Douglas, 2005; Fichman & Cummings, 2003; Hair, Black, Babin, Anderson, & Tatham, 2006; McKnight et al., 2007; Schafer & Graham, 2002). In a review of missing data in selected counseling journals, I investigated the prevalence of missing data in empirically based research articles in the Journal of Counseling & Development (JCD) and Counselor Education and Supervision (CES) from 2004 to 2008. JCD and CES were selected because of their status as premier counseling journals. In JCD, 14 of 66 (21%) articles referenced missing data on some level. Of these 14 articles, 11 mentioned missing data specifically. In the 14 studies, eight used listwise deletion, five used mean substitution, and one used the pairwise deletion procedure. This preliminary information is helpful in that it allows the reader to understand how the missing values were handled. Even though these could be the proper methods for addressing missing data, readers would benefit greatly from additional information, including the type of missing data (e.g., Were the missing data randomly distributed across the data set, or were they specific to one of the study variables?), the amount of missing data (e.g., the amount of missing data will provide some rationale for how the missing data should be managed), and why the specific deletion or imputation procedure was selected (Wilkinson & Task Force on Statistical Inference [TFSI], 1999). Access to this information as well as other reporting considerations provides a clearer picture of the data set, the data analysis steps, and the significance of the missing data on the study results. In the remaining 52 JCD articles, no information was provided on whether missing data existed. It is possible, yet highly unlikely, that all respondents answered each question for each study variable, that there were no data entry errors, and that no other methodological problems resulting in missing data occurred. On the basis of the information provided in these articles, readers could erroneously conclude that no data were missing, which would seem implausible given that missing data are problematic across all disciplines. McKnight et al. (2007) found that approximately 90% of the 300 articles they reviewed over a 3-year period had missing data, with the average amount more than 30% for the studies in the sample. …