Published versus unpublished dissertations in psycho‐oncology intervention research

There are conflicting opinions about whether gray literature, including unpublished doctoral dissertations, should be included in systematic reviews and meta-analyses [1]. On the one hand, doctoral dissertations are not peer reviewed in the same fashion as published journal articles; on the other hand, they are subjected to intense scrutiny by dissertation committees, and their reporting is typically quite thorough. Often publication status is used as a convenient proxy for methodological quality [2] and this practice may have some merit. However, even peer-reviewed journals differ in their methodological rigor, and published articles vary in their reporting quality due to differences in journal space limitations. Finally, some argue for the high quality of doctoral dissertations, asserting, “often the culmination of months or years of work, the dissertation is the concrete manifestation of a doctoral student’s best thinking, guided and refined by the sage suggestions of the student’s committee members” [3, p. 537]. The small number of empirical studies evaluating the quality of gray literature has generally attested that dissertations actually have stronger methodology than published studies [4] or that there is limited evidence that trials in the gray literature are of lower quality than published trials [5]. However, it is unclear whether these conclusions apply to studies in psycho-oncology research. Because searching for and retrieving gray literature, including dissertations, is time-consuming [6], decisions to include them in reviews are complex and should be based on empirical evidence [7]. In a prior, larger review of 488 published and unpublished projects investigating psychosocial interventions for cancer patients we characterized the methodological quality of this literature as a whole and focused on trends over time (Moyer, Sohl, Knapp-Oliver, Schneider, in press). We found that particular strengths of this research, present in a majority of projects, included using randomized designs, testing for baseline group equivalence, and monitoring treatment, the use of which rose significantly over time. In that review we identified a subset of 107 projects that included at least one doctoral dissertation, suggesting that this is a fruitful area for budding investigators. Sixty-five of these projects, however, included dissertations that remained unpublished, prompting the question of whether they were of lower quality than projects with published results. In this review we focus on the characteristics and methodological quality of just these 107 published and unpublished dissertations. Because prior reviews have contrasted unpublished dissertations with published studies in general, they have not explored the potential for publication bias resulting from only some dissertations making their way into the published literature. Thus, this review, in comparing only published versus unpublished research originating from doctoral dissertations, may shed some light on reasons for differential publication. It may also be that publication was more likely when the dissertation reported significant findings. This “publication bias” is a well-documented problem [8] and can result from authors selectively submitting, or from journals selectively accepting, those manuscripts that show significant results [9]. Thus, an additional goal of this review was to examine whether dissertations that reported significant findings were more likely to be published.