' s response to reviews Title : Use of clustering analysis in randomized controlled trials in orthopaedic surgery

and title are poor indicators of what actually happens in a study and thus would be a poor indicator of what the authors actually did or planned to do. Next, the term “cluster randomized” is not the only indicator phrase of an a priori awareness of trial investigators. For example, many planned a prior to account for clustering with statistical analyses in non-cluster-randomized studies. Finally, we searched for many indicators of clustering beyond this one term. Reviewer 1 Comment 4: What were prevalence rates of adjustment for clustering between binary and continuous primary outcomes? Author response: Thank-you for your comment. It was not one of the purposes of our paper to differentiate between clustering in binary (or categorical) vs continuous outcomes . “The primary objective of the present study was to determine the prevalence of reporting of the use of clustering analysis in RCTs published in the top five orthopaedic journals between 2006 and 2010. A secondary objective was to identify factors predicting the use or neglect of use of clustering analysis in the RCTs included in this study. ” (lines 108-112) But we have added a line in the discussion section that makes this recommendation for future research (lines 324-326). “Furthermore, more work is needed on sample size ca lculations and methods of accounting for clustering for binary and count d ata in clinical research. ” Thank-you. Reviewer 1 Comment 5: What proportion of included studies accounted for clustering while calculating sample size using either intra-cluster correlation coefficient (ICC) or design effect? To improve the power of study and reduce bias, it is crucial whether or not the sample size calculation (if reported) accounted for clustering. A trial is classified as meeting the sample size requirement if the sample size calculation is presented and clearly accounted for clustering (such as by using the intracluster correlation, coefficient of variation, or design effect). Author response: Thank-you for your comment. (lines 108-112) Although, we do discuss and emphasize the importance of this in the discussion section (Lines 310-328), the issue of appropriate sample size calculations in these studies was not one of the purposes of our paper and thus we did not extract this information. Also, as noted above, we have added a line to the discussion section that makes this recommendation for future research (lines 324-326). “Furthermore, more work is needed on sample size calculations and methods of a ccounting for clustering for binary and count data in clinical research. ” Thank-you again. Reviewer 1 Comment 5: Nature of analysis used to adjust for clustering was not differentially described by study authors. What percentage of studies carried out cluster level analysis by using summary measures (cluster level analysis), and what proportion of studies used individual level analysis adjusted for clustering. Since type of statistical adjustment for clustering mainly dependent upon number of clusters included in trial, it would be important that authors highlight the distribution of clusters in included RCTs. Author response: Thank-you for your comment. We have added a table (table 2) that describes the nature of the analysis to adjust for clustering. Reviewer 1 Comment 6: Minor Essential Revisions: As per consort guidelines for cluster RCT, it is imperative for any cluster RCT to estimate ICC value for similar future studies. How many studies did indeed report estimation of ICC? Author response: Thank-you for your comment. While the CONSORT guidelines for cluster randomized studies do indeed recommend that cluster RCTs report ICCs, the purpose of our study was not to assess the overall reporting in these included studies, but to determine what methodology was used if clustering was accounted for. Reporting quality is defined as the completeness of information contained within a written report of a piece of scientific work, whereas, methodological quality if the appropriateness of the methods used in the study[1]. Thus, we do not give the numbers of studies that report particular elements of clustering adjustments (e.g. ICCs). Instead, we examine the method used to account for clustering as was consistent with our objectives. Reviewer 1 Comment 7: It would be nice if authors could justify whether sample size of their own study was adjusted for clustering among 5 top orthopedic journals? Author response: Thank-you for your comment. We report in our methods section (line159-160) that “Clustering effects by journal were checked using ge neralized estimating equations (GEE). ”. We did not perform a sample size calculation for this study as it was descriptive and exploratory. Reviewer 1 Comment 8: Discretionary Revisions: Did authors use any CONSORT (Consolidated Standards of Reporting Trials) checklist for analyzing the reporting of clustered randomized controlled trials in order to determine how well each trial adhered to the published reporting guidelines? The extension to CONSORT for cluster randomized trials had indeed resulted in improvements in reporting of cluster RCTs. It is also likely that the extension had a greater impact in journals that actively endorsed (and explicitly enforced) the guideline. Therefore, did authors evaluate or determine whether CONSORT extensions for cluster RCT were endorsed by each participating journal or how they were enforced? Author response: Thank-you for your comment. In fact, the senior author on the submitted paper (Joel Gagnier) is a CONSORT group member and has participated in the development of several reporting guidelines. But, given that we were specifically interested in one aspect of the RCT methodology (accounting for clustering) and not reporting, we did not deem it to be necessary to look at the overall reporting quality in these studies. We did asses the risk of bias in these studies (their methodological quality) with excepted items but we did not report these findings. We would be happy to if the editors felt it to be necessary. Also, we are aware of the extension of the CONSORT statement to cluster randomized studies but again, we were not interested in reporting quality, we were interested in one method, the use of analyses or methods to account for clustering. If the study had clusters, which many of our included studies did, they had to account for variation effects due to these clusters. It is possible that some of the included studies could have accounted for clustering and not describe it, but in no single study did we suspect clustering analyses were done for which the methods were not reported. Of course, in some cases the finite details of the methods were missing, but we know it, accounting for clustering, was done in some general way. We did not assess the validity of the methods used, only if they were used. In the end, the details of the all methods, the global reporting quality of all study aspects, are likely better in endorsing and enforcing journals of CONSORT and related statements, but this was not an objective of our study. Reviewer 2 Comment 1: Discretionary Revisions: If the authors can provide the original dataset extracted from the paper, it would be much easier for the statistician to check the result. Author response: Thank-you for your comment. We would be happy to provide the dataset if you wish. Reviewer 2 Comment 2: In this article, the authors examine the current use of clustering analysis in randomized controlled trials (RCTs) published in the top five journals of orthopaedic surgery. The analysis included the effects of clustering of therapists, centers, year of publication, whether the study team included a statistician, whether the study team included an epidemiologist or clinical trials methodologists, the sample size of the study, which these factors usually be ignored in the CRTs. The logistic regression was conducted using both univariate and multivariate models. The result of the statistical analysis is explicitly showed in the paper. I would recommend accepting the article. Author response: Thank-you for your comment. Again, we would like to thanks the reviewers for their comments and recommendation which have improved our manuscript. Please let us know if you require any further revisions.