## 6 Conclusions and Future Work  
To obtain valid results for an RCT, both the study design and the model for statistical analysis must be chosen judiciously. If the primary outcomes in the two groups are normally distributed with a common standard deviation, then a two-sample \(t\)-test is appropriate for testing the significance of the treatment mean difference. Its application rests on an important assumption that the average values of patient characteristics are approximately equal in the experimental and control groups (which is a reasonable assumption for a randomized trial). In many clinical trials, there are important covariates that are correlated with a primary outcome. Adjusting for these covariates in the analysis can improve precision for estimating treatment effects. For a linear model with normal error terms, ANCOVA is a valid and well-established approach to analyze RCT data, and, in fact, it is advocated by the health authorities [14; 15]. When properly applied, ANCOVA leads to the minimum variance of unbiased estimates of the model parameters, and it is potentially more powerful for testing treatment difference than a simple (undadjusted) analysis using a two-sample \(t\)-test. Furthermore, ANCOVA gains in efficiency are more pronounced for larger values of correlation between the primary outcome and the covariates of interest. In the present paper, we explored these ideas in a context of an RCT evaluating cognitive behavioral therapies, assuming that engagement (quantified as the percentage of successfully completed trainings, measured without error) is an influential covariate. Our findings are in good correspondence with the already available statistical theory. One key contribution of our paper (which, to our knowledge, has not been addressed previously) is the investigation of statistical power and sample size of ANCOVA for different levels and patterns of engagement. A two-sample \(t\)-test matches one specific goal of ANCOVA, namely testing significance of the treatment difference at the average level of engagement observed in the study. In this case, the two-sample \(t\)-test and ANCOVA generally yield  
Figure 5: Distributions of power of the test of significance of the slope (top panel) and power of ANCOVA \(t\)-test of \(H_{0}:\Delta_{0}=0\) (bottom panel). Both tests used a two-sided significance level \(\alpha=5\%\) (two-sided). Responses were simulated from the model (1) with \(\mu_{E}=-1\), \(\mu_{C}=0\), \(\gamma=-0.5\), and \(\sigma=1\). Engagement values were simulated from a normal distribution with \(\text{mean}=0.6\) and \(\text{SD}=0.3\). similar power (which can be observed from our simulation results in Figure 2). However, the average level of engagement is not known upfront, and investigators may wish to compare treatment effects at several levels of engagement (including the average to be observed in the study), to enable generalization of the RCT results to a broader population. Our simulation results demonstrate that larger sample sizes may be required to achieve a given value of power than one would typically expect. The pattern of engagement, the level of engagement at which the treatment contrast is estimated, the value of the linear slope, the mean treatment effects and the standard deviation of the response have an impact on statistical power in this case. Therefore, it is essential that the primary research question is clearly formulated upfront to ensure that the experiment is adequately powered. While our considered approach in this paper was for a rather simple model (a single covariate representing engagement in the experimental group, measured on a continuous scale from 0 to 1), it captures an essential feature of experiments evaluating complex interventions--the need for considering patient engagement in both the design and analysis of the experimental data. However, our assumed model can be challenged on a number of grounds. First, it is rather stringent and it may not hold on the boundary when the engagement is equal to zero.