## 3 Analyzing Experimental Data: An Illustrative Example  
Figure 1 visualizes a dataset from a hypothetical clinical trial with \(n=50\) subjects simulated from model (1) with parameters \(\mu_{E}=-0.5\), \(\mu_{C}=0\), \(\gamma=-0.5\), and \(\sigma=1\). Individual values of engagement in the experimental group were generated from a logit-normal distribution as follows: \(X=\frac{\exp{(U)}}{1+\exp{(U)}}\), where \(U\sim Normal\left(\log\left(\frac{0.6}{0.4}\right),1\right)\). With this approach, we assure that \(X\) is between 0 and 1, with \(\mathrm{E}(X)\sim 0.6\) and \(\mathrm{SD}(X)\sim 0.2\). The sample summary statistics for engagement were \(\overline{x}_{E}=0.6\) and \(S_{xx}=1.13\). Let us analyze these data using ANCOVA, as described in Section 2.1. We have \(\overline{Y}_{E}=-1.04\), \(\overline{Y}_{C}=-0.18\), \(S_{xx}=1.13\), \(S_{xy}=31.12\), and thus \(\widehat{\gamma}=\frac{S_{xy}}{S_{xx}}=-1.788\)  
Figure 1: An example of experimental data from a trial of \(n=50\) subjects (25 per group). Group 1 (experimental) observations are red circles; group 2 (control) observations are green triangles. ANCOVA-fitted linear regression (solid black line) is \(\widehat{\mu}_{E}+\widehat{\gamma}x\). \(\widehat{\mu}_{E}=\overline{Y}_{E}-\widehat{\gamma}\overline{x}_{E}=0.03\), \(\widehat{\mu}_{C}=\overline{Y}_{C}=-0.18\), and \(\widehat{\sigma}^{2}=0.899\). The sample correlation between engagement and response in the experimental group is \(r=\frac{S_{Y_{0}}}{\sqrt{S_{xx}S_{yy}}}=-0.34\). The value of the \(t\)-statistic for testing \(H_{0}:\mu_{E}-\mu_{C}=0\) using Equation (6) is \(t=0.346\), with the corresponding 2-sided \(p\)-value = 0.731, implying that \(\mu_{E}\) and \(\mu_{C}\) are not statistically different. The point estimate of \(\mu_{E}-\mu_{C}\) is \(\widehat{\mu}_{E}-\widehat{\mu}_{C}=0.207\), and the corresponding 95% confidence interval is \((-1.00,1.41)\), which covers both zero and the true value of the mean difference \(\mu_{E}-\mu_{C}=-0.5\). The value of the \(t\)-statistic for testing \(H_{0}:\gamma=0\) using Equation (8) is \(t=-2.005\), with the corresponding two-sided \(p\)-value = 0.051, a marginally significant result. The 95% confidence interval for \(\gamma\) is \((-3.56,0.01)\). The value of the \(t\)-statistic for testing \(H_{0}:\Delta_{\overline{x}_{E}}=0\) using Equation (12) is \(t=-3.24\), with the corresponding two-sided \(p\)-value = 0.0022, which indicates reasonably good evidence against \(H_{0}\). The 95% confidence interval for \(\Delta_{\overline{x}_{E}}\) is \((-1.40,-0.33)\), which covers the true value \(\Delta_{\overline{x}_{E}}=-0.5-0.5\times 0.60=-0.8\). The smallest level of engagement that leads to statistically significant difference between two groups can be found by solving an inequality for the upper limit of the 95% confidence interval for \(\Delta_{x}=(\mu_{E}+\gamma x)-\mu_{C}\); that is: \(0.03-1.78x+0.18+2.01\times 0.944\sqrt{\frac{4}{50}+\frac{(x-0.6)^{2}}{1.13}}<0\). The solution is \(0.4533<x<207.74\), for which only the lower limit is relevant. Thus, we conclude that \(\sim\)46% is the smallest level of engagement required to obtain a statistically significant (using two-sided \(\alpha=0.05\)), larger response in the experimental group compared to the control. A 95% confidence interval for the expected value of the response for a given level of engagement \(x\) is found as \(0.03-1.78x\pm 2.01\times 0.944\sqrt{\frac{2}{50}+\frac{(x-0.6)^{2}}{1.13}}\). For instance, if a subject successfully completes 80% of the dCBT program (\(x=0.8\)), the 95% confidence interval for the mean response is obtained as \((-0.87,0.23)\). A 95% prediction interval for a new observation \(Y(x)\) is found as \(0.03-1.78x\pm 2.01\times 0.944\sqrt{1+\frac{2}{50}+\frac{(x-0.6)^{2}}{1.13}}\). For \(x=0.8\), the 95% prediction interval is \((-2.4,1.77)\). Suppose an investigator decides to perform statistical inference using a two-sample \(t\)-test, as described in Section 2.2.