## 2 Statistical Modeling and Some Theoretical Results
### Two-Sample \(t\)-Test  
It is useful to investigate statistical properties of a two-sample \(t\)-test, which is a conventional way to perform treatment comparison. With this approach, an investigator ignores information on engagement and models' experimental data as  
\[Y_{i}=\delta_{i}\mu_{E}+(1-\delta_{i})\mu_{C}+\varepsilon_{i},\ i=1,\ldots,n. \tag{14}\]  
The least squares estimates of \(\mu_{E}\) and \(\mu_{C}\) using a working model (14) are \(\overline{Y}_{E}\) and \(\overline{Y}_{C}\), the usual sample means. The treatment difference \(\mu_{E}-\mu_{C}\) is estimated as \(\overline{Y}_{E}-\overline{Y}_{C}\), which, under the true model (1), has a normal distribution with \(\mathrm{E}(\overline{Y}_{E}-\overline{Y}_{C})=\mu_{E}-\mu_{C}+\gamma\overline{ x}_{E}\) and \(\mathrm{var}(\overline{Y}-\overline{Y}_{C})=\frac{\sigma^{2}}{n/2}+\frac{ \sigma^{2}}{n/2}=\frac{4\sigma^{2}}{n}\). One may ask a question: what is the estimated under the true model (1) for which \(\overline{Y}_{E}-\overline{Y}_{C}\) can be useful? Note that \(\overline{Y}_{E}-\overline{Y}_{C}\) is biased for \(\mu_{E}-\mu_{C}\), and the bias term \(\gamma\overline{x}_{E}\) can be sizable if \(\gamma\
eq 0\) and \(\overline{x}_{E}>0\). However, \(\mathrm{E}(\overline{Y}_{E}-\overline{Y}_{C})=\Delta_{\overline{x}_{E}}=(\mu _{E}+\gamma\overline{x}_{E})-\mu_{C}\), which implies that the sample mean difference is an unbiased estimate of the contrast between the combined effect of the novel CBT + dCBT at the average level of engagement observed in the trial and the control condition. With a two-sample \(t\)-test, the error variance \(\sigma^{2}\) is estimated by the pooled sample variance  
\[S_{p}^{2}=\frac{1}{n-2}\left(\sum_{i=1}^{n/2}(Y_{i}-\overline{Y}_{E})^{2}+\sum _{i=n/2+1}^{n}(Y_{i}-\overline{Y}_{C})^{2}\right). \tag{15}\]  
Using direct algebraic derivations, it can be shown that the ANCOVA-based estimate of error variance \(\widehat{\sigma}^{2}\) from Equation (5) can be written as  
\[\widehat{\sigma}^{2}=\frac{1}{n-3}\Bigg{\{}(1-r^{2})\sum_{i=1}^{n/2}(Y_{i}- \overline{Y}_{E})^{2}+\sum_{i=n/2+1}^{n}(Y_{i}-\overline{Y}_{C})^{2}\Bigg{\}}, \tag{16}\]  
where \(r^{2}=\frac{S_{xy}^{2}}{S_{xx}^{2}S_{yy}}\), \(S_{xx}=\sum_{i=1}^{n/2}(x_{i}-\overline{x}_{E})^{2}\), \(S_{xy}=\sum_{i=1}^{n/2}(Y_{i}-\overline{Y}_{E})(x_{i}-\overline{x}_{E})\), and \(S_{yy}=\sum_{i=1}^{n/2}(Y_{i}-\overline{Y}_{E})^{2}\). Here, \(0\leq r^{2}\leq 1\) is the squared correlation coefficient (correlation between response and engagement in the experimental group \(E\)). From (15) and (16) one can see that numerically \(S_{p}^{2}\) may be less than \(\widehat{\sigma}^{2}\). For instance, if \(r^{2}=0\), then \(\widehat{\sigma}^{2}=\frac{n-2}{n-3}S_{p}^{2}\), which implies that \(\widehat{\sigma}^{2}>S_{p}^{2}\) (although the difference is very small, especially when \(n\) is large). On the other hand, for larger values of \(r^{2}\), \(\widehat{\sigma}^{2}\) will be less than \(S_{p}^{2}\). For example, suppose \(r=0.8\) and \(\sum_{i=1}^{n/2}(Y_{i}-\overline{Y}_{E})^{2}=\sum_{i=n/2+1}^{n}(Y_{i}-\overline {Y}_{C})^{2}\). Then \(\frac{\widehat{\sigma}^{2}}{S_{p}^{2}}=\frac{n-2}{n-3}\left(1-\frac{r^{2}}{2} \right)\approx 1-\frac{0.8^{2}}{2}=0.68\), which implies that \(\widehat{\sigma}^{2}\) is \(32\%\) lower than \(S_{p}^{2}\). Furthermore, under model (1), we have \(\frac{(n-3)\widehat{\sigma}^{2}}{\sigma^{2}}\sim\chi_{n-3}^{2}\) and \(\mathrm{E}(\widehat{\sigma}^{2})=\sigma^{2}\), which means that \(\widehat{\sigma}^{2}\) is an unbiased estimator of \(\sigma^{2}\). Under the same model, \(\mathrm{E}(S_{p}^{2})=\sigma^{2}+\frac{\gamma^{2}S_{XY}}{n-2}\), which means that \(S_{p}^{2}\) generally overestimates \(\sigma^{2}\). The usual two-sample \(t\)-statistic  
\[t=\frac{\overline{Y}_{E}-\overline{Y}_{\mathrm{C}}}{2S_{p}/\sqrt{n}} \tag{17}\]  
is inappropriate for testing \(H_{0}:\mu_{E}-\mu_{C}=0\) because the numerator of (17) is biased for \(\mu_{E}-\mu_{C}\). However, the test (17) is suitable for testing \(H_{0}:\Delta_{\overline{x}_{E}}=0\).