## 5 Design Aspects
### Sample size requirements for different estimands  
It is instructive to explore sample size requirements for research hypotheses involving different estimands. Let us assume that the underlying distribution of engagement is normal with \(\text{mean}=0.6\) and \(\text{SD}=0.3\), and the true model parameters \(\mu_{E}=-1\), \(\mu_{C}=0\), \(\gamma=-0.5\), and \(\sigma=1\). We consider testing \(H_{0}:\Delta_{x}=0\) for \(x=0,0.3,0.5,0.8\), and \(x=\overline{x}_{E}\).  
Figure 4 shows box-plot distributions of the smallest sample size to achieve 80% power (with a two-sided significance level \(\alpha=5\%\)) of the test of \(H_{0}:\Delta_{x}=0\), and Table 2 shows the corresponding summary statistics. As expected, the sample size is smallest when \(x=\overline{x}_{E}\). By choosing \(n=24\) (12 subjects per arm), we can be 90% confident that our trial has 80% power to reject \(H_{0}:\Delta_{\overline{x}_{E}}=0\) at the 5% significance level. For other values of \(x\), the requisite sample size is larger. For instance, for \(x=0.5\), to achieve 80% power with 90% confidence, one must choose \(n=30\). The similar number for \(x=0.3\) is \(n=50\), and for \(x=0\) it is \(n=114\).  
Note that, in general, when consider testing \(H_{0}:\Delta_{x}=0\), the sample size determination involves the value of \(\Delta_{x}=(\mu_{E}+\gamma x)-\mu_{C}\) under the alternative, which necessitates the choice of \(\mu_{E}\), \(\mu_{C}\), \(\gamma\), and \(x\). In addition, we would need to make an assumption on the underlying distribution of engagement, to account for an inherent uncertainty in the engagement pattern. By contrast, if the objective is to test significance of the contrast between the effect of the experimental treatment at the average observed value of engagement and the control, fewer assumptions are needed; in particular, only the value of the clinically meaningful mean difference (without assumptions on \(\gamma\), \(x\), and engagement distribution) would suffice.  
To draw a parallel between the results presented in Table 2 and the conventional sample size calculation, note that for \(x=\overline{x}_{E}=0.6\), \(\mu_{E}=-1\), \(\mu_{C}=0\), and \(\gamma=-0.5\), we have a mean treatment difference \(\Delta_{x}=-1-0.5\times 0.6=-1.3\). The required sample size for a two-sample \(t\)-test with parameters \(\Delta=-1.3\), \(\sigma=1\), \(\alpha=0.05\), and \(1-\beta=0.8\) can be obtained using standard statistical software, and it is \(n\approx 22\) (11 subjects per arm). Notice that \(n=22\) this is the median sample size (Q50) corresponding to \(x=\overline{x}_{E}\) in Table 2. In other words, for the chosen parameters, a trial design with \(n=22\) subjects has 80% power with 50% probability. If we want to achieve 80% power with 90% probability, the sample size should be increased to \(n=24\) (which is Q90 corresponding to \(x=\overline{x}_{E}\) in Table 2). Moreover, if we are interested in performing statistical testing at other levels of engagement (i.e., \(x\
eq\overline{x}_{E}\)), then the sample size should be further increased to maintain 80% power with due level of confidence.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{4}{c}{**Total Sample Size (\(n\))**} \\ \hline
**Distribution of Engagement (X)** & **Q50** & **Q80** & **Q90** & **Max** \\ \hline \(Beta(0.5,0.5)\) & 64 & 74 & 78 & 98 \\ \hline \(Uniform(0,1)\) & 82 & 92 & 98 & 130 \\ \hline \(Normal(0.6,\text{SD}=0.3)\) & 98 & 112 & 120 & 164 \\ \hline \(Normal(0.6,\text{SD}=0.2)\) & 174 & 194 & 206 & 258 \\ \hline \(\overline{\text{Q50}}=\text{median}\); Q80 = 80 th percentile; Q90 = 90 th percentile; Max = maximum. & & & \\ \end{tabular}
\end{table}
Table 1: Summary statistics for the total sample size distributions (cf. Figure 3).