## 5 Design Aspects
### Sample Size Considerations  
Suppose we are interested in testing \(H_{0}:\Delta_{0}=0\), which posits that there is no difference between the effects of the novel CBT alone and the control intervention, and we want to determine the sample size to achieve some desired level of statistical power for this test. The sample size is determined as a function of several design parameters whose values must be pre-specified upfront:  
* \(\alpha=\) chance of a false positive result;\(\beta\) = chance of a false negative result;
* \(\widetilde{\Delta}_{0}=\) the \"clinically relevant\" mean treatment difference that we would not like to miss;
* \(\sigma=\) the presumed standard deviation of the primary outcome. Assuming equal (1:1) allocation, the total sample size \(n\) is determined to satisfy two major conditions: \(\alpha=\Pr(\text{Reject }H_{0}|\Delta_{0}=0)\) and \(1-\beta=\Pr(\text{Reject }H_{0}|\Delta_{0}=\widetilde{\Delta}_{0})\). Using the test statistic in Equation (6), \(H_{0}\) is rejected if \(|t|>t_{\alpha/2,n-3}\). One difficulty is that the distribution of this test statistic depends on engagement--via \(\overline{x}_{E}\) and \(S_{xx}\)--that are calculated for a particular value of the sample size, using individual engagement values that are unknown upfront. Therefore, we have a conundrum: for sample size planning we need summary statistics of engagement, which are derived from an experiment of a given size. If we have some prior knowledge on the probability distribution of engagement, we can use it to calibrate the sample size for a future study. A sensible approach for this purpose is Monte Carlo simulation. There are two major steps in the simulation process that should be implemented to calibrate the requisite sample size and quantify the associated uncertainty. **Step 1: Sample size for a given set of engagement measurements**  
Suppose, based on a pilot study, it is plausible to assume that individual values of engagement follow a normal distribution with mean \(\lambda\) and standard deviation \(\xi\). We simulate a vector of engagement values: \(\mathbf{X}=(X_{1},\dots,X_{M})\), where \(X_{i}\sim\) i.i.d. \(N(\lambda,\xi^{2})\) and \(M\) is some pre-specified large positive integer (say, \(M=200\)). Let \(m\leq M\) denote the sample size for the experimental group (the total sample size is \(n=2m\)). For a given \(m\) (say, \(m=25\)) we take the first \(m\) components of the vector \(\mathbf{X}\), i.e., \(\mathbf{X}^{(m)}=(X_{1},\dots,X_{m})\), calculate \(\overline{x}_{E}=\frac{1}{m}\sum_{i=1}^{m}X_{i}\) and \(S_{xx}=\sum_{i=1}^{m}(X_{i}-\overline{x}_{E})^{2}\). Next, we run 10,000 simulations to generate datasets from model (1) with the chosen parameter values \((\mu_{E},\mu_{C},\gamma,\sigma)\), total sample size \(n=2m\), keeping fixed within each simulation run the individual values of engagement and treatment assignment indicators. For each simulation, the test statistic is computed using Equation (6), and the test decision is recorded (reject \(H_{0}\), if \(|t|>t_{\alpha/2,2m-3}\); or fail to reject \(H_{0}\), if \(|t|\leq t_{\alpha/2,2m-3}\). The proportion of simulation runs that lead to rejection of \(H_{0}\) is taken as a Monte Carlo estimate of power for the given \(m\). The above procedure is repeated for different values of \(m\) (say, \(m=25,\ 26,\dots,\ M\)), and for each \(m\) the simulated power is obtained. The smallest sample size \(n=2m\) for which simulated power is equal to or exceeds the target level \(1-\beta\) provides the requisite sample size. A plot of power (\(y\)-axis) vs. sample size (\(x\)-axis) can be helpful to visualize the \(n\)'s that yield different values of power (80%, 90%, etc.) **Step 2: Distribution of the requisite sample size**  
The described Step 1 provides a single value of the sample size to achieve power \(=1-\beta\). However, this sample size is obtained for a fixed set of engagement values.