# Integrated Model Selection and Scalability in Functional Data Analysis Through Bayesian Learning
## Appendix E Fast Initialization
### Maximum Likelihood Estimation  
We apply maximum likelihood estimation for point estimates of \(\bar{Z},\beta,\sigma\). \[\bar{Z}^{*},\beta^{*},\sigma^{*}\leftarrow\arg\min_{Z,\beta,\sigma}\mathcal{P}, \tag{116}\]  
where \(\mathcal{P}=-\ln\Pr[Y|\bar{Z},\beta,\sigma]\). Conditioned on these estimates, we can calculate the expectation of \(Z\). **Optimization of \(\beta,\bar{Z}\)**We set the differentiation to zero, i.e., \(\frac{\partial\mathcal{P}}{\partial\mathcal{P}_{k}}=0\), and obtain  
\[\beta_{k}\leftarrow \begin{cases}\theta_{k},\text{if}\,\theta_{k}>0\\ \infty,\text{otherwise}\end{cases}  \tag{108}\]  
where  
\[\theta_{k} =\left(\frac{\sum_{i=1}^{p}s_{ik}^{2}}{\sum_{i=1}^{p}(q_{ik}^{2}- s_{ik})}\right) \tag{109}\] \[q_{ik} =\[PERSON]_{ik}\mathcal{C}_{i/_{k}}^{-1}(Y-Z\[PERSON]_{i})^{T}\] (100) \[s_{ik} =\[PERSON]_{ik}\mathcal{C}_{i/_{k}}^{-1}\[PERSON]_{k}^{T}\] (101) \[\mathcal{C}_{i/_{k}} =\mathcal{C}_{i}-\[PERSON]_{ik}^{T}\beta_{k}^{-1}\[PERSON]_{ik}\] (102) \[\mathcal{C}_{i} =\[PERSON]_{i}^{T}\text{diag}(\beta^{-1})\[PERSON]_{i}+\sigma^{2}I=\sum_{k =1}^{K}\[PERSON]_{ik}^{T}\beta_{k}^{-1}\[PERSON]_{ik}+\sigma^{2}I \tag{103}\]  
We differentiate \(\mathcal{P}\) with respect to \(\bar{Z}\) and zero the derivative, i.e., \(\frac{\partial\mathcal{P}}{\bar{Z}}=0\), to obtain  
\[\bar{Z}\leftarrow\sum_{i=1}^{p}\[PERSON]{(}Y_{i}\mathcal{C}_{i}^{-1}\[PERSON]_{i}^{T} \[PERSON]{)}\left(\sum_{i=1}^{p}(\[PERSON]_{i}\mathcal{C}_{i}^{-1}\[PERSON]_{i}^{T})\right) ^{-1} \tag{104}\]  
We approximate Equation (104) by \(\bar{Z}_{A}\leftarrow\sum_{i=1}^{p}\Bigl{(}Y_{i}\mathcal{C}_{i}^{-1}\[PERSON]_{iA} ^{T}\[PERSON]{)}\Bigl{(}\sum_{i=1}^{p}(\[PERSON]_{iA}\mathcal{C}_{i}^{-1}\[PERSON]_{iA}^{T} )\[PERSON]{)}^{-1}\) and \(\bar{Z}_{B}\gets 0\). This way, we can apply the update with only the active basis functions.  
**Optimization of \(\sigma\):**  
We use EM to optimize \(\sigma\). In the E-step,  
\[\mathbb{E}_{\mathcal{Q}_{2}}[\bar{Z}_{i}] \leftarrow\sigma^{-2}(Y_{i}-Z\[PERSON]_{i})\Phi_{i}^{T}S_{i} \tag{105}\] \[\mathbb{E}_{\mathcal{Q}_{2}}[\bar{Z}_{i}^{T}\bar{Z}_{i}] \leftarrow\mathcal{S}_{i}+\mathbb{E}_{\mathcal{Q}_{2}}[\bar{Z}_{i}]^{T} \mathbb{E}_{\mathcal{Q}_{2}}[\bar{Z}_{i}], \tag{106}\]  
where \(\mathcal{S}_{i}=(\
abla_{i}\sigma^{-2}+\text{diag}(\beta))^{-1}\). In the M-step,  
\[\sigma^{-2} \leftarrow\frac{\sum_{i=1}^{p}\mathbb{E}_{\mathcal{Q}_{2}}\bigl{[} ||Y_{i}-(\bar{Z}_{i}+\bar{Z})\Phi_{i}||_{2}^{2}\bigr{]}}{\sum_{i=1}^{p}N_{i}}\] \[=\frac{\sum_{i=1}^{p}(Y_{i}-\bar{Z}\Phi_{i})(Y_{i}-\bar{Z}\Phi_{i} -2\Phi_{i}^{T}\mathbb{E}_{\mathcal{Q}_{2}}\bigl{[}\bar{Z}_{i}\bigr{]}^{T})^{T} +\text{Tr}(\mathbb{E}_{\mathcal{Q}_{2}}\bigl{[}\bar{Z}_{i}^{T}\bar{Z}_{i} \bigr{]}\Psi_{i})}{\sum_{i=1}^{p}N_{i}} \tag{107}\]  
The optimization iterates between the E-step Equations (105) and (106) and the M-step Equation (107). In practice, we need only \(\mathbb{E}_{\mathcal{Q}_{2}}[\bar{Z}_{iA}],\mathbb{E}_{\mathcal{Q}_{2}}[\bar{ Z}_{iA}^{T}\bar{Z}_{iA}]\), and \(\mathcal{S}_{iA}\), and they can be calculated using the \(K^{(\text{a})}\) active basis functions. Thus, similarly to [39], all computations can be operated with only the active basis functions and thus it is computationally efficient. This is described in Algorithm 2.