# Integrated Model Selection and Scalability in Functional Data Analysis Through Bayesian Learning
## 3 Methods
### Variational Bayesian Inference
Equation (20) indicates that the eigenvectors of \(\Sigma_{\bar{Z}}\) are solely determined by the sum of Gram matrices \(\sum_{i=1}^{p}\Psi_{i}\), where \(\Psi_{i}=\Phi_{i}\Phi_{i}^{T}\), while the eigenvalues of \(\Sigma_{Z}\) have a negative correlation with the scale of \(\sum_{i=1}^{p}\Psi_{i}\), the prior \(\eta\operatorname{diag}(\beta)\), and data-dependent term \(\sigma^{-2}\). This is sensible because, for instance, large noise would result in large uncertainty in \(\bar{Z}\). In Equation (21), the data residuals, excluding the component scores, are projected into the \(K\)-dimensional space through the inner product, with \(\Phi_{i}\) and summed over all sample functions to calculate the mean weights. **Updates for the parameters of the posterior of the weights \(\mathcal{Q}_{W}\)**:  
Figure 1:9 \[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]0 \[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]1 \[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]2  
Equation (22) is similar to Equation (20), because it is correlated with \(\Phi_{i}\), its prior \(\operatorname{diag}(\beta)\otimes\operatorname{diag}(\alpha)\), and data-dependent terms \(\sigma^{-2}\) and \(Z_{i}\). In Equation (23), the data residual excluding the mean function is used to estimate the expectation of \(W\). **Updates for the parameters of the posterior of the component scores \(\mathcal{Q}_{Z_{i}}\)**:  
\[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]3 \[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]4 \[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]5 (25) \[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]6  
where \(H_{i}\) is a temporary variable denoting the Gram matrix of weighted kernel functions. \(W\Phi_{i}\), and \(\Sigma_{[W_{k},W_{j}]}\) denotes the covariance between \(W_{k}^{T}\) and \(W_{j}\) in \(\mathcal{Q}\). **Updates for the parameters of the posterior of the noise \(\mathcal{Q}_{\sigma}\)**:  
\[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]7 \[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]8 \[\mathcal{Q}_{Z}(Z) =\prod_{i}\mathcal{Q}_{Z_{i}}(Z_{i})=\prod_{i}\mathcal{N}(Z_{i}|\mu_{ Z_{i}^{\prime}},\Sigma_{Z_{i}}) \tag{6}\]9 \[\mathcal{Q}_{W}(W) =\mathcal{N}(\text{vec}(W)|\mu_{\text{vec}(W)},\Sigma_{\text{vec }(W)})\]0where \(H\) is a temporary variable that is updated by  
\[\mathcal{Q}_{W}(W) =\mathcal{N}(\text{vec}(W)|\mu_{\text{vec}(W)},\Sigma_{\text{vec }(W)})\]1  
Nearly noninformative (vague) priors, i.e., with almost zero \(a_{0},b_{0}\), introduce an inherent identifiability ambiguity in our formulation--specifically, in the product of the precision parameters \(\alpha,\beta\), and \(\eta\) (Equations (20) and (22)). In our model, scaling \(\alpha\) and \(\eta\) by a specific factor while inversely scaling \(\beta\) leaves the product (and hence the lower bound in Equation (5) unchanged. This inherent ambiguity can lead \(\alpha\), \(\beta\), and \(\eta\) to converge to extreme values, thereby challenging the numerical stability during optimization. To mitigate this issue, we adopt a heuristic constraint to ensure that the smallest values of \(\alpha\) and \(\beta\) remain within one order of magnitude of each other. Specifically, we enforce \(\left|\log_{10}^{\left(\frac{\text{min}(\alpha)}{\text{min}(\beta)}\right)} \right|\leq 1\).