## 4 Bayesian Identification
### Hyperparameter Optimization  
The hyperparameter estimation plays a similar role to the model order selection in classic system identification methods and its essence is to determine a suitable model complexity based on measurement data. There exist several methods to optimize the hyperparameters, such as empirical Bayes, cross-validation, and \(C_{p}\) statistics methods [17]. Among them, the empirical Bayes method is a very natural one which utilizes the stochastic property of the output (\(\mathbf{Y}\)) conditioned on the hyperparameters. The empirical Bayes method relates to the concept of Bayesian evidence and the [PERSON]'s razor principle, i.e., the unnecessarily complex models are automatically penalized [24]. An alternative way to tune hyperparameters is to implement a fully Bayesian approach that is developed by utilizing stochastic simulation techniques, e.g., the Markov chain Monte Carlo (MCMC) method [25]. In such a context, the posterior distribution of the FRF and hyperparameters is recovered in sampled form, from which the sample mean and Bayesian interval are obtained. The empirical Bayes method does not require MCMC simulations and hence uses much less computation; therefore, it is preferred here. The full Bayes approach can be used to verify whether the empirical Bayes method is sufficient for the present problem, and it can be seen as a future interesting extension of the present work. As \(\mathbf{H}\) and \(\mathbf{N}_{\mathbf{Y}}\) are both Gaussian distributed, \(\mathbf{Y}\) follows a zero-mean complex Gaussian distribution with the covariance matrix  
\[\underline{\mathbf{\Sigma}}_{\mathbf{Y}}=\mathbf{\Psi}\underline{\mathbf{ \Sigma}}_{\mathrm{H}}\mathbf{\Psi}^{\mathrm{H}}+\underline{\mathbf{\Sigma}}_{N} \tag{22}\]The probability distribution of \(\mathbf{Y}\) is known up to \(\boldsymbol{\theta}\) and \(\kappa\), which implies that these hyperparameters can be obtained by the maximum likelihood identification based on the following cost function:  
\[\arg\min V(\boldsymbol{\theta},\kappa)=\underline{\mathbf{Y}}^{\mathrm{H}} \underline{\mathbf{\Sigma}}^{-1}\underline{\mathbf{Y}}+\ln\det\underline{ \mathbf{\Sigma}}_{\mathrm{Y}} \tag{23}\]  
Note that \(V(\boldsymbol{\theta},\kappa)\) is a non-convex function, and the gradient-based optimization may converge to a local solution. A concentrated cost function of \(V(\boldsymbol{\theta},\kappa)\) is developed in an attempt to improve the convergence. Let us rewrite \(V(\boldsymbol{\theta},\kappa)\) as  
\[V(\boldsymbol{\theta},\kappa)=\underline{\mathbf{Y}}^{\mathrm{H}}\underline{ \mathbf{\Sigma}}^{-1}\underline{\mathbf{Y}}+\ln\det\underline{\mathbf{\Sigma}} +D\ln\kappa \tag{24}\]  
\[\underline{\mathbf{\Sigma}}=\boldsymbol{\bar{\mathbf{\Sigma}}}_{\mathrm{H}} \mathbf{\Psi}^{\mathrm{H}}+\underline{\mathbf{P}} \tag{25}\]  
with \(\underline{\mathbf{\Sigma}}_{\mathrm{H}}\) having the same form as \(\underline{\mathbf{\Sigma}}_{\mathrm{H}}\) in Equation (16) except that \(\gamma_{r}\) is replaced by \(\rho_{r}=\gamma_{r}/\kappa\). The estimate of \(\kappa\) is obtained by fixing other parameters,  
\[\hat{\kappa}=\frac{1}{D}\underline{\mathbf{Y}}^{\mathrm{H}}\underline{\mathbf{ \Sigma}}^{-1}\underline{\mathbf{Y}} \tag{26}\]  
Let us insert \(\hat{\kappa}\) into Equation (24) to obtain \(V(\boldsymbol{\theta},\hat{\kappa})\), further define \(V_{1}(\boldsymbol{\theta})=\exp\left(V(\boldsymbol{\theta},\hat{\kappa})\right. /D-1)\), then  
\[V_{1}(\boldsymbol{\theta})=h(\boldsymbol{\theta})V_{0}(\boldsymbol{\theta}) \tag{27}\]  
where the parameter vector \(\boldsymbol{\theta}\) is redefined as \(\boldsymbol{\theta}=\left[\alpha,\beta,\rho_{0},\cdots,\rho_{n_{\boldsymbol{ \theta}}}\right]\) after removing \(\kappa\) and  
\[\left\{ \begin{array}{l}h(\boldsymbol{\theta})=\exp\left(\frac{1}{D}\ln\det \underline{\mathbf{\Sigma}}\right)\\ V_{0}(\boldsymbol{\theta})=\frac{1}{D}\underline{\mathbf{Y}}^{\mathrm{H}} \underline{\mathbf{\Sigma}}^{-1}\underline{\mathbf{Y}}\end{array} \right. \tag{28}\]  
It is noted that \(\hat{\kappa}=V_{0}(\boldsymbol{\theta})\).