# Extrinsic Bayesian Optimization on Manifolds
## 4 Examples and Applications
### Positive Definite Matrices  
Lastly, we apply the eBO method to a regression problem with the response on positive definite matrices. Let \((z_{1},y_{1}),\ldots,(z_{n},y_{n})\) be \(n\) points from a regression model in which \(z_{i}\in\mathbb{R}^{d}\) and \(y\in M=SPD(p)\), where \(SPD(p)\) stands for \(p\) by \(p\) positive semi-definite matrices. We are interested in modeling the regression map between \(z\) and \(y\). Let \(K(\cdot,\cdot):\mathbb{R}^{d}\times\mathbb{R}^{d}\rightarrow\mathbb{R}\) be a kernel function defined on covariate space \(\mathbb{R}^{d}\). For example, one can take the standard Gaussian kernel. Let \(g(z)\in M\) be the regression map evaluated at a covariate level \(z\). We propose to estimate \(g\) as  
\[g(z)=\arg\min_{y\in M}\sum_{i=1}^{n}\frac{1}{h}K_{h}(z,z_{i})\rho^{2}(y,y_{i}), \tag{17}\]  
where \(h\) is the bandwidth parameter of the kernel function \(K\). We take \(\rho\) to be the extrinsic distance on the \(SPD(p)\) manifold. For the given input or covariate \(z\), we denote \(f_{n}(y)=\sum_{i=1}^{n}\frac{1}{h}K_{h}(z,z_{i})\rho^{2}(y,y_{i}))\) as the objective function and look for the minimizer \(g(z)\) as the weighed Frechet mean on the \(SPD(p)\) manifold. In order to calculate the extrinsic distance on the \(SPD(p)\) manifold, we choose the embedding \(J\) as the log-map on the matrix, which can map the \(SPD(p)\) into \(p\) by \(p\) real symmetric matrices \(Sym(3)\):  
\[\log:SPD(p)\to Sym(p). \tag{18}\]  
For example, let \(A\in SPD(p)\) with a spectral decomposition \(A=U\Lambda U^{-1}\); we have the log-map of \(A\) as \(\log(A)=U\log(\Lambda)U^{-1}\) where \(\log(\Lambda)\) denotes the diagonal matrix whose diagonal entries are the logarithms of the diagonal entries of \(\Lambda\). Moreover, embedding \(J\) is a diffeomorphism, equivariant with respect to the actions of \(GL(p,\mathbb{R})\), the \(p\) by \(p\) general linear group. That is, for \(h\in GL(p,\mathbb{R})\), we have \(\log(h\Lambda h^{-1})=h\log(A)h^{-1}\). On the other hand, the inverse of embedding \(J^{-1}\) is the exp-map on the matrix, which takes the exponential function on the diagonal entries under the spectral decomposition. Then we obtain the extrinsic distance of two matrices \(A_{1},A_{2}\in SPD(p)\):  
\[\rho(A_{1},A_{2})=\|\log(A_{1})-\log(A_{2})\|, \tag{19}\]  
where \(\|\cdot\|\) denotes the Frobenius norm of the matrix (i.e. \(\|A\|=(\mathit{Tr}(AA^{T})^{1/2})\)). In our analysis, we focus on the case of \(p=3\) and \(d=1\), which have important applications in diffusion tensor imaging (DTI), designed to measure the diffusion of water molecules in the brain. In more detail, diffusion represents the direction along the white matter tracks or fibers, corresponding to structural connections between brain regions along where brain activity and communications happen. DTI data are collected routinely in human brain studies. People are interested in using DTI to build predictive models of cognitive traits and neuropsychiatric disorders [45]. The diffusions are characterized in terms of diffusion matrices, represented by 3 by 3 positive definite matrices. The data set consists of 46 subjects with 28 \(HIV+\) subjects and 18 healthy controls. Those 3 by 3 diffusion matrices were extracted along one atlas fiber tract of the splenium of the corpus callosum. All the DTI data were registered in the same atlas space based on arc lengths, with 75 tensors obtained along the fiber tract of each subject, which has been studied in [45] in a regression and kernel regression setting. We aim to show the difference between the positive (HIV+) and control samples based on kernel regression results. We consider the arc length tensor as the covariant \(z\in\mathbb{R}\) and the diffusion matrix as \(y\in SPD(3)\) in our framework at each arc length \(z\). In more detail, the arc length variable \(z\) ranges from 0 to 48 with 75 different locations.