# A Novel Technique for Semantic Segmentation of Hyperspectral Images Using Multi-View Features
## 3 Materials and Methods
### Canonical Correlation Analysis  
CCA is a statistical technique employed to examine the associations between two sets of variables. It seeks linear combinations of variables in each set, referred to as canonical variables, such that the correlation between corresponding canonical variables is maximized. CCA is particularly useful when exploring relationships between multivariate datasets, aiming to uncover latent structures and patterns that exist between them. The primary objective of CCA is :  
* Maximize Correlation: Find pairs of canonical variables, one from each set, such that the correlation between these pairs is maximized. These canonical variables capture the shared variance between the sets. * Sequential Independence: Subsequent canonical variables are sought in a way that ensures they are uncorrelated with previous canonical variables. This ensures that each new canonical variable captures distinct, independent information. Let \(X=[x_{1},x_{2},x_{3},\ldots,x_{N}]\in R^{d_{x}\times N}\) and \(Y=[y_{1},y_{2},y_{3},\ldots,y_{N}]\in R^{d_{y}\times N}\) represent two data matrices with number of instances as 'N' and \(d_{x},d_{y}\) represent the features of \(X\) and \(Y\) data matrices, respectively. CCA works towards finding '\(K\)' pairs of linear projections, which are called Canonical Vectors represented as \(W_{x}=[w_{x,1},w_{x,2},\ldots w_{x,K}]\in R^{d_{x}\times K}\) and \(W_{y}=[w_{y,1},w_{y,2},\ldots w_{y,K}]\in R^{d_{y}\times K}\). These vectors are such that the correlation between \(W_{x}{}^{T}X\) and \(W_{y}{}^{T}Y\) is maximized. If we consider a canonical vector \(w_{x}\in R^{d\times 1}\) for \(X\) and a canonical vector \(w_{y}\in R^{d\times 1}\) for \(Y\), the correlation coefficient \(\rho\) is maximised by CCA between \(X^{T}w_{x}\) and \(Y^{T}y_{x}\) as shown in Equation (1)  
\[\rho(X^{T}w_{x},Y^{T}y_{x})=\frac{w_{x}{}^{T}XY^{T}w_{y}}{\sqrt{(w_{x}{}^{T}XX ^{T}w_{x})(w_{y}{}^{T}YY^{T}w_{y})}} \tag{1}\]  
The Equation (1) in the constrained form is represented as:  
\[ \begin{split}\max_{w_{x},w_{y}}{w_{x}}^{T}XY^{T}w_{y}\\ s.t.w_{x}{}^{T}XX^{T}w_{x}=1,w_{y}{}^{T}YY^{T}w_{y}=1\end{split}  \tag{2}\]  
The Covariance matrix \(XX^{T}\) or \(YY^{T}\) is singular due to high feature dimensionality which results in undetermined optimization problem. Also, Regularization in CCA helps  
Figure 4: Salinas Valley image (**a**) Sample band image (**b**) Ground Truth. to deal with noise in the data. By introducing Equations (3) and (4), regularizations to the covariance matrices can help with the problem. \[\sum_{xx}=\frac{1}{N}XX^{T}+r_{x}I \tag{3}\]  
\[\sum_{yy}=\frac{1}{N}YY^{T}+r_{y}I \tag{4}\]  
where \(r_{x},r_{y}\) are regularization coefficients. It is assumed that the noise vectors are identically distributed, gaussian, and independent in the \(d_{x}\) and \(d_{y}\) columns of X and Y, respectively. As a result, all possible combinations of the covariances of the columns in X and Y will equal zero, except the covariance between a specific column vector and itself. This variance of each column of X and Y is labeled \(r_{x}\) and \(r_{y}\) are called the regularization coefficients. For choosing these parameters, let \(w_{x}^{i}\) and \(w_{y}^{i}\) be the weights calculated through CCA when data samples \(X_{i}\) and \(Y_{i}\) are removed, where \(i\in\{1,2,\dots,n\}\). The coefficients \(r_{x}\) and \(r_{y}\) are chosen through Grid Search [60] optimization of the following cost function [61] :  
\[max_{r_{x},r_{y}}[corr(\{X_{i}w_{x}^{i}\}_{i=1}^{n},\{Y_{i}w_{y}^{i}\}_{i=1}^{ n})] \tag{5}\]  
where 'corr' refers to the Pearson's Correlation Coefficient. Whenever a sample \(i\) is removed, the above cost function measures changes in \(w_{x}^{i}\) and \(w_{y}^{i}\) and seek optimal values of the regularization coefficients where the change is minimized [62]. Eigen Value decomposition (EVD) and Singular Value Decomposition (SVD) are two major approaches for calculating \(W_{x}\) and \(W_{y}\). 1. Generalised EVD is formulated using: \[