## 4 A Framework for Feature Curves on SPD Manifold
### Features Curves on SPD Manifold  
Although ROI is a semantic feature region in a frame image, such a feature region is directly composed of image pixels. From the perspective of image recognition, such features are original and rough. Our proposed framework uses a regional covariance descriptor to extract the SPD feature matrix from ROI, so as to transform a family of time series of ROIs into a family of feature curves on the Riemannian manifold. The specific methods are as follows:  
For a square ROI \(ROI=[r_{1},r_{2},\cdots,r_{d}]\in R^{D\times d}\), we can simply compute the corresponding RCD by  
\[C=\frac{1}{d}\underset{i=1}{\overset{d}{\sum}}(r_{i}-\mu)(r_{i}-\mu)^{T}, \tag{23}\]  
where \(C\in R^{D\times D}\), \(\mu=\frac{1}{d}\underset{i=1}{\overset{d}{\sum}}r_{i}\). When the ROI is no longer square, or the size is not uniform and the shape is irregular, we use each pixel of the ROI to generate a feature vector to calculate the covariance matrix. For a ROI with \(\lambda\) pixels, each pixel generates a \(D\)-dimensional vector. The ROI can be represented as \(\{v_{1},\cdots,v_{\lambda}\}\in R^{D\times\lambda}\), and the corresponding RCD is given by  
\[C=\frac{1}{\lambda}\underset{i=1}{\overset{\lambda}{\sum}}(v_{i}-\mu_{v})(v_ {i}-\mu_{v})^{T}, \tag{24}\]  
where \(C\in R^{D\times D}\), \(\mu_{v}=\frac{1}{\overset{\lambda}{\lambda}}\underset{i=1}{\overset{\lambda }{\sum}}v_{i}\). The method of generating feature vectors is open and can vary with different applications. For example, each pixel can generate a nine-dimensional feature vector, which is composed of RGB values and the first order gradients of RGB values in X and Y directions, respectively. By doing so, no matter how different the shape and size of ROI are, the size of the SPD feature matrix generated by RCD is certainly \(9\times 9\). In this way, all the resulting curves are on the SPD manifold shared the same dimension, avoiding the influence of sizes of original ROIs. Following [3], to avoid the singularity, we adjusted the original RCD as \(C*=C+\xi I\), where \(I\) is an identity matrix and \(\xi\) is \(10^{-3}\times tr(C)\). Then, we followed the information geometry theory [37] to transform the Gaussian model \(\mathcal{N}(\mu,C*)\) into a \((D+1)\times(D+1)\) dimensional SPD matrix as the final RCD:  
\[\mathcal{N}(\mu,C*)\sim X=\left[ \begin{array}{cc}C*+\mu\mu^{T}&\mu\\ \mu^{T}&1\end{array} \right], \tag{25}\]  
which means the space of \(D\)-dimensional Gaussian models has been embedded into the SPD manifold \(Sym_{++}^{D+1}\). By doing so, one \(ROI_{i,j}\) becomes an SPD matrix \(X_{i,j}\), and a family of time series of ROIs \(\{\mathbb{R}\mathbb{O}\mathbb{I}_{i}\}_{i=1}^{m}=\left\{ROI_{i,1},\cdots,ROI _{i,L_{i}}\right\}_{i=1}^{m}\) are transformed as a family of time series of their embedding SPD matrices \(\{\mathbb{X}_{R_{i}}\}_{i=1}^{m}=\left\{X_{R_{i},1},\cdots,X_{R_{i},L_{i}} \middle|X_{i,j}\in Sym_{++}^{D+1}\right\}_{i=1}^{m}\). Consider that the geometry of RCD generated from ROI is a point on SPD manifold. A time series of SPD matrices \(\mathbb{X}=\{X_{1},\cdots,X_{N}\}\) can be defined as a feature curve \(\Gamma(t)_{0\leq t\leq N-1}\) on the SPD manifold that passes through all SPD matrices belonging to \(\mathbb{X}\) in sequence from \(\Gamma(0)=X_{1}\) to \(\Gamma(N-1)=X_{N}\). Among them, two adjacent SPD matrices are connected by geodesics:  
\[C_{\Gamma(t)\to\Gamma(t+1)}\leftarrow\Gamma_{X_{t+1}}^{X_{t+2}},\ t=0,\cdots,N-2, \tag{26}\]  
where \(\Gamma_{X_{t+1}}^{X_{t+2}}(T)=X_{t+1}^{\frac{1}{2}}exp\bigg{(}T\log X_{t+1}^{ -\frac{1}{2}}X_{t+2}X_{t+1}^{-\frac{1}{2}}\bigg{)}X_{t+1}^{\frac{1}{2}},0<T\leq 1\). In other words, the feature curve representing a time series of ROIs is spliced by multiple geodesics one by one.