## 4 A Framework for Feature Curves on SPD Manifold
### Stringing Time Series of ROIs Based on Motion Estimation  
Given a k-frames video \(\mathbb{F}=\{F_{1},\cdots,F_{k}\}\), where \(F_{i}\) is the image matrix, the spatial features of video are reflected within the frame image of a video. Our proposed framework takes the ROIs of the frame image as the major spatial feature of video. ROI is originally a concept in video encoding. In video encoding, the image quality of non-concerned regions can be sacrificed, and only high-resolution coding can be carried out for key regions, i.e., ROI, to meet the requirements of users' high-definition video monitoring, while saving network bandwidth, processing time, and video storage space. ROI can be square, round, irregular shape, and so on. In our framework, we first extract a key frame from video, then determine a group of ROIs as a pattern. The ROI possesses semantic features and varies according to specific applications. As an algorithm framework, the solution of ROI selection and ROI detection are open. Meanwhile, the weight and number of ROIs can be adjusted to optimize the algorithm. The temporal features of video are reflected in the temporal correlation between video frames. Given a pattern of \(m\) ROIs \(\{ROI_{i}\}_{i=1}^{m}\), our proposed framework takes the \(m\) corresponding time series of ROIs as the spatiotemporal feature of video. Moreover, we use the motion estimation method in video compression coding to trace ROIs. Tracing can be forward, backward, or bidirectional, depending on the position of key frames in the frame sequence. Taking backward tracing as an example, we introduced a specific strategy for stringing time series of ROIs in the following, including the continuous frame extraction strategy and inter-frame extraction strategy. Let \(ROI_{i,1}\) represent the \(i\)-th ROI in the key frame \(F_{s}(1\leq s\leq k)\). For the continuous frame extraction strategy, we take \(F_{s}\) as starting frame for motion estimation and trace the region closest to \(ROI_{i,1}\) in the next frame \(F_{s+1}(s\leq s+1\leq k)\) as \(ROI_{i,2}\). The difference between regions can be calculated by Equations (14)-(16). Then, frame \(F_{s+1}\) becomes the new starting frame for next motion estimation, and so on. Each ROI traced is preserved in a time series of ROIs. However, the ROIs between adjacent frames may be too similar in some smoothly changing videos and preserving ROIs per frame will cause data redundancy. To tackle this problem, we employed an inter-frame extraction strategy. This strategy still traces ROIs per frame, but if the differences between the traced ROI and \(ROI_{i,1}\) in \(F_{s}\) is below a certain lower limit \(\tau\), the traced ROI is not preserved and the starting frame is kept at \(F_{s}\). Only when the differences between the traced ROI in \(F_{s+\eta}\) and \(ROI_{i,1}\) exceed \(\tau\) can the traced ROI be strung into the time series as \(ROI_{i,2}\); then, the starting frame is replaced by \(F_{s+\eta}\) for next motion estimation. Since both strategies will face the distortion of the prediction result due to the accumulation of error caused by each estimation, our proposed framework stops motion estimation in the ending frame \(F_{e}\) where the error accumulates to an upper limit \(\mu\). Thus far, we call it a cycle of motion estimation. After a cycle, we go back to the space domain to detect the ROIs in \(\{F_{e+1},\cdots,F_{k}\}\) and redefine the starting frame to repeat a new cycle of motion estimation. By looping the cycle until the end of the video to find all the \(i\)-th ROIs on the video timeline, we can construct a time series of the \(i\)-th ROIs \(\mathbb{ROI}_{i}=\left\{ROI_{i,1},\cdots,ROI_{i,L_{i}}\right\}(1\leq i\leq m)\), where \(L_{i}\) is the length of the \(i\)-th time series. Therefore, the whole video can be represented by a family of time series of ROIs \(\left\{\mathbb{ROI}_{i}\right\}_{i=1}^{m}=\left\{ROI_{i,1},\cdots,ROI_{i,L_{ i}}\right\}_{i=1}^{m}\), where \(m\) represents the number of ROIs in the pattern.