# Multi-Object Tracking with Grayscale Spatial-Temporal Features
## 3 Method
### Spatial-Temporal Feature Matching  
The spatial-temporal feature matching process is shown in Figure 4. The spatial-temporal matching module includes two parts. It can be seen from the visualization results of \(\textbf{Ps}_{1}^{i}\) and \(\textbf{Rs}_{1}^{i}\). The red rectangular box is the set of target-STF. The green rectangular box is the set of region-STFs. The black rectangular box is the set of the targets.  
Figure 3: The process of obtaining spatial-temporal features. In the image \(\textbf{PsI_{t}}\) and \(\textbf{RsI_{r}}\) for the convenience of observation, different grayscale values are chosen for the target.  
For the first part, we match the target-STF. According to the number of targets \(\mathbf{P}_{t}^{i}\), we create the set \(\mathbf{P}s_{t,i+1}^{k}\). The grayscale value in the target-ST map of target \(\mathbf{p}_{t}^{j}\) is \(j+1\), so the target-STF matching results of target \(\mathbf{p}_{t}^{j}\) is \(\mathbf{p}s_{t,j+1}^{k}\).  
Then, select different points \(\mathbf{D}_{i}(x,y)\) based on the bounding box of each target using Equation (3):  
\[\mathbf{D}_{i}(\mathbf{x},\mathbf{y})=\mathbf{S}(\mathbf{h}_{i},\mathbf{w}_{i},\mathbf{v}_{i},\mathbf{s}_{i}) \tag{3}\]  
\(h_{i}\) is the height of \(\mathbf{P}b_{t}^{i}\), \(w_{i}\) is the width of \(\mathbf{P}b_{t}^{i}\), \(v_{i}\) is the velocity of \(\mathbf{C}\mathbf{P}b_{t}^{i}\), and \(s_{i}\) is the area of \(\mathbf{P}b_{t}^{i}\). \(\mathbf{S}(h_{i},v_{i},v_{i},s_{i})\) is a function to obtain points \(\mathbf{D}_{i}(x,y)\).  
Then, obtain the grayscale values of these points \(\mathbf{D}_{i}(x,y)\) in the target-STF map \(\mathbf{P}s_{t-1}\). If the grayscale values of these points are all zero, zero is stored in the set \(\mathbf{P}s_{t,i}^{k}\). If there is a non-zero gray value, the non-zero gray value is stored in the set \(\mathbf{P}s_{t,i}^{k}\), and the same value is recorded only once. After the operation is completed, we will obtain the target-STF matching results \(\mathbf{P}s_{t,i}^{k}\).  
For the second part, we match the region spatial-temporal features. This process is similar to the first part; according to the number of target-STF created, set \(\mathbf{R}\mathbf{r}s_{t}^{i}\) to record the region corresponding relationship between two frames. We create set \(\mathbf{R}s_{t}^{i}\) to record to which region the target belongs. \(\mathbf{R}s_{t}^{i}\) can be obtained from Equation (4):  
\[\mathbf{r}s_{t}^{i}=\mathbf{g}\mathbf{v}_{\mathbf{R}s_{t}}(\mathbf{D}_{i}(\mathbf{cty},\mathbf{ctx})),\bm {0}\leq\mathbf{i}<\mathbf{P}_{t}\mathbf{.size}. \tag{4}\]  
\(\mathbf{D}_{i}(\mathbf{c}ty,\mathbf{ctx})\) is the center point of the bounding box of target \(\mathbf{p}_{t}^{i}\). After the operation is completed, we know the region to which the target belongs.  
Last, if the target-STF matching results \(\mathbf{p}s_{t,j+1}^{k}\) of target \(\mathbf{p}_{t}^{j}\) are not zero, for target \(\mathbf{p}_{t}^{j}\), select the same points \(\mathbf{D}_{i}(x,y)\) as in the first part. Obtain the grayscale values of these points in the target-STF map \(\mathbf{R}s_{t-1}\), and store the non-zero and non-repeating grayscale values in \(\mathbf{R}\mathbf{r}s_{t,j}^{k}\). After this operation is completed, we will obtain \(\mathbf{R}\mathbf{r}s_{t,i}^{k}\). According to \(\mathbf{r}s_{t}^{j}\) and \(\mathbf{R}\mathbf{r}s_{t,j}^{k}\), the region to which target \(\mathbf{p}_{t}^{j}\) belongs in the previous frame can be obtained.  
Figure 4: The spatial-temporal feature matching process. The blue rectangular box is the detected target. The red rectangular box is the set of target-STF. The green rectangular box is the set of region-STFs. The black rectangular box is the set of targets.