# River Surface Space-Time Image Velocimetry Based on Dual-Channel Residual Network
## 2 Materials and Methods
### Dual-Channel Residual Network Model  
The process of MOT detection is considered an image regression prediction problem:  
\[\delta=F(I) \tag{2}\]  
where \(I\) is the STI to be detected, and \(F\) is the regression prediction function. The residual network is used to construct the regression model, while the prediction function is  
Figure 1: STI synthesis. constructed by its powerful image feature extraction and regression ability. Without pre-processing the STI, the MOT is directly regressed from the STI. The process of our method is shown in Figure 2. Put the STIs into the trained model to detect the MOT and calculate the velocity vector. The idea of ResNet is to assume that there is an optimal number of network layers in a deep network, and then the network contains some redundant network layers, defined as redundant layers. Setting these redundant layers as identity layers can complete the identity mapping of the input and output, and the identity layers are learned adaptively in the process of network training [28]. Thus, the number of layers of neural networks can reach dozens, hundreds, or even thousands of layers. Figure 3 is the concrete structure of the residual block with a jump connection called the shortcut connection. The output \(H(x)\) of the residual block includes the residual mapping \(F(x)\) and the superposition of the input \(x\). Assuming that the function mapping to be fitted is \(H(x)\), then another residual mapping \(F(x)\) can be defined, and \(F(x)\) is equal to \(H(x)-x\). It is easier to optimize the residual mapping than to optimize the original function mapping [29]. In the extreme case where the identity mapping \(H(x)=x\) is optimal, the residual network only needs to learn the residual mapping to 0 (\(F(x)=0\)), which means it does not need to use the superimposed nonlinear network to fit the identity mapping. So ResNet can solve the problem of gradient disappearance and network degradation when building deep networks [30]. STIV mainly relies on the texture features with significant directionality formed by the tracer movement. Since edge information can well reflect the texture orientation, an edge channel is added to the gray channel to form a dual-channel residual network (DCResNet) structure. As shown in Figure 4, it mainly includes two channels and a regression layer composed of three fully connected layers. A single-channel network whose input is the original STI is denoted as Gray-Channel, while the Edge-Channel is input by the STI after Sobel detection. The weighted fusion of the above two channels is denoted as Dual-Channel. Figure 3: Two-layer identity residual block. Figure 2: STIV based on DCResNet. Both channels use the feature extraction layer of ResNet18; that is, the original ResNet18 classification model removes 17 convolutional layers other than the fully connected layer. The network structure parameters are shown in Figure 5. k is the size of the convolution kernel, s is the sliding step size of the spatial domain, and p is the spatial domain filling. The structure of the 17-layer network can be regarded as composed of a conv1 layer and layer1-layer4 layers. The conv1 layer is a convolution layer with a convolution kernel size of \(7*7\), the step size is 2, and the spatial domain filling is 3. Layer1-layer4 are composed of two Basic Block structures. As shown in Figure 6, each Basic Block structure mainly includes two convolution layers with a convolution kernel size  
Figure 4: The structure of DCResNet. Figure 5: Structure and parameters of ResNet18 feature extraction layer. of \(3*3\). In addition to the Basic Block in layer1, layer2-layer4 perform the down-sampling operation in the first Basic Block to ensure that the number of channels in the feature map skip connection is consistent with the number of channels in the output feature map in the Basic Block. The last convolutional layer outputs a \(512*7*7\) feature map, which is processed through avgpool and flatten operations to convert it into a vector suitable for regression tasks.