# Loop Closure Detection in RGB-D SLAM by Utilizing Siamese ConvNet Features
## 2 Materials and Methods
### Siamese ConvNet Architecture for Loop Closure Detection  
For the Siamese ConvNet architecture, the proposed Siamese Convolutional Neural Networks (SCNN) takes a similar structure to AlexNet [32], which is composed of five convolutional layers and three fully connected layers. However, in contrast to the classical Convolutional Neural Networks (CNNs), it takes a pair of images as input and then feeds forward by two identical branch convolutional structures, which share the same parameters. An illustration of the enhanced deep architecture can be seen in Figure 4 and the parameters of each layer are listed in Table 1.  
The architecture takes an image pair with pixels of \(227\times 227\times 3\) as input including eight layers. The first five layers are convolutional layers, and the last three layers are the inner product (full connected) layers. Relu, pooling, and normalization follow after the output of each convolutional layer. The first convolutional layer filters the \(227\times 227\times 3\) input with kernels of size \(11\times 11\times 3\) with a stride of 4 pixels and then, the output of the first convolutional layer is rescaled to \(27\times 27\times 96\) by the \(3\times 3\) max pooling. The second convolutional layer takes the feature maps of the first layer as input, filters it with \(256\) kernels of size \(5\times 5\times 96\), and acquires feature maps with a size of \(13\times 13\times 256\). The third convolutional layer takes the feature maps of the second layer as input, filters it with \(384\) kernels of size \(3\times 3\times 256\), and acquires feature maps with a size of \(13\times 13\times 384\)  
Figure 3: An example of an RGB-D image collected by a Kinect camera. Note that the parameters of other layers can be looked up in Table 1, which are similar to the Alexnet. * Pre-train The Siamese ConvNet on the Place dataset [40] contains more than 2.5 million images of 205 scene classes. Formally, we feed image pairs with a size of \(227\times 227\times 3\) as input and define training samples of \(x_{1}^{k}=\left\{x_{1}^{k},x_{2}^{k},l_{1}^{k},l_{2}^{k}\right\}\), where \(x_{1}^{k}\) and \(x_{2}^{k}\) represent the \(k\)-th input pairs, and \(l_{1}^{k}\) and \(l_{2}^{k}\) are its corresponding labels. Then, the architecture can automatically extract 2048 dimensions feature vectors \(\mathbf{f}^{k}=\left\{\mathbf{f}_{1}^{k},\mathbf{f}_{2}^{k}\right\}\): \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer** & **Layer Type** & **Parameter** \\ \hline input1\_1, input1\_2 & input & Image size:227 \(\times\) 227 \(\times\) 3 \\ conv1\_1, conv1\_2 & convolution & Filter size:11 \(\times\) 11, Filter Num:96, Stride: 4 \\ pool1\_1, pool1\_2 & pooling & Pooling method: Max, Kernel size:3 \(\times\) 3, Stride: 2 \\ conv2\_1, conv2\_2 & convolution & Filter size:5 \(\times\) 5, Filter Num:256, Stride: 1 \\ pool2\_1, pool2\_2 & pooling & Pooling method: Max, Kernel size:3 \(\times\) 3, Stride: 2 \\ conv3\_1, conv3\_2 & convolution & Filter size:3 \(\times\) 3, Filter Num:384, Stride: 1 \\ conv4\_1, conv4\_2 & convolution & Filter size:3 \(\times\) 3, Filter Num:384, Stride: 1 \\ conv5\_1, conv5\_2 & convolution & Filter size:3 \(\times\) 3, Filter Num:256, Stride: 1 \\ pool5\_1, pool5\_2 & pooling & Pooling method: Max, Kernel size:3 \(\times\) 3, Stride: 2 \\ full6\_1, full\_6\_2 & fully connected & Neurons output:4096 \\ full7\_1, full\_7\_2 & fully connected & Neurons output:2048 \\ softmax\_1, softmax\_2 & fully connected & Neurons output:205 \\ feat1\_1, feat1\_2 & fully connected & Neurons output:1000 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The relevant layers and their parameters of SCNN. Figure 4: Siamese ConvNet Structure for visual loop closure detection. \[\mathbf{f}^{k}=Conv(x^{k}\Big{|}\theta_{net}) \tag{1}\]  
where \(Conv(\cdot|\theta_{net})\) is the feature extraction function defined by the Siamese ConvNet, and \(\theta_{net}\) is the shared parameter to be learned. Then, the problem can be reduced to select an appropriate loss function as supervision.