# Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation
## 3 Methodology
### KPConv with LSTM Cells in Latent Space  
The KPConv layer is inspired by the KPConv model introduced by [8]. The idea behind the Kernel Point Convolutions is to model the spatial relationships in a point cloud directly. Sliding a standard convolutional kernel over an unordered space is not feasible. The KPConv layer applies convolutional weights at Kernel Point locations with a distance factor to neighboring points of the input point cloud. The number of kernel points \(K\) is fixed, while the number of neighboring points \(N_{x}\) and the respective association function \(h_{ij}\) varies with the number of points in the local neighborhood. The considered neighbor points lie within a specified radius \(r\) from the center point of the convolution. The output feature dimension \(d_{out}\) for every center point is determined by the number of KPConv filters applied. Here, the KPConv layer follows the same principle as grid-based convolutions. Figure 2 shows a graphical representation for the processing of a center point \(x\).  
Due to the sparsity of the radar data, the maximum radius for neighbor points that affect the feature output of the current center point in the first KPConv layer is set to \(8\,\mathrm{m}\). In the original KPConv implementation [8] for lidar data, this radius is set to a maximum of \(0.15\,\mathrm{m}\) for outdoor scenarios. The larger radius empirically provided the best results on our data set. From a theoretical perspective, the greater sparsity of the radar data motivates a greater radius of influence to include sufficient information of the surroundings. In this radius, all radar reflections of a passenger vehicle could influence all other reflections from the same object. For larger objects, such as trucks and buses, this is not always the case. Nonetheless, an even greater radius did not result in better detection scores, as at the same time more clutter information could be associated in the neighborhood. Figure 1 shows that radar points do not necessarily originate from the visible edges of an object.  
Figure 3 shows the high-level model structure of a KPConv model with a ConvLSTM center layer (KPConv-CLSTM). The inputs to the network are the \(n\) input radar points consisting of the last three radar time steps with five feature dimensions \(feat\): Location \(x\), \(y\), \(z\); radial velocity components \(v_{x}\), \(v_{y}\); and the radar signal strength, called radar cross section \(RCS\). Following the original implementation, each _KPConv ResNet Block_ consists of an MLP layer, a KPConv layer and another MLP layer. The skip link branch consists of an MLP layer to adapt the feature dimension before adding it to the processed KPConv branch. The number of points is compressed with a farthest point sampling method. Once the input point cloud has been encoded, convolutional LSTM cells [27] are applied to the features in the latent space representing the entire input point cloud. The decoding or upsampling of the point cloud is performed by a combination of three-nearest neighbors upsampling and MLP layers for feature generation. When features have been obtained for every input point, three MLP layers serve as a classification head. The upsampling and classification head is inspired by [20]. A class-weighted focal loss function [29] is used for training to handle class imbalances, e.g., in the _Moving Vehicle Data Set_ used below over 97% of the points belong to the background class.  
Due to the modularity of the model, we can replace the ConvLSTM layer in the latent space with a standard LSTM cell or omit the time dependency altogether for our evaluation in Section 4. Additional details of the implementation can be found directly in the provided configurations in the repository released with this publication.