# Research on Negative Road Obstacle Detection Based on Multimodal Feature Enhancement and Fusion
## 3 The Proposed Network
### Data Enhancement  
We propose a stochastic image enhancement method designed to optimize the representation of image features in the negative obstacle detection training set. The method significantly enhances the representation of these features in the training samples by randomly splicing images from two categories: potholes and cracks. We use the existing NPO++ dataset [1], which was captured by real vehicles in urban and rural scenes, covering a variety of weather and lighting conditions. The dataset contains a total of 4600 images, including two types of targets: potholes and cracks. Since the proportion of potholes and cracks in the original images is small, the model's ability to learn these key features is limited. Therefore, we adopt a random splicing strategy to generate a training set with a richer feature distribution, thereby effectively improving the generalization ability and robustness of the model.  
Specifically, the implementation of random image enhancement is carried out to select the original image containing potholes and cracks and geometrically transform it to generate a variety of enhanced samples. These transformations include rotation, scaling, and translation, and the proportion of negative features of obstacles such as potholes and cracks in the scene is adjusted. Finally, the data features of the randomly selected images are reconstructed with a splicing probability of 0.5 and fused with the images of different scenes in the training set to generate a new synthetic image. As shown in Equation (8), the process employs dynamically generated splicing parameters, which not only improves the diversity of samples but also provides richer learning signals for the model.  
\[I_{c}=0.5\cdot(I_{t}\oplus(P\cdot T(I_{f},\theta,\delta x,\delta y)\cdot 1_{[ rand\leq P]})) \tag{8}\]  
\(I_{c}\) denotes the enhanced image, \(I_{t}\) denotes the target image, \(I_{f}\) is the feature image, \(T\) denotes the transformation function (e.g., translation, rotation, etc.), \(\theta\) denotes the angle, \(\delta x,\delta y\) denote the translation offset, \(P\) is the frequency of feature splicing, and \(1_{[ rand<P]}\) is the indicator function, which takes the value of 1 when the random number rand is less than or equal to \(P\), and 0 otherwise.  
A comprehensive representation of eigentransformations is shown in Equation (9):  
\[T(I)=I\odot(R(\theta)\otimes\delta) \tag{9}\]  
\(R(\theta)\) denotes the rotational transformation of the image, with the parameter being the rotation angle, \(\delta\) denotes the translation parameter, which can usually be expressed as (dx, dy), \(\otimes\) denotes the affine transformation operation of the image (both translation and rotation can be realized by a combination of affine transformations), and \(\odot\) denotes the superimposition operation on the pixel level.  
As shown in Figure 4, data enhancement reconstructs and splices multimodal and diverse features, making these features appear frequently at different positions, angles, and scales. This allows the model to pay more attention to key features, which not only solves the problem of a small proportion of negative obstacle features in the original dataset but also greatly improves the detection performance of the model in complex scenarios.