# Road Segmentation and Environment Labeling for Autonomous Vehicles
## 1 Introduction  
Autonomous vehicles rely on machine learning algorithms [1] and efficient computational software to analyze data from various sensors to understand surrounding obstacles to navigate on the road. Each type of sensor provides a different dataset, suitable for understanding the surrounding environment and vehicle movement on the road. Video cameras offer streaming video and signal information. LiDAR sensors provide better-defined images, even under unfavorable weather conditions. The bus data comprise acceleration, velocity, GPS coordinates, brake pressure, pitch, and roll angles; blending all these signals achieves high reliable data suitable for determining the distance of the obstacles from the autonomous vehicle's view. Autonomous vehicles require many human labels for detecting the objects in the scene and eventually use these labels for supervised learning techniques. Consequently, the autonomous vehicle industry spends considerable capital to collect human-labeled datasets in different environmental conditions. Improving the systems' efficiency in detecting objects in various geographical conditions involves collecting new labels with 3D bounding boxes. In recent years, many datasets have been collected in various cities to solve the different comprehensive visual navigation tasks, such as instance segmentation, object detection, 3D geometry estimation, and localization [2]. The most comparable datasets [3, 4] focusing on multimodal sensors are KITTI, Lyft Level 5, nuScenes, Mapillary Vistas, Cityscapes,ApolloScape, Waymo OD, A2D2, and Euro-PVI. These datasets contain LiDAR 3D point clouds and images collected from various sensors. Among these, A2D2 [4] emphasizes more LiDAR data and bus signals for segmentation. The A2D2 dataset is used in our research to segment and label 3D point cloud data. Object detection and segmentation of road scenes using 3D point cloud data is a substantial problem in computer vision. [PERSON] et al. [5] proposed an adaptive space partitioning scheme that focuses computations on data structures on 3D voxel grids. The voxel-graph network proposed in [6] learns the feature representation by constructing the KNN graph through all voxels. Octrees are one of the most popular techniques on voxel grids to learn the representations of high-resolution 3D data. PointTrackNet [7] tracks the object displacement point-wise for any detected object. The network is evaluated using the KITTI tracking dataset for various scenarios by processing the adjacent data frames. Point cloud labeling [8] is an essential step in the segmentation process, where each point is assigned a class, and later it serves as a cue in target detection and scene segmentation. Supervised learning algorithms require numerous labeled data and more computational power due to the high dimensional data. The whole process of manual labeling is time, effort and cost intensive, as it involves a lot of human effort [9]. As an alternative, weakly supervised learning techniques segment the scenes with fewer labels. [PERSON] and [PERSON] [10] proposed weakly supervised learning for point cloud segmentation of 3D CAD shapes and indoor scenes by identifying spatially adjacent points with similar colors and assuming that point cloud prediction is rotation and view angle invariant. However, in reality, the segmentation algorithms for road scenes should handle multiple frames from different sensors, capturing objects in multiple view angles around the AVs. In [11], the authors proposed an unsupervised 3D object detection module for leveraging the underlying 3D geometric structure of the point cloud and integrated it with the knowledge transfer network pre-trained with the existing images. Generally, most systems perform the object classification algorithm using color images, such as pixel-prototype contrastive learning [12]. However, LiDAR provides road information as first-hand data when the self-driving system detects surrounding objects.