# Improving Domain Generalisable LiDAR Semantic Segmentation In Off-Road Environments with Auxiliary Tasks

Abhay Dayal Mathur, Alexandre Chapoutot, David Filliat

► To cite this version:

Abhay Dayal Mathur, Alexandre Chapoutot, David Filliat. Improving Domain Generalisable LiDAR Semantic Segmentation In Off-Road Environments with Auxiliary Tasks. 2025 European Conference on Mobile Robots (ECMR), Sep 2025, Padova, Italy. pp.1-6, 10.1109/ECMR65884.2025.11163269. hal-05321746

HAL Id: hal-05321746

https://hal.science/hal-05321746v1

Submitted on 9 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Improving Domain Generalisable LiDAR Semantic Segmentation In Off-Road Environments with Auxiliary Tasks

Abhay Dayal Mathur<fnref n="1" />,<fnref n="2" />, Alexandre Chapoutot<fnref n="2" /> and David Filliat<fnref n="2" />

**Abstract**— Autonomous navigation for unmanned ground vehicles (UGVs) in complex environments requires robust perception for reliable traversability estimation and path planning. Traditional geometric methods often fail in unstructured terrains, necessitating robust scene understanding. Current 3D semantic segmentation methods are mostly directed towards structured environments and are not generalisable for off-road domains. To address this, we propose modifications to LiDAR semantic segmentation by incorporating spatial context and adding an auxiliary task of learning point-wise ground-relative height to capture robust features. Experiments on both real-to-real and synthetic-to-real transfer demonstrate that the proposed approach outperforms state-of-the-art methods in segmentation accuracy and adaptability, offering a scalable solution for UGV perception in diverse outdoor environments.

## I. INTRODUCTION

Navigation in unstructured outdoor environments presents numerous challenges. Autonomous vehicles must navigate varying terrains, avoid obstacles in real-time, and maintain stability in diverse conditions. Traditional approaches to perception and navigation, i.e., without machine learning, rely heavily on geometric representations of the environment, often assuming structured settings and smooth terrains. However, these methods fall short in unstructured environments, where varying slopes, dense vegetation, and unpredictable obstacles can significantly degrade the effectiveness of geometric-based solutions.

Recent advances in deep learning and sensor fusion techniques have shown promise in improving the perception capabilities of UGVs by leveraging rich sensor data like LiDAR point clouds. Specifically, LiDAR-based semantic segmentation can provide a detailed understanding of the surrounding terrain, identifying traversable areas and potential obstacles. Despite this progress, existing segmentation methods often face challenges in domain generalisation, especially when deployed in unseen environments with substantial domain shifts, which can be caused by changes in terrain, weather, or sensor configurations.

This paper introduces an auxiliary learning task to enhance the generalisability of LiDAR-based semantic segmentation in unstructured environments. By learning point-wise ground-relative height in parallel to the primary task of semantic segmentation, our approach aims to encourage the model to capture domain-agnostic features. We hypothesise that incorporating height estimation improves the model's

robustness to unseen domains, increasing its ability to generalise across different terrains without requiring prior information about the target environment. Further, we modify the existing architecture for semantic segmentation by adding a module that captures spatial context to improve the model's understanding of environmental features. Our experiments show that the proposed method significantly improves segmentation accuracy and generalisability on challenging off-road datasets.

Our contributions can be summarised as follows.

1. We propose a novel approach to improving domain generalisation by incorporating an auxiliary task of learning point-wise ground-relative height, and improve upon the existing architecture by adding a spatial context module.

2. We demonstrate the effectiveness of our method through experiments on both synthetic to real and real to real domain shifts, showing significant improvements in segmentation accuracy and generalisability.

3. We benchmark existing methods for domain generalisation in LiDAR semantic segmentation on datasets designed for off-road navigation.

## II. RELATED WORK

### A. LiDAR Semantic Segmentation

Several methods have been proposed to address the task of LiDAR Semantic Segmentation (LSS), mostly dealing with structured outdoor environments, such as those pertinent to autonomous driving. Several methods project point clouds onto range images, and restore contextual information in pixels from corresponding frustum LiDAR points [1], or boost performance on sparse scans through residual convolution blocks [2]. SqueezeSegV3 [3] introduced the use of context aggregation modules and a new loss function to handle noise and occlusions in LiDAR data. PointNet++ [4] builds on the original PointNet architecture by using a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set, capturing local structures at different scales. Finally, voxel-based methods convert the point cloud data into a voxel grid and apply 3D convolutions. These methods include architectures like MinkowskiNet [5], which uses sparse convolutions to process voxelised point clouds efficiently.

### B. LSS for Outdoor Navigation

3D semantic segmentation for unstructured environments, however, remains relatively unexplored in literature, and very few publicly available datasets are appropriate for the

<footnote n="1">¹Institut Polytechnique de Paris, Palaiseau, France abhay.mathur@ip-paris.fr</footnote>

<footnote n="2">²U2IS, ENSTA, Institut Polytechnique de Paris, Palaiseau, France {alexandre.chapoutot, david.filliat}@ensta.fr</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 1: Overview of the proposed LiDAR semantic segmentation training pipeline. The encoder extracts latent features from the point cloud P, which are processed by two parallel decoders: a semantic decoder for segmentation ŷ and an auxiliary decoder for ground-relative height prediction ŷaux. The combined pipeline is supervised through a composite loss. For the context module, parameters k, d, and BN signify kernel size, dilations and batch-normalization, respectively. At inference, only the semantic decoder is used.</figcaption></figure>

task. Recent methods [6], [7] use semantic segmentation as an upstream task for estimating traversability from LiDAR scans, by grouping the semantic classes into ‘tr’ and ‘not-tr’, or through other simplified classification schemes based on grouping terrain categories as in [8]. Recently, multi-modal architectures such as [9] have been proposed for traversability estimation, but domain generalisation for unstructured environments has so far not been addressed.

### C. Datasets for Unstructured Environments

Despite significant progress in LSS, datasets suitable for unstructured environments remain scarce. Most existing ones either lack LiDAR data, semantic labels, or both, making them unsuitable for developing and benchmarking models aimed at 3D semantic segmentation in such environments.

A majority of available datasets are designed for navigation and mapping tasks, primarily focusing on geometric reconstruction or traversability estimation without semantic annotations (e.g., Tartan Drive 2.0 [10]). As a result, they do not support the upstream task of semantic segmentation, which is critical for more comprehensive scene understanding.

Among the few datasets that do include both LiDAR data and semantic labels, many exhibit highly domain-specific class distributions that limit generalisability. For example, FinnWoodlands [11] provides fine-grained semantic labels but focuses predominantly on different tree species while omitting several relevant object categories. This specificity

restricts its applicability to diverse real-world settings and makes it unsuitable for training models intended for deployment across varied environments. We will therefore rely on two datasets that match our requirements including GOOSE [12] and RELLIS-3D [13].

Furthermore, there is a notable shortage of publicly available synthetic datasets for unstructured environments. While simulators such as Gazebo and the CARLA simulator [14] have been used to generate synthetic LiDAR datasets, these datasets are typically not released, preventing their use for benchmarking and model development [15], [16].

Two synthetic datasets have been made publicly available:

1. The **SynPhoRest** [17] dataset includes classes such as background, soil, traversable regions, canopy, fuel, and trunks. However, its classification scheme is highly specific, with the “fuel” category encompassing all flammable objects. This lack of general-purpose labeling reduces its utility for broader applications beyond fire risk assessment.

2. The **OUTBACK** [18] dataset provides a more diverse set of classes, including branch, fence, grass, grasstree, ground, log, robot (ego vehicle), rock, and tree. The class distribution is more representative of real-world unstructured environments, making it more suitable for domain generalisation studies and synthetic-to-real transfer experiments.

Given these limitations, we select the OUTBACK dataset as the primary synthetic dataset for evaluating domain transfer performance, as its class diversity and semantic richness
align more closely with the requirements of generalisable LSS models.

### D. Domain Generalisation for LSS

Domain generalisation for LSS has lately gathered interest in literature due to its potential to improve model robustness across varied and unseen environments, especially in exploratory navigation where prior knowledge of the environment and ambient conditions is rarely available.

Most efforts in this direction, such as LiDOG [19] or SALUDA [20], aim at encouraging models to learn domain-agnostic representations through auxiliary tasks, by leveraging feature density in bird-eye-view (BEV) projections, or through training parallel decoders on surface reconstruction. These methods, however, are specific to structured environments and work under the assumption of either receiving a particular structure in the LiDAR scans [19], or of the surroundings majorly consisting of surfaces [20]. Other methods, such as CosMix [21] use patch-mixing to augment training samples by concatenating points and labels of different scans into a single scene, relying on pseudolabels obtained from a model trained on the source dataset for patches from the target domain.

Further, most current methods assume some knowledge of the target domain, either intrinsics for the target sensors [22], a small dataset collected from the target domain to fine-tune the model with, or knowledge of conditions in the target domain to construct bridge domains through simulation [23]. These assumptions rarely hold for exploration in the wild, necessitating robust methods that can generalise over unseen environments.

## III. METHOD

Our method focuses on learning robust representations for LSS through an auxiliary task of learning point-wise ground-relative height (de-skewed with respect to sensor vibrations). To this end, we train a decoder on the height-prediction task parallel to the decoder for semantic segmentation. This reduces the reliance of the encoder on domain-specific correlations between point cloud features and semantic labels. An overview of the training pipeline for our method can be found in Figure 1. Additionally, we incorporate some architectural modification to the Minkowski U-Nets backbone [5] to also take spatial context into account.

### A. Spatial Context

We encourage the encoder model to pay greater attention to the positional information from the point cloud in two ways. Firstly, following our empirical results with using the sparse tensor representation of point clouds in Minkowski Engine [5], we duplicate the coordinates of each point in the point cloud in its feature tensor. Our pilot studies showed that this simple modification significantly improves the performance of the model, and obviates the need for hand-engineered features obtained from storing scan statistics in a global 3D map as in other methods [6].

Secondly, we add a spatial context module to the encoder, drawing from projection-based methods like SalsaNext [2]. This module comprises small kernel convolutions (with a kernel size of 1 or 3), facilitating the propagation by the model of global patterns through an increased number of channels.

### B. Auxiliary Task

The auxiliary task of learning point-wise true (de-skewed) height relative to the ground is particularly useful in unstructured environments, where the terrain is not flat and the LiDAR scans are not perfectly aligned with the ground plane. We posit that learning to predict the ground-relative heights as an auxiliary task results in the encoder learning features more robust to domain-shifts.

Predicting ground-relative height for each point is intuitively a regression task, but to allow for more stable training, we formulate the auxiliary training objective as regression through classification, training the model to predict each point's ground-relative height using discrete height bins. This transforms the continuous regression problem into a discrete one, reducing the impact of outliers and noise to improve convergence [24], and allowing for an estimation of confidence in the prediction.

The classification head for the auxiliary decoder therefore estimates the distribution of the ground-relative height of each point in the point cloud across the height bins. The loss for the auxiliary task is computed as:

$$ \mathcal{L}_{aux} = \frac{1}{N} \sum_{j=1}^{N} \|\hat{y}_j \cdot h - y_j\|_2^2, \quad (1) $$

where N is the number of points in the point cloud, $\hat{y}_j \in \mathbb{R}^k$ the prediction from the auxiliary head, $h \in \mathbb{R}^k$ is the vector of centroids of the height bins and $y_j$ the ground truth.

The combined training loss for the model is given by:

$$ \mathcal{L} = \mathcal{L}_{seg} + \lambda \mathcal{L}_{aux}, \quad (2) $$

where $\mathcal{L}_{seg}$ is the segmentation DICE Loss [25] and $\lambda$ is a hyperparameter that controls the weight of the auxiliary task in the overall training loss. Only the semantic decoder is used to generate predictions for inference.

## IV. EXPERIMENTS AND RESULTS

### A. Datasets

For our experiments, we consider two real-world datasets and one synthetic dataset, each designed to support semantic segmentation in off-road environments.

**RELLIS-3D [13]:** RELLIS-3D is a multimodal dataset designed for off-road semantic scene understanding, comprising 13,556 annotated LiDAR scans and 6,235 images. Data was gathered on the Rellis Campus of Texas A&M University, presenting challenges related to class imbalance and environmental topography. We use the point clouds recorded with the 64 channel Ouster OS1 LiDAR sensor.

**GOOSE [12]:** The German Outdoor and Offroad Dataset (GOOSE) offers 10,000 labeled pairs of images and point
<table><caption>TABLE I: Results on Real to Real Transfer</caption><thead><tr><th rowspan="2">Metric</th><th colspan="4">GOOSE → Rellis-3D</th><th colspan="4">Rellis-3D → GOOSE</th></tr><tr><th>Cos-Mix</th><th>LiDOG</th><th>SALUDA</th><th>Ours</th><th>Cos-Mix</th><th>LiDOG</th><th>SALUDA</th><th>Ours</th></tr></thead><tbody><tr><td>Class-wise IoU</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>dirt</td><td>0.2888</td><td>0.3030</td><td>0.3040</td><td><strong>0.4780</strong></td><td>0.1008</td><td>0.1027</td><td>0.1034</td><td><strong>0.2430</strong></td></tr><tr><td>grass</td><td>0.1034</td><td>0.0903</td><td>0.1088</td><td><strong>0.1800</strong></td><td>0.3620</td><td>0.3001</td><td>0.3750</td><td><strong>0.4790</strong></td></tr><tr><td>tree</td><td>0.5290</td><td>0.5579</td><td>0.5568</td><td><strong>0.6180</strong></td><td><strong>0.5430</strong></td><td>0.4050</td><td>0.4035</td><td>0.5356</td></tr><tr><td>pole</td><td>0.0565</td><td>0.0502</td><td><strong>0.0595</strong></td><td>0.0452</td><td>0.0079</td><td>0.0067</td><td>0.0080</td><td><strong>0.0743</strong></td></tr><tr><td>water</td><td>0.0097</td><td>0.0085</td><td>0.0102</td><td><strong>0.0350</strong></td><td>0.0205</td><td>0.0178</td><td>0.0216</td><td><strong>0.0274</strong></td></tr><tr><td>vehicle</td><td>0.0565</td><td>0.0504</td><td>0.0595</td><td><strong>0.0651</strong></td><td>0.0641</td><td>0.0543</td><td>0.0663</td><td><strong>0.1030</strong></td></tr><tr><td>object</td><td>0.0010</td><td>0.0009</td><td>0.0010</td><td><strong>0.0352</strong></td><td>0.0749</td><td>0.0631</td><td>0.0766</td><td><strong>0.1500</strong></td></tr><tr><td>building</td><td>0.0561</td><td>0.0502</td><td>0.0590</td><td><strong>0.2076</strong></td><td>0.0191</td><td>0.0164</td><td>0.0200</td><td><strong>0.0552</strong></td></tr><tr><td>person</td><td>0.1074</td><td>0.0943</td><td>0.1130</td><td><strong>0.2080</strong></td><td>0.1574</td><td>0.1347</td><td>0.1630</td><td><strong>0.2207</strong></td></tr><tr><td>fence</td><td>0.0561</td><td>0.0497</td><td>0.0590</td><td><strong>0.1050</strong></td><td>0.0035</td><td>0.0029</td><td>0.0036</td><td><strong>0.1116</strong></td></tr><tr><td>bush</td><td>0.0207</td><td>0.0176</td><td>0.0218</td><td><strong>0.1600</strong></td><td>0.0627</td><td>0.0535</td><td><strong>0.0645</strong></td><td>0.0101</td></tr><tr><td>concrete</td><td>0.0191</td><td>0.0173</td><td>0.0201</td><td><strong>0.0240</strong></td><td>0.1607</td><td>0.1387</td><td><strong>0.1653</strong></td><td>0.0115</td></tr><tr><td>barrier</td><td>0.0247</td><td>0.0211</td><td>0.0260</td><td><strong>0.0480</strong></td><td>0.0113</td><td>0.0094</td><td><strong>0.0115</strong></td><td>0.0040</td></tr><tr><td>rubble</td><td>0.0565</td><td>0.0501</td><td>0.0595</td><td><strong>0.0774</strong></td><td>0.0059</td><td>0.0051</td><td>0.0061</td><td><strong>0.0102</strong></td></tr><tr><td>mIOU</td><td>0.0990</td><td>0.0972</td><td>0.1042</td><td><strong>0.1630</strong></td><td>0.1138</td><td>0.0936</td><td>0.1063</td><td><strong>0.1453</strong></td></tr></tbody></table>

<table><caption>TABLE II: Results on Synthetic to Real Transfer</caption><thead><tr><th rowspan="2">Metric</th><th colspan="4">OUTBACK → Rellis-3D</th><th colspan="4">OUTBACK → GOOSE</th></tr><tr><th>Cos-Mix</th><th>LiDOG</th><th>SALUDA</th><th>Ours</th><th>Cos-Mix</th><th>LiDOG</th><th>SALUDA</th><th>Ours</th></tr></thead><tbody><tr><td>Class-wise IoU</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>fence</td><td>0.0102</td><td>0.0096</td><td>0.0111</td><td><strong>0.0153</strong></td><td>0.0380</td><td>0.0325</td><td>0.0400</td><td><strong>0.0460</strong></td></tr><tr><td>grass</td><td>0.1320</td><td>0.1223</td><td>0.1447</td><td><strong>0.3450</strong></td><td>0.3569</td><td>0.3093</td><td>0.3746</td><td><strong>0.4550</strong></td></tr><tr><td>ground</td><td>0.0915</td><td>0.0852</td><td>0.1000</td><td><strong>0.1080</strong></td><td>0.1196</td><td>0.1067</td><td>0.1248</td><td><strong>0.1466</strong></td></tr><tr><td>log</td><td>0.0001</td><td>0.0001</td><td>0.0001</td><td><strong>0.0050</strong></td><td>0.1580</td><td>0.1410</td><td><strong>0.1659</strong></td><td>0.1358</td></tr><tr><td>robot</td><td>0.0922</td><td>0.0856</td><td>0.1001</td><td><strong>0.1053</strong></td><td>0.0286</td><td>0.0251</td><td>0.0299</td><td><strong>0.0344</strong></td></tr><tr><td>rock</td><td>0.0013</td><td>0.0012</td><td>0.0014</td><td><strong>0.0140</strong></td><td>0.0011</td><td>0.0010</td><td>0.0012</td><td><strong>0.0500</strong></td></tr><tr><td>tree</td><td>0.5140</td><td>0.5189</td><td>0.5161</td><td><strong>0.5353</strong></td><td>0.4702</td><td>0.4961</td><td>0.4949</td><td><strong>0.5424</strong></td></tr><tr><td>road</td><td>0.0198</td><td><strong>0.0210</strong></td><td>0.0204</td><td>0.0120</td><td>0.1535</td><td>0.1592</td><td><strong>0.1606</strong></td><td>0.1411</td></tr><tr><td>mIOU</td><td>0.1076</td><td>0.1055</td><td>0.1117</td><td><strong>0.1425</strong></td><td>0.1657</td><td>0.1589</td><td>0.1740</td><td><strong>0.1939</strong></td></tr></tbody></table>

clouds, specifically tailored for unstructured outdoor environments. Captured over a year to encompass various seasons and weather conditions, the dataset features data from an RGB+NIR camera and a 128-channel roof-mounted LiDAR system. Annotations cover 64 semantic classes, aiming to enhance perception capabilities of robots operating in complex terrains.

**OUTBACK [18]:** The OUTBACK dataset is a multimodal synthetic collection designed for rural Australian off-road robot navigation. Developed using a novel framework to generate varied photorealistic synthetic off-road data, it supports multiple sensor modalities and environmental conditions. Point clouds are recorded by simulating a Velodyne VLP-16 LiDAR sensor. Samples are annotated with 10 semantic classes, including branches, fences, grass, grasstrees, ground, logs, robots (ego vehicle), rocks, and trees.

We study the performance of our method by comparing its performance on the target dataset when trained on a source dataset. We consider two kinds of domain transfer scenarios: *Real to Real* and *Synthetic to Real*. In the former, we train the model on one real-world dataset and test it on another. In the latter, we train the model on a synthetic dataset and test it on a real-world dataset.

## B. Implementation Details

In our experiments, we use the Minkowski UNet architecture [5] with residual blocks, implemented with TorchSparse [26] as the backbone for the model. The network

consists of 4 hidden layers, with 16 channels in the first layer, doubling in each subsequent layer. We use a TorchSparse implementation for all the baselines, and keep the depth of the networks consistent to ensure the only variation in the pipeline is the domain generalisation method.

We tune the hyperparameter $\lambda$ for the auxiliary loss in the range [0.001, 1] over 30 trials using Optuna [27] on the Rellis-3D to GOOSE transfer setup, and we set it to 0.01 for all subsequent experiments.

1) Labelling for the Auxiliary Task: We label the ground-relative height of each point in the point cloud by estimating the ground plane in each point cloud using the RANSAC algorithm and computing the distance of all points from this estimated plane. We then discretise the height values into $k$ bins, each of width 20 cm.

2) Augmentations: We use a combination of augmentations to improve the generalisability of the model. These include small random rotations and jitter. We also add Gaussian noise to the point cloud to promote invariance to sensor idiosyncrasies such as point densities and distributions.

## C. Real to Real Transfer

We evaluate the performance of our method on the real-to-real transfer scenario between the GOOSE and Rellis-3D datasets. We compare our method with three state-of-the-art methods: Cos-Mix [21], LiDOG [19], and SALUDA [20].

Class mappings between the two datasets are performed manually based on the semantic similarity of the classes and
<figure><img src="image_3.png" /><figcaption>Fig. 2: Qualitative Comparison of results of real-to-real transfer from Rellis-3D to the GOOSE dataset. The first column shows the ground truth labels, the second column shows the results of the model trained without our improvements, and the third column shows the results of our method. The baseline model confounds the ground with grass while our method is able to distinguish between the two.</figcaption></figure>

<table><caption>TABLE III: Ablation study showing the percentage improvement in mIoU with the successive addition of each component in our pipeline. Each modification significantly enhances the model's performance.</caption><thead><tr><th rowspan="2">Method</th><th colspan="2">mIoU</th></tr><tr><th>Rellis-3D → GOOSE</th><th>GOOSE → Rellis-3D</th></tr></thead><tbody><tr><td>Residual MinkUNet</td><td>0.068</td><td>0.056</td></tr><tr><td>+ Augmentations</td><td>0.081</td><td>0.079</td></tr><tr><td>+ Spatial Context</td><td>0.099</td><td>0.103</td></tr><tr><td>+ Auxiliary Task</td><td>0.144</td><td>0.163</td></tr></tbody></table>

can be found in Table A.1. Classes with no corresponding matches are ignored.

The results are presented in Table I. Our method outperforms the state-of-the-art methods in most classes and has the highest mean Intersection over Union (mIoU) score for both transfer cases. We achieve a significant improvement over the next best method (SALUDA) by 56.4% for GOOSE → Rellis-3D and 27.7% for Rellis-3D → GOOSE.

### D. Synthetic to Real Transfer

Next, we evaluate our method on synthetic-to-real transfer by training on the OUTBACK dataset and testing on the Rellis-3D and GOOSE datasets. Relevant class mappings can be found in Table A.2.

The results in Table II demonstrate the superior performance of our method in synthetic-to-real domain transfer. As in the real to real transfer case, our approach achieves the highest mean IoU for both target datasets, with a 27.5% improvement over SALUDA for OUTBACK → Rellis-3D and an 11.4% improvement for OUTBACK → GOOSE.

Our auxiliary model attains a loss of 7.8cm for real-to-real transfer and 8.6cm for synthetic-to-real transfer, indicating that the auxiliary task effectively aids in capturing meaningful information, and helps in learning domain-agnostic features through more than just regularisation.

### E. Ablations

We investigate the effect of each of our proposed modifications by running ablative tests on the performance of the model for real-to-real transfer in Table III. Each modification shows a considerable effect on improving the performance of the model.

A qualitative comparison of our baseline (Residual MinkUNet) with our complete approach can be found in Figure 2. It can be observed that using our method enables the model to correctly distinguish between the ground and grass in an unseen environment, while the baseline model confounds the two.

## V. CONCLUSIONS

This paper introduces a novel approach to improving performance and generalisability in LiDAR-based semantic segmentation for autonomous navigation of unmanned ground vehicles (UGVs) in unstructured environments. By introducing an auxiliary task of learning point-wise true (demskewed) height above ground, the model is encouraged to learn features that enhance its robustness to variations in terrain and environmental conditions.

Current methods for domain generalisation rely on assumptions on the structure of the environment and even prior information about the nature of the target domain. Auxiliary tasks based on such methods can therefore at best act as regularisers for the model in domains where the assumptions don't hold, and indeed might even be detrimental for performance if the resultant self-supervised signals are not representative of the intended features. On the contrary, our method is designed to be agnostic to the structure of the environment and does not require any prior information about the target domain. By learning to predict the ground-relative height of each point in the point cloud, the model is encouraged to learn features that are robust to variations in terrain and environmental conditions. This approach further allows the model to implicitly perform ground-plane detection which is essential to many downstream tasks in navigation.

Experimental evaluations on both real to real transfer and synthetic to real transfer demonstrate that our method consistently outperforms existing state-of-the-art approaches in terms of segmentation accuracy and domain adaptation.

## REFERENCES

[1] X. Xu, L. Kong, H. Shuai, and Q. Liu, "Frnet: Frustum-range networks for scalable lidar segmentation," 2024. [Online]. Available: https://arxiv.org/abs/2312.04484
[2] T. Cortinhal, G. Tzelepis, and E. E. Aksoy, "Salsanext: Fast, uncertainty-aware semantic segmentation of lidar point clouds for autonomous driving," 2020. [Online]. Available: https://arxiv.org/abs/2003.03653

[3] B. Wu, A. Wan, X. Yue, and K. Keutzer, "Squeezezeg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud," 2017. [Online]. Available: https://arxiv.org/abs/1710.07368

[4] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space," 2017. [Online]. Available: https://arxiv.org/abs/1706.02413

[5] C. Choy, J. Gwak, and S. Savarese, "4d spatio-temporal convnets: Minkowski convolutional neural networks," 2019. [Online]. Available: https://arxiv.org/abs/1904.08755

[6] F. Ruetz, N. Lawrance, E. Hernandez, P. Borges, and T. Peynot, "Foresttrav: 3d lidar-only forest traversability estimation for autonomous ground vehicles," IEEE Access, 2024. [Online]. Available: https://ieeexplore.ieee.org/document/10458917

[7] A. Santo, J. J. Cabrera, D. Valiente, C. Viegas, and A. Gil, "Te-next: A lidar-based 3d sparse convolutional network for traversability estimation," 2024. [Online]. Available: https://arxiv.org/abs/2406.01395

[8] T. Guan, D. Kothandaraman, R. Chandra, A. J. Sathyamoorthy, K. Weerakoon, and D. Manocha, "Ga-nav: Efficient terrain segmentation for robot navigation in unstructured outdoor environments," IEEE Robotics and Automation Letters, vol. 7, pp. 8138-8145, 7 2022.

[9] Z. Chen, C. Sun, S. Nie, C. Min, C. Ning, H. Li, and B. Wang, "Semantic scene completion based 3d traversability estimation for off-road terrains," 2024. [Online]. Available: https://arxiv.org/abs/2412.08195

[10] M. Sivaprakasam, P. Maheshwari, M. G. Castro, S. Triest, M. Nye, S. Willits, A. Saba, W. Wang, and S. Scherer, "Tartandrive 2.0: More modalities and better infrastructure to further self-supervised learning research in off-road driving tasks," in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 12 606-12 606.

[11] J. Lagos, U. Lempiö, and E. Rahtu, FinnWoodlands Dataset. Springer Nature Switzerland, 2023, p. 95-110.

[12] P. Mortimer, R. Hagmanns, M. Granero, T. Luettel, J. Petereit, and H.-J. Wuensche, "The GOOSE Dataset for Perception in Unstructured Environments," in Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2024. [Online]. Available: https://arxiv.org/abs/2310.16788

[13] P. Jiang, P. Osteen, M. Wigness, and S. Saripalli, "Rellis-3d dataset: Data, benchmarks and analysis," 2022. [Online]. Available: https://arxiv.org/abs/2011.12954

[14] A. Dosovitskiy, G. Ros, F. Codevilla, A. M. López, and V. Koltun, "CARLA: an open urban driving simulator," CoRR, vol. abs/1711.03938, 2017. [Online]. Available: http://arxiv.org/abs/1711.03938

[15] X. Zhou, Y. Feng, X. Li, Z. Zhu, and Y. Hu, "Off-road environment semantic segmentation for autonomous vehicles based on multi-scale feature fusion," World Electric Vehicle Journal, vol. 14, no. 10, 2023. [Online]. Available: https://www.mdpi.com/2032-6653/14/10/291

[16] W. Qian, Z. Xia, J. Xiong, Y. Gan, Y. Guo, S. Weng, H. Deng, Y. Hu, and J. Zhang, "Manipulation task simulation using ros and gazebo," in 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014), 2014, pp. 2594-2598.

[17] R. Nunes, J. Ferreira, and P. Peixoto, "Synphorest - synthetic photorealistic forest dataset with depth information for machine learning model training," Mar. 2022.

[18] L. Wijayathunga, D. Dabare, A. Rassau, D. Chai, and S. M. S. Islam, "Outback: A multimodal synthetic dataset for rural australian off-road robot navigation," in 2024 International Conference on Digital Image Computing: Techniques and Applications (DICTA), 2024, pp. 397-402.

[19] C. Saltori, A. Osep, E. Ricci, and L. Leal-Taixé, "Walking your lidog: A journey through multiple domains for lidar semantic segmentation," 2023. [Online]. Available: https://arxiv.org/abs/2304.11705

[20] B. Michele, A. Boulch, G. Puy, T. H. Vu, R. Marlet, and N. Courty, "Saluda: Surface-based automotive lidar unsupervised domain adaptation," Proceedings - 2024 International Conference on 3D Vision, 3DV 2024, pp. 421-431, 2024.

[21] C. Saltori, F. Galasso, G. Fiameni, N. Sebe, E. Ricci, and F. Poiesi, "Cosmix: Compositional semantic mix for domain adaptation in 3d lidar segmentation," 2022.

[22] J. Kim, J. Woo, J. Kim, and S. Im, "Rethinking lidar domain generalization: Single source as multiple density domains," 2024. [Online]. Available: https://arxiv.org/abs/2312.12098

[23] H. Zhao, J. Zhang, Z. Chen, S. Zhao, and D. Tao, "Unimix: Towards domain adaptive and generalizable lidar semantic segmentation in adverse weather," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024, pp. 14 781-14 791.

[24] L. Stewart, F. Bach, Q. Berthet, and J.-P. Vert, "Regression as classification: Influence of task formulation on neural network features," 2023. [Online]. Available: https://arxiv.org/abs/2211.05641

[25] S. Jadon, "A survey of loss functions for semantic segmentation," in 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CICCB). IEEE, Oct. 2020, p. 1-7. [Online]. Available: http://dx.doi.org/10.1109/CICCB48159.2020.9277638

[26] H. Tang, Z. Liu, X. Li, Y. Lin, and S. Han, "Torchsparse: Efficient point cloud inference engine," 4 2022. [Online]. Available: http://arxiv.org/abs/2204.10319

[27] T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, "Optuna: A next-generation hyperparameter optimization framework," in The 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 2623-2631.

# APPENDIX

<table><caption>TABLE A.1: Appendix Table: Class mappings between the GOOSE and Rellis-3D datasets.</caption><thead><tr><td>GOOSE</td><td>Rellis-3D</td></tr></thead><tbody><tr><td>water</td><td>water</td></tr><tr><td>vehicle</td><td>car, bus, motorcycle, truck, military_vehicle, ego_vehicle</td></tr><tr><td>tree</td><td>tree_crown, tree_trunk, tree_root</td></tr><tr><td>rubble</td><td>debris</td></tr><tr><td>pole</td><td>pole, pipe</td></tr><tr><td>person</td><td>person</td></tr><tr><td>object</td><td>obstacle, traffic_cone, rock, barrel</td></tr><tr><td>grass</td><td>low_grass, high_grass</td></tr><tr><td>fence</td><td>fence</td></tr><tr><td>dirt</td><td>soil</td></tr><tr><td>concrete</td><td>curb</td></tr><tr><td>bush</td><td>hedge, bush</td></tr><tr><td>building</td><td>building, wall</td></tr><tr><td>barrier</td><td>road_block, boombarrier, guard_rail</td></tr></tbody></table>

<table><caption>TABLE A.2: Appendix Table: Class mappings between the OUTBACK, Rellis-3D and GOOSE datasets for Synthetic-to-Real Experiments.</caption><thead><tr><td>OUTBACK</td><td>Rellis-3D</td><td>GOOSE</td></tr></thead><tbody><tr><td>fence</td><td>fence, barrier</td><td>fence, guard_rail</td></tr><tr><td>grass</td><td>grass</td><td>low_grass, high_grass</td></tr><tr><td>ground</td><td>dirt, uphill, downhill</td><td>gravel, soil</td></tr><tr><td>log</td><td>log</td><td>—</td></tr><tr><td>robot</td><td>vehicle</td><td>ego_vehicle</td></tr><tr><td>rock</td><td>rubble</td><td>debris, rock</td></tr><tr><td>tree</td><td>tree</td><td>tree_crown, tree_trunk, tree_root</td></tr><tr><td>road</td><td>concrete</td><td>curb, asphalt</td></tr></tbody></table>

# ACKNOWLEDGMENT

This research was funded in whole or in part by the French National Research Agency (ANR) under the "ANR-23-MOXE-0003" project.