<table><caption>Table 5.4: **Panoptic Segmentation Experiment.** We report class-averaged panoptic metrics: SQ, RQ, PQ. The U-TAE appears essential to the quality of the prediction.</caption><thead><tr><th></th><th>SQ</th><th>RQ</th><th>PQ</th></tr></thead><tbody><tr><td>U-TAE + PaPs</td><td>81.3</td><td>49.2</td><td>40.4</td></tr><tr><td>U-ConvLSTM + Paps</td><td>80.9</td><td>40.8</td><td>33.4</td></tr></tbody></table>

true occupancy mask of the parcels. We combine all losses without scaling.

<figure><img src="image_56.png" /><figcaption>Figure 5.11: **Panoptic Segmentation.** The local maxima of the predicted centerness heatmap define *M* tentative parcels. We then predict a bounding box size, a semantic class, and an *S* × *S* shape patch for each detected centerpoint. Combined with a global saliency map, the shape patch allows us to predict pixel-precise masks.</figcaption></figure>

**Converting to Panoptic Segmentation.** Panoptic segmentation consists in associating to each pixel a semantic label and, for non-background pixels (our only *stuff* class), an instance label [KGHD19]. We resolve the overlaps of the previously described instance proposals with Non-Maximum Suppression. We report in Table 5.4 the performance of our approach, constituting the first state-of-the-art of SITS panoptic segmentation. See Figure 5.12 for qualitative illustrations of the advantage of our approach.

We adapt our temporal attention encoder to analyze remote sensing image time series. We propose a hybrid convolutional transformer architecture which defines a new state-of-the-art for semantic segmentation of agricultural parcels. We also propose the first dataset for evaluating the task of agricultural parcel panoptic segmentation, for which our approach sets up the first state-of-the-art.
<figure><img src="image_57.png" /><figcaption>Figure 5.12: **Qualitative Panoptic Segmentation Results.** We present a single image from the sequence using the RGB channels (a), and whose ground truth parcel boundaries and types are known (b). We then present our predicted panoptic segmentation (c), and semantic segmentation (d). Large, fragmented parcel are sometime declared as one single field ○, making their recovery difficult. Conversely, fragmented parcels ○ are correctly predicted as a single instance, suggesting that our network is able to use the temporal dynamics to recover ambiguous borders.</figcaption></figure>
# 4 Multi-Modal TAEs

The various sensors of Earth Observation satellites capture information of different natures and distinct spatial and temporal resolutions, with varying resilience to atmospheric conditions. In particular, C-band radar and optical images possess well-known synergies for automated crop mapping [VTGGP18, SWNW18, CTGHM$^{+19}$]. Optical time series contain highly relevant information for monitoring the evolution of plant phenology [VMD$^{+18}$, SBAK20], but are highly susceptible to cloud cover and atmospheric distortion [STAL20]. Conversely, radar acquisitions are impervious to cloud cover, which makes them uniquely well-suited for monitoring rapidly changing biological processes [MKL$^{+14}$]. However, radar is influenced by extrinsic factors such as humidity and terrain, making it harder to extract robust features.

The fusion of optical and radar time series has been extensively explored with traditional machine learning methods [VTGGP18, SWNW18, GBLC20], and more recently, recurrent neural networks [IIGM19]. As illustrated in Figure 5.13, we leverage this multimodality with temporal attention networks. We extend the work of [OAPL21] by assessing the benefit of combining optical for improving the precision and robustness to cloud cover of several crop mapping tasks. We also evaluate the effect of two simple training enhancements.

**Fusion Schemes.** As represented in Figure 5.14, we list four different schemes for merging image time series of different modalities:

* Early Fusion. We choose a pivot modality (e.g. optical) and temporally interpolate all other modalities to the acquisition dates of this pivot modality. We stack the resulting images in the channel dimension and process the sequence with a single spatio-temporal encoder.

* Mid Fusion. Each modality is processed by a dedicated spatial encoder. We merge the resulting embeddings into a single time series processed by a single temporal encoder.

* Late Fusion. Each modality is processed by a dedicated spatio-temporal encoder, and the resulting feature maps are concatenated channel-wise before classification.

* Decision Fusion. Each modality is classified independently, and we average their class scores.

**Training Enhancements.** Optical images contain richer information for crop mapping than radar acquisitions. This imbalance can cause the decision to overly rely on the optical modality, resulting in a weak supervisory signal for the other modality. We present two simple training enhancements to mitigate this issue:
<figure><img src="image_58.png" /><figcaption>Figure 5.13: **MultiModal Dataset.** We introduce the PASTIS-R dataset containing 2433 multimodal image time series of Sentinel-2 optical and Sentinel-1 radar data. We evaluate different fusion strategies and enhancements on parcel-based classification, semantic segmentation, and panoptic segmentation.</figcaption></figure>

* Auxiliary Supervision. All schemes except early fusion can benefit from the addition of auxiliary loss supervising each modality independently. This ensures that all modalities contain enough information to perform meaningful classification on their own.

* Temporal Dropout. During training, we randomly drop some acquisitions from the time series, with an increased probability for the optical modality. This prevents the network from overly relying on a single modality. Note that this enhancement also reduces the memory required for training, which can prove necessary for large models or image sequences.

**PASTIS-Radar.** To evaluate the benefit of multimodality, we extend the PASTIS dataset with corresponding Sentinel-1 observations. We separate the observations made in ascending and descending orbits into two different time series of around 70 3-channel images: vertical polarization (VV), horizontal polarisation (VH), and the ratio of vertical over horizontal polarization (VV/VH). We use the Ground Range Detected format processed into backscattering coefficient in decibels, orthorectified at a 10m spatial resolution. We do not apply spatial or temporal speckle filtering or radiometric terrain correction. The resulting data consists of 339k radar images and is available at github.com/VSainteuf/pastis-benchmark.

**Multi-Task Evaluation.** We report in Table 5.5 the improvements brought by the proposed fusion strategies and enhancements. We observe a consistent increase in precision compared to the individual modalities across all schemes, with late fusion being particularly well suited for parcel classification and semantic segmentation, while early fusion gives the best panoptic results. See Figure 5.16a for qualitative illustrations of our results.
<figure><img src="image_59.png" /><figcaption>Figure 5.14: **Evaluated Fusion Schemes.** We implemented four ways to embed time series of different modalities.</figcaption></figure>
<table><caption>**Table 5.5: Quantitative Evaluation.** We report the 5-fold cross-validated performance of the individual modalities and fusion schemes across three tasks. When it is possible to isolate their effect, we report the improvement of our proposed enhancements in parenthesis. ‡ indicates that the enhancements are detrimental, and * that the enhancements are necessary for fitting the model into memory. A dash — means that this model was not evaluated for memory issues or because its design is incompatible with the task.</caption><thead><tr><th rowspan="2">Modality</th><th rowspan="2">Parcel<br>Classification<br>mIoU</th><th rowspan="2">Semantic<br>Segmentation<br>mIoU</th><th colspan="3">Panoptic<br>Segmentation</th></tr><tr><th>SQ</th><th>RQ</th><th>PQ</th></tr></thead><tbody><tr><td>S1D</td><td>64.7 (+0.2)</td><td>54.9 ‡</td><td>77.0</td><td>39.3</td><td>30.9</td></tr><tr><td>S1A</td><td>63.3 ‡</td><td>53.8 ‡</td><td>77.4</td><td>38.8</td><td>30.6</td></tr><tr><td>S2</td><td>74.5 (+0.6)</td><td>63.6 (+0.5)</td><td>81.3</td><td>49.2</td><td>40.4</td></tr><tr><td>Early Fusion</td><td>76.5 (+1.6)</td><td>65.8 (+0.9)</td><td><strong>82.2</strong></td><td><strong>50.6</strong></td><td><strong>42.0</strong></td></tr><tr><td>Mid Fusion</td><td>76.5 (+1.4)</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>Late Fusion</td><td><strong>77.2 (+4.2)</strong></td><td><strong>66.3</strong> *</td><td>81.6</td><td>50.5</td><td>41.6</td></tr><tr><td>Decision Fusion</td><td>75.8 (+3.3)</td><td>64.3 *</td><td>-</td><td>-</td><td>-</td></tr></tbody></table>

**Robustness to Clouds.** In Figure 5.15b, we report the robustness to cloud cover brought by the fusion schemes and enhancements by artificially masking optical acquisitions during inference for parcel classification and semantic segmentation. Even with 70% to 90% of optical acquisitions missing, the fusion schemes outperform the model operating on radar time series.

Optical and radar acquisitions have known synergies for crop mapping. We design several multimodal schemes and related enhancements, resulting in higher precision and robustness to varying cloud cover across different crop mapping tasks.
<figure><img src="image_60.png" /><figcaption>Figure 5.15: **Varying Cloud Cover Experiment.** We evaluate different models with varying ratios of available optical observations remaining. In both parcel-based classification (a) and semantic segmentation (b), the fusion models prove robust to a reduced number of optical observations.</figcaption></figure>
<figure><img src="image_61.png" /><figcaption>Figure 5.16: **Qualitative Results for Panoptic Segmentation.** We compare the predictions made by unimodal models operating on S1D (a), S2 (b), the late fusion model (c), and the ground truth annotations (d). Some parcels are misclassified when only using the optical modality but are successfully recovered by the radar and fusion models (green circle ○). Some parcels are only detected using both modalities (red circle ○).</figcaption></figure>
# 5 Modelling Crop Rotations

<figure><img src="image_62.png" /><figcaption>Figure 5.17: **Multiyear Sentinel-2 Data.** Crop type for a part of our area of interest across three years.</figcaption></figure>

The impact of crop rotations is extensively studied in the agricultural optimization literature [DSG+12, DRVI03, BF15]. Given their measurable impact on crop yields [KKN+15], it is natural to assume that crop rotations significantly influence the choice of cultivated species. Consequently, taking past cultures into account when classifying parcels should improve our precision.

However, most crop mapping methods operate on a single year's worth of temporal acquisitions and ignore the inter-annual crop dynamics represented in Figure 5.17. Previous works propose to use probabilistic Markov models [OID15] or Conditional Random Field (CRF) [GBLC20] to learn transition statistics explicitly. Yaramasu et. al [YBP20] analyze multi-year data with a deep convolutional-recurrent model, but only select one image per year, thus ignoring intra-year dynamics. In contrast, we propose a model that operates at the intra-year scale through yearly sequences of observations and at the inter-annual scale by considering past cultures.

**Multi-Year Dataset.** In order to evaluate the impact of crop rotation, we present Sentinel2Agri-Multi, a version of the Sentinel2Agri dataset spanning 2018, 2019, and 2020. We select 103 602 *stable* parcels only, i.e. whose contours only undergo minor changes across the three studied years. Our dataset associates each parcel with three image time series and three crop annotations corresponding to the years 2018, 2019, and 2020.

**Inter-Year Training.** Given several years of annotated acquisitions, we observe that training a single model on a single multi-year dataset is ben-
eficial compared to training different models for each year independently. Indeed, the increased volume of data and varying temporal domains result in a network with better generalization, and mitigate the rare class issue. Implemented with a PSE+L-TAE baseline, the mixed-year approach outperforms specialized networks trained on a single year worth of data even for their year of specialization by up to 5 mIoU points for parcel classification. These results should encourage practitioners to combine several years of annotations when constituting SITS datasets.

**Modeling Rotations.** We consider the problem of classifying a parcel given SITS corresponding to several years, along with ground truth labels for the *past years*. We propose three strategies to exploit this data:

* $M_{\text{obs}}$: we process all yearly SITS independently and concatenate their spatio-temporal embeddings before classification.

* $M_{\text{lab}}$: we only process the SITS of the target year and concatenate to the resulting spatio-temporal feature the sum of the one-hot-encoded past labels.

* $M_{\text{CRF}}$: we only process the SITS of the target year and use a second-order CRF to model crop rotations from past labels.

These models can learn to resolve ambiguous observations by considering the history of a parcel. By processing past knowledge of labels and current observations end-to-end, the model $M_{\text{lab}}$ performs best. As shown in Table 5.6, this results in an appreciable improvement of over 6.3 points compared to only considering the last SITS. Further analysis shows that the most significant increase (16.9%) is for permanent crops (e.g. vineyards, meadow), followed by crops with statistically significant rotation patterns (e.g. rapeseed, soybeans) with 7.6%. Even for crops with no apparent rotation rules, this model provides an increase of 2.3% on average.

> We model multi-year patterns of agricultural cultivation by considering the parcel crop type history. Combined with our temporal attention model, this leads to a significant improvement across all cultures.

<table><caption>Table 5.6: **Performance by model.** Performances of the models $M_{\text{obs}}$ and $M_{\text{lab}}$ for the year 2020. We compare with the model $M_{\text{single}}$ trained on only the target year, and a CRF baseline $M_{\text{CRF}}$.</caption><thead><tr><th>Model</th><th>Description</th><th>OA</th><th>mIoU</th></tr></thead><tbody><tr><td>$M_{\text{single}}$</td><td>last SITS only with L-TAE</td><td>96.8</td><td>68.7</td></tr><tr><td>$M_{\text{obs}}$</td><td>SITS for all 3 years with L-TAE</td><td>96.8</td><td>69.3</td></tr><tr><td>$M_{\text{CRF}}$</td><td>last SITS with L-TAE + past labels with CRF</td><td>96.8</td><td>74.4</td></tr><tr><td>$M_{\text{lab}}$</td><td>last SITS + past labels with L-TAE</td><td>97.5</td><td>75.0</td></tr></tbody></table>
# 6 Leveraging Class Hierarchies

<figure><img src="image_63.png" /><figcaption>Figure 5.18: Class hierarchy for S2-Agri.</figcaption></figure>

Classification models typically focus on maximizing prediction accuracy, regardless of the semantic nature of errors. A step towards more reliable and interpretable algorithms would be to explicitly model and measure this gravity of errors [BMT+20]. In many classification problems, we can organize the class set according to a tree whose structure encapsulates the semantic similarity and discrepancy between classes in a hierarchical fashion. For a classification task over a set $\mathcal{K}$ of $K$ classes, we represent the class tree by a finite metric $D \in \mathbb{R}_+^{K \times K}$ such that $D[k, l]$ is the length of the shortest path between class $k$ and $l$ in the nomenclature tree. For a dataset indexed by $\mathcal{N}$, the Average Hierarchical Cost (AHC) between class predictions $y \in \mathcal{K}^N$ and the true labels $z \in \mathcal{K}^N$ is defined as [RDS+15, DBLFF10]:

$$ \text{AHC}(y, z) = \frac{1}{|\mathcal{N}|} \sum_{n \in \mathcal{N}} D[y_n, z_n]. \qquad (5.7) $$

Crop mapping especially benefits from predictions with a low hierarchical cost. Indeed, payment agencies monitor the allocation of agricultural subsidies and whether crop rotations follow best practice recommendations [Gra97]. The monetary and environmental impact of misclassifications are typically reflected in the class hierarchy designed by domain experts [BF15, Bul92], see Figure 5.18. By achieving a low AHC, we ensure that these downstream tasks can be meaningfully realized.
<figure><img src="image_64.png" /><figcaption>(a) Linear Classifier,
= 0.42,
ER= 15.2%, AHC = 0.81</figcaption></figure>

<figure><img src="image_65.png" /><figcaption>(b) Prototypical Classifier,
distortion= 0.47,
ER= 14.2%, AHC = 0.75</figcaption></figure>

<figure><img src="image_66.png" /><figcaption>(c) Guided Prototypes,
distortion= 0.22,
ER= 11.9%, AHC = 0.52</figcaption></figure>

<figure><img src="image_67.png" /></figure>

Figure 5.19: **Metric Guided Prototype.** We propose a hierarchical tree rep-
resenting the visual similarity between digits. The baselines (a) linear clas-
sifier and (b) prototypical classifier ignore this prior. The proposed regu-
larization term used in (c) leads to prototypes whose arrangement is con-
sistent with the class hierarchy. This results in a decrease of *Average Hierar-
chical Cost* (AHC), as well as *Error Rate* (ER), indicating that the taxonomy
may contain useful information for learning better visual features.

Beyond reducing the AHC, the class hierarchy may also contain infor-
mation about the data structure. Although it is not always the case, co-
hyponyms (i.e. siblings) in a class tree tend to share some structural prop-
erties. Encouraging such classes to have similar representations could lead
to more efficient learning, e.g. by leveraging shared feature detectors, as
exemplified in Figure 5.19.

**Metric-Guided Regularization.** We propose to integrate a pre-defined
class hierarchy into a prototype-based network [YZYL18, CLT+19]. We in-
troduce a regularization term encouraging the pairwise distance between
prototypes to reflect the error cost defined by the class tree. We define a
function $f : \mathcal{X} \to \Omega$ mapping inputs in $\mathcal{X}$ to the representation space $\Omega$.
Each class $k$ is associated with a prototypes $\pi_k \in \Omega$. A sample $x_n$ is classi-
fied with the label $k$ if $f(x_n)$ is closer to $\pi_k$ than all other prototypes. Both
$f$ and $\pi$ can be learned jointly with log-likelihood maximization [SSZ17].

We equip the embedding space Ω with a distance function *d*: Ω ×
Ω ↦→ ℝ+, such that (Ω, *d*) forms a continuous metric space. We say that
the prototypes π are consistent with the finite metric *D* if their pairwise
distance in (Ω, *d*) reflects their distance in *D*: the mapping *k* ↦→ π_k has
low distortion, as defined by De Sa *et. al* [DSGRS18]. We argue that this
property should lead to a lower hierarchical error: *near* misclassifications
are more likely to result in a *related* class. However, the scale induced by
the tree may be in conflict with the optimal arrangement of prototypes to
minimize the classification loss. We propose a new regularizer which is a
<figure><img src="image_68.png" /><figcaption>Figure 5.20: **Quantitative Results.** Error Rate (ER) in % and Average Hierarchical Cost (AHC) on four datasets for Guided-proto, the Linear Classifier baseline (Cross-Entropy), and competing approaches. Methods that use the hierarchical knowledge are indicated with the symbol * . We plot the best performances on each dataset in green. Our guided prototype approach improves the ER and AHC across the four datasets compared to the baseline. *: would not meaningfully converge.</figcaption></figure>

scale-free and differentiable surrogate of distortion:

$$
\mathcal{L}_{\text{disto}}(\pi) = \frac{1}{K(K-1)} \min_{s \in \mathbb{R}^{+}} \sum_{k,l \in \mathcal{K}^2, k \neq l} \left( \frac{sd(\pi_k, \pi_l) - D[k, l]}{D[k, l]} \right)^2 . \quad (5.8)
$$

Minimizing this regularizer encourages the pairwise distances between
prototypes $d(\pi_k, \pi_l)$ and their classes $D[k, l]$ to be close. Note that the inner
minimization problem can be efficiently solved in closed form.

**6.1 Numerical Experiments.**

We evaluate our approach across different tasks and datasets with fine-
grained class hierarchies: image classification on CIFAR100 [KH+09] and
the 1010-class iNaturalist-19 [VHMAS+18], RGB-D image segmentation on
NYUDv2 [NSF12], and image sequence classification on S2-Agri. We de-
fine class hierarchies for each dataset, such as the one represented in Fig-
ure 5.18. We only use classical backbone networks: ResNet18 [HZRS16]
for CIFAR100 and iNaturalist-19, FuseNet [HMDC16] for NYUDv2, and
PSe+LTAE for S2-Agri.
<figure><img src="image_69.png" /><figcaption>Figure 5.21: **Effect of Regularization.** Partial confusion matrices for the “living organism” class subset of CIFAR100 for the linear classifier baseline (a) and our approach (b). For readability, we only display (in black) entries of the matrices with at least one confusion. We also represent the cost of confusing different classes in shades of reds (c). We note that our approach yields fewer confusions between pairs of classes with high costs, such as plants and animals.</figcaption></figure>

We compare our approach with a linear classifier baseline and several reimplemented methods designed to leverage extrinsic class hierarchies. As seen in Figure 5.20, the benefits provided by our approach appear on all datasets. Compared to the linear classifier baseline, our model improves the AHC by 3% on NYUDv2 and S2-Agri, and up to 9% and 14% for CIFAR100 and iNat-19, respectively. While some methods perform on par or better than ours for some datasets, only our metric-guided prototypes consistently reduce the hierarchical cost across all tasks and datasets. We also observe a relative decrease of the error rate by 3 to 4% across all datasets compared to the linear classifier baseline. This indicates that cost matrices derived from the class hierarchies can help neural networks learn more discriminative representations.

In Figure 5.21, we represent the partial confusion matrices of the cross entropy baseline and our approach on CIFAR100, illustrating that guiding prototypes with a finite metric lead to fewer “high-cost” errors.

> We model the agreement between a prediction and a tree-shaped class hierarchy as the distortion between two metric spaces. Combined with a versatile prototype-based approach, our model signifi-
cantly reduces the weighted and unweighted error rates across vari-
ous datasets and tasks.

**Conclusion**

SITS analysis is a high-impact and complex machine learning task with
unique challenges. We propose a series of methods leveraging the particu-
lar structure of SITS for crop-type mapping applications. Our models lead
to higher precision, faster inference, and lower memory requirement for
parcel classification problems. We also propose the first method for panop-
tic segmentation on SITS, and new ways to exploit the synergy between
remote sensing modalities and the semantic structure of complex class sets.
We release 4 unique and novel datasets aiming to popularize SITS analysis
as a machine learning task and to encourage more reproducible science.
CHAPTER 6

# Perspectives

I identify two main directions for future work on geospatial machine learning: efficient learning with hierarchical partitions and reciprocal learning with many modalities.

# 1 Efficient Learning with Hierarchical Partitions

The cost of training state-of-the-art neural networks has reached unprecedented levels, effectively pricing out most public actors and practitioners [BHA<sup>+</sup>21]. Indeed, recent deep learning advances favour generality and scalability over efficiency [JGB<sup>+</sup>21], raising questions regarding the sustainability of the field. Exploiting the spatial regularity of geospatial processes could be the key to more sober and efficient machine learning. When analyzing a scene, we do not give the same attention to each square centimetre of every surface. Instead, we instinctively group our environment into homogeneous regions of various sizes and analyze their relationships at different scales: *this car is composed of four tires, a hood, and a roof; is on a road and under a tree; is in the middle of a commercial district*. However, traditional deep learning methods process the input's atomic elements (pixels, voxels, 3D points) *uniformly* despite the highly redundant information they carry, which leads to wasteful computations. Furthermore, this restricts the quantity of data considered simultaneously and prevents the modelling of long-range interactions. We could achieve more efficient learning by mimicking this two-step process: first compute a hierarchical partition of the input into homogeneous regions dubbed super-elements (e.g., super-pixel, super-points), then perform a multi-scale analysis of the super-elements. This would allow efficient and parsimonious high-level and large-scale reasoning without considering millions of individual atoms.

This approach goes exactly against the trend of large, generic, and powerful models [BHA<sup>+</sup>21] which aim to be free from any sort of inductive bias or human expertise [JGB<sup>+</sup>21]. The main idea is to leverage the spatial regularity of many spatial processes [Tob70]. This property translates into the spatial regularity of many latent variables of interest such as semantic or instance labels, or surface parameterization. Consequently, the solution to many spatial analysis problems is constant with respect to a partition of the input with much fewer components than the number of atomic elements. By computing such partition at multiple nested scale, we can define
<figure><img src="image_70.png" /><figcaption>Figure 6.1: **Architecture from Structure.** Instead of using cookie-cutter grid partitions (a), we build a hierarchical partition of a scene (b). We can then define a network whose architecture (c) is directly derived from the partition itself instead of from uniformly spaced convolutions. This allows the analysis of the input data to adapt to its local complexity. The red shaded rectangles indicate a step in which super-elements of the same level exchange information, e.g. with a transformer architecture.</figcaption></figure>

a hierarchical partition encompassing both small details and large struc-
tures. Instead of using this prior to improve atomic predictions *ex-post*
[KK11, LRV$^+$17, MKH19], we could define a network whose entire struc-
ture adapts to the data, while reducing the complexity of learning problems
by several orders of magnitude. This would significantly accelerate infer-
ence, training, and lead to more compact models with lighter hardware
requirements. Furthermore, multi-level hierarchical partitions allow us to
model long-range interactions, which would be particularly beneficial for
geospatial data.

2 Cross-Modal Reciprocal Learning

The supervised learning paradigm requires costly annotations, which can
be particularly expensive in some geospatial applications. For exam-
ple, forestry labels must be collected *in situ* in poorly accessible areas
[KLMC22a, KLMC22b], making large-scale annotation prohibitively costly.
Instead, we could train models by exploiting the diversity of remote sens-
ing sensors, their global scope, and systematic georeferencing.

Text-Image contrastive pretraining [CHL05] has shown impressive re-
sults in computer vision [OLV18], leading to influential foundation mod-
els [RKH+21, YCC+21]. Contrastive learning has been recently explored
for EO by exploiting the spatial alignment between time series [AUM+21]
and for cross-modal localization [TLB+22]. We could generalize this ap-
proach to the multi-modal setting with georeferenced observations. In other words, we want to align spatially the features extracted from acquisitions with different modalities: the descriptor of an area should be consistent regardless of the scanning sensor and different from other areas. By forcing spatial alignment across sensors capturing different information, the features must describe the only shared latent variable: the actual content of the considered area.

Generalizing contrastive learning to the multi-domain setting raises several theoretical and technical challenges. First, the classic two-modalities formulation leads to an exponential complexity w.r.t the number of sensors, quickly becoming impractical. Furthermore, current work produces a single descriptor per image, whereas we will consider pixel descriptors, worsening the combinatorics of the problem. Second, cross-modal learning implies the simultaneous training of several large networks and the manipulation of costly multi-modal batches. This raises technical issues such as inefficiency in memory use and prolonged training times. Lastly, if we only optimize the encoders associated with different sensors to produce spatially aligned features, sensor-specific information may be ignored. This would result in weaker individual representations discarding the unique strength of each sensor at the benefit of the lowest common denominator. However, such training would have a considerable impact as we could train single and multi-modal models using nothing but geolocalized observations. The resulting feature extractors could be fine-tuned on downstream tasks with much fewer annotations than if trained from scratch.

## 3 Other Works

I have also had the pleasure of collaborating with Mathieu Aubry (ENPC) on unsupervised object discovery in object-level [LMAL21] and large-scale [LVAL23] 3D datasets. Our approach involves learning representations in input space, rather than in an abstract feature space, allowing the models to provide interpretable and editable visualizations of their reasoning. This property is beneficial for application to industrial and public-policy settings. We also propose an application to sound discovery [LBT+22], showcasing the versatility of our proposed approach.
<figure><img src="image_71.png" /><figcaption>**Figure 6.2: Cross-Modal Contrastive Learning.** We consider a set of encoders for different inputs: image time series, LiDAR scans, hyperspectral images, etc. Each multi-modal patch is embedded by a dedicated encoder into a small 5 × 5 feature map of width 128, whose cells are trained to be co-linear with the corresponding cell in the same modality and contrasted with others.</figcaption></figure>
Short CV

**Summary** I am a machine learning/computer vision researcher with a strong interest in geospatial applications. My multi-disciplinary research aims to exploit the specific structure of complex tasks to develop more efficient solutions. I strive to create links between machine learning and geospatial communities by organizing workshops and conferences and co-chairing working groups.

Positions and Education

Since 2012, I have the status of civil servant of the French Ministry of
Ecology (IPEF).

• 2023-, IMAGINE, LIGM, ENPC : *Researcher*,

• 2015–2023, IGN/ENSG, LASTIG, UGE : *Researcher*,

* Sep 2012 - Sep 2016, ENPC ParisTech - INRIA, PhD
  *Learning structured models on weighted graphs, with applications to spatial data analysis. Advisors: Francis Bach and Guillaume Obozinski.*

• 2011 - 2012, ENS Cachan, MSc. Master MVA, machine learning.

* 2011 - 2012, ENPC ParisTech, MSc. Master IMI, computer science.

* 2007 - 2011, Ecole Polytechnique, MSc. Algorithmics.

Research

**Software Development** I am committed to reproducible research: every published article comes with its open-source implementation. I authored or participated as an advisor to over 30 repositories, including the following highlights:

• [O]loicland/superpoint-graph 642★ 207¥

• [O]loicland/cut-pursuit 64★ 18¥

• [O]torch-points3d/torch-points3d 2001 ★ 338 ¥

• [O]VSainteuf/pytorch-psetae 128 ★ 29 ¥

• [O]drprojects/DeepViewAgg 148 ★ 15 ¥
<figure><img src="image_72.png" /><figcaption>Breakdown of my post-PhD Publications across different fields.</figcaption></figure>

**Datasets** My students and I released 7 open-access datasets, totalling 2500+ downloads or over 300To :

* **HelixNet:** largest 3D datasets (10B points) for autonomous driving, contains unique sensor information for precise latency measurements.

* **PASTIS:** first dataset of satellite image time series with parcel borders and content (4000km², 2B pixels).

* **PASTIS-R & Multi-Year Sentinel:** companion datasets focusing on multimodality and multi-year acquisition, respectively.

* **StrataNet & WildForest3D:** first densely and weakly annotated aerial LiDAR datasets for forest inventory (over 2000 individual trees).

**Invited Talks** I have given over 20 invited talks, both for **academic institutions** (ETH Zurich, Ecole Polytechnique, Univ. Bologna, Politecnico di Milano, Univ. Erlangen, Tech. Univ. Danemark, Norwegian Inst. for Bioeconomics Research, Univ. Aix-Marseille, Univ. Montpelier, Univ. Paris-Est, SIAM symposium) and **leading industry research centers** (Valeo AI, Facebook AI Research, Sony CSL, ENGIE CRIGEN, Quantcube Tech.).

I also gave **keynote addresses** to the Urban3D ICCV Workshop (2021), the French Robotic Research Symposium (2019), the French Workshop on Mobile and Airborne LiDAR (2022), the 2nd International Workshops on
Point Cloud Processing (2019,2023), and the French Workshop on Graph Signals Processing (2019).

## Supervision and Community

**Graduated PhD students (with percentage of supervision):**

<table><tr><td>Raphael Sulzer</td><td>2018-2022</td><td>40%</td><td>now at INRIA Titane, FRA</td></tr><tr><td>Vivien S.F Garnot</td><td>2018-2021</td><td>90%</td><td>now at Univ. of Zurich, CHE</td></tr><tr><td>Stéphane Guinard</td><td>2017-2020</td><td>50%</td><td>now at Univ. of Laval, CAN</td></tr><tr><td>Mohamed Boussaha</td><td>2017-2020</td><td>50%</td><td>now at Gambi-M, FRA</td></tr></table>

**Ongoing students (with percentage of supervision):**

<table><tr><td>Damien Robert</td><td>2020-2023</td><td>100%</td><td>with ENGIE & B. Vallet</td></tr><tr><td>Romain Loiseau</td><td>2020-2023</td><td>80%</td><td>with M. Aubry (ENPC)</td></tr></table>

**Ongoing post-Doctoral Fellow:** Ekaterina Kalinincheva Jan 2021-Jan 2023

**Internships:** 14 master & industry interns

## Organization & Community

* **Program chair** of the XXIV ISPRS Congress (2022), **main event of the Remote Sensing community** (every 4 years), 743 papers published and 1500 participants from 59 countries, coordinating 200+ area chairs [LRE+22].

* **Organizing committee** of Earth Vision (2021,2022,2023), a **leading CVPR worskhop** at the intersection between remote sensing and computer vision.

* **Program chair** of the Conference on IGN Research (2019, 2020), (800+ participants) and LASTIG Research Seminar (250+ participants).

* **Editorial.**

- Editorial advisory board of IJPRS (Elsevier, IF:11.8)

- Reviewing Committee of Remote Sensing (MDPI, IF:5.3)

- Guest editor for IJPRS (Elsevier, IF:11.8).

* **Scientific Society:**

- Co-chair of the ISPRS WG *Temporal Geospatial Data Understanding*.

- Co-lead of the IEEE GRSS WG *Image and Signal Processing*.

* **Reviewing:** ICML, CVPR, ISPRS Congress, NeurIPS, ICCV, ECCV, ICLR, BMVC, IJCV, PAMI, IJDSA, IJPRS.

* **Awards:**

- Outstanding reviewer: ICML2021 (top 10%), CVPR2021 (top 10%), ECCV2022 (top 5%), ISPRS Congress 2022 (top 5%).

- CVPR 2022 Best paper finalist (top 33 / 2065).

* **Expertise:** ANR Grants, Dutch Research Council (NWO), Canadian Centres of Excellence (Mitacs).

* **Jury Member:** 3 PhD jury, 3 PhD committees, 1 assistant professor Jury.

* **Scientific Advisory Board** of the AI-based startup SAMP ( samp.ai).
## Teaching

* ENSG, *Course Instructor and Creator*, Deep Learning for Remote Sensing and Introduction to Machine Learning (24 hours yearly).

* EduSERV for EUROSDR, MOOC *instructor and course creator*, Deep learning for remote sensing (2 weeks).

* Master IMI (ENPC), *Teaching assistant*, Machine learning.

* Master MVA (ENS), *Teaching assistant*: Probabilistic graphical models, *Invited Speaker*: 3D Deep Learning

* Master AMMI (Rwanda), *Teaching assistant*, Probabilistic Graphical Models (2 weeks intensive training).

## Projects and Grants

* **Principal investigator** of the ANR JCJC **READY3D**: Real-Time Analysis of Dynamic LiDAR 3D Point Clouds (total cost: 476k€, 194k€ subsidy).

* Participant of the **BIOM** ANR Project (total cost: 1776k€, 723k€ subsidy).

* DGA PhD Grant (100k€ subsidy).

* ASP Agence de Services et de Paiement - 300k€ research Grant.

* Joint PhD between ENGIE, IGN and Univ. Paris Est (250k€).
# Synthèse en Français

De par leur richesse, échelle et complexité, les données géospatiales sont un excellent terrain d'inv-estigation pour l'apprentissage automatique. Plutôt que d'utiliser des outils existant issus de la vision par ordinateur ou de l'apprentissage machine, notre motivation est d'exploiter la structure particulière des données géospatiales pour proposer des algorithmes adaptés et innovants. En accordant nos méthodes aux tâches considérées, nous atteignons d'important gains de vitesses, de performance, et une réduction drastique de la taille des modèles considérés.

## 1 Contributions Scientifiques

Je présente dans cette section un rapide aperçu de mes contributions scientifiques et de mes charges d'encadrements et organisationnelles à l'IGN au cours de ces 6 dernières années.

J'ai ensuite rejoint l'IGN (LASTIG, ENSG, UGE), où je me suis intéressé à l'apprentissage automatique pour les données géospatiales.

**Publications.** Je situe mes recherches à l'interface entre la vision par ordinateur et l'apprentissage automatique d'une part, et la télédétection et photogrammétrie d'autre part. Je publie avec mes étudiants dans des journaux spécialisés et thématiques de la communauté de télédétection et de photogrammétrie (IJPRS, JAEOG, PERS, Remote Sensing) mais aussi dans les conférences méthodologiques très sélectives de vision et d'apprentissage. En particulier, j'ai depuis ma thèse publié à CVPR (4 papiers dont 2 oraux et un best paper finalist, 3 workshops), ICML (1 papier et 2 workshops), et 2 papier à I/E.CCV. J'ai aussi publié dans des conférences plus spécialisées comme 3DV (2 papiers dont 1 oral), BMVC, SGP Eurographics et ICPR.

**Logiciel et Donnée Libre.** Mes étudiants et moi avons publié plus de 22 dépôts de code libres liés à nos projets, accumulant plus 3000 étoiles sur GitHub et 500 branches. Nous avons aussi constitué 7 jeux de données libres aux propriétés uniques, totalisant plus de 2000 téléchargements.

**Organisation.** J'ai été Program Chair du congrès 2022 de l'International Society of Photogrammetry and Remote Sensing (ISPRS), le plus grand évènement de la communauté, qui a lieu une fois tous les 4 ans et a rassemblé 1500 chercheurs de 68 pays différents. Je participe aussi à l'organisation du Workshop CVPR EarthVision (2021 et 2022), qui a pour but de rassembler les chercheurs en vision intéressés par les
problématiques d'observation de la Terre. J'ai aussi organisé les journées de la recherche de l'IGN sur le thème de l'IA pour la géomatique (800 participants, virtuels) et plusieurs séminaires inter-disciplinaires sur le sujet rassemblant jusqu'à 250 participants (virtuellement).

**Responsabilité Éditoriale.** Je suis au comité de rédaction du journal de l'ISPRS (IF: 9.0) et de Remote Sensing (IF: 5.3). Je fait partie des comités de lecture de nombreuses conférences et journaux de vision par ordinateurs (CVPR, ECCV, ICCV, ACCV, BMVC, PAMI, IJCV) et d'apprentissage (ICLR, NeurIPS, ICML). J'ai été distingué plusieurs fois comme outstanding reviewer: CVPR21, ICML21, ISPRS22. Depuis 2022, je suis également Working Group Officer pour le pôle Understanding Temporal Data de l'ISPRS, et Co-lead du Working Group IEEE GRSS sur Image and Signal Processing.

**Encadrements.** J'ai co-encadré 3 doctorants qui ont défendus avec succès leur thèse, et je co-encadre actuellement 3 autres étudiants et une post-doctorante. J'ai encadré 14 stages de master ou industriels.

**Financements.** Je suis porteur principal de l'ANR JCJC ReADy3D sur la perception 3D temps réel pour la conduite autonome (476k€, 194k€ de subvention). Je participe aussi à ANR (BIOM, 1776k€), et ai contribué à l'obtention de financements auprès d'acteurs privés (ENGIE: 250k €) et publics (ASP: 300k €, DGA: 100k €).

## 2 Spécificités des Données Géospatiales

L'analyse des données géospatiales a plusieurs atouts qui en font un excellent terrain d'application pour l'apprentissage automatique. En particulier, ces données sont typiquement acquises à grande échelle par une variété de capteurs, et possèdent une structure complexe qui mêle les dimensions spatiale, temporelle, et spectrale. Il existe de nombreuses sources de données géospatiales et d'annotations en libre accès. Leur analyse automatique à grande échelle mène de nombreuses applications à fort impact, autant économique qu'environnemental et social.

Malgré une ressemblance superficielle, les données géospatiales présentent des différences importantes avec les images ou vidéos naturelles typiquement traitées en vision par ordinateur. La structure unique des données géospatiales nécessite donc de développer des méthodes adaptées. Le principe guidant l'ensemble de mes travaux est d'exploiter cette structure pour mettre au point des algorithmes et architectures adaptés afin d'améliorer la vitesse et précision de leur analyse.

Dans ce manuscrit, nous présentons certains des travaux que j'ai conduit avec mes étudiants depuis l'obtention de mon doctorat en septembre
2016. En particulier, nous développons la manière dont nous avons exploité les structures suivantes:

* *Structure de régularité sur Graphe:* nous proposons une approche mathématique pour résoudre certains problèmes d'optimisation en tirant parti d'une propriété de régularité habituellement rencontrée lors de l'analyse de données géospatiales.

* *Structure de Régularité des Données 3D:* ces méthodes exploitent le même principe que le chapitre précédent mais dans le cadre de l'apprentissage profond pour l'analyse automatique de grands nuages de points.

* *Structure des Capteurs de Données 3D:* nous proposons plusieurs approches exploitant la structure particulière de certains capteurs 3D pour améliorer la rapidité et la précision de leur analyse.

* *Structure des Séquences Temporelles Satellite:* nous présentons une série d'algorithmes pour l'analyse automatique de séries temporelles d'images satellites, améliorant significativement l'état de l'art de ce domaine.

## 3 Structure de Régularité sur Graphe

Cette première section présente nos travaux les plus théoriques, où nous montrons comment une propriété abstraite de régularité définie par rapport à un graphe général peut être exploitée pour considérablement accélérer la résolution d'une large classe de problèmes d'optimisation. Dans certaines circonstances, notre approche apporte également des garanties théoriques inédites et nécessitant peu d'hypothèses sur les fonctions considérées. Nos travaux s'appliquent à de nombreux problèmes géo-spatiaux, pour lesquels nous parvenons à réduire les temps de calculs de plusieurs ordres de grandeur, c.f. Figure 7.1. Cette section reprend les publications suivantes:

[LO17] Loic Landrieu, Guillaume Obozinski, "Cut Pursuit: Fast Algorithms to Learn Piecewise Constant Functions on General Weighted Graphs", SIAM Journal of Imaging Science, 2017

[RL18] Raguet Hugo, Loic Landrieu, "Cut-Pursuit Algorithm for Regularizing Nonsmooth Functionals with Graph Total Variation", ICML, 2018

[RL19] Raguet Hugo, Loic Landrieu, "Parallel Cut Pursuit For Minimization of the Graph Total Variation", ICML Workshop on Graph Reasonning, 2019

[GLCV19] Stéphane Guinard, Loic Landrieu, Laurent Caraffa, Bruno Vallet, "Piecewise-Planar Approximation of Large 3D Data on Graph-Structured Optimization", ISPRS Annals, 2019
<figure><img src="image_73.png" /><figcaption>Figure 7.1: **Cut Pursuit.** Évolution de la distance à la solution pour différentes versions de Cut Pursuit (CP, PCP, PCP-bal) et autres méthodes (PFDR [RL15], PMF [CD09]) et différentes tâches. L'accélération peut atteindre plusieurs ordres de grandeurs.</figcaption></figure>
**Contexte.** Nous considérons le problème d'optimisation consistant à minimiser une fonction dont les variables $x \in \Omega^V$ sont définies selon les sommets d'un graphe $G = (V, E, w)$ avec des arêtes aux poids $w \in \mathbf{R}_+^E$. En particulier, nous nous intéressons aux fonctions $F$ qui peuvent se décomposer de la manière suivante :

$$F(x) := f(x) + \sum_{(u,v) \in E} w_{u,v}h(x_u, x_v), \qquad (7.1)$$

avec $f: \Omega^V \to \mathbf{R}$ et $h: \Omega \to \mathbf{R}$ une fonction qui atteint son minimum uniquement quand $x_u = x_v$. Ces fonctions apparaissent naturellement lors de l'analyse de problèmes ayant une composante spatiale, comme les données géospatiales ou l'imagerie médicale. Cette formulation couvre une large famille de problèmes d'optimisation fonctionnelle, telle que la régularisation par variation totale sur un graphe général ($h = \| \cdot \|$) et la régularisation par longueur des contours ($h = [\cdot \neq 0]$).

**Cut Pursuit.** Les points critiques $x^*$ de $F$ exhibent typiquement une forme de régularité par rapport au graphe $G$: les sommets connectés ont souvent des valeurs identiques. Par exemple, le label sémantique associé à un point 3D est partagé par la majorité de ses voisins. En conséquence, les solutions $x^*$ sont constantes par rapport à une partition $\mathcal{V}$ de $V$ avec un faible nombre de composantes. En imposant que les variables $x$ soient constantes par rapport à $\mathcal{V}$, la minimisation de $F$ devient plus simple. En effet, ce problème réduit n'a que $|\mathcal{V}|$ variables au de $|V|$, tout en gardant une structure similaire.

Cette propriété peut être exploitée algorithmiquement pour accélérer la minimisation de $F$. En effet, chaque solution est entièrement définie par une partition et la valeur associée à chacune de ses composantes. L'algorithme cut pursuit alterne entre la minimisation du problème réduit contraint par la partition courante $\mathcal{V}$ et le raffinement de $\mathcal{V}$ en composantes plus petites. La première étape peut être effectuée efficacement de par sa faible dimension, et la seconde peut se formuler sous la forme d'un problème de coupe minimale dans un graphe de flot bien choisi.

**Performance.** L'approche proposée est avantageuse quand il existe une solution $x^*$ constante par rapport à une partition $\mathcal{V}$ telle que $|\mathcal{V}| \ll |V|$. Dans ce cas, l'algorithme cut pursuit apporte une accélération pouvant atteindre 1000x, même pour des problèmes inverses complexes. De plus, notre approche est entièrement parallèle grâce à la première implémentation *multi-thread* de l'algorithme de Ford-Fulkerson [FF56].

**Garanties.** L'algorithme cut pursuit s'applique à une grande variété de problèmes. Pour le cas de la variation totale, notre approche permet
<figure><img src="image_74.png" /><figcaption>Figure 7.2: **Superpoint Graph.** Un nuage de points (a) est partitionné en formes géométriquement simples (b). Un graphe reliant les superpoints adjacents est construit (c), permettant leur classification (d).</figcaption></figure>

non seulement une résolution efficace, mais apporte un certificat de crit-
icalité à précision machine et une garantie de convergence en un nombre
fini d'itérations. Contrairement à la plupart des méthodes existantes, nos
preuves ne nécessitent ni la convexité ni la differentiablité de f.

**Versatilité.** Notre approche peut aussi se décliner pour la régularisation par longueur des contours des zones constantes, qui n'est ni continue, ni différentiable, ni convexe. Néanmoins, notre approche reste plus rapide que le populaire algorithme α-expansion quand la solution recherchée est simple. Nous proposons aussi une variation de notre approche pour l'approximation de grands nuages de points 3D en un faible nombre de plans. Ici encore, quand la solution est bien simple (i.e. peu de plans), notre méthode apporte une accélération de plus d'un ordre de grandeur comparé à l'état de l'art.

4 Structure de Régularité des Données 3D

Dans cette section, nous proposons des approches pour la segmentation sémantique et la reconstruction de surface pour de très grands nuages de points. Ces méthodes sont basées sur les mêmes idées que la section précédente, mais adaptées au cadre de l'apprentissage profond. Cette section est principalement basée sur les articles suivants:

[LS18]: Loic Landrieu, Martin Simonovsky, "Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs", CVPR, 2018

[LB19]: Loic Landrieu, Mohamed Boussaha, "Point Cloud Oversegmentation with Graph-Structured Deep Metric Learning", CVPR, 2019

[SLMV21]: Raphael Sulzer, Loic Landrieu, Renaud Marlet, Bruno Vallet, "Scalable Surface Reconstruction with Delaunay-Graph Neural Networks", Symposium on Geometry Processing (SGP), 2021
**SuperPoint Graph.** Nous introduisons l'approche Superpoint Graph pour la segmentation sémantique de nuages de points 3D à large échelle. Cet algorithme consiste à calculer une partition du nuage de points en formes simples, appelées superpoints, et la création d'une structure de graphe liant les superpoints voisins. Au lieu de classifier chaque point indépendamment, nous classifions les superpoints eux-même, réduisant ainsi la complexité du problème de plusieurs ordres de grandeur. En d'autre terme, nous transformons le problème de segmentation sémantique d'un grand nuage de points en la classification des sommets d'un graphe avec peu de sommets. Cela nous permet d'utiliser des puissants outils de convolution sur graphe pour analyser le contexte de grandes scènes, cf. Figure 7.2. Notre approche permet de classifier avec d'excellents résultats des nuages contenant des dizaines de millions de points avec des réseaux de neurones efficaces en temps d'inférences et en nombre de paramètres.

**Segmentation Supervisée.** La limitation de l'approche précédente est que toute erreur dans la partition entraîne des mauvaises classifications. Nous proposons donc d'entraîner un réseau de neurones à segmenter un grand nuage de points en superpoints sémantiquement purs. Nous mettons au point une nouvelle fonction de perte contrastive permettant d'apprendre une représentation de points qui soit homogène au sein des objets et présente de forts contrastes à leur interface. La partition en découlant aboutit à une amélioration importante de la qualité de la segmentation sémantique.

**Reconstruction Large Échelle.** Nous proposons une approche pour la reconstruction grande de surface à grande échelle basée sur une discrétisation de l'espace 3D par une tetraèdrisation de Delaunay. Nous adaptons les classiques modèles d'énergie sous-modulaire, qui peuvent être efficacement résolus par minimisation de flots, mais dont les potentiels sont appris par un réseau de neurones sur graphe. Notre méthode combine donc la richesse des méthodes d'apprentissage et la rapidité et les garanties des méthodes énergétiques. Entraînée sur un faible nombre de scans virtuelles, notre approche généralise à de nouveaux domaines et à de très grandes scènes avec une qualité supérieure aux modèle d'apprentissage et énergétiques.

# 5 Structure des Capteurs de Données 3D

Dans cette section, nous présentons nos travaux ayant pour but d'exploiter la géométrie particulière des acquisitions de télédétection 3D pour améliorer la précision et la vitesse d'algorithmes de segmentation
<figure><img src="image_75.png" /><figcaption>Figure 7.3: **Structure des Capteurs.**</figcaption></figure>

Nous exploitons la structure d'acquisition des LiDARs rotatifs, la complémentarité images/LiDAR, et les informations de visibilités qui peuvent être déduites de la pose des capteurs.

sémantique et de reconstruction de surface. Ce chapitre est basé sur les
publications suivantes:

[LAL22]: Romain Loiseau, Mathieu Aubry, Loic Landrieu, "Online Segmentation of LiDAR Sequences: Dataset and Algorithm", ECCV, 2022

[RVL22]: Damien Robert, Bruno Vallet, Loic Landrieu, "Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation", CVPR, 2022

[SLB+22]: Raphael Sulzer, Loic Landrieu, Alexandre Boulch, Renaud Marlet, Bruno Vallet, "Deep Surface Reconstruction from Point Clouds with Visibility Information", ICPR, 2022

**LiDAR Rotatif Mobile.** Les LiDARs rotatifs sont très utilisés dans le cadre de la conduite autonome, motivant ainsi le besoin d'outils d'analyse temps réel de séquences temporelles de points 3D. La structure de ces séquences est complexe: le capteur tourne rapidement sur lui-même, la plate-forme d'acquisition est en mouvement, et les environnements urbains sont typiquement dynamiques; voir Figure 7.3a pour une illustration.

La plupart des jeux de données et algorithmes proposés pour l'analyse
de telles séquences opèrent sur un découpage en trames couvrant 3600°,
ce qui conduit à une latence d'acquisition incompatible avec les applica-
tions temps réels. Nous répondons à ce problème avec deux contributions
principales. Tout d'abord, nous présentons HelixNet, un dataset de 10 mil-
liards de points 3D annotés individuellement avec labels sémantiques et
les informations de capteur nécessaires à la mesure précise de la capacité
temps réel des algorithmes de segmentation. Nous proposons également Helix4D, un réseau spatio-temporel compact et efficace spécialement conçu pour l'analyse de séquences de points 3D issues d'un LiDAR rotatif. Helix4D opère sur des tranches d'acquisition couvrant une fraction de tour du capteur sur lui-même, réduisant significativement la latence totale.

Nous évaluons la performance et latence de plusieurs algorithmes de l'état de l'art sur HelixNet et SemanticKITTI. Helix4D atteint une précision comparable aux algorithmes de segmentations les plus récents avec une réduction de plus de 5× en terme de latence, et 50× du nombre de paramètres.

**Complémentarité Image/LiDAR.** Les images et les nuages de points contiennent des informations différentes et complémentaires: les images capturent la texture et le contexte des objets, alors que les nuages de points reflètent leur géométrie avec précision, c.f. Figure 7.3b. Les approches hybrides 2D/3D proposent de combiner ces informations pour améliorer la précision de l'analyse de grandes scènes.

Combiner images et nuages de points issus de grandes scènes présente de nombreuses difficultés, telles que la mise en relation des points et des pixels et l'agrégation des descripteurs issus de différentes images. Les méthodes actuelles nécessitent souvent la construction d'un maillage ou l'usage de capteurs spécialisés pour trouver les occlusions, et utilisent des heuristiques pour combiner les images.

Nous proposons une méthode d'agrégation multi-vues exploitant les conditions d'observations (angle de vue, distance, occlusion, etc.) des points 3D dans les images. Notre approche permet de combiner des réseaux 2D et 3D standards et d'obtenir des résultats supérieurs aux réseaux classiques ou hybrides sans nécessiter de colorisation, de maillage, ni de capteur de profondeur. Notre approche opère sur les nuages de points bruts, et un nombre arbitraire d'images avec pose. Nous définissons un nouvel état de l'art pour un jeu de donnée de segmentation de scènes d'intérieur et un autre jeu de données centré sur la conduite autonome.

**Reconstruction avec Information de Visibilité.** Les méthodes actuelles de reconstruction de surface à partir de nuages de points 3D ignorent les informations de pose et n'utilisent que la position des points. Cependant, la pose des capteurs permet de déduire des informations de visibilité (ligne de vue) qui sont précieuses pour la reconstruction et l'orientation des surfaces prédites. Nous proposons deux manières simples d'enrichir chaque point 3D avec ses informations de visibilité: (i) ajout d'un vecteur pointant vers le capteur, (ii) ajouts de points virtuels le long de la ligne de vue, c.f. Figure 7.3c. Ces deux modifications peuvent s'intégrer facilement à de nombreuses approches existantes avec modifications minimales.
Notre méthode améliore la précision des surfaces générées mais aussi les capacités de généralisation des réseaux à des nouveaux domaines: nouvelles classes, scènes entières, nouveaux capteurs.

# 6 Structure des Séquences Temporelles Satellite

Cette section présente nos travaux sur l'analyse automatique de séries temporelles d'images satellites (SITS) comme représentée à la Figure 7.4. De par l'accessibilité de grandes quantités de données annotées, l'analyse automatique de SITS est un exemple de premier plan de l'intérêt des approches d'apprentissage pour l'analyse des données géospatiales. Cependant, les SITS suivent une structure particulière nécessitant des approches adaptées: multimodalité, faible résolution spatiale, spatio-spectro-temporalité, et grande échelle. Nous proposons une série d'algorithmes exploitant cette structure pour améliorer la vitesse et précision de l'analyse des SITS. Cette section est basée sur les articles suivants:

[GLGC20]: Vivien Sainte Fare Garnot, Loic Landrieu, Sebastien Giordano, Nesrine Chehata, "Satellite image time series classification with pixel-set encoders and temporal self-attention", CVPR, 2020

[GL21b]: Vivien Sainte Fare Garnot, Loic Landrieu, "Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks", ICCV, 2021

[GL20]: Vivien Sainte Fare Garnot, Loic Landrieu, "Lightweight Temporal Self-Attention for Classifying Satellite Image Time Series", ECML Workshop on Advanced Analysis and Learning on Temporal Data, 2020

[GL21a]: Vivien Sainte Fare Garnot, Loic Landrieu, "Leveraging Class Hierarchies with Metric-Guided Prototype Learning", BMVC, 2021

[GL22]: Vivien Sainte Fare Garnot, Loic Landrieu, "Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series", ISPRS Journal, 2021

[QL21]: Félix Quinton, Loic Landrieu, "Crop Rotation Modeling for Deep Learning-Based Parcel Classification from Satellite Time Series", Remote Sensing, 2021

[GBLC20]: Giordano, Sébastien and Bailly, Simon and Landrieu, Loic and Chehata, Nesrine, "Improved crop classification with rotation knowledge using Sentinel-1 and -2 time series", Photogrammetric Engineering & Remote Sensing, 2021

[GLGC19]: Vivien Sainte Fare Garnot, Loic Landrieu, Sebastien Giordano, Nesrine Chehata, "Time-space trade-off in deep learning models for crop classification on satellite multi-spectral image time series", IGARSS, 2020

**Classification de Parcelles.** De par leur grande accessibilité, les séries temporelles d'images satellites sont au centre d'un effort d'automatisation
<figure><img src="image_76.png" /><figcaption>Figure 7.4: **Segmentation Panoptique de SITS.** Notre objectif est de prédire automatiquement les cultures et contours de parcelles agricoles à partir de séries temporelles d'images satellite.</figcaption></figure>

des données d'observation de la terre. En particulier, le suivi des cultures agricoles est un sujet d'importance écologique, économique, et politique majeure. En effet, plus de 57 milliards de subvention par an de subvention sont distribuées chaque année en Europe aux agriculteurs, et la rotation des cultures a un important impact environnemental mais aussi le rendement des cultures.

Les approches les plus couramment utilisées pour cette tâche mêlent des réseaux convolutionnels pour l'aspect spatial et des réseaux récurrents pour la dimension temporelle. Observant que la résolution des images satellites à faible temps de revisite ne permet pas de capturer la texture des cultures, nous proposons de considérer chaque parcelle comme un ensemble non ordonné d'observations radiométriques multi-spectrales. Cette approche permet d'apprendre efficacement des descripteurs de la réponse spectrale des parcelles sans avoir à les redimensionner, ce qui économise à la fois du temps de calcul et de la mémoire.

Nous adaptons également les méthodes d'auto-attention si efficace en traitement de la langue au contexte des séquences d'images satellite. Nous proposons une série de modifications visant à améliorer la performance et de la vitesse de ces méthodes pour notre cas d'usage, aboutissant à une architecture légère et performante qui est maintenant largement utilisée dans le domaine de la classification de séries temporelles. Pour l'évaluation, nous mettons à disposition un dataset de parcelles annotées et leurs séries temporelles correspondantes.

**Segmentation Panoptique de Parcelles.** Nous proposons une extension de nos travaux précédents pour prédire non seulement les cultures mais aussi la forme des parcelles agricoles. Ce problème peut se formuler comme la segmentation panoptique de séries temporelles d'images: chaque pixel est associé à un label sémantique, et les pixels de cultures sont associés à l'index de leur parcelle.

Ce problème a été étudié pour les images satellite, mais la dimension
temporelle reste ignorée alors qu'elle est critique pour la classification des espèces végétales. Nous présentons la première approche d'apprentissage pour la segmentation panoptique de série temporelle d'images satellites. Un premier module combine des convolutions spatiales et l'approches d'attention temporelle décrite ci-dessus pour extraire de riches descripteurs spatio-temporels multi-échelles. Un second module permet de prédire efficacement et précisément le contour des parcelles agricoles.

Nous introduisons aussi PASTIS, le premier jeu de données libre de séries temporelles d'images satellites avec annotation panoptique, et couvrant plus de 4000 mk². Évalué sur cet dataset, nous démontrons la supériorité de notre encodeur spatio-temporel pour la segmentation sémantique, et proposons le premier état de l'art de la segmentation panoptique de séries d'images satellite.

**Extension Multi-Années.** Les rotations de cultures jouent un rôle important sur l'impact environnemental et le rendement des cultures. Nous proposons un modèle capable d'apprendre les rotations de cultures pour améliorer leur classification. Nous introduisons également le premier jeu de données multi-années de séries temporelles d'images satellite. Notre approche améliore les performances de classification comparés aux modèle mono-années, mais aussi par rapport aux approches existantes pour prendre en compte les rotations.

**Multimodalité Optique-Radar.** Les capteurs optiques passifs superspectraux et le radar à synthèse d'ouverture permettent d'acquérir depuis l'espace des informations complémentaires sur les cultures. Nous proposons une nouvelle approche pour combiner ces informations pour l'analyse de SITS. Notre méthode repose sur les modules d'attention temporelle décrits ci-dessus, et peut combiner différentes modalités sans nécessiter d'alignement temporel, d'interpolation, ou de pré-traitement.

Nous proposons également une extension du jeu de données PASTIS avec les séries radar associées à chaque série optique. Nous évaluons nos approches et ses variations sur différentes tâches, allant de la classification de parcelles à la segmentation panoptique. Nous définissons un nouvel état de l'art pour chacune des tâches considérées grâce à l'apport de la multimodalité.

**Classification Hiérarchique.** Les espèces agricoles peuvent être organisées selon une hiérarchie de classes, à l'instar de nombreuses autres nomenclatures. Cette structure induit une distance sémantique entre classes, et définit ainsi une métrique discrète.

Nous proposons une régularisation permettant d'apprendre conjointement des descripteurs et des prototypes de classes qui suivent un arrange-
ment en accord avec la distance sémantique entre leur classes. Formelle-
ment, nous minimisons la distorsion enter la métrique discrète entre
classes et la métrique continues entre prototypes. Nous évaluons notre
approche sur différentes tâches, de la classification de séries temporelles
à la segmentation sémantique d'images de profondeur, et observons une
baisse systématique du taux d'erreur pondéré par leur coût. De façon
plus surprenante, dans certains cas le taux d'erreur non pondéré diminue
également, démontrant que l' injection de connaissance sur la hiérarchie des
classes permet aussi d'améliorer la qualité des descripteurs appris.
Bibliography

Defendant's References

[CCHL20] Thomas Chaton, Nicolas Chaulet, Sofiane Horache, and **Loic Landrieu**. Torch-Points3D: A modular multi-task framework for reproducible deep learning on 3D point clouds. 3DV, 2020.

[GBLC20] Sébastien Giordano, Simon Bailly, **Loic Landrieu**, and Nesrine Chehata. Improved crop classification with rotation knowledge using sentinel-1 and-2 time series. *Photogrammetric Engineering & Remote Sensing*, 2020.

[GL17] Stéphane Guinard and **Loic Landrieu**. Weakly supervised segmentation-aided classification of urban scenes from 3D LiDAR point clouds. *ISPRS Workshops 2017*, 2017.

[GL20] Vivien Sainte Fare Garnot and **Loic Landrieu**. Lightweight temporal self-attention for classifying satellite images time series. *ECML Workshop on Advanced and Learning on Temporal Data*, 2020.

[GL21a] Vivien Sainte Fare Garnot and **Loic Landrieu**. Leveraging class hierarchies with metric-guided prototype learning. *BMVC*, 2021.

[GL21b] Vivien Sainte Fare Garnot and **Loic Landrieu**. Panoptic segmentation of satellite image time series with convolutional temporal attention networks satellite image time series with convolutional temporal attention networks. *ICCV*, 2021.

[GL22] Vivien Sainte Fare Garnot and **Loic Landrieu**. Multi-modal temporal attention models for crop mapping from satellite time series. *ISPRS Journal of Photogrammetry and Remote Sensing*, 2022.

[GLCV19] Stéphane Guinard, **Loic Landrieu**, Laurent Caraffa, and Bruno Vallet. Piecewise-planar approximation of large 3D data as graph-structured optimization. *ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences*, 2019.
[GLGC19] Vivien Sainte Fare Garnot, Loic Landrieu, Sebastien Gior-
dano, and Nesrine Chehata. Time-space trade-off in deep
learning models for crop classification on satellite multi-
spectral image time series. IGARSS, 2019.

[GLGC20] Vivien Sainte Fare Garnot, **Loic Landrieu**, Sebastien Gior- dano, and Nesrine Chehata. Satellite image time se- ries classification with pixel-set encoders and temporal self- attention. CVPR, 2020.

[KLMC22a] Ekaterina Kalinicheva, Loic Landrieu, Clément Mallet, and Nesrine Chehata. Multi-layer modeling of dense vegetation from aerial LiDAR scans. CVPR Workshops, 2022.

[KLMC22b] Ekaterina Kalinicheva, **Loic Landrieu**, Clément Mallet, and Nesrine Chehata. Predicting vegetation stratum occupancy from airborne LiDAR data with deep learning. *Journal of Applied Earth Observation and Geoinformation*, 2022.

[LAL22] Romain Loiseau, Mathieu Aubry, and Loic Landrieu. On-line segmentation of LiDAR sequences: Dataset and algorithm. ECCV, 2022.

[LB19] Loic Landrieu and Mohamed Boussaha. Point cloud over-segmentation with graph-structured deep metric learning. CVPR, 2019.

[LBT+22] Romain Loiseau, Baptiste Bouvier, Yann Teytaut, Elliot Vincent, Mathieu Aubry, and Loic Landrieu. A model you can hear: Audio identification with playable prototypes. *arXiv preprint arXiv:2208.03311*, 2022.

[LMAL21] Romain Loiseau, Tom Monnier, Mathieu Aubry, and Loic Landrieu. Representing shape collections with alignment-aware linear models. In 3DV, 2021.

[LO14] Loic Landrieu and Guillaume Obozinski. Continuously indexed potts models on unoriented graphs. UAI, 2014.

[LO16] Loic Landrieu and Guillaume Obozinski. Cut pursuit: Fast algorithms to learn piecewise constant functions. AISTATS, 2016.

[LO17] Loic Landrieu and Guillaume Obozinski. Cut pursuit: Fast algorithms to learn piecewise constant functions on general weighted graphs. SIAM Journal on Imaging Sciences, 2017.
[Loi16] Landrieu Loic. *Learning structured models on weighted graphs, with applications to spatial data analysis.* PhD thesis, Paris Sciences et Lettres (ComUE), 2016.

[LRE<sup>+</sup>22] **Loic Landrieu**, Ewelina Rupnik, S Oude Elberink, Clément Mallet, and Nicolas Paparoditis. Preface: the 2022 edition of the xxivth isprs congress. In *XXIVth ISPRS Congress*, 2022.

[LRV<sup>+</sup>17] **Loic Landrieu**, Hugo Raguet, Bruno Vallet, Clément Mallet, and Martin Weinmann. A structured regularization framework for spatially smoothing semantic labelings of 3D point clouds. *ISPRS Journal of Photogrammetry and Remote Sensing*, 2017.

[LS18] **Loic Landrieu** and Martin Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. *CVPR*, 2018.

[LVAL23] Romain Loiseau, Elliot Vincent, Mathieu Aubry, and **Loic Landrieu**. Learnable earth parser: Discovering 3d prototypes in aerial scans. *arXiv preprint arXiv:2304.09704*, 2023.

[QL21] Félix Quinton and **Loic Landrieu**. Crop rotation modeling for deep learning-based parcel classification from satellite time series. *Remote Sensing*, 2021.

[RL15] Hugo Raguet and **Loic Landrieu**. Preconditioning of a generalized forward-backward splitting and application to optimization on graphs. *SIAM Journal on Imaging Sciences*, 2015.

[RL18] Hugo Raguet and **Loic Landrieu**. Cut-pursuit algorithm for regularizing nonsmooth functionals with graph total variation. *ICML*, 2018.

[RL19] Hugo Raguet and **Loic Landrieu**. Parallel cut pursuit for minimization of the graph total variation. *CVPR Workshop on Graph Reasoning*, 2019.

[RVL22] Damien Robert, Bruno Vallet, and **Loic Landrieu**. Learning multi-view aggregation in the wild for large-scale 3D semantic segmentation. *CVPR*, 2022.

[SLB<sup>+</sup>22] Raphael Sulzer, **Loic Landrieu**, Alexandre Boulch, Renaud Marlet, and Bruno Vallet. Deep surface reconstruction from point clouds with visibility information. *ICPR*, 2022.
[SLMV21] Raphael Sulzer, Loic Landrieu, Renaud Marlet, and Bruno Vallet. Scalable surface reconstruction with delaunay-graph neural networks. Symposium on Geometry Processing, 2021.

Other References

[AAM$^{+11}$] I Aziz, M Ashraf, T Mahmood, KR Islam, et al. Crop rotation impact on soil quality. Pakistan Journal of Botany, 2011.

[AHK$^{+21}$] Elias Ayrey, Daniel J. Hayes, John B. Kilbride, Shawn Fraver, John A. Kershaw, Bruce D. Cook, and Aaron R. Weiskittel. Synthesizing disparate LiDAR and satellite datasets through deep learning to generate wall-to-wall regional inventories for the complex, mixed-species forests of the Eastern United States. Remote Sensing, 2021.

[AJG96] Chester L Arnold Jr and C James Gibbons. Impervious surface coverage: the emergence of a key environmental indicator. Journal of the American planning Association, 1996.

[ASM$^{+22}$] Sheikh Kamran Abid, Noralfishah Sulaiman, Nur Putri Nadjwa Mahmud, Umber Nazir, and Nur Azhani Adnan. A review on the application of remote sensing and geographic information system in flood crisis management. Conference on Broad Exposure to Science and Technology, 2022.

[ASS$^{+12}$] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk. SLIC superpixels compared to state-of-the-art superpixel methods. Transactions on Pattern Analysis and Machine Intelligence, 2012.

[ASZ$^{+16}$] Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3D semantic parsing of large-scale indoor spaces. CVPR, 2016.

[AUM$^{+21}$] Kumar Ayush, Burak Uzkent, Chenlin Meng, Kumar Tanmay, Marshall Burke, David Lobell, and Stefano Ermon. Geography-aware self-supervised learning. ICCV, 2021.

[Bac11] Francis Bach. Shaping level sets with submodular functions. NeurIPS, 2011.

[BAC$^{+14}$] Hanna Becker, Laurent Albera, Pierre Comon, Rémi Gribonval, and Isabelle Merlet. Fast, variation-based methods for
the analysis of extended brain sources. *European Signal Processing Conference*, 2014.

[Bat18] Michael Batty. Digital twins. *Environment and Planning B: Urban Analytics and City Science*, 2018.

[BD05] DS Boyd and FM Danson. Satellite remote sensing of forest resources: three decades of research development. *Progress in Physical Geography*, 2005.

[BdLGM14] Alexandre Boulch, Martin de La Gorce, and Renaud Marlet. Piecewise-planar 3D reconstruction with edge and corner regularization. *Computer Graphic Forum*, 2014.

[BF15] Gerhard Brankatschk and Matthias Finkbeiner. Modeling crop rotation in agricultural lcas—challenges and potential solutions. *Agricultural Systems*, 2015.

[BGLSA18] Alexandre Boulch, Joris Guerry, Bertrand Le Saux, and Nicolas Audebert. Snapnet: 3D point cloud semantic labeling with 2D deep segmentation networks. *Computers & Graphics*, 2018.

[BGM$^{+}$19] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall. SemanticKITTI: A dataset for semantic scene understanding of LiDAR sequences. *ICCV*, 2019.

[BGT15] Piero Boccardo and Fabio Giulio Tonolo. Remote sensing role in emergency mapping for disaster response. *Engineering Geology for Society and Territory-Volume 5*, 2015.

[BHA$^{+}$21] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. *arXiv preprint arXiv:2108.07258*, 2021.

[BJL$^{+}$02] Ian J Bateman, Andrew P Jones, Andrew A Lovett, IR Lake, and BH Day. Applying geographical information systems (GIS) to environmental and resource economics. *Environmental and Resource Economics*, 2002.

[BJMO11a] Francis Bach, Rodolphe Jenatton, Julien Mairal, and Guillaume Obozinski. Convex optimization with sparsity-inducing norms. *Optimization for Machine Learning*, 2011.
<table>
  <tr>
    <td>[BJMO11b]</td>
    <td>Francis Bach, Rodolphe Jenatton, Julien Mairal, and Guil-laume Obozinski. Optimization with sparsity-inducing penalties. <i>Foundations and Trends in Machine Learning</i>, 2011.</td>
  </tr>
  <tr>
    <td>[BJMO12]</td>
    <td>Francis Bach, Rodolphe Jenatton, Julien Mairal, and Guil-laume Obozinski. Structured sparsity through convex optimization. <i>Statistical Science</i>, 2012.</td>
  </tr>
  <tr>
    <td>[BK04]</td>
    <td>Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. <i>Transactions on Pattern Analysis and Machine Intelligence</i>, 2004.</td>
  </tr>
  <tr>
    <td>[BLBK17]</td>
    <td>Benjamin Brede, Alvaro Lau, Harm M. Bartholomeus, and Lammert Kooistra. Comparing RIEGL RiCOPTER UAV Li-DAR derived canopy height and DBH with terrestrial Li-DAR. <i>Sensors</i>, 2017.</td>
  </tr>
  <tr>
    <td>[BLN<sup>+</sup>13]</td>
    <td>Matthew Berger, Joshua A. Levine, Luis Gustavo Nonato, Gabriel Taubin, and Claudio T. Silva. A benchmark for surface reconstruction. <i>Transaction on Graphics</i>, 2013.</td>
  </tr>
  <tr>
    <td>[BM22]</td>
    <td>Alexandre Boulch and Renaud Marlet. POCO: Point convolution for surface reconstruction. <i>CVPR</i>, 2022.</td>
  </tr>
  <tr>
    <td>[BMKS14]</td>
    <td>Christopher Boshuizen, James Mason, Pete Klupar, and Shannon Spanhake. Results from the planet labs flock constellation. <i>AIAA/USU Conference on Small Satellites</i>, 2014.</td>
  </tr>
  <tr>
    <td>[BMT<sup>+</sup>15]</td>
    <td>Adeline Bailly, Simon Malinowski, Romain Tavenard, Laetitia Chapel, and Thomas Guyet. Dense bag-of-temporal-sift-words for time series classification. <i>ECML Workshop on Advanced Analytics and Learning on Temporal Data</i>, 2015.</td>
  </tr>
  <tr>
    <td>[BMT<sup>+</sup>20]</td>
    <td>Luca Bertinetto, Romain Mueller, Konstantinos Tertikas, Sina Samangooei, and Nicholas A. Lord. Making better mistakes: Leveraging class hierarchies with deep networks. <i>CVPR</i>, 2020.</td>
  </tr>
  <tr>
    <td>[Bou19]</td>
    <td>Alexandre Boulch. Generalizing discrete convolutions for unstructured point clouds. <i>3DOR Eurographics</i>, 2019.</td>
  </tr>
  <tr>
    <td>[BRV16]</td>
    <td>A. Bódis-Szomorú, H. Riemenschneider, and L. Van Gool. Efficient volumetric fusion of airborne and street-side data for urban reconstruction. <i>ICPR</i>, 2016.</td>
  </tr>
  <tr>
    <td>[BS14]</td>
    <td>Alvaro Barbero and Suvrit Sra. Modular proximal optimization for multidimensional total-variation regularization. <i>The Journal of Machine Learning Research</i>, 2014.</td>
  </tr>
</table>
[Bul92] Donald G Bullock. Crop rotation. *Critical Reviews in Plant Sciences*, 1992.

[BVZ01] Yuri Boykov, Olga Veksler, and Ramin Zabih. Fast approximate energy minimization via graph cuts. *Transactions on Pattern Analysis and Machine Intelligence*, 2001.

[BYPC16] Nicolas Ballas, Li Yao, Chris Pal, and Aaron Courville. Delving deeper into convolutional networks for learning video representations. *ICLR*, 2016.

[CBFAB97] Pierre Charbonnier, Laure Blanc-Féraud, Gilles Aubert, and Michel Barlaud. Deterministic edge-preserving regularization in computed imaging. *Transactions on Image Processing*, 1997.

[CBV17] Laurent Caraffa, Mathieu Brédif, and Bruno Vallet. 3D watertight mesh generation with uncertainties from ubiquitous data. *ACCV*, 2017.

[CC89] Emilio Chuvieco and Russell G Congalton. Application of remote sensing and geographic information systems to forest fire hazard mapping. *Remote Sensing of Environment*, 1989.

[CD09] Antonin Chambolle and Jérôme Darbon. On total variation minimization and surface evolution using parametric maximum flows. *International Journal of Computer Vision*, 2009.

[CFG+15] Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: an information-rich 3D model repository. Technical report, Stanford University, Princeton University, Toyota Technological Institute at Chicago, 2015.

[CG+04] Frédéric Cazals, Joachim Giesen, et al. *Delaunay triangulation based surface reconstruction: a short survey*. PhD thesis, INRIA, 2004.

[CGN+13] Camille Couprie, Leo Grady, Laurent Najman, Jean-Christophe Pesquet, and Hugues Talbot. Dual constrained TV-based regularization on graphs. *SIAM Journal on Imaging Sciences*, 2013.

[CGS19] Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4D spatio-temporal convnets: Minkowski convolutional neural networks. *CVPR*, 2019.
<table>
  <tr>
    <td>[Cha08]</td>
    <td>Kang-Tsung Chang. Introduction to geographic information systems. McGraw-Hill Boston, 2008.</td>
  </tr>
  <tr>
    <td>[CHL05]</td>
    <td>Sumit Chopra, Raia Hadsell, and Yann LeCun. Learning a similarity metric discriminatively, with application to face verification. CVPR, 2005.</td>
  </tr>
  <tr>
    <td>[CHLS17]</td>
    <td>Ziyun Cai, Jungong Han, Li Liu, and Ling Shao. RGB-D datasets using Microsoft Kinect or similar sensors: a survey. Multimedia Tools and Applications, 2017.</td>
  </tr>
  <tr>
    <td>[CK16]</td>
    <td>Siddhartha Chandra and Iasonas Kokkinos. Fast, exact and multi-scale inference for semantic image segmentation with deep gaussian crfs. ECCV, 2016.</td>
  </tr>
  <tr>
    <td>[CKR04]</td>
    <td>Simon Clode, Peter J Kootsookos, and Franz Rottensteiner. The automatic extraction of roads from LiDAR data. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2004.</td>
  </tr>
  <tr>
    <td>[CLLH19]</td>
    <td>Hung-Yueh Chiang, Yen-Liang Lin, Yueh-Cheng Liu, and Winston H Hsu. A unified point-based framework for 3D segmentation. 3DV, 2019.</td>
  </tr>
  <tr>
    <td>[CLT+19]</td>
    <td>Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: Deep learning for interpretable image recognition. NeurIPS, 2019.</td>
  </tr>
  <tr>
    <td>[CP08]</td>
    <td>Patrick L Combettes and Jean-Christophe Pesquet. A proximal decomposition method for solving convex variational inverse problems. Inverse Problems, 2008.</td>
  </tr>
  <tr>
    <td>[CRC05]</td>
    <td>Matthew L Clark, Dar A Roberts, and David B Clark. Hyperspectral discrimination of tropical rain forest tree species at leaf to crown scales. Remote Sensing of Environment, 2005.</td>
  </tr>
  <tr>
    <td>[CRPW12]</td>
    <td>Venkat Chandrasekaran, Benjamin Recht, Pablo A Parrilo, and Alan S Willsky. The convex geometry of linear inverse problems. Foundations of Computational Mathematics, 2012.</td>
  </tr>
  <tr>
    <td>[CS95]</td>
    <td>Roger N Clark and Gregg A Swayze. Mapping minerals, amorphous materials, environmental materials, vegetation, water, ice and snow, and other materials: the USGS Tri-corder algorithm. JPL, Summaries of the Fifth Annual JPL Airborne Earth Science Workshop, 1995.</td>
  </tr>
  <tr>
    <td>[CSAD04]</td>
    <td>David Cohen-Steiner, Pierre Alliez, and Mathieu Desbrun. Variational shape approximation. Transactions on Graphics, 2004.</td>
  </tr>
</table>
[CTA20] Tiago Cortinhal, George Tzelepis, and Eren Erdal Ak- soy. SalsaNext: Fast, uncertainty-aware semantic seg- mentation of LiDAR point clouds for autonomous driving. arXiv:2003.03653, 2020.

[CTGHM$^{+19}$] Manuel Campos-Taberner, Francisco Javier García-Haro, Beatriz Martínez, Sergio Sánchez-Ruíz, and María Amparo Gilabert. A Copernicus Sentinel-1 and Sentinel-2 classifi- cation framework for the 2020+ European common agricul- tural policy: A case study in València (Spain). *Agronomy*, 2019.

[CvMG$^{+14}$] Kyunghyun Cho, Bart van Merriënboer, Çağlar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. *EMNLP*, 2014.

[CWB08] Emmanuel J Candes, Michael B Wakin, and Stephen P Boyd. Enhancing sparsity by reweighted $ℓ_1$ minimization. *Journal of Fourier Analysis and Applications*, 2008.

[DBC$^{+11}$] Fabio Dell'Acqua, Christian Bignami, Marco Chini, Gianni Lisini, Diego Aldo Polli, and Salvatore Stramondo. Earth- quake damages rapid mapping by satellite remote sensing data: L'aquila april 6th, 2009 event. *Journal of Selected Topics in Applied Earth Observations and Remote Sensing*, 2011.

[DBG$^{+20}$] Ralph Dubayah, James Bryan Blair, Scott Goetz, Lola Fa- toyinbo, Matthew Hansen, Sean Healey, Michelle Hofton, George Hurtt, James Kellner, Scott Luthcke, et al. The global ecosystem dynamics investigation: High-resolution laser ranging of the earth's forests and topography. *Science of Remote Sensing*, 2020.

[DBLFF10] Jia Deng, Alexander C Berg, Kai Li, and Li Fei-Fei. What does classifying more than 10,000 image categories tell us? *ECCV*, 2010.

[DCLT18] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional trans- formers for language understanding. *NAACL*, 2018.

[DDBC$^{+12}$] Matthias Drusch, Umberto Del Bello, Sébastien Carlier, Olivier Colin, Veronica Fernandez, Ferran Gascon, Bianca
Hoersch, Claudia Isola, Paolo Laberinti, Philippe Marti-
mort, et al. Sentinel-2: ESA's optical high-resolution mis-
sion for GMES operational services. *Remote Sensing of Envi-
ronment*, 2012.

[DDN20]
Angela Dai, Christian Diller, and Matthias Nießner. SG-NN: Sparse generative neural networks for self-supervised scene completion of RGB-D scans. CVPR, 2020.

[DG12]
Fabio Dell'Acqua and Paolo Gamba. Remote sensing and earthquake damage assessment: Experiences, limits, and perspectives. *Proceedings of the IEEE*, 2012.

[DGF+19]
Theo Deprelle, Thibault Groueix, Matthew Fisher, Vladimir Kim, Bryan Russell, and Mathieu Aubry. Learning elementary structures for 3D shape generation and matching. *NeurIPS*, 2019.

[DMDV11]
Jérôme Demantké, Clément Mallet, Nicolas David, and Bruno Vallet. Dimensionality based scale selection in 3D LiDAR point clouds. *Laserscanning*, 2011.

[DN18]
Angela Dai and Matthias Nießner. 3DMV: Joint 3D-multi-view prediction for 3D semantic scene segmentation. ECCV, 2018.

[DPS15]
Charles-Alban Deledalle, Nicolas Papadakis, and Joseph Salmon. On debiasing restoration algorithms: Applications to total-variation and nonlocal-means. *International Conference on Scale Space and Variational Methods in Computer Vision*, 2015.

[DRVI03]
S Dogliotti, WAH Rossing, and MK Van Ittersum. Rotat, a tool for systematically generating crop rotations. *European Journal of Agronomy*, 2003.

[DSG+12]
Jérôme Dury, Noémie Schaller, Frédérick Garcia, Arnaud Reynaud, and Jacques Eric Bergez. Models to support cropping plan and crop rotation decisions. a review. *Agronomy for Sustainable Development*, 2012.

[DSGRS18]
Christopher De Sa, Albert Gu, Christopher Ré, and Frederic Sala. Representation tradeoffs for hyperbolic embeddings. *Proceedings of Machine Learning Research*, 2018.

[EBD21]
Nico Engel, Vasileios Belagiannis, and Klaus Dietmayer. Point transformer. *ICCV*, 2021.
[EFP$^{+13}$] Damian H Evans, Roland J Fletcher, Christophe Pottier, Jean-Baptiste Chevance, Dominique Soutif, Boun Suy Tan, Sokrithy Im, Darith Ea, Tina Tin, Samnang Kim, et al. Uncovering archaeological landscapes at Angkor using Li-DAR. *Proceedings of the National Academy of Sciences*, 2013.

[EHJT04] Bradley Efron, Trevor Hastie, Iain Johnstone, and Robert Tibshirani. Least angle regression. *The Annals of Statistics*, 2004.

[EKHL17] Francis Engelmann, Theodora Kontogianni, Alexander Hermans, and Bastian Leibe. Exploring spatial context for 3D semantic segmentation of point clouds. *ICCV Workshops*, 2017.

[EOMW20] Philipp Erler, Stefan Ohrhallinger, Niloy Mitra, and Michael Wimmer. Points2Surf: Learning implicit surfaces from point clouds. *ECCV*, 2020.

[ESR$^{+16}$] Isaque Daniel Rocha Eberhardt, Bruno Schultz, Rodrigo Rizzi, Ieda Del'Arco Sanches, Antonio Roberto Formaggio, Clement Atzberger, Marcio Pupin Mello, Markus Immitzer, Kleber Trabaquini, William Foschiera, et al. Cloud cover assessment for operational crop monitoring systems in tropical areas. *Remote Sensing*, 2016.

[EZE07] Noha Youssry El-Zehiry and Adel Elmaghraby. Brain MRI tissue classification using graph cut optimization of the Mumford-Shah functional. *Proceedings of the International Vision Conference of New Zealand*, 2007.

[FB81] Martin A Fischler and Robert C Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. *Communications of the ACM*, 1981.

[FBJ$^{+12}$] António Ferraz, Frédéric Bretar, Stéphane Jacquemoud, Gil Gonçalves, Luisa Pereira, Margarida Tomé, and Paula Soares. 3D mapping of a multi-layered Mediterranean forest using ALS data. *Remote Sensing of Environment*, 2012.

[FF56] Lester Randolph Ford and Delbert R Fulkerson. Maximal flow through a network. *Canadian journal of Mathematics*, 1956.

[FLE$^{+22}$] Christopher Fisher, Stephen Leisz, Damian Evans, Diana H Wall, Kathleen Galvin, Melinda Laituri, Geoffrey Henebry,
James Zeidler, Juan Carlos Fernandez-Diaz, Shrideep Pallickara, et al. Creating an earth archive. *Proceedings of the National Academy of Sciences*, 2022.

[FMJ$^{+15}$] António Ferraz, Clément Mallet, Stéphane Jacquemoud, Gil Gonçalves, Margarida Tomé, Paula Soares, Luísa Pereira, and Frédéric Bretar. Canopy density model: A new als-derived product to generate multilayer crown cover maps. *Transactions on Geoscience and Remote Sensing*, 2015.

[For08] Eric Kwabena Forkuo. *Digital terrain modeling in a GIS environment*. Citeseer, 2008.

[FSM$^{+16}$] António Ferraz, Sassan Saatchi, Clément Mallet, Stéphane Jacquemoud, Gil Gonçalves, Carlos Alberto Silva, Paula Soares, Margarida Tomé, and Luisa Pereira. Airborne LiDAR estimation of aboveground forest biomass in the absence of field inventory. *Remote Sensing*, 2016.

[GAMP12] Etienne Gaujour, Bernard Amiaud, Catherine Mignolet, and Sylvain Plantureux. Factors and processes affecting plant biodiversity in permanent grasslands. a review. *Agronomy for Sustainable Development*, 2012.

[GBB$^{+20}$] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. *arXiv preprint arXiv:2101.00027*, 2020.

[GCB07] Megandhren Govender, Kershani Chetty, and Hartley Bulcock. A review of hyperspectral remote sensing and its application in vegetation and water resource studies. *Water SA*, 2007.

[GCVY21] Matthew G. Gale, Geoffrey J. Cary, Albert I.J.M. Van Dijk, and Marta Yebra. Forest fire fuel through the lens of remote sensing: Review of approaches, challenges and future directions in the remote sensing of biotic determinants of fire behaviour. *Remote Sensing of Environment*, 2021.

[GFK$^{+18}$] Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, and Mathieu Aubry. AtlasNet: A papier-mâché approach to learning 3D surface generation. *CVPR*, 2018.
[GKM$^{+20}$] Jakob Geyer, Yohannes Kassahun, Mentar Mahmudi, Xavier Ricou, Rupesh Durgesh, Andrew S Chung, Lorenz Hauswald, Viet Hoang Pham, Maximilian Mühlegg, Sebastian Dorn, et al. A2D2: Audi autonomous driving dataset. *arXiv preprint arXiv:2004.06320*, 2020.

[GLSU13] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The KITTI dataset. *The International Journal of Robotics Research*, 2013.

[Gon08] Georges Gonthier. Formal proof-the four-color theorem. *Notices of the AMS*, 2008.

[Goo10] Michael Goodchild. Twenty years of progress: GIScience in 2010. *Journal of Spatial Information Science*, 2010.

[GPGMLS17] Angel Garcia-Pedrero, Consuelo Gonzalo-Martin, and M Lillo-Saavedra. A machine learning approach for agricultural parcel delineation through agglomerative segmentation. *International Journal of Remote Sensing*, 2017.

[GR92] Donald Geman and George Reynolds. Constrained restoration and the recovery of discontinuities. *Transactions on Pattern Analysis and Machine Intelligence*, 1992.

[Gra97] Wyn Grant. *The common agricultural policy*. Macmillan International Higher Education, 1997.

[GRM$^{+19}$] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. *ICLR*, 2019.

[GWCV16] Adrien Gaidon, Qiao Wang, Yohann Cabon, and Eleonora Vig. Virtual worlds as proxy for multi-object tracking analysis. *CVPR*, 2016.

[GWH$^{+20}$] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for 3D point clouds: A survey. *Transactions on Pattern Analysis and Machine Intelligence*, 2020.

[GYH$^{+20}$] Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. *Machine Learning and Systems*, 2020.
[GYL$^{+20}$] Jianhao Gao, Qiangqiang Yuan, Jie Li, Hai Zhang, and Xin Su. Cloud removal with fusion of high resolution optical and sar images using generative adversarial networks. *Remote Sensing*, 2020.

[H$^{+73}$] Peter J Huber et al. Robust regression: Asymptotics, conjectures and monte carlo. *The Annals of Statistics*, 1973.

[HCZ17] Hamid Hamraz, Marco Contreras, and Jun Zhang. Forest understory trees can be segmented accurately within sufficiently dense airborne laser scanning point clouds. *Scientific Reports*, 2017.

[HDW$^{+06}$] Peter Hyde, Ralph Dubayah, Wayne Walker, J Bryan Blair, Michelle Hofton, and Carolyn Hunsaker. Mapping forest structure for wildlife habitat analysis using multi-sensor (LiDAR, SAR/InSAR, ETM+, Quickbird) synergy. *Remote Sensing of Environment*, 2006.

[HGDG17] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask R-CNN. *ICCV*, 2017.

[HL03] John E Hasse and Richard G Lathrop. Land resource impact indicators of urban sprawl. *Applied Geography*, 2003.

[HMDC16] Caner Hazirbas, Lingni Ma, Csaba Domokos, and Daniel Cremers. FuseNet: Incorporating depth into semantic segmentation via fusion-based CNN architecture. *ACCV*, 2016.

[HMGCO20] Rana Hanocka, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. Point2Mesh: A self-prior for deformable meshes. *Transaction on Graphics*, 2020.

[HSL$^{+17}$] Timo Hackel, Nikolay Savinov, Lubor Ladicky, Jan D Wegner, Konrad Schindler, and Marc Pollefeys. Semantic3D.net: A new large-scale point cloud classification benchmark. *ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 2017.

[HUL04] Jean-Baptiste Hiriart-Urruty and Claude Lemaréchal. *Fundamentals of convex analysis*. Springer Science & Business Media, 2004.

[HvOW03] Werner Härdtle, Goddert von Oheimb, and Christina Westphal. The effects of light and soil conditions on the species richness of the ground vegetation of deciduous forests in northern Germany (Schleswig-Holstein). *Forest Ecology and Management*, 2003.
[HWB<sup>+</sup>13] Armin Hornung, Kai M Wurm, Maren Bennewitz, Cyrill Stachniss, and Wolfram Burgard. OctoMap: An efficient probabilistic 3D mapping framework based on octrees. *Autonomous robots*, 2013.

[HYC<sup>+</sup>21] Le Hui, Jia Yuan, Mingmei Cheng, Jin Xie, Xiaoya Zhang, and Jian Yang. Superpoint network for point cloud over-segmentation. *ICCV*, 2021.

[HYL17] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. *NeurIPS*, 2017.

[HYX<sup>+</sup>20a] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham. Randla-net: Efficient semantic segmentation of large-scale point clouds. *CVPR*, 2020.

[HYX<sup>+</sup>20b] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham. RandLA-Net: efficient semantic segmentation of large-scale point clouds. *CVPR*, 2020.

[HZJ<sup>+</sup>21] Wenbo Hu, Hengshuang Zhao, Li Jiang, Jiaya Jia, and Tien-Tsin Wong. Bidirectional projection network for cross dimension scene understanding. *CVPR*, 2021.

[HZRS16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. *CVPR*, 2016.

[IIGM19] Dino Ienco, Roberto Interdonato, Raffaele Gaetano, and Dinh Ho Tong Minh. Combining Sentinel-1 and Sentinel-2 satellite image time series for land cover mapping via a multi-source deep learning architecture. *ISPRS Journal of Photogrammetry and Remote Sensing*, 2019.

[IK98] John L Innes and Barbara Koch. Forest biodiversity and its assessment by remote sensing. *Global Ecology & Biogeography Letters*, 1998.

[IS15] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. *ICML*, 2015.

[Jag13] Martin Jaggi. Revisiting frank-wolfe: Projection-free sparse convex optimization. *International Conference on Machine Learning*, 2013.
[Jal16] Khalid Jalalzai. Some remarks on the staircasing phenomenon in total variation-based image denoising. *Journal of Mathematical Imaging and Vision*, 2016.

[JGB$^{+}$21] Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: general perception with iterative attention. *ICLR*, 2021.

[JGS19] Maximilian Jaritz, Jiayuan Gu, and Hao Su. Multi-view pointNet for 3D scene understanding. *CVPR Workshops*, 2019.

[JOWS21] Peng Jiang, Philip Osteen, Maggie Wigness, and Srikanth Saripalli. Rellis-3D dataset: data, benchmarks and analysis. *ICRA*, 2021.

[JP11] Michal Jancosek and Tomas Pajdla. Multi-view reconstruction preserving weakly-supported surfaces. *CVPR*, 2011.

[JP14] Michal Jancosek and Tomas Pajdla. Exploiting visibility information in surface reconstruction to preserve weakly supported surfaces. *International Scholarly Research Notices*, 2014.

[JR08] Junchang Ju and David P Roy. The availability of cloud-free landsat ETM+ data over the conterminous United States and globally. *Remote Sensing of Environment*, 2008.

[JSL$^{+}$18] Varun Jampani, Deqing Sun, Ming-Yu Liu, Ming-Hsuan Yang, and Jan Kautz. Superpixel sampling networks. *ECCV*, 2018.

[JSM01] Juha Jantunen, Kimmo Saarinen, and Olli Marttila. Effects of forest management on field layer vegetation: a comparison between finnish and russian karelia. *Scandinavian Journal of Forest Research*, 2001.

[JSM$^{+}$20] Chiyu Max Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, and Thomas Funkhouser. Local implicit grid representations for 3D scenes. *CVPR*, 2020.

[JT92] Jerzy W Jaromczyk and Godfried T Toussaint. Relative neighborhood graphs and their relatives. *Proceedings of the IEEE*, 1992.

[KBJ$^{+}$15] KS Kumar, Alvaro Barbero, Stefanie Jegelka, Suvrit Sra, and Francis Bach. Convex optimization for parallel energy minimization. *arXiv preprint arXiv:1503.01563*, 2015.
[KDB<sup>+19</sup>] B Koetz, P Defourny, S Bontemps, K Bajec, C Cara, L de Vendiatis, L Kucera, P Malcorps, G Milcinski, L Nicola, et al. Sen4cap sentinels for cap monitoring approach. *JRC IACS Workshop*, 2019.

[KFC<sup>+19</sup>] Nadezhda Kurganova, Michael Filin, Dmitry Cherniaev, Artem Shaklein, and Dmitry Namiot. Digital twins introduction as one of the major directions of industrial digitalization. *International Journal of Open Information Technologies*, 2019.

[KGHD19] Alexander Kirillov, Ross Girshick, Kaiming He, and Piotr Dollár. Panoptic feature pyramid networks. *CVPR*, 2019.

[KH<sup>+09</sup>] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.

[KH13] Michael Kazhdan and Hugues Hoppe. Screened Poisson surface reconstruction. *Transaction on Graphics*, 2013.

[KK11] Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected CRFs with Gaussian edge potentials. *NeurIPS*, 2011.

[KKN<sup>+15</sup>] Chris Kollas, Kurt Christian Kersebaum, Claas Nendel, Kiril Manevski, Christoph Müller, Taru Palosuo, Cecilia M Armas-Herrera, Nicolas Beaudoin, Marco Bindi, Monia Charfeddine, et al. Crop rotation modelling—a European model intercomparison. *European Journal of Agronomy*, 2015.

[KL51] Solomon Kullback and Richard A Leibler. On information and sufficiency. *The Annals of Mathematical Statistics*, 1951.

[KL18] Wouter M Kouw and Marco Loog. An introduction to domain adaptation and transfer learning. *arXiv preprint arXiv:1812.11806*, 2018.

[KLG<sup>+16</sup>] Nataliia Kussul, Guido Lemoine, Francisco Javier Gallego, Sergii V Skakun, Mykola Lavreniuk, and Andrii Yu Shelestov. Parcel-based crop classification in ukraine using Landsat-8 data and Sentinel-1A data. *Journal of Selected Topics in Applied Earth Observations and Remote Sensing*, 2016.

[KLSS17] Nataliia Kussul, Mykola Lavreniuk, Sergii Skakun, and Andrii Shelestov. Deep learning classification of land cover and crop types using remote sensing data. *Geoscience and Remote Sensing Letters*, 2017.
[KMMB20] Shunichi Koshimura, Luis Moya, Erick Mas, and Yanbing Bai. Tsunami damage detection with remote sensing: a review. Geosciences, 2020.

[KNL+20] Bernd Ketzler, Vasilis Naserentin, Fabio Latino, Christopher Zangelidis, Liane Thuvander, and Anders Logg. Digital twins for cities: A state of the art review. Built Environment, 2020.

[KTR+21] Lukas Kondmann, Aysim Toker, Marc Rußwurm, Andrés Camero, Devis Peressuti, Grega Milcinski, Pierre-Philippe Mathieu, Nicolas Longépé, Timothy Davis, Giovanni Marchisio, et al. DENETHOR: The DynamicEarthNET dataset for Harmonized, inter-operable, analysis-Ready, daily crop monitoring from space. NeurIPS Datasets and Benchmarks Track, 2021.

[KYF+20] Abhijit Kundu, Xiaoqi Yin, Alireza Fathi, David Ross, Brian Brewington, Thomas Funkhouser, and Caroline Pantofaru. Virtual multi-view fusion for 3D semantic segmentation. ECCV, 2020.

[Lec89] Yvan G Leclerc. Constructing simple stable descriptions for image partitioning. International Journal of Computer Vision, 1989.

[LKA+22] Nico Lang, Nikolai Kalischek, John Armston, Konrad Schindler, Ralph Dubayah, and Jan Dirk Wegner. Global canopy height regression and uncertainty estimation from GEDI LIDAR waveforms with deep ensembles. Remote Sensing of Environment, 2022.

[LKZ+17] Måns Larsson, Fredrik Kahl, Shuai Zheng, Anurag Arnab, Philip H. S. Torr, and Richard I. Hartley. Learning arbitrary potentials in CRFs with gradient descent. CoRR, abs/1701.06805, 2017.

[LL19] gregory Lepère and Mathias Lemmens. Laser scanning of damaged historical icons. GIM International, 2019.

[LP15] Dirk A Lorenz and Thomas Pock. An inertial forward-backward algorithm for monotone inclusions. Journal of Mathematical Imaging and Vision, 2015.

[LPK09] P. Labatut, J. P. Pons, and R. Keriven. Robust and efficient surface reconstruction from range data. Computer Graphics Forum, 2009.
[LR12] Yan Lu and Christopher Rasmussen. Simplified Markov random fields for efficient semantic labeling of 3D point clouds. *International Conference on Intelligent Robots and Systems*, 2012.

[LS19] A Lubin and A Saleem. Remote sensing-based mapping of the destruction to aleppo during the syrian civil war between 2011 and 2017. *Applied Geography*, 2019.

[LSL+17] Zhiwei Li, Huanfeng Shen, Huifang Li, Guisong Xia, Paolo Gamba, and Liangpei Zhang. Multi-feature combined cloud and cloud shadow detection in GaoFen- wide field of view imagery. *Remote Sensing of Environment*, 2017.

[LSvdHR16] Guosheng Lin, Chunhua Shen, Anton van den Hengel, and Ian D. Reid. Efficient piecewise training of deep structured models for semantic segmentation. *CVPR*, 2016.

[Lu06] Dengsheng Lu. The potential and challenge of remote sensing-based biomass estimation. *International Journal of Remote Sensing*, 2006.

[LWG+19] Lei Luo, Xinyuan Wang, Huadong Guo, Rosa Lasaponara, Xin Zong, Nicola Masini, Guizhou Wang, Pilong Shi, Houcine Khatteli, Fulong Chen, et al. Airborne and space-borne remote sensing for archaeological and cultural heritage applications: A review of the century (1907–2017). *Remote Sensing of Environment*, 2019.

[LWZ+18] Yangbin Lin, Cheng Wang, Dawei Zhai, Wei Li, and Jonathan Li. Toward better boundary preserved supervoxel segmentation for 3D point clouds. *ISPRS Journal of Photogrammetry and Remote Sensing*, 2018.

[LWZ20] Rong Liu, Jinling Wang, and Bingqi Zhang. High definition map for automated driving: Overview and analysis. *The Journal of Navigation*, 2020.

[LXG21] Yiyi Liao, Jun Xie, and Andreas Geiger. KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D. *Transactions on Pattern Analysis and Machine Intelligence*, 2021.

[LZS20] Minghua Liu, Xiaoshuai Zhang, and Hao Su. Meshing point clouds with predicted intrinsic-extrinsic ratio guidance. *ECCV*, 2020.
[Mac96] David A MacLean. Forest management strategies to reduce spruce budworm damage in the fundy model forest. *The Forestry Chronicle*, 1996.

[Man18] Cedric Manlhiot. Machine learning for predictive analytics in medicine: real opportunity or overblown hype? *European Heart Journal-Cardiovascular Imaging*, 2018.

[MBVH09] Daniel Munoz, J Andrew Bagnell, Nicolas Vandapel, and Martial Hebert. Contextual classification with functional max-margin Markov networks. *CVPR*, 2009.

[MD03] Carsten Moenning and Neil A Dodgson. Fast marching farthest point sampling. *Eurographics*, 2003.

[Mea82] Donald Meagher. Geometric modeling using octree encoding. *Computer Graphics and Image Processing*, 1982.

[MKH19] Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? *NeurIPS*, 2019.

[MKL+14] Heather McNairn, Angela Kross, David Lapen, R Caves, and Jiali Shang. Early season monitoring of corn and soybeans with terrasar-x and radarsat-2. *International Journal of Applied Earth Observation and Geoinformation*, 2014.

[MLL+19] Qingfei Min, Yangguang Lu, Zhiyong Liu, Chao Su, and Bo Wang. Machine learning based digital twin framework for production optimization in petrochemical industry. *International Journal of Information Management*, 2019.

[MLRF+21] Jorge Andres Chamorro Martinez, Laura Elena Cué La Rosa, Raul Queiroz Feitosa, Ieda Del'Arco Sanches, and Patrick Nigri Happ. Fully convolutional recurrent networks for multidate crop recognition from multitemporal image sequences. *ISPRS Journal of Photogrammetry and Remote Sensing*, 2021.

[MON+19] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: learning 3D reconstruction in function space. *CVPR*, 2019.

[MPH+22] Matti Mõttus, Phu Pham, Eelis Halme, Matthieu Molinier, Hai Cu, and Jorma Laaksonen. TAIGA: a novel dataset for multitask learning of continuous and categorical forest variables from hyperspectral imagery. *Transactions on Geoscience and Remote Sensing*, 2022.
[MPT20] Khairiya Mudrik Masoud, Claudio Persello, and Valen-
tyn A Tolpekin. Delineation of agricultural field boundaries
from Sentinel-2 images using a novel super-resolution con-
tour detector based on fully convolutional networks. *Remote sensing*, 2020.

[MR11] Donald McKenzie and Crystal Raymond. Modeling under-
story vegetation and its response to fire. *Models for Planning
Wildlife Conservation in Large Landscapes*, 2011.

[MRB<sup>+</sup>19] Cheikh Mbow, Cynthia Rosenzweig, Luis G Barioni, Tim G Benton, Mario Herrero, Murukesan Krishnapillai, Emma Li-wenga, Prajal Pradhan, M-G Rivera-Ferre, T Sapkota, et al. *Food security*. Intergovernmental Panel on Climate Change, 2019.

[MS89] David Bryant Mumford and Jayant Shah. Optimal approximations by piecewise smooth functions and associated variational problems. *Communications on Pure and Applied Mathematics*, 1989.

[MSF20] Saba Mehmood, Muhammad Shahzad, and Muhammad Moazam Fraz. Dcarn: Deep context aware recurrent neural network for semantic segmentation of large scale unstructured 3D point cloud. *Neural Processing Letters*, 2020.

[MV21] Rohit Mohan and Abhinav Valada. Efficientps: Efficient panoptic segmentation. *International Journal of Computer Vision*, 2021.

[MVG<sup>+</sup>09] Sebastian Martinuzzi, Lee Vierling, William Gould, Michael Falkowski, Jeffrey Evans, Andrew Hudak, and Kerri Vierling. Mapping snags and understory shrubs for a LiDAR-based assessment of wildlife habitat suitability. *Remote Sensing of Environment*, 2009.

[MWJ13] Kevin Murphy, Yair Weiss, and Michael I Jordan. Loopy belief propagation for approximate inference: An empirical study. *UAI*, 2013.

[MYWC18] Thomas Möllenhoff, Zhenzhang Ye, Tao Wu, and Daniel Cremers. Combinatorial preconditioners for proximal algorithms on graphs. *AISTATS*, 2018.

[Nag01] Harini Nagendra. Using remote sensing to assess biodiver-
sity. *International Journal of Remote Sensing*, 2001.
[NNSP14] Mohammad Najafi, Sarah Taghavi Namin, Mathieu Salzmann, and Lars Petersson. Non-associative higher-order Markov networks for point cloud classification. *ECCV*, 2014.

[NPA22] Joachim Nyborg, Charlotte Pelletier, and Ira Assent. Generalized classification of satellite image time series with thermal positional encoding. *CVPR Workshops*, 2022.

[NSF12] Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. Indoor segmentation and support inference from RGBD images. *ECCV*, 2012.

[NSGW21] Steven A Niederer, Michael S Sacks, Mark Girolami, and Karen Willcox. Scaling digital twins from the artisanal to the industrial. *Nature Computational Science*, 2021.

[OAPL21] Stella Ofori-Ampofo, Charlotte Pelletier, and Stefan Lang. Crop type mapping from optical and radar time series using attention-based deep learning. *Remote Sensing*, 2021.

[OID15] Julien Osman, Jordi Inglada, and Jean-François Dejoux. Assessment of a markov logic model of crop rotations for early crop mapping. *Computers and Electronics in Agriculture*, 2015.

[OLV18] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. *arXiv preprint arXiv:1807.03748*, 2018.

[PASW13] Jeremie Papon, Alexey Abramov, Markus Schoeler, and Florentin Wörgötter. Voxel cloud connectivity segmentation - supervoxels for point clouds. *CVPR*, 2013.

[PBI$^{+22}$] Heiko Prümers, Carla Jaimes Betancourt, José Iriarte, Mark Robinson, and Martin Schaich. LiDAR reveals pre-Hispanic low-density urbanism in the Bolivian Amazon. *Nature*, 2022.

[PBM$^{+18}$] Laura Pastonchi, Anna Barra, Oriol Monserrat, Guido Luzi, Lorenzo Solari, and Veronica Tofani. Satellite data to improve the knowledge of geohazards in world heritage sites. *Remote Sensing*, 2018.

[PC11] Thomas Pock and Antonin Chambolle. Diagonal preconditioning for first order primal-dual algorithms in convex optimization. *ICCV*, 2011.
[Pea01] Karl Pearson. On lines and planes of closest fit to systems of points in space. *The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science*, 1901.

[Pea82] Judea Pearl. *Reverend Bayes on inference engines: A distributed hierarchical approach*. Cognitive Systems Laboratory, School of Engineering and Applied Science, 1982.

[PFS⁺19] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. CVPR, 2019.

[PI18] Alexander Y Prosekov and Svetlana A Ivanova. Food security: The challenge of the present. *Geoforum*, 2018.

[PJL⁺21] Songyou Peng, Chiyu "Max" Jiang, Yiyi Liao, Michael Niemeyer, Marc Pollefeys, and Andreas Geiger. Shape as points: A differentiable poisson solver. *NeurIPS*, 2021.

[PKPG12] François Petitjean, Camille Kurtz, Nicolas Passat, and Pierre Gançarski. Spatio-temporal reasoning for the classification of satellite image time series. *Pattern Recognition Letters*, 2012.

[PNM⁺20] Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. Convolutional occupancy networks. ECCV, 2020.

[Pot52] Renfrey Burnard Potts. Some generalized order-disorder transformations. *Mathematical Proceedings of the Cambridge Philosophical Society*, 1952.

[PPC⁺12] Nicolas Paparoditis, Jean-Pierre Papelard, BertrandCNelle, Alexandre Devaux, Bahman Soheilian, Nicolas David, and Erwann Houzay. Stereopolis II: A multi-purpose and multi-sensor 3D mobile mapping system for street visualization and 3D metrology. *Revue Française de Photogrammétrie et de Télédétection*, 2012.

[PRG⁺16] Pierre Potin, Betlem Rosich, Patrick Grimont, Nuno Miranda, Ian Shurmer, Alistair O'Connell, Ramon Torres, and Mike Krassenburg. Sentinel-1 mission status. *European Conference on Synthetic Aperture Radar*, 2016.

[PTB16] Eann A Patterson, Richard J Taylor, and Mark Bankhead. A framework for an integrated nuclear digital environment. *Progress in Nuclear Energy*, 2016.
[PVI<sup>+</sup>16] Charlotte Pelletier, Silvia Valero, Jordi Inglada, Nicolas Champion, and Gérard Dedieu. Assessing the robustness of random forests to map land cover with high resolution satellite image time series over large areas. *Remote Sensing of Environment*, 2016.

[PVK21] Maria Papadomanolaki, Maria Vakalopoulou, and Konstantinos Karantzalos. A deep multi-task learning framework coupling semantic segmentation and fully convolutional LSTM networks for urban change detection. *Transactions on Geoscience and Remote Sensing*, 2021.

[PWP19] Charlotte Pelletier, Geoffrey I Webb, and François Petitjean. Temporal convolutional neural network for the classification of satellite image time series. *Remote Sensing*, 2019.

[PZL17] Jian Peng, Huijuan Zhao, and Yanxu Liu. Urban ecological corridors construction: A review. *Acta Ecologica Sinica*, 2017.

[QSMG17a] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3D classification and segmentation. *CVPR*, 2017.

[QSMG17b] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning on point sets for 3D classification and segmentation. *CVPR*, 2017.

[QYSG17] Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. *NeurIPS*, 2017.

[Rag18] Hugo Raguet. A note on the forward-douglas-rachford splitting for monotone inclusion and convex optimization. *Optimization Letters*, 2018.

[RBG19] Santiago Royo and Maria Ballesta-Garcia. An overview of LiDAR imaging systems for autonomous vehicles. *Applied Sciences*, 2019.

[RCVS<sup>+</sup>19] Markus Reichstein, Gustau Camps-Valls, Bjorn Stevens, Martin Jung, Joachim Denzler, Nuno Carvalhais, et al. Deep learning and process understanding for data-driven earth system science. *Nature*, 2019.

[RCW<sup>+</sup>19] Rose Rustowicz, Robin Cheong, Lijing Wang, Stefano Ermon, Marshall Burke, and David Lobell. Semantic segmentation of crop type in Africa: A novel dataset and analysis of deep learning methods. *CVPR Workshops*, 2019.
[RDG18] Xavier Roynard, Jean-Emmanuel Deschaud, and François Goulette. Paris-Lille-3D: A large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification. *The International Journal of Robotics Research*, 2018.

[RDS<sup>+</sup>15] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San-jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. *International Journal of Computer Vision*, 2015.

[RFB15] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. *MICCAI*, 2015.

[RFP13] Hugo Raguet, Jalal Fadili, and Gabriel Peyré. A generalized forward-backward splitting. *SIAM Journal on Imaging Sciences*, 2013.

[RHV<sup>+</sup>18] Johannes Reiche, Eliakim Hamunyela, Jan Verbesselt, Dirk Hoekman, and Martin Herold. Improving near-real time deforestation monitoring in tropical dry forests by combining dense sentinel-1 time series with landsat and alos-2 palsar-2. *Remote Sensing of Environment*, 2018.

[RK18] Marc Rußwurm and Marco Körner. Convolutional LSTMs for cloud-robust segmentation of remote sensing imagery. *NeurIPS Workshops*, 2018.

[RK19] Marc Rußwurm and Marco Körner. Self-attention for raw optical satellite time series classification. *ISPRS Journal of Photogrammetry and Remote Sensing*, 2019.

[RKH<sup>+</sup>21] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. *ICML*, 2021.

[RMM20] Luka Rumora, Mario Miler, and Damir Medak. Impact of various atmospheric corrections on sentinel-2 land cover classification accuracy using machine learning classifiers. *ISPRS International Journal of Geo-Information*, 2020.

[ROF92] Leonid I Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. *Physica D: Nonlinear Phenomena*, 1992.
[RRW19] Tom R Robinson, Nick Rosser, and Richard J Walters. The spatial and temporal influence of cloud cover on satellite-based emergency mapping of earthquake disasters. *Scientific Reports*, 2019.

[SBAK20] Joel Segarra, Maria Luisa Buchaillot, Jose Luis Araus, and Shawn C Kefauver. Remote sensing for precision agriculture: Sentinel-2 improved features and applications. *Agricultropy*, 2020.

[SBK21] Maja Schneider, Amelie Broszeit, and Marco Körner. Eurocrops: A pan-European dataset for time series crop type classification. *Proceedings of the 2021 Conference on Big Data from Space*, 2021.

[SBT+05] William D Shuster, James Bonta, Hale Thurston, Elizabeth Warnemuende, and DR Smith. Impacts of impervious surface on watershed hydrology: A review. *Urban Water Journal*, 2005.

[SIBD11] Charles Soussen, Jérôme Idier, David Brie, and Junbo Duan. From bernoulli-gaussian deconvolution to sparse signal restoration. *Transactions on Signal Processing*, 2011.

[SK17] Martin Simonovsky and Nikos Komodakis. Dynamic edge-conditioned filters in convolutional neural networks on graphs. *CVPR*, 2017.

[SK20] Maja Schneider and Marco Körner. [re] satellite image time series classification with pixel-set encoders and temporal self-attention. *ML Reproducibility Challenge*, 2020.

[SL08] Trenton F Stanger and Joseph G Lauer. Corn grain yield response to crop rotation and nitrogen over 35 years. *Agricultropy Journal*, 2008.

[SM98] James A Slater and Stephen Malys. WGS 84—past, present and future. *Advances in Positioning and Reference Frames*, 1998.

[SNRS14] Alena Schmidt, Joachim Niemeyer, Franz Rottensteiner, and Uwe Soergel. Contextual classification of full waveform LiDAR data in the Wadden Sea. *Geoscience and Remote Sensing Letters*, 2014.

[SO20] Nicholas Sharp and Maks Ovsjanikov. PointTriNet: Learned triangulation of 3D point sets. *ECCV*, 2020.
[SOC01] David V. Sandberg, Roger D. Ottmar, and Geoffrey H. Cushon. Characterizing fuels in the 21st century. *International Journal of Wildland Fire*, 2001.

[SPI+19] Andrei Stoian, Vincent Poulain, Jordi Inglada, Victor Poughon, and Dawa Derksen. Land cover maps production with high resolution satellite image time series and convolutional neural networks: Adaptations and limits for operational systems. *Remote Sensing*, 2019.

[SSG17a] Nico Schertler, Bogdan Savchynskyy, and Stefan Gumhold. Towards globally optimal normal orientations for large point clouds. *Computer Graphics Forum*, 2017.

[SSG+17b] Thomas Schöps, Johannes L. Schönberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. *CVPR*, 2017.

[SSZ17] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. *NeurIPS*, 2017.

[STAL20] Martin Sudmanns, Dirk Tiede, Hannah Augustin, and Stefan Lang. Assessing global sentinel-2 coverage dynamics and data availability for operational earth observation (eo) applications using the eo-compass. *International Journal of Digital Earth*, 2020.

[Str82] Wolfgang Strasser. Fast curve and surface generation for interactive shape design. *Computers in Industry*, 1982.

[SU15] Alexander G. Schwing and Raquel Urtasun. Fully connected deep structured networks. *CoRR*, 2015.

[SVB10] Roman Shapovalov, Er Velizhev, and Olga Barinova. Nonassociative Markov networks for 3D point cloud classification. *International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 2010.

[SWNW18] Max J Steinhausen, Paul D Wagner, Balaji Narasimhan, and Björn Waske. Combining sentinel-1 and sentinel-2 data for improved land use and land cover mapping of monsoon regions. *International Journal of Applied Earth Observation and Geoinformation*, 2018.

[TC96] Italo Tamanini and Giuseppe Congedo. Optimal segmentation of unbounded functions. *Rendiconti del Seminario Matematico della Universita di Padova*, 1996.
[TDH<sup>+13</sup>] Mustafa Teke, Hüsne Seda Deveci, Onur Haliloğlu, Sevgi Zübeyde Gürbüz, and Ufuk Sakarya. A short survey of hyperspectral remote sensing applications in agriculture. *International Conference on Recent Advances in Space Technologies*, 2013.

[TGB19] Daniel Tenbrinck, Fjedor Gaede, and Martin Burger. Variational graph methods for efficient point cloud sparsification. *arXiv preprint arXiv:1903.02858*, 2019.

[TGD<sup>+18</sup>] Hugues Thomas, François Goulette, Jean-Emmanuel Deschaud, Beatriz Marcotegui, and Yann LeGall. Semantic classification of 3D point clouds with multiscale spherical neighborhoods. *3DV*, 2018.

[TGLM21] Neil C Thompson, Kristjan Greenewald, Keeheon Lee, and Gabriel F Manso. Deep learning's diminishing returns: the cost of improvement is becoming unsustainable. *Spectrum*, 2021.

[Tho52] Paul D Thomas. *Conformal projections in geodesy and cartography*. US Government Printing Office, 1952.

[TL16] Hui-Ting Tang and Yuh-Ming Lee. The making of sustainable urban development: a synthesis framework. *Sustainability*, 2016.

[TLB<sup>+22</sup>] Wei-Hsin Tseng, Hoàng-Ân Lê, Alexandre Boulch, Sébastien Lefèvre, and Dirk Tiede. Croco: Cross-modal contrastive learning for localization of earth observation data. *International Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 2022.

[TLZ<sup>+20</sup>] Haotian* Tang, Zhijian* Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, and Song Han. Searching efficient 3D architectures with sparse point-voxel convolution. *ECCV*, 2020.

[TMC<sup>+17</sup>] Romain Tavenard, Simon Malinowski, Laetitia Chapel, Adeline Bailly, Heider Sanchez, and Benjamin Bustos. Efficient temporal kernels between feature sets for time series classification. *ECML Workshop on Advanced Analytics and Learning on Temporal Data*, 2017.

[Tob70] Waldo R Tobler. A computer movie simulating urban growth in the detroit region. *Economic Geography*, 1970.
[TQD<sup>+19</sup>] Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. KPConv: Flexible and deformable convolution for point clouds. CVPR, 2019.

[TQM<sup>+20</sup>] Weikai Tan, Nannan Qin, Lingfei Ma, Ying Li, Jing Du, Guorong Cai, Ke Yang, and Jonathan Li. Toronto-3D: A large-scale mobile LiDAR dataset for semantic segmentation of urban roadways. CVPR Workshops, 2020.

[TRW<sup>+21</sup>] Devis Tuia, Ribana Roscher, Jan Dirk Wegner, Nathan Jacobs, Xiaoxiang Zhu, and Gustau Camps-Valls. Toward a collective agenda on AI for earth science data analysis. Geoscience and Remote Sensing Magazine, 2021.

[TSA19] Pavel Tokmakov, Cordelia Schmid, and Karteek Alahari. Learning to segment moving objects. International Journal of Computer Vision, 2019.

[TVB<sup>+10</sup>] Mao-Ning Tuanmu, Andrés Viña, Scott Bearer, Weihua Xu, Zhiyun Ouyang, Hemin Zhang, and Jianguo Liu. Mapping understory vegetation using phenological characteristics derived from remotely sensed data. Remote Sensing of Environment, 2010.

[VAG20] Nina Varney, Vijayan K Asari, and Quinn Graehling. DALES: A large-scale aerial LiDAR data set for semantic segmentation. CVPR Workshops, 2020.

[VBR<sup>+17</sup>] Ján Vachálek, Lukás Bartalský, Oliver Rovný, Dana Šišmišová, Martin Morháč, and Milan Lokšík. The digital twin of an industrial production line within the industry 4.0 concept. International Conference on Process Control, 2017.

[VBS<sup>+15</sup>] Bruno Vallet, Mathieu Brédif, Andrés Serna, Beatriz Marcotegui, and Nicolas Paparoditis. TerraMobilita/iQmulus urban point cloud analysis benchmark. Computers & Graphics, 2015.

[VHMAS<sup>+18</sup>] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The iNaturalist species classification and detection dataset. CVPR, 2018.

[VLPK12] Hoang Hiep Vu, Patrick Labatut, Jean Philippe Pons, and Renaud Keriven. High accuracy and visibility-consistent dense multiview stereo. Transactions on Pattern Analysis and Machine Intelligence, 2012.
[VM98] Robert J Valkenburg and Alan M McIvor. Accurate 3D measurement using a structured light system. Image and Vision Computing, 1998.

[VMD+18] Anton Vrieling, Michele Meroni, Roshanak Darvishzadeh, Andrew K Skidmore, Tiejun Wang, Raul Zurita-Milla, Kees Oosterbeek, Brian O'Connor, and Marc Paganini. Vegetation phenology from Sentinel-2 and field cameras for a Dutch barrier island. Remote Sensing of Environment, 2018.

[VN20] Richard Van Noorden. The ethical questions that haunt facial-recognition research. Nature, 2020.

[VOOR19] Cláudia M Viana, Sandra Oliveira, Sérgio C Oliveira, and Jorge Rocha. Land use/land cover change detection and urban sprawl analysis. Spatial modeling in GIS and R for Earth and environmental sciences, 2019.

[VSP+17] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 2017.

[VTGGP18] Kristof Van Tricht, Anne Gobin, Sven Gilliams, and Isabelle Piccard. Synergistic use of radar sentinel-1 and optical sentinel-2 imagery for crop mapping: A case study for Belgium. Remote Sensing, 2018.

[WD20] François Waldner and Foivos I Diakogiannis. Deep learning on edge: Extracting field boundaries from satellite images with a convolutional neural network. Remote Sensing of Environment, 2020.

[WH94] Nigel P Weatherill and Oubay Hassan. Efficient three-dimensional delaunay triangulation with automatic point creation and imposed boundary constraints. International Journal for Numerical Methods in Engineering, 1994.

[WH18] Yuxin Wu and Kaiming He. Group normalization. ECCV, 2018.

[WHL08] Qihao Weng, Xuefei Hu, and Dengsheng Lu. Extracting impervious surfaces from medium spatial resolution multispectral and hyperspectral imagery: a comparison. International Journal of Remote Sensing, 2008.

[WJHM15] Martin Weinmann, Boris Jutzi, Stefan Hinz, and Clément Mallet. Semantic point cloud interpretation based on op-
timal neighborhoods, relevant features and efficient classifiers. *ISPRS Journal of Photogrammetry and Remote Sensing*, 2015.

[WMG14] Michael Waechter, Nils Moehrle, and Michael Goesele. Let there be color! Large-scale texturing of 3D reconstructions. *ECCV*, 2014.

[WPC+21] Kan Wu, Houwen Peng, Minghao Chen, Jianlong Fu, and Hongyang Chao. Rethinking and improving relative position encoding for vision transformer. *ICCV*, 2021.

[WSM+15] Martin Weinmann, Alena Schmidt, Clément Mallet, Stefan Hinz, Franz Rottensteiner, and Boris Jutzi. Contextual classification of point cloud data by exploiting individual 3D neigbourhoods. *ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 2015.

[WXS+20] Yuqing Wang, Zhaoliang Xu, Hao Shen, Baoshan Cheng, and Lirong Yang. Centermask: Single shot instance segmentation with point representation. *CVPR*, 2020.

[YBP20] Raghu Yaramasu, Varaprasad Bandaru, and Koutilya Pnvr. Pre-season crop type mapping using deep neural networks. *Computers and Electronics in Agriculture*, 2020.

[YCC+21] Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, et al. Florence: A new foundation model for computer vision. *arXiv preprint arXiv:2111.11432*, 2021.

[YCHNP19] Derek S Young, Xi Chen, Dilrukshi C Hewage, and Ricardo Nilo-Poyanco. Finite mixture-of-gamma distributions: estimation, inference, and model-based clustering. *Advances in Data Analysis and Classification*, 2019.

[YFST18] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. FoldingNet: Point cloud auto-encoder via deep grid deformation. *CVPR*, 2018.

[YK09] Lexiang Ye and Eamonn Keogh. Time series shapelets: A new primitive for data mining. *ACM SIGKDD*, 2009.

[YL22] Mulin Yu and Florent Lafarge. Finding Good Configurations of Planar Primitives in Unorganized Point Clouds. *CVPR*, 2022.
[YZYL18] Hong-Ming Yang, Xu-Yao Zhang, Fei Yin, and Cheng-Lin Liu. Robust classification with convolutional prototype learning. CVPR, 2018.

[ZJR<sup>+</sup>15] Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip H. S. Torr. Conditional random fields as recurrent neural networks. ICCV, 2015.

[ZKR<sup>+</sup>17] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan R Salakhutdinov, and Alexander J Smola. Deep sets. NeurIPS, 2017.

[ZSH19] Yang Zhou, Shuhan Shen, and Zhanyi Hu. Detail preserved surface reconstruction from point cloud. Sensors, 2019.

[ZWK19] Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl. Objects as points. arXiv preprint arXiv:1904.07850, 2019.

[ZZD<sup>+</sup>20] Yang Zhang, Zixiang Zhou, Philip David, Xiangyu Yue, Zerong Xi, Boqing Gong, and Hassan Foroosh. Polarnet: An improved grid representation for online LiDAR point clouds semantic segmentation. CVPR, 2020.

[ZZF21] Zixiang Zhou, Yang Zhang, and Hassan Foroosh. Panoptic-polarnet: Proposal-free LiDAR point cloud panoptic segmentation. CVPR, 2021.

[ZZW<sup>+</sup>21] Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, and Dahua Lin. Cylindrical and asymmetrical 3D convolution networks for LiDAR segmentation. CVPR, 2021.

Miscellaneous

[BDT] BD TOPO. https://geoservices.ign.fr/documentation/donnees/vecteur/bdtopo. Accessed: 2022-06-24.

[CAP] The common agricultural policy at a glance. https://ec.europa.eu/info/food-farming-fisheries/key-policies/common-agricultural-policy/cap-glance_en. Accessed: 2021-09-24.

[geo] La démarche géocommuns. https://www.ign.fr/la-demarche-geocommuns. Accessed: 2022-07-01.
[Ico] Iconem. Iconem. https://iconem.com/en/. Accessed: 2022-06-24.

[IGN] IGN. LIDAR HD: une cartographie 3D du sol et du sursol de la france. https://geoservices.ign.fr/lidarhd. Accessed: 2022-07-24.

[Lyf] Lyft. Lyft self-driving safety report. https://autonomous. lyft.com/wp-content/uploads/2020/06/Safety_Report_ 2020.pdf. Accessed: 2022-07-24.

[Map] Open Street Map. Open street map. https://www. openstreetmap.org/. Accessed: 2022-07-01.

[Rie17] Christoph Rieke. Deep learning for instance segmenta- tion of agricultural fields. https://github.com/chrieke/ InstanceSegmentation_Sentinel2, 2017. Accessed: 2022-09- 27.

[RPG] Registre parcellaire graphique (RPG) : Contours des par- celles et îlots culturaux et leur groupe de cultures majori- taire. https://www.data.gouv.fr/en/datasets/registre- parcellaire-graphique-rpg-contours-des-parcelles-et-ilots- culturaux-et-leur-groupe-de-cultures-majoritaire/. Ac- cessed: 2021-09-24.

[RWC⁺¹⁹] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsu- pervised multitask learners. *OpenAI Blog*, 2019. Accessed: 2021-10-21.

[Val] Valeo.ai. Valeo drive4u, the first autonomous car to be demonstrated on the streets of paris. https://www.valeo. com/en/valeo-drive4u-the-first-autonomous-car-to-be- demonstrated-on-the-streets-of-paris/. Accessed: 2022- 07-24.

[Zoo] Zoox. Zoox: Built for riders, not drivers. https://zoox. com/vehicle/. Accessed: 2022-07-24.