# Apprentissage par transfert pour discriminer le bois/feuille des Unmanned aerial vehicle Laser Scanning (ULS)

Yuchen BAI, Jean-Baptiste Durand, Grégoire Vincent, Florence Forbes

► To cite this version:

Yuchen BAI, Jean-Baptiste Durand, Grégoire Vincent, Florence Forbes. Apprentissage par transfert pour discriminer le bois/feuille des Unmanned aerial vehicle Laser Scanning (ULS). TRIDIFOR 2022 - Méthodes innovantes d'analyse de données 3D en forêt, GIP ECOFOR, Sep 2022, Online, France. 1 p. hal-04837364

HAL Id: hal-04837364

https://inria.hal.science/hal-04837364v1

Submitted on 13 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Apprentissage par transfert pour discriminer le bois/feuille des Unmanned aerial vehicle Laser Scanning (ULS)

PhD student:
Yuchen BAI

Supervisor:
Jean-Baptiste DURAND
Grégoire VINCENT
Florence FORBES

UGA
Université
Grenoble Alpes

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>
# Outline

- [ ] Introduction

- [ ] Three identified questions

- [ ] Prototype model

- [ ] Perspectives
# Introduction

<figure><img src="image_5.png" /></figure>

## Airborne Laser Scanning (ALS):

<figure><img src="image_6.png" /></figure>

* RIEGL LMSQ780

<figure><img src="image_7.png" /></figure>

* point density : ~70/m², pulse density : ~40/m²

<figure><img src="image_8.png" /></figure>

## Terrestrial Laser Scanning (TLS):

<figure><img src="image_9.png" /></figure>

* RIEGL VZ-400

<figure><img src="image_10.png" /></figure>

* point density : >200,000/m², pulse density : >200,000/m²

<figure><img src="image_11.png" /></figure>

## Unmanned aerial vehicle (UAV) Laser Scanning (ULS):

<figure><img src="image_12.png" /></figure>

* RIEGL miniVUX-1UAV

<figure><img src="image_13.png" /></figure>

* point density : >750/m², pulse density : >500/m²

<figure><img src="image_14.png" /><figcaption>Figure 1. Leaf Area Index (LAI)</figcaption></figure>

The leaf area density (LAD) is defined as the total one-sided leaf area of photosynthetic tissue per unit canopy volume. The Leaf area index (LAI) is then derived by integrating the leaf area density over the canopy height. It corresponds to the one sided leaf area per unit horizontal ground surface area.
<figure><img src="image_15.png" /><figcaption>**Figure 2.** LiDAR (Light Detection and Ranging) in a cuboid, 20 m x 20 m x 50 m, *"More is different"* -- P. W. Anderson</figcaption></figure>
# Three identified questions

* How to do semantic segmentation (wood-leaf discrimination) on ULS data?

* TLS -> dense, small footprint and fewer mixed points

* no good approach for drone data (Unmanned aerial vehicle Laser Scanning (ULS))

* How to get a more accurate and unbiased point estimator of vegetation density?

* clumping (aggregate distribution of leaves),

* occlusion (limited penetration and incomplete exploration of voxels)

* develop estimates which are less biased and have lower variance

* How to correct the censorship bias of undetected targets?

* undetected hits bias the calculated signal attenuation and the estimated leaf areas.

* model the effect of this censorship and correct this bias

* in dense canopies, systematic overestimation of Leaf Area Index (LAI) 10~30% in magnitude

* simulation!

attenuation_PPL_ML
E profile

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>
# State of the Arts methods

<figure><img src="image_18.png" /></figure>

## Methods based on geometric and spatial properties

<figure><img src="image_19.png" /></figure>

- Treeseg[1], LeWos[2], Quantitative Structure Models (QSMs) based[3], Superpoint based[4], Itakura et al. [10]

<figure><img src="image_20.png" /></figure>

## Methods based on Random forest :

<figure><img src="image_21.png" /></figure>

- 'Deep Points Consolidation algorithm'[5]

<figure><img src="image_22.png" /></figure>

## Methods based on Deep Learning :

<figure><img src="image_23.png" /></figure>

- Point-wise methods : FSCT[6], Deep-RBN[7], Morel et al.'s method [8]

<figure><img src="image_24.png" /></figure>

- Voxel-wise methods : Windrim et al.'s method[9], Yang et al. [11]

<figure><img src="image_25.png" /><figcaption>Fig 3. Lewos on TLS data</figcaption></figure>
# Prototype model : Point-Voxel based neural network

## Semantic segmentation on ULS dataset

How to get training data?

- label transfer from TLS to ULS data

- training with TLS and fine-tuning with ULS

- downsample the TLS data

<figure><img src="image_26.png" /><figcaption>Fig 4. ULS-like data</figcaption></figure>
Use Canopy Height Model (CHM) to qualify co-registration quality

<figure><img src="image_27.png" /><figcaption>Figure 5. CHM comparaison</figcaption></figure>

$$ \text{Diff CHM} = \text{ULS CHM} - \text{TLS CHM} $$
<figure><img src="image_28.png" /><figcaption>Figure 6. TLS and ULS co-registration</figcaption></figure>
<figure><img src="image_29.png" /><figcaption>Figure 7. TLS and ULS co-registration</figcaption></figure>
The architecture of Prototype Model

Input

<figure><img src="image_30.png" /><figcaption>Figure 8. The architecture of the prototype model</figcaption></figure>
<figure><img src="image_31.png" /><figcaption>Figure 9. Trilinear interpolation</figcaption></figure>

It approximates the value of a function at an intermediate point (x,y,z) within the local axial rectangular prism linearly, using function data on the lattice points. Find 8 nearest voxels in the space.
<figure><img src="image_32.png" /><figcaption>Figure 10. Preliminary results : overprediction!</figcaption></figure>
<figure><img src="image_33.png" /><figcaption>Figure 11. Failure prediction for some parts, need to improve</figcaption></figure>
<figure><img src="image_34.png" /><figcaption>Fig 12-1. Training on tls, p@intensity, v@ratio point density, 0.2m, acc<50%</figcaption></figure>

<figure><img src="image_35.png" /><figcaption>Fig 12-2. p@elevation, v@ratio point density, 0.2m, acc<50%</figcaption></figure>

<figure><img src="image_36.png" /><figcaption>Fig 12-3. p@intensity, v@ratio point density, 0.1m, acc<50%</figcaption></figure>

<figure><img src="image_37.png" /><figcaption>Fig 12-4. p@intensity, v@ratio point density, 0.1m, without rescaling, acc<70%</figcaption></figure>

<figure><img src="image_38.png" /><figcaption>Fig 12-5. p@intensity, v@ratio point density, 0.1m, rescaling, acc=85%</figcaption></figure>

<figure><img src="image_39.png" /><figcaption>Fig 12-6. p@intensity, v@ratio point density, 0.1m, rescaling, acc=80%</figcaption></figure>

<figure><img src="image_40.png" /><figcaption>Fig 12-7. p@intensity, v@ratio point density, 0.1m, rescaling, acc <50%</figcaption></figure>

<figure><img src="image_41.png" /><figcaption>Fig 12-8. p@intensity, v@std intensity, 0.1m, rescaling, acc<50%</figcaption></figure>

Fig 12: Still no desired result
<figure><img src="image_42.png" /><figcaption>Fig 12-9. p@intensity, v@occupancy 1 or 0, 0.1m, acc<50%</figcaption></figure>

<figure><img src="image_43.png" /><figcaption>Fig 12-10. p@intensity, v@intensity moyen, 0.1m, acc<50%</figcaption></figure>

<figure><img src="image_44.png" /><figcaption>Fig 12-11. p@intensity + elevation, v@ratio point number, 0.1m, acc<50%</figcaption></figure>

<figure><img src="image_45.png" /><figcaption>Fig 12-12. p@intensity + elevation, v@ratio point number, 0.1m, **pointnet**, acc=90%</figcaption></figure>

<figure><img src="image_46.png" /><figcaption>Fig 12-13. p@intensity + elevation, v@ratio point number, 0.1m, pointnet, acc=92.061%</figcaption></figure>

<caption>Fig 12-14. p@intensity, v@ratio point number, 0.1m, pointnet, class weights added</caption>

<caption>Fig 12-15. p@intensity, v@ratio point number, 0.1m, pointnet, ensure the proportion of wood points when sampling</caption>

Fig 12: Result on test data, improvement needed
<figure><img src="image_47.png" /><figcaption>Figure 13. reflectance ratio (intensity), ULS (905 nm) vs TLS (1550 nm), Greg Vincent</figcaption></figure>
<figure><img src="image_48.png" /><figcaption>(a) accuracy, 300 epochs</figcaption></figure>

<figure><img src="image_49.png" /><figcaption>(b) loss, 300 epochs</figcaption></figure>

Figure 14. Plot loss and accuracy for training 300 epochs
## 2.4 Matthew Coefficient Correlation (MCC)

The MCC can be calculated directly from the confusion matrix using the formula:

$$MCC = \frac{TP \times TN - FP \times FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}$$

<table><caption>Fig 13. Matthew Coefficient Correlation / Phi coefficient</caption><thead><tr><th rowspan="2"></th><th colspan="2">Predicted condition</th></tr><tr><th>Positive (PP)</th><th>Negative (PN)</th></tr></thead><tbody><tr><th rowspan="2">Actual condition</th><th>Total population = P + N</th><th></th></tr><tr><th>Positive (P)</th><th>True positive (TP)</th><th>False negative (FN)</th></tr><tr><th></th><th>Negative (N)</th><th>False positive (FP)</th><th>True negative (TN)</th></tr></tbody></table>

<figure><img src="image_50.png" /><figcaption>Fig 15. MCC is a good metric for class imbalance case</figcaption></figure>
<table><caption>Figure 13. Confusion Matrix for the best result so far (sample size=5000 point, epoch = 546)</caption><thead><tr><th></th><th></th><th colspan="2">Prediction</th></tr></thead><tbody><tr><td></td><td>Total test point = 100000</td><td>Wood</td><td>Leaf</td></tr><tr><td rowspan="2">True situation</td><td>Wood</td><td>328</td><td>3182</td></tr><tr><td>Leaf</td><td>4325</td><td>92165</td></tr></tbody></table>
Perspectives

Improving existing DL prototype model

Limits of current estimators of vegetation density

estimation of clumping parameter required

study spatial dependencies by using hierarchical Bayesian models.

The censorship bias of undetected intercepts

undetected intercepts biases the calculated signal attenuation and the
estimated leaf areas

model the effect and correct this bias
<figure><img src="image_51.png" /><figcaption>Figure 16. DART simulation</figcaption></figure>

DART has 3 major modes, we may use DART-RC which simulates LiDAR signals with a Ray-Carlo (RC) approach that combines ray tracking and forward Monte Carlo (MC) methods.
Reference

1. A. Burt, M. Disney, and K. Calders, "Extracting individual trees from lidar point clouds using treeseg," Methods in Ecology and Evolution, vol. 10, no. 3, pp. 438-445, 2019.

2. D. Wang, S. Momo Takoudjou, and E. Casella, "Lewos: A universal leaf-wood classification method to facilitate the 3d modelling of large tropical trees using terrestrial lidar," Methods in Ecology and Evolution, vol. 11, no. 3, pp. 376-389, 2020.

3. P. Raumonen, E. Casella, K. Calders, S. Murphy, M. Åkerblom, and M. Kaasalainen, "Massive-scale tree modelling from tls data," ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. II-3/W4, pp. 189-196, 2015.

4. D. Wang, "Unsupervised semantic and instance segmentation of forest point clouds," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 165, pp. 86-97, 2020.

5. S. T. Digumarti, J. Nieto, C. Cadena, R. Siegwart, and P. Beardsley, "Automatic segmentation of tree structure from point cloud data," Oct 2018.

6. S. Krisanski, M. S. Taskhiri, S. Gonzalez Aracil, D. Herries, A. Muneri, M. B. Gurung, J. Mont-gomery, and P. Turner, "Forest structural complexity tool—an open source, fully-automated tool for measuring forest point clouds," Remote Sensing, vol. 13, no. 22, 2021.

7. H. Liu, X. Shen, L. Cao, T. Yun, Z. Zhang, X. Fu, X. Chen, and F. Liu, "Deep learning in forest structural parameter estimation using airborne lidar data," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 1603-1618, 2021.

8. J. Morel, A. Bac, and T. Kanai, "Segmentation of unbalanced and in-homogeneous point clouds and its application to 3d scanned trees," Sep 2020.

9. L. Windrim and M. Bryson, "Detection, segmentation, and model fitting of individual tree stems from airborne laser scanning of forests using deep learning," Remote Sensing, vol. 12, no. 9, 2020.

10. K. Itakura, S. Miyatani, and F. Hosoi, "Estimating tree structural parameters via automatic tree segmentation from lidar point cloud data," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 15, pp. 555-564, 2022.

11. J. Yang, Z. Kang, S. Cheng, Z. Yang, and P. H. Akwensi, "An individual tree segmentation method based on watershed algorithm and three-dimensional spatial distribution analysis from airborne lidar point clouds," 2020.
Annexe A. Compare 12 different methods

<table><thead><tr><th rowspan="2"></th><th rowspan="2">Algo</th><th colspan="4">Train Set</th><th colspan="4">Test Set</th></tr><tr><th>Name</th><th>Number of tree</th><th>Number of points</th><th>Type de LiDAR</th><th>Name</th><th>Number of tree</th><th>Number of points in each class</th><th>Type de LiDAR</th></tr></thead><tbody><tr><td rowspan="7">Geometric</td><td>treeseg</td><td>1 ha of tropical forest (moist, Terra Firma, lowland, mixed species, old growth) in Nouragues Nature Reserve, French Guiana/ 0.25 ha of Eucalyptus spp. open forest in Karawatha Forest Park, Australia</td><td>425/40</td><td>4 billion/50 million</td><td>TLS</td><td></td><td></td><td></td><td></td></tr><tr><td>automatic reconstruction for plot level</td><td>30-m diameter English oak plot and a 80-m diameter Australian eucalyptus plot</td><td></td><td>124 million/71 million</td><td>TLS</td><td></td><td></td><td></td><td></td></tr><tr><td>new classification algo</td><td>Mulligans Flat Nature Reserve (Australian Capital Territory, Australia)</td><td>28</td><td>1,379,173 points</td><td>MLS</td><td></td><td></td><td></td><td></td></tr><tr><td>robot detect tronc superpoint, geometric only</td><td>any forest</td><td></td><td>10000 points/second</td><td>TLS</td><td></td><td></td><td></td><td></td></tr><tr><td>Random forest</td><td>Synthetic dataset</td><td>30</td><td>20 million triangle mesh</td><td>TLS</td><td></td><td></td><td></td><td></td></tr><tr><td>LeWoS</td><td>simulated data</td><td></td><td>50000/second</td><td>TLS</td><td></td><td></td><td></td><td></td></tr><tr><td>LeWoS</td><td>Eastern estimating of Cameroon</td><td>61</td><td>700,000 scan point clouds per second</td><td>ULS(unnamed aerial vehicle)</td><td>Qishan scenic area</td><td>522, 160, 456, and 167</td><td></td><td>ULS</td></tr><tr><td rowspan="4">Supervised</td><td>a novel individual tree segmentation method</td><td>Qishan scenic area, 4 places, 1947.16, 44,596.64, 60,601.78 and 14,780.11 square meters</td><td>501 (trees), 168 (trees)/334 (buildings), 426 (trees), and 166 (trees)</td><td>10240 after the data augmentation</td><td>ULS(unnamed aerial vehicle)</td><td>Qishan scenic area</td><td>522, 160, 456, and 167</td><td></td><td>ULS</td></tr><tr><td>FSCT</td><td>6 20m^2 plots and 1 circular 40 m</td><td>177 trees</td><td>individual tree point clouds</td><td>TLS</td><td></td><td></td><td></td><td></td></tr><tr><td>an innovative method</td><td></td><td></td><td>20.000 to 100.000</td><td>TLS</td><td>a semi-deciduous forest of Eastern Cameroon</td><td></td><td></td><td>TLS</td></tr><tr><td>australia, helicopter</td><td>two Radiata pine forests in NSW, Australia Tumut, NSW, Australia (collected in November 2016) and Carabost. NSW, Australia (collected in February, 2018)</td><td>400 stems/ha and 600 stems/ha - 60 + 70</td><td>300-700 points per m2</td><td>ALS (helicopter)</td><td></td><td>- ~10 + ~15</td><td></td><td>ALS</td></tr><tr><td></td><td>3D CNN</td><td>eight different sites across the Northern New England/Acadian Forest</td><td>714 trees/ha</td><td>9 pls/m2</td><td>ALS</td><td></td><td></td><td></td><td></td></tr></tbody></table>
Annexe A-bis. Compare 12 different methods

<table><thead><tr><th></th><th>Algo</th><th>Descriptor/Method</th><th>Performance</th><th>Notes</th><th>Articles</th><th>Author</th><th>Year</th><th>conference/Journal</th></tr></thead><tbody><tr><td rowspan="10">Geometric</td><td>treeseg</td><td>Euclidean clustering, principal component analysis, region-based segmentation, shape fitting and connectivity testing</td><td>96 % and 70 %</td><td>Not full automatically<br>30% required further manual segmentation</td><td>Extracting individual trees from lidar point clouds using treeseg</td><td>Andrew Burt et al.</td><td>2018</td><td>Methods in Ecology and Evolution</td></tr><tr><td>automatic reconstruction for plot level</td><td>build quantitative structure models (QSMs) for every tree, this method is based on morphological rules, a cover-set approach, and geometric primitives.</td><td>N/A</td><td>quantitative structure model of every tree<br>Calculate the biomass, not classification</td><td>MASSIVE-SCALE TREE MODELLING FROM TLS DATA</td><td>P. Raumone n et al.</td><td>2015</td><td>ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences</td></tr><tr><td>new classification algo</td><td>structural characteristics of the vegetation objects</td><td>98 % for trees and 80 % for vegetation</td><td>data set is too small, recognize tree and vegetation not classification, and failed to correctly classify three of the trees and three of the elevated vegetation objects</td><td>Deriving comprehensive forest structure information from mobile laser scanning observations using automated point cloud classification</td><td>Suzanne M. Marselis et al.</td><td>2016</td><td>Environmental Modelling & Software</td></tr><tr><td>robot detect tronic</td><td>3-D geometry analysis of point clouds and geometric primitives fitting</td><td>N/A</td><td>Tech report, let robot be able to traverse the perceived terrain. Not too related to us.</td><td>Automatic Three-Dimensional Point Cloud Processing for Forest Inventory</td><td>Jean-François Lalonde et al.</td><td>2006</td><td>N/A</td></tr><tr><td>superpoint, geometric only</td><td>Super point graph + unsupervised</td><td>87.7%</td><td>based on super point, the conception that I have seen in Paris<br>Simulated data not real</td><td>Unsupervised semantic and instance segmentation of forest point clouds</td><td>Di WANG</td><td>2020</td><td>ISPRS Journal of Photogrammetry and Remote Sensing</td></tr><tr><td>Random forest</td><td>Random forest classifier<br>Deep Points Consolidation, meso-skeleton</td><td>~ 91%</td><td>The algorithm works at the level of the input point cloud itself, preserving quantitative accuracy in the resulting model.<br>Use labelled data manual thresholds or heuristics based on tree allometry do not scale well across different species of trees and</td><td>Automatic Segmentation of Tree Structure From Point Cloud Data</td><td>Sundara Tejaswi Digumarti</td><td>2018</td><td>IEEE</td></tr><tr><td>LeWoS</td><td>graph based point cloud segmentation technique<br>Manual fine-tuning some thresholds, class probability<br>Graph-structured class regularization oeprates on class probability</td><td>~ 91%</td><td>STOA?</td><td>LeWoS: A Universal Leaf-wood Classification Method to Facilitate the 3D Modelling of Large Tropical Trees Using Terrestrial LiDAR</td><td>Di WANG</td><td>2019</td><td>Methods in Ecology and Evolution</td></tr><tr><td rowspan="4">Supervised</td><td>a novel individual tree segmentation method</td><td>Deep learning, supervised, PointNet++</td><td>~ 90%</td><td>similar to FSCT,<br>See more details about data on the page 7 of 22</td><td>Individual Tree Crown Segmentation Directly from UAV-Borne LiDAR Data Using the PointNet of Deep Learning</td><td>Xinxin Chen et al.</td><td>2021</td><td>MDPI</td></tr><tr><td>FSCT</td><td>Deep learning, supervised, PointNet++</td><td>~ 95.4%</td><td>FSCT</td><td>Sensor Agnostic Semantic Segmentation of Structurally Diverse and Complex Forest Point Clouds Using Deep Learning</td><td>Sean Krisanski et al.</td><td>2021</td><td>MDPI</td></tr><tr><td>an innovative method</td><td>Deep learning, supervised, PointNet++<br>Poisson disk sampling, local PCA,</td><td>~ 90%</td><td>subsample the raw point clouds by using Poissondisk sampling our method classifies 90 to 95% of the points as good as a human</td><td>Segmentation of unbalanced and in-homogeneous point clouds and its application to 3D scanned trees</td><td>Jules Morel</td><td>2020</td><td>Springer Nature</td></tr><tr><td>australia, helicopter</td><td>low-flying aircraft at high-resolutions (hundreds of points per m2)<br>A stem segmentation approach extended by incorporating voxel representations that include LiDAR return intensity into the learning representation</td><td>~ 72% - 92%</td><td>Individual tree need manually label</td><td>Detection, Segmentation, and Model Fitting of Individual Tree Stems from Airborne Laser Scanning of Forests Using Deep Learning</td><td>Lloyd Windrim</td><td>2020</td><td>MDPI</td></tr><tr><td></td><td>3D CNN</td><td>All LiDAR data were acquired between 2012 and 2016 in leaf-on conditions between June and August</td><td></td><td></td><td>The Use of Three-Dimensional Convolutional Neural Networks to Interpret LiDAR for Forest Inventory</td><td>Elias Ayrey</td><td>2018</td><td>MDPI</td></tr></tbody></table>
<figure><img src="image_52.png" /><figcaption>Annexe B. Small branches can also be detected</figcaption></figure>
Annexe C. merged TLS data

<figure><img src="image_53.png" /><figcaption>Figure 2. Different scanning position under tower</figcaption></figure>

<figure><img src="image_54.png" /><figcaption>Figure 3. A part of TLS Digital Terrain Model (DTM)</figcaption></figure>
<figure><img src="image_55.png" /><figcaption>Annexe D. Training on TLS predict on ULS</figcaption></figure>
# Prototype model : Point-Voxel based neural network

## Semantic segmentation on ULS data

<figure><img src="image_56.png" /></figure>

- label transfer from TLS to ULS data

- training with TLS and fine-tuning with ULS

- Subsample ULS-like data from the TLS data
<figure><img src="image_57.png" /></figure>

<figure><img src="image_58.png" /></figure>

<figure><img src="image_59.png" /><figcaption>Figure 7. Subsample ULS-like data from the TLS data</figcaption></figure>
## 1.3 Subsample ULS-like data from the TLS data

<figure><img src="image_60.png" /><figcaption>Fig 4: TLS 36.laz</figcaption></figure>

<figure><img src="image_61.png" /></figure>

<figure><img src="image_62.png" /><figcaption>Fig 5: ULS-like data from TLS 36.laz</figcaption></figure>
## 2.3 Adam or SGD + Nestrov momentum?

moving average for the step in lieu of the gradient. Mathematically, the Adam update equation can be represented as:

$$w_k = w_{k-1} - \alpha_{k-1} \cdot \frac{\sqrt{1 - \beta_2^k}}{1 - \beta_1^k} \cdot \frac{m_{k-1}}{\sqrt{v_{k-1}} + \epsilon}, \quad \text{where} \quad (2)$$

$$m_{k-1} = \beta_1 m_{k-2} + (1 - \beta_1) \hat{\nabla} f(w_{k-1}), \\ v_{k-1} = \beta_2 v_{k-2} + (1 - \beta_2) \hat{\nabla} f(w_{k-1})^2. \quad (3)$$

<caption>Fig 9. Adam update equation</caption>

[*] Nitish Shirish Keskar, Richard Socher, “Improving Generalization Performance by Switching from Adam to SGD”
## 2.3 Adam or SGD + Nesterov momentum?

For ease of analysis, we rewrite update rules of MaSS in Eq.1 in the following equivalent form (introducing an additional variable $v$):

$$ \left\{ \begin{array}{l} \mathbf{w}_{t+1} \leftarrow \mathbf{u}_t - \eta_1 \tilde{\nabla} f(\mathbf{u}_t), \\ \mathbf{u}_{t+1} \leftarrow (1+\gamma)\mathbf{w}_{t+1} - \gamma \mathbf{w}_t + \eta_2 \tilde{\nabla} f(\mathbf{u}_t) \end{array} \right. \iff \left\{ \begin{array}{l} \mathbf{w}_{t+1} \leftarrow \mathbf{u}_t - \eta \tilde{\nabla} f(\mathbf{u}_t), \\ \mathbf{v}_{t+1} \leftarrow (1-\alpha)\mathbf{v}_t + \alpha \mathbf{u}_t - \delta \tilde{\nabla} f(\mathbf{u}_t), \\ \mathbf{u}_{t+1} \leftarrow \frac{\alpha}{1+\alpha} \mathbf{v}_{t+1} + \frac{1}{1+\alpha} \mathbf{w}_{t+1}. \end{array} \right. \quad (\xi) $$

There is a bijection between the hyper-parameters $(\eta_1, \eta_2, \gamma)$ and $(\eta, \alpha, \delta)$, which is given by:

$$ \gamma = (1 - \alpha)/(1 + \alpha), \quad \eta_1 = \eta, \quad \eta_2 = (\eta - \alpha\delta)/(1 + \alpha). \quad (9) $$

**Remark 3 (SGD+Nesterov).** In the literature, the Nesterov's method is sometimes written in a similar form as the R.H.S. of Eq.8. Since SGD+Nesterov has no compensation term, $\delta$ has to be fixed as $\eta/\alpha$, which is consistent with the parameter setting in [15].

Fig 10. Nestrove SGD update equation

<footnote>[*] Chaoyue Liu, Mikhail Belkin, “Accelerating SGD with momentum for over-parameterized learning”</footnote>
Annexe 2. ULS merged data (intensity)

<figure><img src="image_63.png" /></figure>
<figure><img src="image_64.png" /><figcaption>Fig 11. Accuracy of 300 epochs on new dataset, SGD + Nestrov, class weights</figcaption></figure>

<figure><img src="image_65.png" /><figcaption>Fig 12. Loss of 300 epochs on new dataset, SGD + Nestrov, class weights</figcaption></figure>
<figure><img src="image_66.png" /><figcaption>Annexe 3. Tls parcels with labels</figcaption></figure>
attenuation_PPL_MLE
profile

<figure><img src="image_67.png" /></figure>

ULS_195956.vox

ULS_195956_dec90pct.vox

attenuation_PPL_ML
E profile

<figure><img src="image_68.png" /></figure>

TLS.vox

ULS_195956.vox

Attenuation
<figure><img src="image_69.png" /><figcaption>Figure 4. TLS and ULS co-registration</figcaption></figure>
<figure><img src="image_70.png" /><figcaption>Figure 6. label transfer from TLS to ULS data</figcaption></figure>
# NEURAL NETWORKS FOR LEAF/WOOD DISCRIMINATION IN LIDAR FOR LEAF AREA DENSITY ESTIMATION

PhD student:
Yuchen BAI

Supervisor:
Jean-Baptiste DURAND
Grégoire VINCENT
Florence FORBES

UGA
Université
Grenoble Alpes

Inria

Statify

<figure><img src="image_71.png" /></figure>
## 2.2 Weighted loss/class weights for wood&leaf

```python
>>> TrainDataSet is prepared:
>>> device = cpu
>>> print('test label detach()'. numpy() .transpose((0,2,1)). reshape(self.batch_size* self.sample_size, 2)
>>> samples.shape=(60, 5000, 5)
>>> samples_cuboid_index.shape=60
>>> So checkpoint folder exist, the path is: /home/yuchen/Documents/PhD/phd_mission/src/checkpoints
>>> So gradient clipping folder exist, the path is: /home/yuchen/Documents/PhD/phd_mission/src/checkpoints
>>> self train_loader dataset= 212
No checkpoints found at /home/yuchen/Documents/PhD/phd_mission/src/checkpoints
======== Start epoch 0 ===========
epoch : loss = 0.7638216018676758, weighted_loss = 13.978540420532227
[e=0]>>> [Training] - Current test loss: 0.7638216018676758 - test accuracy: 0.502
epoch : loss = 0.6371403336524963, weighted_loss = 7.388162612915039
[e=0]>>> [Training] - Current test loss: 0.6371403336524963 - test accuracy: 0.5047
epoch : loss = 0.5406445264816284, weighted_loss = 3.8675711154937744
[e=0]>>> [Training] - Current test loss: 0.5406445264816284 - test accuracy: 0.49625
epoch : loss = 0.41227322816848755, weighted_loss = 19.190902709960938
[e=0]>>> [Training] - Current test loss: 0.41227322816848755 - test accuracy: 0.50155
epoch : loss = 0.32395246624946594, weighted_loss = 2.7720727920532227
[e=0]>>> [Training] - Current test loss: 0.32395246624946594 - test accuracy: 0.5053
epoch : loss = 0.1903519630432129, weighted_loss = 7.873373985290527
[e=0]>>> [Training] - Current test loss: 0.1903519630432129 - test accuracy: 0.49925
epoch : loss = 0.12450125813484192, weighted_loss = 3.9948694705963135
[e=0]>>> [Training] - Current test loss: 0.12450125813484192 - test accuracy: 0.49805
epoch : loss = 0.1914239078760147, weighted_loss = 1.1349207162857056
[e=0]>>> [Training] - Current test loss: 0.1914239078760147 - test accuracy: 0.50235
epoch : loss = 0.05195620283484459, weighted_loss = 1.7309350967407227
```

<caption>Fig 7: wood point predictions are dominant</caption>

sklearn.class_weight.compute_class_weight(class_weight, *, classes, y) [source]

Estimate class weights for unbalanced datasets.

<table><tr><td>Parameter s:</td><td>class_weight: dict, 'balanced' or None<br/>If 'balanced', class weights will be given by n Samples / (n classes * np.bincount(y)). If a dictionary is given, keys are classes and values are corresponding class weights. If None is given, the class weights will be uniform.<br/>classes: ndarray<br/>Array of the classes occurring in the data, as given by np.unique(y_org) with y_org the original class labels.<br/>y: array-like of shape (n Samples,)<br/>Array of original class labels per sample.</td></tr><tr><td>Returns:</td><td>class_weight_vector: ndarray of shape (n_classes,)<br/>Array with class_weight_vector[i] the weight for i-th class.</td></tr></table>

References

The "balanced" heuristic is inspired by Logistic Regression in Rare Events Data, King, Zen, 2001.

<caption>Fig 8: Class weights given by sklearn are better than preset</caption>