dataesr/scientific-paragraphs-categorization huggingface dataset:
Baseline Parameters The first baseline is based on the Pointnet++ architecture, commonly used in deep learning applications. We selected the architecture provided by the authors  [29]. It is composed of three abstraction layers as the feature extractor and three MLP as the last part of the model. The number of points and neighborhood radius by layer were taken from the PointNet++ authors for outdoor and dense environments using MSG passing. The second baseline is based on the KPConv architecture. We selected the KP-FCNN architecture provided by the authors for outdoor scenes  [30]. It is composed of a five-layer network, where each layer contains two convolutional blocks, as originally proposed by the ResNet authors  [32]. We used dl 0 = 6 cm, inspired by the value used by the authors for the Semantic3D dataset.
Implementation Details We fixed similar training parameters between both baselines (Pointnet++  [29]  and KPConv  [30] ) in order to compare their performance. As pre-processing, point clouds are sub-sampled on a grid, keeping one point per voxel (voxel size of 6 cm). Models learn, validate, and test with these data. Then, when testing, we perform inference with the under-sampled point clouds and then give the labels in "full resolution" with a KNN of the probabilities (not the labels). Spheres were computed in pre-processing (before the training stage) in order to reduce the computational cost. During training, we selected the spheres by class (class of the center point of the sphere) so that the network considered all the classes at each epoch, which greatly reduces the problem of class imbalance of the dataset.
At each epoch, we took one point cloud from the dataset (T i for CARLA and S i for Paris) and set the number of spheres seen in this point cloud to 100. Two features were included as input: RGB color information and height of points (z). In order to prevent overfitting, we included geometric data augmentation techniques: elastic distortion, random Gaussian noise with σ = 0. 1 m and clip at 0. 05 m, random rotations around z, anisotropic random scale between 0. 8 and 1. 2, and random symmetry around the x and y axes. We included the following transformations to prevent overfitting due to color information: chromatic jitter with σ = 0. 05, and random dropout of RGB features with a probability of 20%. For training, we selected the loss function as the sum of Cross Entropy and Power Jaccard with p = 2  [33].
Quantitative Results Prediction of test point clouds was performed by using a sphere-based approach using a regular grid and maximum voting scheme. In this case, the spheres' centers were calculated to keep the intersections of spheres at 1/3 of their radius (r = 10 m as done during the training). We report the obtained results in Table  2. We obtained an overall 13. 9 % mIoU for Point-Net++ and 37. 5 % mIoU for KPConv. This remains low for state-of-the-art architectures. This shows the difficulty and the wide variety of classes present in this Paris-CARLA-3D dataset. We can also see poorer results on synthetic data due to the greater variety of objects between CARLA cities, while, for the real data, the test data are very close to the training data.
Qualitative Results Semantic segmentation of point clouds is better on Paris than on CARLA in all evaluated scenarios. This is an expected behavior because class variability and scene configurations are much more complex in the synthetic dataset. By way of an example, Figures  3 4 5 6 display the predicted labels and ground truth from the test sets of Paris and CARLA data. These images were obtained from the KPConv architecture. From the qualitative results of semantic segmentation, it is evidenced the complexity of our proposed dataset. In the case of Paris data, color information is discriminant enough to separate sidewalks, roads, and road-lines. This is an expected behavior because the point clouds were from the same town and were acquired the same day. However, in CARLA point clouds, color information in ground-like classes changed between different towns.
Influence of Color We studied the influence of color information during training in the PC3D dataset. In Table  3 , we report the obtained results on the test set of semantic segmentation using the KPConv architecture without RGB features. The rest of the training parameters were the same as in the previous experiment. We can see that even if the colorization of the point cloud can create artifacts during the projection step (from the difference in point of view between the LiDAR sensor and the cameras or from the presence of moving objects), the use of the color modality in addition to geometry clearly improved the segmentation results.
Task Protocol We introduce the task protocol to evaluate the instance segmentation methods in our dataset. Evaluation of the performance in the instance segmentation task is different to that in the semantic segmentation task. Inspired by  [35]  on things, we report Segment Matching (SM) and Panoptic Quality (PQ), with IoU = 0. 5 as the threshold to determine well-predicted instances. We also report the mean IoU, based on IoU by instance i (IoU i ), calculated as follows: IoU i = IoU IoU ≥ 0. 5 0, otherwise (2) A common issue in LiDAR scanning is the presence of far objects that are unrecognizable due to the small number of points. In the semantic segmentation task, such objects do not affect evaluation metrics, due to their low rate. However, in the instance segmentation task, they may considerably affect the evaluation of the algorithms.
For now, we only evaluate the geometry prediction, as we leave the prediction of simultaneous geometry, semantics, and color for future work. The vast majority of the existing methods of scene completion (SC) work focus on small indoor scenes, while, in our case, we have a dense outdoor environment with our Paris-CARLA-3D dataset. Completing outdoor LiDAR point clouds is more challenging than data obtained from RGB-D images acquired in indoor environments, due to the sparsity of points obtained using LiDAR sensors. Moreover, larger occluded areas are present in outdoor scenes, caused by static and temporary foreground objects, such as trees, parked vehicles, bus stops, and benches. SemanticKITTI  [7]  is a dataset conducting scene completion (SC) and semantic scene completion (SSC) on LiDAR data, but they use only one single scan as input, with a target (ground truth) being the accumulation of all LiDAR scans.
To evaluate the completed scene, we use the Chamfer Distance (CD) between the original P 1 and predicted P 2 point clouds: CD = 1 |P 1 | ∑ x∈P 1 min y∈P 2 ||x -y|| 2 + 1 |P 2 | ∑ y∈P 2 min x∈P 1 ||y -x|| 2 (3) In a self-supervised context, not having the ground truth and having the predicted point cloud more complete than the target places some limitations on using the CD metric. For this, we introduce a mask that needs to be used to compute the CD only on the points that were originally available. The mask is simply a binary occupancy grid on the original point cloud. We extract the random chunks as explained previously for Paris (1000 chunks per point cloud) and CARLA (3000 chunks per town) and provide them along with the dataset for future research on scene completion.
7. We can see that the Chamfer Distance (CD) is better for the model trained only on synthetic CARLA data: the network is attempting to fill a local plane in large missing regions and smoothing the rest of the geometry. This is an expected behavior, because of the handcrafted geometry present in CARLA, where planar geometric features are predominantly present. Point clouds of real outdoor scenes are not easily obtained and the need to complete missing is becoming increasingly important in vision-related tasks; here, we can see the value of leveraging the large amount of synthetic data present in CARLA to pre-train the network and fine-tune it on other smaller datasets such as Paris when not enough data are available. As we can see in Table  7 , pre-training on CARLA and then fine-tuning on Paris allows us to obtain the best predicted TSDF ( 1 and mIoU) and point cloud (Chamfer Distance).
Conclusions We presented a new dataset called Paris-CARLA-3D. This dataset is made up of both synthetic data (700 M points) and real data (60 M points) from the same LiDAR and camera mobile platform. Based on this dataset, we presented three classical tasks in 3D computer vision (semantic segmentation, instance segmentation, and scene completion) with their evaluation protocol as well as a baseline, which will serve as starting points for future work using this dataset. On semantic segmentation (the most common task in 3D vision), we tested two stateof-the-art methods, PointNet++ and KPConv, and showed that KPConv obtains the best results (37. 5% overall mIoU). We also presented a first instance detection method on dense point clouds from mapping systems (with vehicle and pedestrian instances for synthetic data and vehicle instances for real data).
Results in semantic segmentation task using PointNet++ and KPConv architectures on our dataset, Paris-CARLA-3D. Results are mIoU in %. For S 0 and S 3 , training set is S 1 , S 2. For T 1 and T 7 , training set is T 2 , T 3 , T 4 and T 5. Overall mIoU is the mean IoU on the whole test sets (real and synthetic).
[30]lts in semantic segmentation task using KPConv[30]architecture on our PC3D dataset with and without RGB colors on LiDAR points. Results are mIoU in %. For S 0 and S 3 , training set is S 1 , S 2. For T 1 and T 7 , training set is T 2 , T 3 , T 4 and T 5. Overall mIoU is the mean IoU on the whole test sets (real and synthetic).
Results in transfer learning for the semantic segmentation task using KPConv architecture on our PC3D dataset. Results are mIoU in %. Pre-training was done using urban towns from CARLA (T 2 , T 3 , and T 5 ). No fine-tuning: the model was pre-trained on CARLA data without fine-tuning on Paris data. No frozen parameters: the model was pre-trained on CARLA without frozen parameters during fine-tuning on Paris data. No transfer: the model was trained only on the Paris training set.
Although it has been shown that phenology can respond to temporal environmental variation in free ranging populations of several species, little is known about the mechanisms of these responses and their effects on demography, and in particular on survival. Exploring phenological responses and their associated consequences on survival can be achieved at two distinct scales: the population scale, which focusses on a set of common responses to environmental conditions, and the individual scale, focusing on the relative position of each individual in the distribution of survival and phenology under particular conditions. In this study, we apply capture-mark-recapture multistate modelling on a 38-year monitoring dataset of blue tits (Cyanistes caeruleus) to investigate the effects of breeding phenology and some demographic covariates (breeding density, average and individual breeding success) on adult survival, at both population and individual scales.
In this study, using a long-term monitoring dataset, we investigated the effect of breeding phenology on adult survival in a population of a temperate passerine bird, the Blue tit (Cyanistes caeruleus). The data comprised 1562 adult individuals captured and ringed in nest boxes during the reproduction period across 38 years. In this population, breeding phenology is influenced by meteorological conditions. Median laying date is earlier in warmer years  (Bonamour et al. , 2019)  and has an impact on breeding success: earlier breeding is correlated with higher number of fledglings at the individual and population scales  (Marrot et al. , 2018). Long-term effects of climate on the phenology of the studied population also occurred over the period of the study  (Marrot et al. , 2018) , during which spring temperatures have significantly increased (1979-2016: +0.
Population Monitoring Our data was collected over 38 years  in a natural population of blue tits Cyanistes caeruleus in an evergreen oak (Quercus ilex) forest on the Mediterranean island of Corsica (E-Pirio site: Lat 42. 38; Long: 8. 75; see  Blondel et al. , 2006 and Charmantier et al. , 2016 for details). Blue tits are cavity-nesting non-migratory passerine birds that readily breed in nest boxes, which they often prefer to natural cavities  (Newton, 1994). The high density of nest boxes compared to the rarity of natural cavities in the E-Pirio site (A. C. , unpublished data) allowed us to monitor almost all breeding individuals in the E-Pirio study area. During the whole study period, nest boxes were checked every year at least weekly during the breeding season (March to June), allowing to accurately assess the date of the first laid egg for all clutches (hereafter, laying date), as well as the clutch size and the number of fledglings.
This 38 day window has been identified in a previous study using a sliding window analysis, as the time window most correlated with the average annual laying date in the focal population (see Table  1  in  Bonamour et al. , 2019). The temperature 6 data were obtained by regressing daily temperature measurements over 4 years in the E-Pirio station with temperature data from the meteorological station of Calvi (Lat: 42. 52; Long: 8. 79, 15. 9 km away from the population) and inferring the temperature in E-Pirio over the remaining years. 3 Capture-Mark-Recapture modelling Individual capture-recapture histories for the 1562 breeding individuals were analysed to provide robust estimates of survival and recapture probabilities (respectively φ and p,  Lebreton et al. , 1992) , using a logit-link function.
All analyses were conducted using the program E-SURGE  (Choquet et al. , 2009b). Goodness-of-fit of models to the data was ensured for each dataset using the program U-CARE  (Choquet et al. , 2009a) , based on the Cormack Jolly Seber model for monostate models and on the Jolly Move model for multistate models. In several years, experiments were conducted in the population including brood size manipulations that can highly alter adult survival  (Dijkstra et al. , 1990; Nur, 1984). Capturerecapture histories of the corresponding individuals were censored right after the first time they experienced such a fitness-changing experiment. However, nestlings born during these experiments and later recaptured as adults were not removed from the analysis. This necessary censoring resulted in a significant decrease in observation numbers (-23.
Data analysis and model selection The Akaike's Information Criterion corrected for small sample size (AICc) was used for model selection  (Burnham and Anderson, 1998). A low AICc was considered revealing a good compromise between the fit to the data (likelihood of the model) and the number of parameters used by the model. The threshold for significant difference between two models was set at two AICc points. In case of lower difference, the model with the lowest number of parameters was selected. The four main questions addressed are summarised in Table  1  and details are provided in the sections below. a Link between population-level phenology and average annual adult survival 7 Prior to assessing the effects of population-level covariates on survival, a preliminary model selection assessed the effects of year, sex and age on recapture (p) and survival (φ) probabilities with the monostate dataset of capture histories (Table  2A ).
The starting model (Model 22) included potential effects of sex, age and yearly variations (hereafter, year) on survival and potential effects of sex and year on recapture probabilities. Only simple (first order) interactions between variables influencing survival and recapture probabilities were considered, to enable robust biological interpretation of the results and avoid overcomplexity. All subsequent models were nested in this one. The model selection started with simplifying constraints on p. As a second step, we retained the best model structure for recapture and considered simpler survival models. We then used the best model to test the effects of our three demographic temporal covariates (medLD, popD, repS) and their interactions (Table  2B ) on survival. b Link between individual timing of breeding (relative laying date) and adult survival To assess the effect of intra-population variation in laying dates, a multistate dataset of capture histories for our population was created.
d Links between individual breeding success, relative laying date and adult survival 8 To assess the role of intra-population reproductive investment in the system, another multistate dataset of capture histories was used. The 6 states took into account combinations of the individual differences in reproduction in three categories (Failure: clutch abandoned or predated, 0 fledglings; Low reproduction: number of fledglings below the median number of fledglings per brood in the population this year; High reproduction: number of fledglings above this median) and individual differences in phenology in two categories (Early breeders: first quartile of laying date (state 1 in the previous dataset), Late breeders: second to fourth quartiles of laying date (states 2,3 and 4 in the previous dataset)).
Goodness-of-fit No significant violation of standard models assumptions was found while performing the goodness-of-fit tests using program U-CARE  (Choquet et al. , 2009a) : Detailed goodness-of-fit results are provided in Appendix 1. a Link between population-level phenology and average annual adult survival The best model from our preliminary selection on the single state capture histories (without covariates) is presented in When adding the three population-level annual demographic covariates (median laying date of the population (medLD), population density (popD) and mean number of fledglings per attempted brood in the population (repS)) as determinants of survival, we observed that adding each covariate separately to Model 1 improved the fit to the data (Table  2B , models  35, 36 and 37).
11 -3 -2. 5 -2 -1. 5 -1 -0. 5 0 0. 5 1 1. 5 2 0. 1 0. 3 0. 5 0. 7 0. 9 Standardized Median Laying Date (year t) Annual Survival Probability (year t to t+1)   2A ) for age 1 with 95% confidence intervals, the solid line is for age 1 from Model 36 (Table  2B ) which has a continuous effect of median laying date on survival, its 95% CI are represented by the dashed lines. B: Survival is negatively correlated with population density during the previous spring (ß=-0. 14; SE=0. 07). The points are the estimates of Model 3 (Table  2A ) for age 1 with 95% confidence intervals, the solid line is for age 1 from Model 35 (Table  2B ) which has a continuous effect of population density on survival, its 95% CI are represented by the dashed lines. The model selection on the state-separated laying date quartiles capture histories dataset did not show any model stand out in terms of AICc.
All individuals of our datasets were captured in the nest boxes during reproduction: they have all at least attempted reproduction and reached the nestling stage, i. e. successfully hatched at least one nestling. This means that individuals with failed reproduction (0 fledglings) encountered strong impediments during the breeding event (scarceness of 25 resources, harsh climatic events, predation of the brood, abandonment by one or the other parent, or death). A high proportion of these brood failures may be linked to the death of one or both parents: as it occurs after the recapture and identification of the parents (otherwise they would not be in the dataset), it will be analysed as death during the subsequent year and therefore we expected to observe low survival for adults with failed reproduction compared to others.
Acknowledgements We are grateful to  Emmanuelle Cam ,  Aurélie Cohas ,  Olivier Gimenez ,  François Chiron ,  François Sarrazin ,  Roger Pradel  and  Pierre-Yves Henry  for their constructive comments on an earlier version of this manuscript. We thank all the people who helped conduct the blue tit monitoring in the past four decades, in particular  Jacques Blondel ,  Phillippe Perret ,  Marcel Lambrechts ,  Claire Doutrelant ,  Denis Réale ,  Samuel Caro , and  Christophe de Franceschi  (who also managed the database). We thank the  APEEM , the  ONF and Achille Sanroma  for field logistics, and the  Fango valley MAB reserve. The long term project has been supported by the  OSU-OREME  since 2009.
Data acquisition and processing (TLS) The data of all study sites have been acquired with a terrestrial 3D long-range laser scanner. Two systems were used: the Riegl LMS-Z420i and the Riegl VZ-4000. Each scanner works on the same principle of time of flight, but due to laser configurations, the scanning distance of the VZ-4000 is 4 times longer with 4000 m compared to 1000 m. Both systems provide colour information due to integrated camera systems in order to colorize the point clouds. The RGB values of the pictures allow the filtering of vegetation in the pre-processing of the data and simplify the visualization of e. g. individual blocks. All important technical information of both devices is listed in Table  2. Due to the long distance between scanner and target and the poor reflectance of the volcanic material in the Dolomieu crater, we used the VZ-4000 for this study site.
All other test sites were surveyed using the LMS-Z420i. To minimize shadowing effects, several scan positions were necessary at each site, which had to be referenced using manual adjustment and ICP algorithms (Table  3 ). Both tools are implemented in the software RiSCAN PRO (v2. 2. 1, http://www. riegl. com, last access date: 8 February 2021). After the referencing, the data were exported as ASCII files containing x, y and z coordinates as well as RGB values for further analysis in SAGA GIS LIS  (Conrad et  Table  2. Technical data of the two terrestrial laser scanning systems Riegl LMS-Z420i and VZ-4000  (RIEGL Laser Measurement Systems GmbH, 2010 , 2020). The values of the VZ-4000 refer to measurements at a rate of 30 kHz.
Determination of block size, block shape and runout length As the block size and shape have a great influence on the runout distance of the falling rock material, we measured according to the workflow of  Haas et al. (2012)  the dimensions of the three axes (a, b and c) of single boulders for every study area (PF n = 255; GA n = 618; ZBT n = 65; DTS n = 182). We selected blocks that were deposited on the talus surface beginning from the upper and going to the lower parts Table  3. Information about the TLS surveys for each study area. of the talus cones. Based on the coloured lidar point clouds every block larger than approximately ∼ 0. 5 m was manually measured in the software RiSCAN PRO. Figure  2  shows an idealized sketch of a boulder with the three measured axes.
Surface roughness The roughness of the slope can serve as a proxy for the particle size distribution on the talus cone. This parameter could give more information regarding the gravitational sorting pattern where it indicates a coarsening towards the foot of the slope and/or a concentration of coarse material at the foot of the rock wall. This is to determine and analyse the sorting of particle sizes beyond the measured block sizes of the single debris cones. For this purpose, we calculated the roughness based on the high-resolution TLS point clouds. The algorithm used fits a plane for each point of the point cloud in its local neighbourhood with a search radius of 3 m (SAGA surface roughness module (PC); Laserdata Information System, LIS, 2010; https://www. laserdata. at, last access: 8 February 2021).