dataesr/scientific-paragraphs-categorization huggingface dataset:
The trees were assembled in R using the phytools package  (Revell, 2012)  and TreePar  (Stadler, 2011). We set the divergence between mammals and reptiles at 330 Myr based on  Didier and Laurin (2020). For later divergence times, we used the online resource TimeTree  (Kumar et al. , 2017). We set the divergence between Lepidosauria and other reptiles at 281 Myr. We considered turtles as the sister taxon to Archosauria (birds and crocodylians), as suggested by recent molecular studies  (Chiari et al. , 2012; Irisarri et al. , 2017) , and set the age of the turtle+archosaur clade (Archelosauria) at 261 Myr. We set the divergence between Astrochelys radiata and Chelonoidis Note: For each taxon, the reported body mass value (in grams) is the median  (Myhrvold et al. , 2015). Abbreviations: Aq, semi-aquatic; Ar, arboreal; BS/TV, the bone surface area per unit volume; BV/TV, the bone volume fraction; C, crouched; Conn.
| Phylogenetic signal We tested the phylogenetic signal in femoral posture. This was done using the delta statistic  (Borges et al. , 2019) , which is based on the uncertainty associated with ancestral state reconstruction. The delta statistic is proportional to the phylogenetic signal and inversely proportional to the uncertainty at the nodes. We tested the phylogenetic signal in femoral posture with our 100 trees and calculated a p-value each time based on 10 random permutations of femoral posture at the tips of the tree branches. We also searched for a phylogenetic signal in the aforementioned microanatomical parameters using the phylosig function of the R package phytools  (Revell, 2012) , which computes the K-statistic  (Blomberg et al. , 2003). A strong phylogenetic signal, implying that closely related species are more similar to each other than would be expected under a Brownian model of evolution, is indicated by a K-statistic greater than 1.
| Phylogenetic generalized least squares All the microanatomical parameters presented above have been previously associated with femoral posture, body mass or lifestyle in several amniote clades  (Bishop et al. , 2018; Doube et al. , 2011; Mielke et al. , 2018; Plasse et al. , 2019). As mentioned above, the taxa in our sample show a diversity of femoral postures, as well as a variety of lifestyles (semi-aquatic, arboreal, fossorial and terrestrial; see Table  1 ), but also differ greatly in terms of body mass (from 2. 33 g in Suncus etruscus to 3. 22 t in Elaphas maximus). To explore these relationships with our sample, we designed several linear models in R using the gls function from the package nlme  (Pinheiro et al. , 2021). The function fits a linear model using generalized least squares (GLS).
Compositional data are a special kind of data, represented as a proportion carrying relative information. Although this type of data is widely spread, no solution exists to deal with the cases where the classes are not well balanced. After describing compositional data imbalance, this paper proposes an adaptation of the original Synthetic Minority Oversampling TEchnique (SMOTE) to deal with compositional data imbalance. The new approach, called SMOTE for Compositional Data (SMOTE-CD), generates synthetic examples by computing a linear combination of selected existing data points, using compositional data operations. The performance of the SMOTE-CD is tested with three different regressors (Gradient Boosting tree, Neural Networks, Dirichlet regressor) applied to two real datasets and to synthetic generated data, and the performance is evaluated using accuracy, cross-entropy, F1score, R2 score and RMSE.
Context Over the past few years, data imbalance problems have been widely studied in classification tasks  [1]. An imbalance distribution over the classes will often cause the models to prioritize their performance on the majority classes, at the expense of the minority ones. Different methods exist to deal with imbalanced datasets  [2] : algorithm-level methods, where the algorithm reduces the bias by inducing a weight on the classes; data-level methods, where the data are modified to reach a more balanced state; and hybrid methods, combining both algorithm-level methods and data-level methods. Among data-level methods, Synthetic Minority Oversampling TEchnique (SMOTE)  [3] , with all its variations  [4] , is one of the most popular for classification problems. The SMOTE algorithm generates synthetic data points for a particular class by combining the features of two existing points belonging to the same class through linear interpolation.
Most algorithms designed to tackle class imbalance problems, such as SMOTE, are often limited to the classification tasks; for instance  [5] [6] [7] [8] [9]. However, even though regression problems are also very common in real-life problems, only a few resampling strategies exist for regression tasks  [10, 11]. In this paper, we address the special issue of dealing with an imbalanced dataset in regression problems in the case where the labels are compositional. Compositional data are data carrying relative information  [12] , presented as proportions or percentages, making them different from other types of data. Compositional data are encountered in various fields, including biology  [13] [14] [15] , chemistry  [16, 17] , ecology  [18, 19] , geology  [20, 21] , and social sciences  [22] [23] [24] , among others.
However, the class imbalance problem in compositional data regression remains a major challenge in the development of effective models. Existing adaptations of SMOTE and other oversampling techniques have focused on addressing imbalanced datasets in single-label regression  [25] [26] [27] [28] , multi-label classification  [29, 30] , or when the features are compositional data  [31]. However, to the best of our knowledge, no oversampling technique exists for addressing the issue of class imbalance in multi-label regression problems with compositional labels. Therefore, we propose a new oversampling technique called SMOTE for Compositional Data (SMOTE-CD), specifically designed to address this particular situation. Here, we will measure class imbalance by summing the values of the labels (probability values) for each class on the whole dataset, and summarizing it as a percentage.
In that sense, in a perfectly balanced dataset, the percentage of the sum of each class would be 1/K, with K being the number of classes. The proposed method is evaluated using five different performance metrics, including accuracy, cross-entropy, F1-score, R2 score, and RMSE, to three different models (Gradient Boosting tree, Neural Networks, Dirichlet regressor) on both simulated and real datasets. Since no other oversampling algorithm currently exists for compositional data, the evaluation of SMO-TE-CD is limited to comparing its performance against the case where no oversampling technique is applied. The results show that the performance of the models is overall greater when applying SMOTE-CD, thus demonstrating the effectiveness of the proposed method. This is an important contribution to the field, as it provides a solution for dealing with compositional data imbalance, which has not been addressed before.
The use of five different evaluation metrics, as well as the application of three different models to both simulated and real datasets, further strengthens the reliability and generalizability of the proposed method. The entire paper is arranged as follows. The paper's first section introduces the proposed method and the motivation example. Section 2 presents the compositional data and the SMO-TE-CD algorithm. Section 3 presents the metrics, the simulation study and its results. Sections 4 and 5 present the result on the real datasets. Section 6 presents the discussion and conclusion.
Motivation example: Maupiti island Description of Maupiti island. The overall purpose of our research project is to develop an automated mapping tool able to provide a classification map from a given satellite image, with a particular focus on a coral reef-lagoon system. The test field site is the Maupiti island, the westernmost Leeward island of the Society archipelago, French Polynesia. The site has a size of approximately 8km by 8km. Maupiti data, that we use here, is just an example, but compositional data can be found, for instance, in health or chemistry fields. An expert-based mapping of Maupiti island was used as a training dataset to develop the model. The satellite image used is a 4-band image captured on June, 14 2021 by the Pleiades satellite. The expert-based mapping of the image relies on the combination of several field observation campaigns  [32]  and direct examination of the satellite image.
The present analysis focuses on the shallow regions of the lagoon, displaying more interpretable imaging. In the selected areas, four seabed type classes were established (Fig 1a  ):  • Class 1: Coral, marked by a overwhelming dominance of coral reef cover. • Class 2: Sand, describing areas covered by detritic sand. • Class 3: Shorereef, gathering shore reef and transitional shore reef. • Class 4: Mixed, representing area covered by a combination of sand and coral. Automatic mapping. To perform the automatic mapping, the image was first segmented using Felzenszwalb's method  [33] , which gives Fig 1. For each segment, two different operations were applied: • The four statistical moments (mean, variance, skewness, kurtosis) were computed on each band; these 16 values will be the features of the dataset.
• The percentage of pixels belonging to each class were computed, according to the expertbased classification; this results in a vector that sums up to 1 that will be the labels of the dataset. To be able to map the satellite image, the idea was to train a regressor to retrieve, for each segment, the percentage of pixels belonging to each class (i. e. , a vector of probabilities). As shown in Table  1 , the data are not balanced: one of the class represents 49. 5% of the dataset, while another one represents only 3. 6%. To overcome this issue, we developed an oversampling technique in order to improve the performance of the regression model on this special kind of data.
Compositional data Mathematically, we define a D-part compositional dataset as a vector x ¼ ðx 1 ; x 2 ;. ; x D Þ 2 R D such that, x i � 0; 8 i 2 f1; 2;. ; Dg; X D i¼1 x i ¼ 1: 8 > > < > > : A simplex S D is defined as the ensemble of all the D-part compositional data, i. e.
SMOTE for compositional data In this section, we denote by n the number of samples in the dataset, p the number of features and K the number of classes. The matrix X 2 R n�p contains the n observations of the p features and Y 2 R n�K contains their labels. For any i 2 {1,. , n} and j 2 {1,. , K}, we denote by y i,j the value of Y at row i and column j, and y i,� = (y i,1 ,. , y i,K ) the probability vector label of row i. Similarly, for i 2 {1,. , n} and j 2 {1,. , p}, x i,j is the value of X at row i and column j, and x i,� = (x i,1 ,. , x i,p ). In order to simplify the notation, we define argmaxðy i;� Þ ¼ argmax j2f1;. ;Kg ðy i;j Þ; which represents the majority class of a given label y i,� 2 [0, 1] K. We also define the sum vector S 2 R K as the sum of the values for each class, S ¼ X n i¼1 y i;1 ; X n i¼1 y i;2 ;.
; X n i¼1 y i;K ! : ð3Þ The majority class of the dataset is thus defined as argmax(S), and the minority class as argmin (S). Before introducing the SMOTE-CD algorithm, let's first summarize the idea behind the original SMOTE algorithm. As shown in Fig  2(a) , the SMOTE algorithm creates a new point that belongs to class 1 (represented by blue points). To achieve this, the algorithm first selects a point at random (in this case, p 1 ) and identifies its nearest neighbors (p 2 , p 3 , p 4 ). Note that only neighbors with the same label as p 1 (i. e. , class 1) are considered, while points labeled as class 2 (represented by red points) are ignored. The algorithm then chooses one of these neighbors (p 4 ) and creates a new point along the line that connects p 1 and p 4. The features of the new point are determined through a linear combination of the features of p 1 and p 4 , and its label is assigned as 1.
Algorithm 1 describes the SMOTE algorithm. Algorithm 1 Original SMOTE  [3]  Require: X 2 R n�p the features. Require: Y 2 {1,. , J} n the class label outputs. Require: k 2 N the number of neighbors to select for the k-Nearest Neighbors. Ensure: Generated data X new 2 R q�p and Y new 2 {1,. , J} q with q the number of points created. 1: Denote by S j the number of points labeled as class j. 2: M the majority class of dataset. 3: Initialize X new and Y new as empty matrices. 4: for every class m that needs to be oversampled do 5: while S m < S M do 6: Compute D ¼ fi j y i ¼ m}, the set of points labeled as class m.
8: Randomly choose an index r 2 among these neighbors. 9: The SMOTE-CD algorithm keeps the main ideas from the original SMOTE: 1) select a point from the class to be oversampled, 2) select one of its k-Nearest Neighbors (k 2 N specified by the user) and 3) create a synthetic point in-between those two points. Because of the label that is compositional, these three steps have to be adapted: x new w � x r 1;� þ ð1 À wÞ � x r 2;� with w 2 [0, 1. Select a point r 1 whose majority class is m, where m is the minority class of the dataset.
Compute the k-Nearest Neighbors of r 1 among the points that also have m as their majority class. Then select a point r 2 in one of these k neighbors. 3. Randomly draw w 2 [0, 1]. The features of the point to be created is a linear combination of the two points selected before, with w being the weight of r 2 and (1 -w) the weight of r 1. Similarly, the labels of the point to be created is a linear combination, but using the operators from Eqs (1) and (2). As we are dealing with compositional data label, every point p i has a vector label y i. All the blue points are the points having the class m as the majority class of their label y i , where m is the minority class of the dataset. The algorithm computes the 3 nearest neighbors of p 1 only considering the blue points, and then a point is created on the line between p 1 and p 4.
The proof of convergence holds in the fact that, at each iteration, the increase of the major class of S is smaller that the increase of its minor one, causing the sum of the minor class to converge to the sum of the major one. In other words, we have to be assured that, at each iteration, y new m > y new M , with m (resp. M) the minority (resp. majority) class of the dataset. This is straightforward by noticing that the two indices r 1 and r 2 used for generating a new point are chosen in D ¼ fi j argmax(y i,� ) = m}: r 1 ; r 2 2 D ) ( y r 1 ;m > y r 1 ;M y r 2 ;m > y r 2 ;M ) ( w � y r 1 ;m > w � y r 1 ;M ð1 À wÞ � y r 2 ;m > ð1 À wÞ � y r 2 ;M ) w � y r 1 ;m þ ð1 À wÞ � y r 2 ;m > w � y r 1 ;M þ ð1 À wÞ � y r 2 ;M ) y new m > y new M : Simulation study.
; x p Þ 2 R p , we define x 0 ¼ ð1; x 1 ;. ; x p Þ 2 R pþ1 and a vector α as: a ¼ softmaxðB 0;1 þ B 1;1 x 1 þ � � � þ B p;1 x p ;. ; B 0;K þ B 1;K x 1 þ � � � þ B p;K x p Þ ¼ softmaxðx 0 � B �;1 ;. ; x 0 � B �;K Þ: We are then able to randomly draw a label for x with a Dirichlet distribution with parameter α. Algorithm 3 generates a random dataset using this method. To better understand how the regression coefficients B can change the configuration of the data, we give an example of simulated data with 2 features and 2 labels. Two different values B (a)  Each column of a matrix B represents the coefficients for one class. There are 3 lines here because there are 2 features and the first value corresponds to the intercept of the regression. In B (a) , the coefficients of each class are purposely close to each other, while they are easily separable in B (b).
Results First, to investigate the effect of the oversampling technique, synthetic data were generated with 2 features and 2 classes. To make the dataset imbalanced, 90% of the points that had class 0 as a majority class were deleted. We obtain a dataset in which 93% of the points have To evaluate the performance of SMOTE-CD, a 5-fold cross validation was used for three models: Gradient Boosting tree (GB), Neural Network (NN) with one hidden layer, and Dirichlet regression model  [36]. The first and second models are chosen because Random Forest and NN are known to be the most efficient to map coral reefs from multispectral satellites  [37, 38]  and because NN are used in literature for the task of predicting compositional labels  [39, 40] , and the third is chosen because it is used to generate the simulated data.
0006). With the compositional distance on the Neural Network, only the F1-score significantly increases (p � 10 -10 ), while accuracy, RMSE, and R 2 decrease. The GB model shows significant improvement for cross-entropy, F1-score, and RMSE (p < 0. 008), but a decrease in accuracy. The Dirichlet model with oversampling significantly increases accuracy and F1-score (p � 10 -10 ) but decreases cross-entropy, RMSE, and R 2. In order to understand the effects of the imbalance of the dataset on the performance of the oversampling method, three metrics (accuracy, F1 and R 2 ) were evaluated with different imbalance ratios. First, a matrix B was created to generate a balanced dataset with 16 features and 4 classes. Then, the ratio of class 0 was increased by incrementing the value of B 1,1. At each step (for a total of ten steps), the following operation was repeated 100 times: 550 points were created to train the models on the raw or oversampled data, and the models were tested on a set of 11000 points.
The result appears in Fig  5. It is apparent that the efficiency of SMOTE-CD depends on the data and the model used. The oversampling technique only improves the R 2 score when the dataset is slightly imbalanced (largest class representing less than 40%), but performs poorly when it is highly imbalanced. On the other hand, the more the dataset is imbalanced, the more the oversampling technique will improve the F1-score. The improvement in accuracy peaks at a certain value of imbalance (when the largest class represents 50% of the dataset), but drops above that threshold. In order to explain the low R 2 score for the oversampled data, the R 2 per class was calculated for each of the ten steps mentioned above and then averaged. For the largest class, the R 2 score is decreased from 0. 5 to 0.
Application to Maupiti data The performance of the three models on the raw dataset was compared with the oversampled dataset (with either the logratio distance used to create the new labels, or the compositional   3. With the Maupiti dataset, the NN is defined with 2 hidden layers of size 80 and 40, and the relu activation function. With the GB model, all the metrics are significantly improved (p < 0. 03) when using the oversampling technique, excepted for the cross-entropy for which the differences are not statistically significant (p = 0. 14 and p = 0. 38 respectively for the compositional and the logratio distance). The SMOTE-CD shows less results with the NN and Dirichlet model, where only the difference on the F1 is statistically significant (respectively p < 0. 044) and p < 10 -10 ).
Application to Tecator dataset To fully evaluate the effectiveness of the SMOTE-CD technique, we applied it to the Tecator meat sample dataset  [41] , which consists of 240 meat samples. Each sample has absorbance values measured at 100 different wavelengths, as well as corresponding information on the composition of moisture (water), fat, and protein contents. The objective of this analysis is to predict a 3-class compositional data vector from a feature vector of size 100. Because the Dirichlet regression model can be very slow when dealing with a high number of features, we opted to improve its speed by using only the 22 principal components provided in the dataset instead of the 100 features. To account for the small size of the dataset, a 10-fold cross validation is applied for each model, iterated over 100 times to vary the folds.
The results are displayed in Table  4. The neural network is configured with three hidden layers, each having 70 neurons and using the hyperbolic tangent (tanh) activation function, which were selected through hyperparameter tuning. With the NN, the raw data gives slightly better performances than the oversampled data. However, given the really poor performances of the NN (a negative R 2 and a really high RMSE), we also note that this model was probably not suited for this dataset. The analysis of the GB and Dirichlet models reveals interesting differences. In both cases, using either the raw or oversampled datasets leads to statistically significant differences (p < 10 -4 ). Specifically, for the GB model, using the oversampled data results in better performance, while for the Dirichlet model, oversampling decreases the performance.
Notably, among all the models tested, the GB model trained on oversampled data with compositional distance yields the best results. Compared to the Dirichlet model trained on raw data, this approach achieves significantly better accuracy (p < 0. 006), RMSE (p � 10 -10 ), and R 2 (p < 10 -4 ), with only a slight difference of 1% in cross-entropy and F1-score. In light of these results, it is apparent that SMOTE-CD can improve the performance for a model that does not perform too poorly (e. g. a R 2 above 0. 3). Indeed, if a model has low performance, it is more likely that this is due to poor fit to the data than from the imbalance of the dataset.
Discussion The results on the synthetic datasets show that the SMOTE-CD technique can significantly improve the F1-score and accuracy, but it has a mixed effect on other metrics depending on the model and dataset imbalance level. SMOTE-CD improves the overall performance of the model, especially with respect to the accuracy and the F1-score in the cases where the dataset is not too heavily imbalanced. The R 2 score of the majority class remains similar, but the R 2 of a very small class (3% of the dataset) will be decreased. The R 2 of all the other classes is improved, which is the desired goal of the method. The results on the real datasets show that the SMOTE-CD technique can significantly improve the performance of the Gradient Boosting model for all metrics, while it has a less pronounced effect on the other models.
The per-class analysis of the R 2 score reveals that the SMOTE-CD technique can improve the performance for some classes but not for others, depending on the model and distance metric used. Further tests are required with other datasets having compositional labels, but these are often hard to find because they are not publicly available. Our oversampling technique could be used with datasets in biology and metabolomics, in poll studies or in soil analysis, but its effectiveness depends on several factors that should be carefully considered. The original SMOTE paper  [3]  proposes to undersample the dataset before applying the oversampling technique, which we similarly tested here. The synthetic dataset was first undersampled by randomly withdrawing some points from the majority class, until the total sum of the largest class was equal to the sum of the second largest one.
SMOTE-CD was then applied. The results are summarised in S3 Table and compared with those in Table  2  when not using undersampling (S4 Table ). No significant difference can be seen when using undersampling before the oversampling, be it positive or negative. The results are similar when undersampling not only the points having the largest class as their majority class, but the points having one of the n largest classes as their majority class (with n 2 [1,. , 3]). At this point, we are not able to exclude the utility of the undersampling and suggest it could once more depend on the dataset or on the way the removed points are chosen. For instance, when performing random undersampling, consideration could be given to an Edited Nearest Neighbor approach  [42] ; see  [43]. Work has still to be done regarding the initial selection of the points, because it can influence the performance of the original SMOTE algorithm.