dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgments: We thank the people who helped with digitizing information on the distribution of native and alien earthworm species, particularly  E. Pedarros ,  S. Dandrifosse ,  C. Mathieu ,  J. Nabias , and T. Allain. We highly appreciate the comments and suggestions of  Isabelle Gounand ,  Gérard Lacroix  and  Olivier Moine , and the help of  W. Dawson  and  R. Early  regarding the use of their databases, as well as the help of Marvin Wright regarding the use of the package ranger. Finally, we thank the Animal Plant Health Inspection Service from the  United States Department of Agriculture  and the  Invasive Alien Species & Domestic Programs Section  from the  Canadian Food Inspection Agency  for their help regarding the import-export regulation laws. Funding was provided by the  France-Stanford Center for Interdisciplinary Studies.
Data availability Data are available online on the Zenodo datawarehouse. EWINA_RICH, the database of earthworm native and alien species richness in North America at the two spatial resolutions, together with the aboveground alien species richness in TDWG4 units, is available at https://doi. org/10. 5281/zenodo. 6421014. EWINA_IPATHS, the database of intercepted earthworm introduction pathways from 1945 to 1975 is available at https://doi. org/10. 5281/zenodo. 6408609. EWINA_SP, the tabular data of Ecological profile of earthworm species found in North America, including geographical origin, alien status, functional role and reproduction mode is available at https://doi. org/10. 5281/zenodo. 6462831. EWINA_1st_RECORDS, the list of first year of observation of each earthworm species in North America can be found at https://zenodo.
Code availability R scripts used for the analyses are available at https://github. com/JeromeMathieuEcology/GlobalWorming Extended Data Fig. 7  Temporal dynamics of the known geographical range of native and alien earthworm species across North America. CC-BY-NC-ND 4. 0 International license available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint this version posted June 29, 2022. ; https://doi. org/10. 1101/2022. 06. 27. 497722 doi: bioRxiv preprint Extended Data Fig. 8 Model performance. CC-BY-NC-ND 4. 0 International license available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The aim of this study was to describe the spatiotemporal patterns of rabies cases in C. jacchus reported by state health services to Brazil's Ministry of health passive surveillance system between 2008 and 2020, and to identify whether spatial expansions of rabies are occurring in this species. We also combined published and newly collected occurrence data of the common marmoset to estimate its current range and suitable habitat distribution across Brazil, including recent methodological improvements to ecological niche modeling to identify the environmental drivers influencing C. jacchus distribution.
Datasets (i) Reports of rabies infections in C. jacchus to the Brazilian Ministry of Health from 2008 to 2020. Dead C. jacchus suspected to have died from rabies were passively reported by state's health units to the Rabies Technical Group of the Ministry of Health's Zoonoses Surveillance Technical Unit. This unit received post mortem brain samples and performed laboratory confirmation using the fluorescent antibody test (FAT) by partnering with reference laboratories, and performed antigenic characterization to determine rabies variant typing on a subset (39%) of positive samples including at least one sample per outbreak  [4]. We analyzed data as either municipality-level outbreaks (defined as at least one laboratory-confirmed case of rabies submitted by a municipality within a year) or cases (defined as the total number of laboratoryconfirmed cases submitted by a municipality within a year).
To evaluate if temporal trends in rabies-positive C. jacchus were influenced by passive surveillance effort, we also included the total number of dead non-human primates reported to the Ministry of Health's Zoonoses Surveillance Technical Unit, which includes mainly primates tested for Yellow Fever and rabies. Although this data was not detailed by primate species, it can represent a proxy of passive surveillance effort at the national and state level. The dataset is available on S1 and S2 Tables of the Supplementary Material. (ii) C. jacchus occurrence data. A total of 178 GPS locations of C. jacchus were obtained from four different data sources: (i) by direct observation of the species during a field mission in three states in 2019 (number of occurrences N = 21), (ii) from published GPS occurrence data extracted from the ATLANTIC PRIMATES database (N = 117) recorded from 1987 to 2017  [18] , (iii) by requesting GPS locations of two wild C.
jacchus groups studied for more than a decade in the state of Rio Grande do Norte (MFA, pers. comm) (N = 2) recorded in 2019 and (iv) extracting GPS locations from recent published data available in  Moraes et al. 2019  (excluding locations already present in the ATLANTIC PRIMATES database) (N = 38) recorded from 1999 to 2017. GPS locations of C. jacchus were recorded in the field during 10-day field excursions in the states of Bahia (number of occurrences N = 8), Pernambuco (N = 10) and Rio Grande do Norte (N = 3) during January and February 2019. During these excursions, a team including at least two observers with a minimum of one experienced primatologist visually confirmed the presence of C. jacchus from a list of places that were previously suggested as habitat of C. jacchus by either the State Health Secretary that monitors this species or by primatologists working in each state.
Identification of C. jacchus was performed using species-specific characteristics including their distinctive white tufted-ears and vocalizations. Data on locations is available on S3 Table. Occurrence data did not include dead animals from passive surveillance, as GPS locations for these animals were not available. (iii) Data used in the Ecological niche modelling. GPS locations were first converted to an ESRI point shapefile with a defined projection of WGS 1984 and subsequently projected into WGS 1984 World Equidistant Conic projected coordinate system in ArcGIS. The presence of spatial autocorrelation in occurrence datasets is a known problem in niche modeling, whose use without rectification can result in biased estimates of distributions and generate model overfitting  [24]. Therefore, we rarefied the occurrence dataset used in our ecological niche model by randomly removing locations that were within a 10, 25 or 50 km radius using the SDM Toolbox v2.
jacchus can be influenced by bioclimatic variables  [23, 27]. Therefore, similarly to Moraes et al. 2019, we used the 30 arc second (*1 km) resolution CHELSA (Climatologies for the Earth's Land Surface Areas)  [28]  data to model the ecological niche of C. jacchus in Brazil. The choice of this spatial resolution was made with a consideration of the spatial resolution in which the common marmoset data was gathered, also at 1km. The CHELSA data consists of downscaled model output temperature and precipitation estimates at a horizontal resolution of 30 arc second. Downscaling algorithm for temperature is mainly based on statistical downscaling of atmospheric temperatures, and the algorithm for downscaling precipitation incorporates orographic predictors including wind fields, valley exposition, and boundary layer height, with a subsequent bias correction  [28].
The resulting data consist of a monthly temperature and precipitation data and various derived bioclimatic parameters. The 19 bioclimatic variables are specifically developed for species distribution modeling and for related ecological applications, with values for each grid or pixel representing different bioclimatic conditions of annual averages, seasonality, and extreme or limiting environmental factors for a given species. Detailed technical description of CHELSA data can be found in the technical document here: http://chelsa-climate. org/wp-admin/download-page/CHELSA_tech_ specification. pdf.
Spatiotemporal patterns in the distribution of C. jacchus rabies Spatial dynamics of rabies outbreaks/cases in C. jacchus were explored by estimating the annual number of outbreaks/cases from January 2008 to October 2020, the annual number of municipalities reporting outbreaks and the annual number of new municipalities reporting outbreaks for the first time. We also tested the correlation between year and the number of outbreaks/cases or municipalities reporting cases using a Spearman's correlation test in R 3. 6. 1  [29] , since the number of outbreaks/cases was not normally distributed. For this correlation analysis, we excluded the year 2020 given incomplete data. Municipality and state maps were obtained from the open access GADM database (gadm. org), using the getdata and readOGR functions of R.
Ecological niche modelling The present climatic suitability of C. jacchus in Brazil was determined with the rarefied, presence-only occurrence data and CHELSA bioclimatic layers following the maximum entropy modeling approach previously described  [30] [31] [32] , with Maxent 3. 4. 1 software  [32]. The relevance of different environmental layers was tested using the jackknife procedure in Maxent, allowing us to determine a smaller subset of variables needed to adequately describe the environmental niche for this species. First, we built a global model including 15 out of the 19 bioclimatic variables excluding variable 8 (mean temperature of the wettest quarter), 9 (mean temperature of the driest quarter), 18 (precipitation of the warmest quarter), and 19 (precipitation of coldest quarter) as they are known to have spatial artifacts that influence model outcomes  [33].
Drivers of occurrence and predicted habitat suitability for C. jacchus The occurrence dataset used for the ecological niche model included 34 duplicate records with the same geographic coordinates, which were removed. A 50 km distance between occurrences resulted in a visually desirable occurrence dataset that was minimally autocorrelated (S1 Text). Therefore, analyses were performed with this set of occurrences. Rarefaction of occurrences randomly removed 87 occurrences, resulting in 57 unique occurrence locations used in our niche modeling, randomly selecting half of locations (n = 29) for model calibrating and the other half (n = 28) for testing. Five bioclimatic variables were selected by the jackknife procedure as main drivers of the occurrence of C. jacchus including mean diurnal range (Bioclim 2), temperature annual range (Bioclim 7), the mean temperature of coldest quarter (Bioclim 11), precipitation seasonality (Bioclim 15), and precipitation of driest quarter (Bioclim 17).
In Novel Class Discovery (NCD), the goal is to find new classes in an unlabeled set given a labeled set of known but different classes. While NCD has recently gained attention from the community, no framework has yet been proposed for heterogeneous tabular data, despite being a very common representation of data. In this paper, we propose TabularNCD, a new method for discovering novel classes in tabular data. We show a way to extract knowledge from already known classes to guide the discovery process of novel classes in the context of tabular data which contains heterogeneous variables. A part of this process is done by a new method for defining pseudo labels, and we follow recent findings in Multi-Task Learning to optimize a joint objective function. Our method demonstrates that NCD is not only applicable to images but also to heterogeneous tabular data.
While audio and image data have been of great interest in recent scientific publications, tabular data remains a very common information structure that is found in many real world problems, such as the information systems of companies. In this paper, we will therefore focus on NCD for tabular data. Tabular data can come in large unlabeled quantities due to the labelling process being often costly and time-consuming, in part due to the difficulty of visualization, which is why a lot of unlabeled data is often left unexploited. In  [10] , the authors review the primary challenges that come with tabular data and identify three main obstacles to the success of deep neural networks on this type of data. The first is the quality of the data, that often includes missing values, extreme data (outliers), erroneous or inconsistent data and class imbalance.
Then, the lack of spatial correlation between features makes it difficult to use techniques based on inductive biases, such as convolutions or data augmentation  [11]. And finally, the heterogeneous and complex nature of the data (dense continuous and sparse categorical features) that can require considerable pre-processing, leading to information loss compared to the original data  [12]. Our proposal: To address the NCD problem in the challenging tabular data environment, we propose TabularNCD (for Tabular Novel Class Discovery). In the first step of our method, an unbiased latent representation is initialized by taking advantage of the advances in Self-Supervised Learning (SSL) for tabular data  [13]. Then, we build on the idea that the local neighborhood of an instance in the latent space is likely to belong to the same class, and a clustering of the unlabeled data is learned through similarity measures.
The process in this second step is jointly optimized with a classifier on the known classes to include the relevant features from the already discovered classes. Our key contributions are summarized as follows: • We propose TabularNCD, a new method for Novel Class Discovery. To the best of our knowledge, this is the first attempt at solving the NCD problem in the context of tabular data with heterogeneous features. Thus, we do not depend on the spatial inductive bias of features, which other NCD methods rely heavily on when using convolutions and specialized data augmentation techniques. • We empirically evaluate TabularNCD on seven varied public classification datasets. These experiments demonstrate the superior performance of our method over common fully unsupervised methods and a baseline that exploits known classes in a naive way.
II. NOVEL CLASS DISCOVERY AND RELATED WORKS The process of discovering new classes depends on the final application in the real world (offline vs online learning, nature of the data, etc). In the literature of "concept drift"  [14] , changes in the distribution of known classes can appear either gradually or suddenly. Some analogies can be made with the present paper, where we consider the case of the "sudden drift", as the labeled dataset D l and unlabeled dataset D u do not share any classes (as described in Sec. I). Furthermore, we assume that a form of knowledge can be extracted from D l to be then used on D u. In this sense, "Novel Class Discovery (NCD)" methods mainly lie at the intersection of several other lines of research that we briefly review here. Transfer Learning  [15] : To solve a problem faster or with better generalization, transfer learning leverages knowledge from a different (but related) problem.
This is the first major difference with Novel Class Discovery, as the ultimate goal of NCD is to explore the novel samples. The second difference is that in NCD, the known and unknown classes are already split, which is not the case in ND. Novel Class Discovery (NCD): In a large part of the literature related to NCD, the goal is to cluster classes in an unlabeled set when a labeled set of known classes is available. Unlike in semi-supervised learning, the classes of these two sets are disjoint. The labeled data is used to reduce the ambiguity that comes with the many potentially valid clustering criteria of the unlabeled data. It is a challenging problem, as the patterns learned from the labeled data may not be relevant or sufficient to cluster the unlabeled data. In  [21] , the authors explore the assumptions behind NCD and give a formal definition of NCD.
2 ). Despite the fact that for the tasks of classification or regression on tabular data, neither deep nor shallow neural networks have caught up with the performance of classical methods such as XGBoost  [23] , LightGBM  [24]  or CatBoost  [25] , we believe they are still relevant in our context. Neural networks on tabular data suffer from sparse values, high dimensionality and heterogeneous features. The authors of  [10]  insist that encoders are a good solution to these challenges, as data is projected in a homogeneous latent space of reduced dimensionality. Furthermore, in a well-defined latent space, instances close to each other are likely to belong to the same class. This means that data augmentation and similarity measures relying on this inductive bias can be used with more confidence after projection.
B. Joint training on the labeled and unlabeled data We formulate the novel class discovery process as multitask optimization problem. In this step, two new networks are added on top of the previously initialized encoder, each solving different tasks on different data (see Fig. 2 ). The first is a classification network η l (z) ∈ R C l +1 trained to predict 1) the C l known classes from D l with the ground-truth labels and 2) a single class formed of the aggregation of the unlabeled data. The second is another classification network trained to predict the C u novel classes from D u. It will be referred as the clustering network η u (z) ∈ R C u. These two networks share the same embedding and both update it through backpropagation, sharing knowledge with one another. The classification network is optimized with the cross-entropy loss using the ground-truth one-hot labels y: l class.
= - C l +1 c=1 y c log(η l c (z)) (4) where z = ϕ(x). Its role is to guide the representation to include the features of the known classes that are relevant for the supervised classification task. To train the clustering network η u with unlabeled data in a supervised manner, pseudo labels ỹi,j ∈ {0, 1} are generated for each pair (x i , x j ) of unlabeled data in a mini-batch. This is comparable to a pretext task in Self-Supervised Learning. Another possible approach would be to predict the similarity of instances directly, but using pseudo-labels has the advantage of directly associating classes to the instances and does not require an additional clustering of the embedding. In this case, pseudo labels indicate if a pair of instances should belong to the same class or not, independently of the class number predicted by the network.
C. Consistency regularization During the training of the two classification networks, the latent representation of the encoder changes. And since the pseudo labels are defined according to the similarity of 1 Experimentally, see the supplementary material in https://github. com/ ColinTr/TabularNCD instances in the latent space, this causes a moving target phenomenon, where the pairwise pseudo labels can differ from one epoch to another. To limit the impact of this problem, a regularization term is needed. Commonly employed in semisupervised learning  [31] , we use "data augmentation". The idea is to encourage the model to predict the same class for an instance x and its augmented counterpart x. But while data augmentation has become a standard in computer vision, the same cannot be said for tabular data.
The authors of  [10]  consider the lack of research in tabular data augmentation (along with the difficulty to capture the dependency structure of the data) to be one of the main reasons for the limited success of neural networks in tabular data. Nevertheless, we find that consistency regularization is an essential component for the performance of the method proposed here. In this paper we use SMOTE-NC (for Synthetic Minority Oversampling Technique  [32] ), which is one of the most widely used tabular data augmentation techniques. It synthesizes new samples from the minority classes to reduce imbalance, thus improving the predictive performance of models on these classes. Synthetic samples are generated by taking a point along the line between the considered sample and a random instance among the k closest of the same class (in the original input space).
A. Datasets and experimental details Datasets. To evaluate the performances of the method proposed here, a variety of datasets in terms of application domains and characteristics have been chosen (see Table  I ). Six public tabular classification datasets were selected: Forest Cover Type  [35] , Letter Recognition  [36] , Human Activity Recognition  [37] , Satimage  [38] , Pen-Based Handwritten Digits Recognition  [38]  and 1990 US Census Data  [38] , as well as MNIST  [39] , which was flattened to transform the 28 × 28 grayscale images into vectors of 784 attributes. We pre-process Following the same procedure found in Novel Class Discovery articles  [6] ,  [9] ,  [26] , we hide the labels of some classes to create the unknown classes. And the capacity of the compared methods to recover these classes is then evaluated.
Around 50% of the classes are hidden, resulting in further splitting of the training and testing sets into labeled and unlabeled subsets. The partitions of the 7 datasets can be found in Table  I. Evaluation metrics. To thoroughly assess the performance of the methods compared in the experiments described below, we report four different metrics. The first two are the clustering accuracy (ACC) and balanced accuracy (BACC), which can be computed after the optimal linear assignment of the class labels is solved with the Hungarian algorithm  [40]. The use of the balanced accuracy is justified by the unbalanced class distribution of some datasets, which resulted in inflated accuracy scores that were not truly representative of the performance of the method. Another reported metric is the Normalized Mutual Information (NMI).
It measures the correspondence between two sets of label assignments and is invariant to permutations. Finally, we compute the Adjusted Rand Index (ARI). It measures the similarity between two clustering assignments. These four metrics range between 0 and 1, with values closer to 1 indicating a better agreement to the ground truth labels. The ARI is an exception and can yield values in [-1, +1], with negative values when the index is less than the Expected RI. The metrics reported in the next section are all computed on the unlabeled instances from the test sets. Competing methods. As expressed in Section I, to the best of our knowledge, there is no other method that solves the particular Novel Class Discovery problem for tabular datasets. Nonetheless, we can compare ourselves to unsupervised clustering methods.
Compared to the unsupervised approaches, this technique has the advantage of learning the patterns of the known classes in a homogeneous latent space. However, we still expect it to perform sub-optimally as the unique features and patterns of the unlabeled data might be lost in the last hidden layer after training. Implementation details. For all datasets, we employ an encoder composed of 2 dense hidden layers that keeps the dimension of the input with activation functions and dropout values that are optimized hyper-parameters. Following VIME  [13]  and AutoNovel  [6] , we use a single linear layer for the mask estimator, vector estimator, classification network and clustering network. The hyper-parameters (see Table  VI  in appendix) of the proposed method are optimized with a Bayesian search on a validation set which represents 20% of the training set.