dataesr/scientific-paragraphs-categorization huggingface dataset:
Therefore, we compared the solution time and effectiveness of formulations (  6 ), (  7 ), (  8 ) and  (9). We took 447 simplices from a branch and bound process over the functions Hartman 3, 4 and 6 that, as the name suggests, have dimension 3, 4 and 6. We have used the routines linprog and intlinprog in Matlab setting IntegerTolerance to 1e-6 and ConstraintTolerance to 1e-8. The result is given in Table  1. In each line of the table we give for the LP and MIP formulations the percentage of the effectiveness measured as proven monotonicity. For instance, for problem Hartman 3, LP (6) proved in 81. 6% that there is a monotone directional derivative. We can prove there is a monotone decreasing direction from any facet by solving LP  (7)  for all vertices, or by solving any of the MIP-s once.
Structure of the paper In the remainder of this paper, we present a systematic review of the literature on processing techniques for predicting ruminant behaviour from raw accelerometer data. In the processing of raw accelerometer data to predict animal behaviour, most researchers use a methodology involving the following three stages, as illustrated in Fig. 2:  1) Data Collection: Collection of raw accelerometer data from wearable sensors and behavioural observations in the field. 2) Data Pre-processing: Pre-processing of accelerometer data to get a suitable dataset.
Data collection: Animal, sensor and observational options The purpose of this first step is to collect the data needed to develop a classification model in the final stage of the methodology (Fig. 2 ), that is: -Raw accelerometer data, collected at regular time-intervals (also called sampling rate; units: Hertz) using one or more 2-Dimensional or 3-Dimensional accelerometer sensors attached to animals, positioned on varied body-parts. Fig. 1. PRISMA flow diagram for the systematic review of the literature on the prediction of ruminant behaviours using raw accelerometer data. The keywords associated with each species are provided in Table  1. The criteria mentioned in the diagram are those introduced above. -Observations of the equipped animals expressing different behaviours. The behaviours observed and the associated time are recorded.
Accelerometer sensor Of the 66 studies using a wearable accelerometer to predict behaviours, 97% used 3D-accelerometer sensors while the remaining used 2Daccelerometers (e. g. Nadimi et al. , 2008). Accelerometer data were collected either using a study-specific device including a datalogger (e. g. Axivity AX3;  Fogarty et al. , 2020) , a smartphone (e. g. iPhone 4S;  Andriamandroso et al. , 2017)  or a commercial system for ruminants in which raw data is available (e. g. HOBO® loggers;  Benaissa et al. , 2018). It should be noted that additional sensors were sometimes attached to animals and used for behaviour prediction in addition to accelerometer data. The added sensors were most often a 3D-gyroscope  (Walton et al. , 2018) , a 3D-magnetometer  (Dutta et al. , 2015)  or a GPS  (González et al.
Development of a behavioural classification model: Splitting, modelling and validation options In this step, a classification model to predict ruminant behaviours (outputs) from accelerometer features (inputs) previously extracted is trained and validated. The 66 papers are reviewed by addressing the three sub-steps applied in model development (Fig. 2 ), that is, (i) Dataset Splitting, (ii) Model Training and (iii) Model Validation. The approach adopted in the studies varies according to three main sets of choices regarding (i) the technique applied to validate the model and the criterion used to split the dataset, (ii) the classifier used and (iii) the way in which the performance of the model is reported. Fig. 5  summarises the options adopted in this literature. Appendix 4 presents detailed profiles of the 30 core papers (see Supplementary Materials A3 for the details of the remaining 36 papers).
Dataset splitting: Validation techniques and options to split the dataset Among the 66 reviewed papers, 42% split the time-windows dataset into a training-set and a test-set (e. g. Martiskainen et al. , 2009; Fig. 5a ). This step involves using a percentage of the dataset to train the model (training-set) and the remaining percentage to assess the performance of the model independently (test-set). Cross-Validation (CV) and Leave-One-Out validation (LOOV) are used in 45% of the studies (e. g. Mansbridge et al. , 2018; Fig. 5a ). Using CV, the dataset is split into K folds, with K-1 folds used to train the model, and the K th used to test it. The LOOV is the specific case of the CV where there are as many folds as there are observations in the dataset. The procedure is repeated K times, so that all folds have been used to test the model.
Finally, 13% used the whole dataset both for training and validation (e. g. Busch et al. , 2017; Fig. 5a ). Recommendations on the different techniques used and their impact on the reliability of the model validation are discussed in Section 6. When the dataset is split into several portions in the surveyed papers, either into a training data-set and test data-set, or into a cross-validation procedure, one of the following three criteria is usually chosen to split the dataset (Supplementary Data B4): • Random splits: The split is done in a completely random way; that is, the partitioning of the time-windows into training and test sets is done regardless of the animals and periods from which the time-windows came (e. g. Dutta et al. , 2015). This option is used in the 63% of studies (Fig. 5a ).
• Time-based splits: The split is done regardless of the animals from which the time-window came, but the period when the animals are observed is taken into account (e. g. Riaboff et al. , 2020). Validation is therefore carried out by considering the time-period associated to the time-window. This option is used in 5% of the studies (Fig. 5a ). • Animal-based splits: The split is drawn from a different sample of animals than those used to train the model (e. g. Tamura et al. , 2019). This option has been used in 32% of the studies (Fig. 5a ). Recommendations on the criterion to be used for splitting the dataset and its impact on the reliability of the model validation, are also discussed in Section 6.
Model training: Options for the classifiers The objective of this step is to train and validate a classification model that best discriminate the different behaviours (Fig. 2 ). Of the 66 reviewed papers, six main categories of models can be identified: • Supervised Machine Learning (SML): This category of classifiers has been used in 56% of the studies (Fig. 5b ). It mainly includes Linear Discriminant Analysis (LDA;  Giovanetti et al. , 2017)  or Quadratic Discriminant Analysis (QDA;  Barwick et al. , 2018) , Support Vector Machines (SVM;  Martiskainen et al. , 2009) , k-Nearest Neighbour (k-NN;  Sakai et al. , 2019) , Naïve Bayes models (NB;  Benaissa et al. , 2017)  and Decision Trees (DT;  Robert et al. , 2009). The best hyperparameters for each model are usually found using Grid Search and a validation dataset (e.
Model validation: Options to evaluate model performance The validation step aims to assess the accuracy and robustness of the developed model (Fig. 2 ). This evaluation includes the three distinct sub-steps that were typically found across the 66 articles: • Predictive testing: Behaviours are predicted for each time-window from the developed model, using the test-data-set (42% of the studies), CV technique (45% of the studies) or the whole dataset (13% of the studies). • Confusion matrix calculating: The predicted behaviours of each timewindow are compared to the actual observed behaviours (references obtained from the observations) as a confusion matrix. When more than 2 behaviours are predicted, a confusion matrix per behaviour is usually used to calculate true positives, false positives, true negatives and false negatives associated to each behaviour.
Experimental performance may not extend to field applications Although current models report impressive performance, this performance is very much experimental-based and may not extend to commercial deployments. Recall that an original dataset can be split in different ways for model validation purposes (Sections 4. 1). In this respect, the performance obtained with the same model seems to be greatly affected by the split used. Several studies report a substantial decrease in performance when validation is done with animals other than those used to train the model, that is, using an animal-based split, compared to a random or time-based splits. For instance,  Rahman et al. (2018)  reported a 0. 4 decrease in F-score (scale 0-1) using the same original dataset when it was split by animal in a Leave-One-Out-Animal validation technique, compared to the random split using a k-fold Cross-Validation.
In the same way,  Riaboff et al. (2020)  also reported a 13% decrease in accuracy when splitting the dataset by animal as opposed to a time-based split, using the independent test-set in both cases. These decrements in model performance come from high inter-animal variability based on differences in physical characteristics (e. g. , muscles, tendons, joints) that lead to different expressions of the same behaviour (e. g. , motion intensity, speed, posture;  Barwick et al. , 2020) ; hence the same behaviour ends up being expressed by different accelerometer patterns. Consequently, splitting an original dataset by animal can lead to markedly different feature-distributions between the training and test-sets and consequential poor predictive performance  (Rahman et al. , 2018). A decrease in performance can also be expected when validation is applied with a time based-split, compared to a random split  (Riaboff et al.
General recommendations for data collection In data collection, a general recommendation can be made about the number of animals to equip during the experiment. Good performance of prediction have already been reached with few equipped animals (<10 animals; e. g. Arcidiacono et al. , 2017)  but the lack of variability in the dataset can disimprove the generality of the model (see Section 5. 3). While this is not a major problem when the model is used thereafter with the same animals for experimental purposes, it is a critical issue for commercial deployments. Therefore, the more animals that are involved, the greater the variability between individuals in the training set, the more robust the model will be. In this respect, equipping 25 animals  (Smith et al. , 2016)  seems to be a reasonable minimum to achieve both robust prediction, that is, good performance using appropriate technique for validation (see Section 6.
1. 3 to follow). Although rarely seen in the papers surveys, equipping animals from at least 2 different farms is also recommended to improve model robustness. In the same way, the longer the animals are observed, the more inter-and intra-individual variability is taken into account in the dataset, the more robust the model will be. In this regard, about 40 h of observation appears to be the minimum time to achieve both high and robust prediction  (González et al. , 2015). It is worth mentioning that continuous observation ensures correct association between accelerometer data and observed behaviours over continuous intervals, to both train and validate the model properly. This technique is thus to be preferred to scan sampling (e. g. Tamura et al. , 2019)  as it is necessary to extend the observed behaviour between the different observation scans, while the animal may have expressed another behaviour over the interval.
General recommendations for model development Regarding the techniques used to train and test the model, the most important decision seems to involve the criterion used to split the data set. As explained in Section 5. 3, several studies report a substantial decrease in performance when validation is done with other animals than those used to train the model compared to random or time-based splits due to the variability between animals and time periods. Therefore, splitting the dataset randomly does not seem appropriate for testing the robustness of the model and may lead to overly-optimistic performances, not representative of those obtained in a commercial context. In this respect, we recommend to use an animal-based split to evaluate the robustness of the model properly  (Rahman et al. , 2018).
Testing the model with an independent dataset  (Riaboff et al. , 2020)  is also a good way to ensure that the parameters used to train the model (window size, overlap, classifier, hyperparameters, etc. ) are suitable for another dataset than the one used to develop the model and are not too specific, which could lead to overly-optimistic performance. Consistently, testing the model with the same dataset than the one used to train it, as done in 13% of the studies (Fig. 5a ), conflates training and testing, and is thus a validation technique to be proscribed. Another reason leading to unreliable and sub-optimal performance is the unbalanced distribution of time-windows within the different behaviours to predict; that is, with some behaviours over-represented, and other underrepresented. In this way, using stratified CV  (Riaboff et al.
, 2019)  or using under sampling  (Fogarty et al. , 2020; Sakai et al. , 2019)  to improve the balance of the dataset between the different behaviours to predict is also recommended. In studies comparing several classifiers, best predictive performance have often been obtained with SVM, RF and XGB  (Mansbridge et al. , 2018; Riaboff et al. , 2020; Vázquez Diosdado et al. , 2015). Although Deep Learning is seldom used in the community, contrary to Human Activity Recognition (e. g. Song-Mi  Lee et al. , 2017) , excellent performance has been achieved with PNN and LSTM models  (Peng et al. , 2020; Weizheng et al. , 2019)  and more recently with a Convolutional Neural Network  (Pavlovic et al. , 2021). These Machine Learning and Deep Learning classifiers are thus recommended for the classification of ruminant behaviours from accelerometer data.
Data collection: Include as much variability as possible As explained in Section 6. 1. 1, equipping a large number of animals from different farms, each observed continuously during a long time is essential to acquire a training dataset representative of the observed variability between animals  (Hu et al. , 2020). To further develop robust models, including different breeds  (Andriamandroso et al. , 2017) , different parities and lactation periods, observing animals at different times of the year  (Riaboff et al. , 2020)  and possibly under different breeding conditions (e. g. different types of housing, such as straw yards vs. cubicles) or different management and feeding practices (e. g. different grass heights for pasture-based systems;  Guo et al. , 2018)  seems to be a promising way to include as much variability as possible in a training dataset, and consequently to develop models that have greater generality.
Model development: Use classifiers that prevent overfitting and/or adaptive over time As mentioned in Section 6. 1. 3, SVM, RF or XGB classifiers lead to good predictive performance. Moreover, beyond the good performance obtained in an experimental framework, such classifiers include hyperparameters specifically designed to avoid overfitting (i. e. , where a model becomes too adapted to its training dataset, and therefore does not generalise well). For example, the margin width relative to the hyperplane in the SVM model  (Burges, 1998)  can be adapted (hyperparameter C: cost of constraints violation) so that it allows some misclassification in the training dataset for observations that are too far from the mean distribution (i. e. outliers). Ensemble machine learning methods such as RF, Adaboost or XGB use several weak classifiers to avoid overfitting and also include hyperparameters to optimize  (Breiman, 2001; Friedman, 2001).
For that purpose, we encourage researchers to include as much variability as possible in their datasets and to use classifiers that prevent overfitting. Some lines to investigate are also proposed to address these issues, but, in general, we would argue that the techniques used need to be objective-driven (e. g. adapted to the behaviours to predict, to the desired application on farms, etc). In this regard, we recommend that a series of pilot studies be carried out to identify the most appropriate methods and techniques for the objectives of the study. Finally, while the large range of relevant techniques and methods reported in ruminants and related communities should help to solve the mentioned limitations, further work is still needed to combine performance and operability of the systems developed.
(Vázquez Diosdado et al. , 2015)e methods for classification, such as Random Forest (RF;Lush et al. , 2018), eXtreme Gradient Boosting (XGB;Riaboff et al. , 2020)or Adaboost (ADA;Dutta et al. , 2015). The best hyperparameters for each model are usually found using Grid Search and a validation dataset (e. g. Riaboff et al. , 2020)• Unsupervised Machine Learning (UML): This category of classifiers has been used in 3. 2% of the studies (Fig. 5b). It especially includes the kmeans classification(Vázquez Diosdado et al. , 2015). •Deep Learning (DL): These methods have been used in 8% of studies (Fig. 5b). This category of classifiers includes the different varieties of Artificial Neural Networks, including Multilayer Perceptions (MLP; Nadimi et al. , 2012), Convolutional Neural Networks (CNN; Peng et al.
Acknowledgments This publication has emanated from research conducted with the financial support of  (i) Science Foundation Ireland (SFI)  to the  Insight SFI Centre for Data Analytics  under Grant Number  12/RC/2289_P2  and  (ii) SFI  and the  Department of Agriculture, Food and Marine on behalf of the Government of Ireland  to the  VistaMilk SFI Research Centre  under Grant Number  16/RC/3835.
Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Appendix 1. In Data Collection: Detail of the key choices made when predicting ruminant behaviour from accelerometers in each surveyed paper (note -based on the 30 core articles*; see Supplementary data A1 for the other 36 studies). Note: * The 30 papers presented in Appendix 1 out of the 66 reviewed were selected based on their complete methodological practices; that is, there is no main missing elements in the description of the material and methods used and the results are reported in a complete and understandable way. Furthermore, we selected papers predicting at least two distinct behaviours in order to display the predictive performance obtained in a challenging framework.
Simplify solves these issues by extracting the remaining structure from a pruned model, and removing all the zeroed-out neurons from the network. This allows to obtain a model that can be saved, shared and used without any special hardware or software. While, at a first glance, this may seem a straightforward procedure, the removal of zeroed neurons poses some hidden challenges like the presence of bias in said neurons or some constraints in the output's dimensions due to skip or residual connections. Even though the interest of the deep learning community on the matter seems to be quite strong, 2 very few approaches and libraries for simplifying pruned models have been proposed. 3 Moreover, they are usually limited to simpler architectures such as VGG  [23] , and their usage is restricted to the deployment of an already pruned model.
Software description The Simplify library leverages on the main PyTorch packages and is composed of three main modules that, even if designed to function in a predefined order, can be used independently based on the user requirements. We now provide a brief overview of each module functionalities and purpose. A more detailed explanation of the maths involved in each module is provided in the Appendix. Fuse First, we have the fuse module. Here we perform a nonmandatory optimization of the model by merging, in a single Convolutional layer, pairs of consecutive Convolutional and Batch Normalization layers. This process is known as Batch Normalization fusion or folding. This step can be ignored if the presence of Batch Normalization layers in the network is required, i. e. for further training of the simplified model.
Illustrative examples In this section we provide an usage overview for Simplify. We also illustrate the results obtained for the two different use cases discussed in Section 1, namely optimization for model deployment and optimization during training. Please note that our experiments were performed on different image classification networks, made available by the Torchvision library. 4  Such models are defined for the ImageNet  [26]  dataset and therefore expect an input of size B × 3 × 224 × 224, where B represents the batch size. Optimization for deployment This is the most common use case. Here, the simplification procedure is applied on an already trained model, on which a pruning criterion has been previously applied. In most cases, a one-line call to the simplify method is sufficient: the library performs all the three steps autonomously, and takes care of different architectural patterns such as residual connections.
Below, we provide a sample code snippet. As show in the code snippet, the simplify function takes as argument the model to be simplified and a tensor representing a input image, filled with zeros, with batch size of 1 (in this case we are working with models build for ImageNet therefore is of size 1 × 3 × 224 × 224). Is important to note that the model has to be set to evaluation mode before the simplification procedure, so that the automatic update of parameters (such as for Batch Normalization) is disabled. The simplification of the model happens in-place. Table  1  shows the inference times (in milliseconds) of different Torchvision models. In this table we compare the inference speed of the dense models (i. e. models that have not been pruned), the resulting pruned architectures (random, structured pruning with 50% probability) and the simplified model obtained with our proposed library.
Conclusions We propose the PyTorch compatible library Simplify, with the aim of providing a simple-to-use set of procedures to remove zeroed neurons from a neural network architecture. The proposed library solves different issues in the creation of simplified models, such as the propagation of the bias of pruned neurons and the shape constraint of skip connections. The proposed library is composed of three modules that, while designed to work together, can be used independently from one another according to the required functionality for a specific setting.
Artisanal and small-scale mines (ASM) are on the rise. They represent a crucial source of wealth for numerous communities but are rarely monitored or regulated. The main reason being the unavailability of reliable information on the precise location of the ASM which are mostly operated informally or illegally. We address this issue by developing a strategy to map the ASM locations using a convolutional neural network for image segmentation, aiming to detect surface mining with satellite data. Our novel dataset is the first comprehensive measure of ASM activity over a vast area: we cover 1. 75 million km 2 across 13 countries in Sub-Tropical West Africa. The detected ASM activities range from 0. 1 ha to around 2, 000 ha and present a great diversity, yet we succeed in hitting acceptable compromises of performance, as achieving 70% precision while maintaining simultaneously 42% recall.