dataesr/scientific-paragraphs-categorization huggingface dataset:
Data Collection: The training set used for the challenge will be a version of the 2019 training set  [40]  enriched by new contributions from the Xeno-canto network and a geographic extension. It will contain approximately 80K recordings covering between 1500 and 2000 species from North, Central and South America, as well as Europe. This will be the largest bioacoustic dataset used in the literature. For the test set, three sources of soundscapes will be used: (i) 100+ h of manually annotated soundscapes recorded using 30 field recorders between January and June of 2017 in Ithaca, NY, USA by the Cornell Lab of Ornithology [7], (ii) 10+ h of fully annotated dawn chorus soundscapes recorded using solar-powered field recorders between January and June 2018 near Frankfurt (a. M. ), Germany by OekoFor  [19] , (iii) 2 h acquired at high sampling rate (250 kHz) by binaural antenna in Côte d'Azur, France.
Task Description: In 2020, two scenarios will be evaluated: (i) the recognition of all specimens singing in a long sequence (up to one hour) of raw soundscapes that can contain tens of birds singing simultaneously, and (ii) chorus source separation in complex soundscapes that were recorded in stereo at very high sampling rate (250 kHz SR). For the first scenario, participants will be asked to provide time intervals of recognized singing birds. Participants will be allowed to use any of the provided metadata complementary to the audio content (. wav format, 44. 1 kHz, 48 kHz, or 96 kHz sampling rate). The task is focused on developing real-world applicable solutions and therefore requires participants to submit single models trained on none other than the mono-species recordings provided as training data.
For the second task on stereophonic recordings, the goal will be to determine the species singing in chorus simultaneously during a time interval. In contrast to task one, the challengers are invited to run automatic source separation before or jointly to the bird species classification, taking advantage of the multi-channel recordings. Participants will be allowed to use any other data than the provided recordings, but will have to provide the scripts to check that their solution is fully automatic. For both tasks, the evaluation measure will be the classification mean average precision (c-mAP,  [39] ).
Data Collection: The challenge will rely on a collection of millions of occurrences of plants and animals in the US and France (primarily from GBIF  [10] , iNaturalist  [13] , Pl@ntNet  [20]  and a few expert collections). In addition to geocoordinates and species name, each occurrence will be matched with a set of geographic images characterizing the local landscape and environment around the occurrence. In more detail, this will include: (i) high resolution (about 1 m 2 /pixel) remotely sensed imagery (from NAIP [18] for the US and from IGN  [2]  for France), (ii) bio-climatic rasters (1 km 2 /pixel, from WorldClim  [24] ) and (iii), land cover rasters (30 m 2 /pixel from NLCD  [17]  for the US, 10 m 2 /pixel from CESBIO  [3]  for France).
Data Collection: Images of about 100 snake species from all around the world (between 300 and 150,000 images per species) will be aggregated from different data sources (including iNaturalist  [13] ). This will extend the dataset used in a previous challenge  [22]  hosted on the AICrowd platform. The distribution of the number of images between the classes is highly imbalanced. Task Description: Given the set of images and corresponding geographic location information, the goal of the task will be to return for each image a ranked list of species sorted according to the likelihood that they are in the image and might have been observed at that location.
2D FE Model of Healthy and Damaged Beam Equation (  1 ) is discretized using 2-nodes beam elements with 2 degrees of freedom (DOFs) per node, a vertical translation v and a bending rotation θ z. As can be seen in Figure  1 , the beam is divided into N elements of equal size of and the model has N + 1 nodes. The damage is assumed to be placed within the beam elements of the FE model. Consider the damage at location x l within node i and i + 1 of beam. It will be assumed that stiffness reduction falls within one or several elements. Natural frequencies of healthy and damaged 2D beam are obtained from solving the eigenvalue formulation using MATLAB eig () function and their expression (in Hz) is f i = ω i 2π. In this 2D FE model, the damage is simulated by reducing the stiffness of selected element.
The stiffness reduction coefficient (or severity) and the localization of the reduction are updated during this procedure. The minimum value of the FSC is sought during the updating process. The argument of the minimum yield wanted location and severity. Practically, the minimization is performed using particle swarm optimization (PSO)  [26]  in MATLAB global optimization toolbox. If the damage is identified, then we can proceed to the next step for geometry investigation. Step 3: In this step, based on the results of Step 2 and Step 3, the identification of damage physical parameters can be done. The estimated damage severity from Step 2 is searched in the damage library. It corresponds to a (potentially nonunique) set of width and depth for a rectangular geometry. Note that curve fitting may be necessary if the library is not precise enough and that a tolerance can be introduced.
Numerical Assessment of the Strategy Using 3D and 2D Models Following the methodology proposed in Section 3, numerical assessments are performed using 3D and 2D models. In the beginning, beam properties and Young's modulus updating procedure are detailed. Complimenting the 2D model made in MATLAB, a 3D model built with COMSOL is going to be used to be simulated as well as correlated using FSC minimization in order to build a damage library using Step 1. Cases with Single and multiple damage are tested to validate the bending stiffness based identification using Step 2. Furthermore, rectangular damage cases modeled with 3D FE are investigated; Step 3 is used in order to distinguish the damage properties and size (width and depth) based on the obtained severity.
The input excitation is produced through an impact hammer, while monoaxial accelerometers (Brüel et Kjaer) record the output data. Two accelerometers are placed at 0. 35 m and 0. 75 m from the clamped end. Frequencies of a healthy and damaged structure are obtained from frequency response function by applying circle fit method using EasyMod package  [27]. The first seven experimentally identified frequencies for each beam are listed in Table  6. Slight differences in natural frequencies are noticeable.
Times series classification can be successfully tackled by jointly learning a shapelet-based representation of the series in the dataset and classifying the series according to this representation. This shapelet-based classification is both accurate and explainable since the shapelets are time series themselves and thus can be visualized and be provided as a classification explanation. In this paper, we claim that not all shapelets are good visual explanations and we propose a simple, yet also accurate, adversarily regularized EXplainable Convolutional Neural Network, XCNN, that can learn shapelets that are, by design, suited for explanations. We validate our method on the usual univariate time series benchmarks of the UCR repository.
Many methods have been developed to tackle this problem (see  [2]  for a review). One very successful category of methods consists in "finding" discriminative phase-independent subsequences, called shapelets, that can be used to classify the series. In the first papers about shapelet-based time series classification  [3] ,  [4] , the shapelets were directly extracted from the training set and the selected shapelets could be used a posteriori to explain the classifier's decision. However, the shapelet enumeration and selection processes were either very costly or the selection was fast but did not yield good performance (as discussed in Section II). Jointly learning a Elisa Fromont is supported by the HyAIAI (Hybrid Approaches for Interpretable AI) Inria "Défis" project. Romain Tavenard is partly funded by ANR through project MATS (ANR-18-CE23-0006).
shapelet-based representation of the series in the dataset and classifying the series according to this representation  [5] ,  [6]  allowed to obtain discriminative shapelets in a much more efficient way. An example of such learned shapelets, obtained with the method from  [6] , is given in blue in Figure  1  (left). However, if the learned shapelets are definitively discriminative, they are often very different (visually) from actual pieces of a real series in the dataset. As such, these shapelets might not be suited to explain a particular classifier's decision. Note that the same interpretability issue arises with ensemble classifiers such as  [7]  where one decision depends on the presence of multiple shapelets. One of the main challenges nowadays is to provide Machine Learning (ML) methods that are both accurate and self-explanatory, i.
A min-pooling aggregator should then be used for temporal aggregation. Closely related to shapelet-based methods (as stated above), variants of Convolutional Neural Networks (CNN) have been introduced for the TSC task  [11]. These are mostly monodimensional variants of CNN models developed in the Computer Vision field. Note however that most models are rather shallow, which is likely to be related to the moderate sizes of the benchmark datasets present in the UCR/UEA archive  [9]. A review of these models can be found in  [2]. Finally, ensemble-based methods, such as COTE  [7]  or HIVE-COTE  [12] , that rely on several of the above-presented standalone classifiers are now considered state-of-the-art for the TSC task. Note however that these methods tend to be computationally expensive, with high memory usage and difficult to interpret (as stated in Section I) due to the combination of many different core classifiers.
The prediction results are explained by inspecting the input MTS using GradCAM on both the variable and temporal dimensions. [18] has a similar goal as ours (to produce interpretable discriminative shapelets) and build on both the work from  [5]  (in this case the candidate shapelets are extracted with a piecewise aggregate approximation) and from  [6]  to automatically refine the "handcrafted" shapelets. Contrarily to our method, there is no explicit constraint on the learning process that ensures the interpretability of the shapelets. Besides, their experimental validation makes it hard to fully grasp the benefits and limitations of the proposed method since the algorithm is evaluated on a small subset of UCR/UEA datasets  [9]  and they provide visualizations for only a couple of the learned shapelets.
The work from  [19]  is the closest to ours. Contrarily to ours, they decouple the shapelet learning phase and the classification process resulting in a quite different adversarial architecture. Their classification process is made using the shapelet transform method  [5]  but, in this case, the candidate shapelets are dynamically generated for each input time series. In our case, this is learned by a simple CNN for all the dataset. In  [19] , an adversarial regularization is also used to constrain the generated shapelets to be similar to real pieces of the series. However, the regularization is imposed on the result of the convolutions (i. e. the feature maps) and not on the convolutions themselves as we propose to do in this paper. This is a different philosophy: we believe that the pattern detectors, i.
A. Experimental Setting 1) Competitors: We provide experiments about the quality (for explanations) of our learned shapelets as well as their quality for classification. As explained in Section II, our most relevant competitor is Learning Shapelets (LS) from  [6]  as it also describes a shapelet-based model where the shapelets are learned and where a single model is used for classification. The quality (for explanations) of the shapelets produced by  [3]  and  [4]  is, by design, perfect since the shapelets are true subpart of the original series so we do not compare with them but only with the shapelets learned by  [6]. However, we compare our classification performance to  [3] , Fast Shapelets  [4]  and the recent ELIS  [18]. 2) Datasets: We use the 85 univariate time series datasets from the UCR/UEA repository  [9]  for which most of our baselines results are already available.
1 ote that our CNN-based method may also be suited for multivariate time series but giving "intuitive" explanations for multivariate data is far from obvious and we decided to focus only on univariate ones in this paper. The datasets are significantly different from one to another, including seven types of data with various number of instances, lengths, and classes. The splits between training and test sets are provided in the repository. 3) Architecture details and parameter setting: We have implemented the XCNN model using TensorFlow  [23]  following the general architecture illustrated in Fig. 2. The classifier is composed of one 1D convolution layer with ReLU activation, followed by a max-pooling layer along the temporal dimension and a fully connected layer with a soft-max activation.
The shapelets use a Glorot uniform initializer  [24]  while the other weights are initialized uniformly (using a fixed range). For each dataset, three different shapelet lengths are considered, inspired by the heuristic from  [6]  but without resorting to hyperparameter search: we consider 3 groups of 20 × n cl shapelets of length 0. 2T , 0. 4T and 0. 6T , where n cl is the number of classes in the dataset and T is the length of the time series at stake. The convolution filters of the classifier, i. e. the shapelets, are given as input to the discriminator which has the same structure as the classifier, but with shorter convolution filters (100 filters of size 0. 06T , 0. 12T and 0. 18T ) and a singleneuron tanh activation instead of the soft-max in the last layer. For optimization, we use Adam optimizer with a standard parameterization (α = 10 -3 , β 1 = 0.
In this figure, we show the evolution of a given shapelet for the Wine dataset at epochs 20, 200, 800 and 8,000. One can see from the loss values reported in Figures  3a  and 3d  that these correspond to different stages in our learning process. At epoch 20, the Wasserstein loss is far from the 0 value (L d = 0 corresponds to a case where the discriminator cannot distinguish between shapelets and real subseries), and this indeed corresponds to a shapelet that looks very different from an actual subseries. As epochs go, both the Wasserstein loss L d and the cross-entropy one L c get closer 0, leading to both realistic and discriminative shapelets. 2) Shapelet contributions: The computation of the contribution of a shapelet to a decision is based on GradCAM ("Gradient-weighted Class Activation Mapping")  [16].
where Ãk is a vector of all zeros but at position t = arg max t A k t (where A k t is stored). To obtain the global positive contribution of a shapelet k given a set of n time series examples, we compute gp k = 1 n n i=1 p k (Z i ). (2) The shapelets shown in Fig. 4  are the 3 most contributing shapelets, according to this global criterion. In Fig. 4 , the shapelets learned by XCNN seem visually closer to the time series than the shapelets learned by LS. We then computed the average L 2 between a shapelet and a subpart of a time series over all the shapelets learned by XCNN and by LS for a given dataset, computed at the best matching point of the closest time series in the dataset (also in terms of L 2 ). The results are given in Fig. 5. This scatter-plot shows that, even if the optimized distance between the shapelets the input series in the neural network is not the L 2 one (it is the dot product), our adversarial regularization allows XCNN to obtain closer (in terms of L2) shapelets than LS which are deemed more suited for explanations.
To illustrate this, we overlay in Fig. 6  and 7  the three most positively (resp. negatively on the right) contributing shapelets on the time series at their best matching location (using L2 distance), with number of total positively and negatively shapelets noted in the captions. Note that on the left side, the horizontal axis gives the length of the series (in black) while on the right, it gives the length of the shapelets which is at most 60% of the length of the series. We do not show the original series for the negative shapelets since, by definition, they are very far from the original series. In Fig. 7  there is no negative shapelet used to discriminate the series of this dataset. This is due to the fact Fig. 4 : Three most discriminative shapelets obtained for the datasets Beef, ECG200, GunPoint, Herring, OliveOil (rows 1 to 5, resp.
) using (left column) LS or (right column) XCNN. The average discriminative power of the shapelets is evaluated using Eq. 2 and each shapelet is superimposed over its best matching time series in the test set. that the series for all the classes are very similar except for very small changes in the slope of the bump or in the size of the plateau at the top of the bump. These small changes can be captured by the positive shapelets but many of them are used to succeed in discriminating the classes. We can also use our method to show the shapelets that rp k (c) = ReLU     1 N c N c i=1     p c k (Z i ) - 1 n cl -1 ncl-1 j=1 j =c p j k (Z i )         where N c is the number of examples in class c, and n cl is the total number of classes in the dataset. One can compute rn k (c) similarly by replacing p j k with n j k.
) of the DiatomSizeReduction test set. Note that there were different positive and negative for different series, e. g. there were in total 240 positive shapelets used for the decision of the first test series and 0 negative ones. Shapelets  [3] ), much more accurate than very fast random shapelet selection-based methods (our baseline is FS  [4] ) and as accurate and as fast as single model shapelet learning methods (our baselines are LS  [6]  and ELIS  [18] ). 1) Accuracy: We analyze the accuracies obtained by FS, LS, ELIS and our XCNN method on the 85 datasets using scatter plots. We compare FS versus XCNN in Fig. 10 , LS versus XCNN in Fig. 11  and ELIS versus XCNN in Fig. 12. We also show how a simple CNN (without the adversarial regularization) compares against LS in Fig. 13. We indicate the number of win/tie/loss for our method and we provide a Wilcoxon significance test  [26]  with the resulting p-value (> 0.
01: none of the two methods is significantly better than the other). The points on the diagonal are datasets for which the accuracy is identical for both competitors. Fig. 10  shows that, as expected, our method yields significantly better performance than FS. Fig. 11 : Accuracy comparison between Learning Shapelets (LS) and XCNN on 85 datasets (each point is a dataset) of the UCR/UEA repository  [9]. LS, for most datasets, the difference in accuracy is low, with a small edge (significant) for LS. On three datasets (namely HandOutlines, NonInvasiveFetalECGThorax1 and OliveOil), our XCNN method and its regularization seems to be strongly positive (and detrimental on one dataset), in terms of generalization. A simple CNN that would correspond to the classifier of our XCNN alone seems to give slightly better (non significant) results than LS (and thus than our XCNN).
To have a better grasp on the actual training time of all methods, we ran the methods on a single dataset (ElectricDevices) and recorded the CPU time. The experiments were conducted on a Debian Cluster using Intel(R) Xeon(R) CPU E5-2650 v4 Processor (12 core 2. 20 GHz CPU) with 32GB memory. The results are averaged over five runs. The implementation code of our baselines is taken from  [2]  (as for the accuracy results). As expected, the original Shapelet  [3]  method does not finish in 48 hours for this medium size dataset. FS finishes in 12. 1 minutes, LS finishes in 2323 minutes, and our method takes 142 minutes. The theoretical complexity of LS and XCNN is identical so these results were surprising. We suspected that the JAVA implementation of LS was not well optimized and we used the implementation of LS method from tslearn  [27]  using Keras 2 with TensorFlow as backend.
Model description and developments We use the Earth system model of intermediate complexity iLOVECLIM, which is a code fork of the LOVECLIM model  (Goosse et al. , 2010). It includes a representation of the atmosphere, ocean, sea ice, terrestrial biosphere vegetation and the carbon cycle. The ocean component (CLIO) consists of a free-surface primitive equation ocean model (3 • ×3 • horizontal grid, 20 depths layers) coupled to a dynamicthermodynamic sea-ice model. iLOVECLIM includes a land vegetation module (VECODE)  (Brovkin et al. , 1997)  and a marine carbon cycle model  (Bouttes et al. , 2015) , both computing the evolution of 13 C and 14 C. Previous work has shown that the simulated oceanic δ 13 C and 14 C distribution is in reasonable agreement with observations and with available general circulation model (GCM) simulations  (Bouttes et al.
As reported in other model studies, the increased 14 C in the upper ocean reflects the accumulation of radiocarbon in the atmosphere and the upper ocean in the absence of carbon entrainment in deep water through North Atlantic deep-water formation  (Butzin et al. , 2005). The generally good ventilation state of the upper ocean indicated by both δ 13 C and 14 C may also be related to an increase in the vertical mixing (and ventilation) in the intermediate waters in the absence of North Atlantic deep convection cell  (Schmittner, 2007). The pattern we obtain in the intermediate waters (Figs. 3  and 5b ) is consistent with a δ 13 C dataset on the Brazilian margin  (Lund et al. , 2015; Tessin and Lund, 2013)  showing increasing δ 13 C between 1000 and 1300 m and decreasing between 1600 and 2100 m water depth.
Missiaen, 2019)ource code is based on the LOVECLIM model version 1. 2 whose code is accessible at http://www. elic. ucl. ac. be/modx/elic/index. php?id=289 (last access: 17 April 2019, UCLouvain, 2019). The developments in the iLOVECLIM source code are hosted at https://forge. ipsl. jussieu. fr/ludus(lastaccess:17April 2020, LUDUS, 2020) but are not publicly available due to copyright restrictions. Access can be granted on demand by request to Didier M. Roche (didier. roche@lsce. ipsl. fr). The model output related to this article is available on Pangaea (https://doi. org/10. 1594/PANGAEA. 900617,Missiaen,2019). The supplement related to this article is available online at: https://doi. org/10. 5194/cp-16-867-2020-supplement. Author contributions. LM, CW and DMR designed the research. LM, DMR, NB, JCD and AQ developed the iLOVECLIM model.
LM performed the simulations. LM and JYP developed the postprocessing algorithm. JCD and SP contributed with expert knowledge of Pa/Th. LM wrote the paper with the inputs from all the coauthors. The authors declare that they have no conflict of interest. Acknowledgements. This is a contribution to ERC project AC-CLIMATE; the research leading to these results has received funding from the European Research Council under the European Union Seventh Framework Programme (FP7/2007-2013), ERC grant agreement 339108. Lise Missiaen acknowledges funding from the Australian Research Council grant DP180100048. We thank Santiago Moreira and Fanny Lhardy for their help with Python. Elisabeth Michel is thanked for expert knowledge discussion of 14 C. This is a LSCE contribution 6949. This research has been supported by the European Research Council (ACCLIMATE (grant no.