dataesr/scientific-paragraphs-categorization huggingface dataset:
Geothermal heating. The model modern boundary conditions include a realistic map of geothermal heating, whose impact on the ocean abyssal dynamics is non-negligible (Emile-Geay and  Madec, 2009). As the geothermal heat flow q(t) can be related to the ocean lithosphere age t, we construct a geothermal heating map for the Turonian based on the reconstruction of the Turonian ocean crustal age of  Müller et al. (2008). We derive the geothermal heat flux using the age-heat flux relationship of the  Stein and Stein (1992)  model. For t ≤ 55 Ma, q(t) = 510t 1/2. ( 2 ) For t > 55 Ma, q(t) = 48 + 96e -0. 0278t. (3) In regions where crust age is missing in the Müller et al. (  2008 ) dataset, we impose the minimal geothermal heat flux obtained with the above equations. The geothermal heating map created is then remapped on the PALEORCA grid and we limit the maximal heat flux to 400 mW m -2 because the  Stein and Stein (1992)  parameterization becomes singular for young ages (Emile-Geay and  Madec, 2009).
The final topography/bathymetry is shown in Fig. (18b ). LMDZ requires a 10 min global elevation dataset to compute subgrid-scale parameters (elevation standard deviation, slope, peaks and valleys) that are needed to compute surface roughness length as well as to activate orographic drag parameterizations  (Lott and Miller, 1997). As the paleotopography provided by Sewall et al. (  2007 ) is 2. 8 • × 1. 4 • in resolution, we simply artificially upscaled it to a 10 min resolution through the use of climate data operator (cdo) remapping tools  (Schulzweida, 2019). River routing. The river routing system (i. e. , length of rivers, direction of flows, locations of river outflows) is constructed by first upscaling the low-resolution topography of  Sewall et al. (2007)  to a 1 • × 1 • resolution and then using the  Wang and Liu (2006)  algorithm within the QGIS software (QGIS Geographic Information System, Open Source Geospatial Foundation project; http://qgis.
"No coupling" indicates atmospheric model only. -Data output cost is the cost of performing I/O, computed as the ratio of CHSY with and without I/O: (CHSY -CHSY no I/O)/CHSY. Compared to IPSL-CM5A, which could be run on 38 cores only, hybrid parallelization for IPSL-CM5A2 permitted to design two configurations using more than 100 cores: a highthroughput version (IPSL-CM5A2-T) on 160 cores and a "fast" version (IPSL-CM5A2-S) running on 302 cores. IPSL-CM5A2-T configuration allows to reach 38 SYPD (3. 8 times faster than IPSL-CM5A) with a 10. 8 % increase in computing cost (CHSY, Table  A1  and Fig. 2 ). IPSL-CM5A2-S reaches 56 SYPD, for a CHSY 42 % larger than for IPSL-CM5A (129 000 h for 1000 simulated years). Besides, CHSY is greater for IPSL-CM5A2-S, which means there is a better use of a resource allocation than with IPSL-CM5A.
The replacement of the sequential OASIS3. 3 coupler by the parallel OASIS3-MCT library in IPSL-CM5A2 also allows reducing the cost of the coupling from 12 % to 1 % of the total CPU time of a simulation. Regarding input/output aspects, IPSL-CM5A used the IOIPSL library, which performed sequential writing and imposed a rebuild post-processing step to merge multiple files into one single file. Instead, IPSL-CM5A2 uses the XIOS library that writes out data asynchronously using dedicated I/O servers. Parallel writing allows users to obtain directly one single file without any time-consuming rebuild step. Still, although the writing is performed asynchronously, the I/O cost remains important in IPSL-CM5A2 (10 %) due to operations (temporal operations, variables combinations) that are now performed by the library (on the client side, i.
Acknowledgements. The setup of the model strongly benefited from the pioneer work of  Gurvan Madec  and  Alistair Sellar  on the ocean grid. The authors thank  Daniel J. Lunt  and an anonymous reviewer for their constructive remarks. We also acknowledge  Didier Roche  for his comments and suggestions regarding EMICs. This work was granted access to the HPC resources of TGCC under allocation 2019-A0050102212 made by  GENCI. Financial support. Jean-Baptiste Ladant  and  Yannick Donnadieu  acknowledge the support of the  Anox-Sea  project funded by the  Agence Nationale de la Recherche (ANR)  (grant no. ANR-12-BS06-0011-03 ). Sophie Szopa  and  Pierre Sepulchre  acknowledge the support of the  PALEOx  project, funded by  ANR  (grant no. ANR-16-CE31-0010 ). Review statement. This paper was edited by  Fiona O'Connor  and reviewed by  Dan Lunt  and one anonymous referee.
The login/password combination requested at first use to download the ORCHIDEE component is anonymous/anonymous. We recommend to refer to the project website (http://forge. ipsl. jussieu. fr/igcmg_doc/wiki/Doc/Config/IPSLCM5A2, last access: 25 June 2020; Caubel and Sepulchre, 2019) for a proper installation and compilation of the environment. Data availability. All model outputs analyzed in this paper are available as NetCDF files on the Zenodo repository with the following DOI: https://doi. org/10. 5281/zenodo. 3549769 (Sepulchre  et al. , 2019). We also published a Jupyter notebook gathering most of the scripts used to create the figures. This notebook is embedded in a binder container that allows anyone to run it and analyze the code. It is published on the Zenodo repository with the following DOI: https://doi.
org/10. 5281/zenodo. 3549652 (Sepulchre and  Brockmann, 2019). Most of the figures were done using NOAA pyferret within Jupyter notebooks, thanks to the ferretmagic add-on developed at LSCE by Patrick Brockmann. Ferret is a product of NOAA's Pacific Marine Environmental Laboratory. (Information is available at http://ferret. pmel. noaa. gov/Ferret (last access: 1 June 2020; NOAA's Pacific Marine Environmental Laboratory, 2020), distributed under the Open Source Definition. The Jupyter notebook is an open-source web application. Most colored figures in this paper were made with perceptually uniform, color-vision-deficiency-friendly scientific color maps, developed and distributed by Fabio Crameri (Crameri, 2018a, b), to prevent visual distortion of the data. We also used the C-ESM-EP (Climate Model Assessment Framework (CliMAF) Earth System Model Evaluation Platform) developed at LSCE by Jérôme Servonnat https://github.
In this work, we present MorphoCluster, a software tool for data-driven, fast and accurate annotation of large image data sets. While already having surpassed the annotation rate of human experts, volume and complexity of marine data will continue to increase in the coming years. Still, this data requires interpretation. MorphoCluster augments the human ability to discover patterns and perform object classification in large amounts of data by embedding unsupervised clustering in an interactive process. By aggregating similar images into clusters, our novel approach to image annotation increases consistency, multiplies the throughput of an annotator and allows experts to adapt the granularity of their sorting scheme to the structure in the data. By sorting a set of 1. 2M objects into 280 data-driven classes in 71 hours (16k objects per hour), with 90 % of these classes having a precision of 0.
The annotation rate of human experts is long surpassed by the amount of data that is generated  [11]. Therefore, advanced automatic image recognition techniques are indicated. These should liberate taxonomy experts from the tedious task of routine identifications  [12]. However, to extract valuable insights from the data, moderation of automatic techniques is imperative  [9]. Published marine image annotation software  [13]  tools include photoQuad  [14] , VARS  [15] , Seascape  [16]  and BIIGLE  [17]. Beyond that, there are several tools not formally published like SQUIDLE+  [18]  and EcoTaxa  [19]  or the older Plankton Identifier  [20]  and ZooImage  [21]. Some tools address the annotation of whole frames where objects of interest have to be localized first to be classified afterward. However, plankton image data has usually a uniform background so no semantic segmentation is needed.
MorphoCluster In this work, we present MorphoCluster, a tool for data-driven, fast and accurate annotation of large data sets of single object images. Although we present and discuss the tool in the context of marine image annotation, it should be applicable in many areas with similar data sets (images of individual objects). Considering the strength of deep neural networks to learn distinctive features  [38] , we hypothesize that it is feasible to cluster these features to partition a plankton image data set in a meaningful way. We therefore combine unsupervised clustering with an interactive tool to revise the initial clusters, arrange them hierarchically, manually correct the hierarchy and annotate the clusters. The annotator therefore can explore the groupings inherent in the data and spot novelties and unexpected patterns.
General overview of the MorphoCluster process The MorphoCluster process is outlined in fig. 1. First, a deep feature extractor is trained to obtain features that encode relevant object properties for the task of plankton recognition and therefore enable efficient clustering. Then, the entire data set is clustered using HDBSCAN* with settings that allow for the creation of large and homogeneous clusters. In the cluster approval phase, visually pure clusters are validated and mixed clusters are rejected manually. During cluster growing, the remaining pure clusters are used as seeds to find additional visually similar objects. The samples that are not assigned to a cluster after the growing step are re-clustered with a less restrictive setting that yields smaller clusters than in the previous round.
Cluster approval and growth steps are thereafter repeated. The described process is conducted iteratively with less and less restrictive settings until no further meaningful clusters are found. Thereafter, the identified clusters are hierarchically arranged using agglomerative clustering to group similar clusters. The clusters and branches of the resulting tree can then be inspected manually, very similar clusters can be merged and clusters and branches named in a user-defined manner. Validation, growing and naming are conducted in a specially developed web application available at https://github. com/morphocluster.
Accuracy Using MorphoCluster, a very large fraction of classes was sorted with high precision. Figure  4  shows the class size and individual precision per class which is consistently higher for L MC compared to L 0. Roughly a tenth of the objects in each class in L MC was already labeled in L 0 (red) which allows calculating the agreement between both labelings. Table  3  shows this agreement (L MC vs L v ) and also the macro precision of L MC and L 0 individually. For the calculation of the agreement between the MorphoCluster labeling L MC and the initial labeling L 0 , only L v was used to avoid overly optimistic results coming from data which the feature extractor was trained on. We computed the proportions of objects from all initial classes in L v for every MorphoCluster category in L MC.
Fine-grained data set exploration Figure  4  compares the initial labeling L 0 to the resulting labeling L MC. Using MorphoCluster, the data set could be sorted into 280 categories in contrast to the initial 65 categories. Also, the relative class abundances of the indicator classes C i were misestimated in the initial sorting. The high ranking of Poeobius in L 0 likely originates from the high effort that was put into finding examples for this class after it had been discovered  [23]. Although the largest part of the data set was sorted in the early steps (see section 3. 2), Figure  5  shows that the later steps were nevertheless required to achieve this large number of categories. Spiking the data with labeled objects from the validation set L v allowed the calculation of relative overlap between initial and new classes L 0 and L MC.
Thus we need to develop techniques that exploit and augment the human ability to perform object classification and novelty detection by accelerating annotation and increasing consistency  [12]. MorphoCluster excels at cluster-based manual mass allocation of images into homogeneous groups, followed by hierarchical ordering in a semantic tree for easy naming of classes. By paying attention to the cluster structure of a data set, we achieve an outstanding combination of properties: MorphoCluster is at the same time fast, allows for a flexible, fine-grained and data-driven classification, is accurate, consistent, and enables novelty detection. MorphoCluster is available as open-source software at https: //github. com/morphocluster. We expect that the approach can be adapted to any kind of image collection where individual objects can be extracted and useful features that enable meaningful clustering can be calculated using a deep convolutional neural network (CNN).
g. the distinction of man-made objects like bicycles, cars and trains)  [27, 57, 58]. We therefore tested in some preliminary experiments if we could train a feature extractor with ImageNet  [43]  data. However, this did not produce well-defined clusters and we fine-tuned the network with plankton images so it could learn the characteristic appearances of different kinds of plankton. The CNN features extracted using this auxiliary training set then allowed efficient clustering and transformation into a hierarchy by agglomerative clustering. We use the advantageous characteristics of the CNN features to provide a complete workflow to separate and classify plankton images in a real-world data set. By merging supervised and unsupervised tools with human intervention, MorphoCluster enables flexible, fine-grained mass annotation of images and detection of novel classes in a data-driven way.
Funding: This project was funded by the  Cluster of Excellence 80 "Future Ocean" (CP1733). " Future Ocean " is funded within the framework of the Excellence Initiative by the  Deutsche Forschungsgemeinschaft (DFG)  on behalf of the German federal and state governments. R. Kiko  was furthermore supported by the  SFB 754  " Climate-Biogeochemistry Interactions in the Tropical Ocean " (www. sfb754. de,grant no. 27542298  of the  German Science Foundation DFG)  and via a  "Make Our Planet Great Again" grant  of the  French National Research Agency  within the " Programme d'Investissements d'Avenir "; reference  ANR-19-MPGA-0012.
clusters of symptoms and complaints of HA patients (topics). The probability of distribution of words within a topic uniquely characterizes it. The convergence of the log-likelihood of the LDA-model has been reached after 2,000 iterations. Several statistical parameters have been tested for topic modeling and word relevance factor within each topic. Results: Despite a rather small dataset, this exploratory study demonstrates that patients' free speeches available on the Internet constitute a valuable material for machine learning and statistical analysis aimed at categorizing ST/HA complaints. The LDA model with K = 15 topics seems to be the most relevant in terms of relative weights and correlations with the capability to individualizing subgroups of patients displaying specific characteristics.
Design of the Study This paper presents a preliminary statistical analysis of freely posted comments on an internet forum (TinnitusTalk. com). After having contacted the founder of the Tinnitus Hub (administrator of TinnitusTalk. com) and gained verbal permission of doing so, we extracted all data from a publicly available source and anony-Palacios/Noreña/Londero Audiol Neurotol 4 DOI: 10. 1159/000504741 mized it before statistical analysis. Moreover, no personally identifiable information was included and no individual dataset is discussed, maintaining the anonymity of the responses. Thus, considering the minimal potential risk to individual participants, the study did not require ethical approval. All the codes used in this paper are freely available and downloadable: https://github. com/GuillaumePK/LDATinnitusHyperacusis.
Data Extraction The first step was to aggregate the textual and meta-data from the TinnitusTalk. com website. We downloaded all the relevant web pages in HTML format. Then, we wrote a Python script to automate the parsing of the HTML data. For each post in the forum, we extracted the following information: (a) the "post-id," which is a unique identifier of a post on the forum, (b) the hyperlink to the thread and the post on the forum stored with the label "url," (c) the actual textual content of the message posted by the user in string format stored in the field "text," and (d) the timestamp stating when the post was sent on the forum with the label "post-date. " In order to anonymize the dataset and guarantee the users' privacy, we have hashed each "user" to protect the pseudonym that he/she uses on the forum.
Hence, no personal information has been stored, processed or analyzed. The same Python script used to parse the data was used to save it in a locally-hosted document-based database (MongoDB). The data was stored in Javascript Object Notation (JSON). A sample of data input is shown in Table  1. For the present study, we focused on a discussion thread titled "Hyperacusis" that brings together 3,785 posts from 200 contributors (as of March 2016).
Pre-Processing Before analyzing the textual data, it is mandatory to clean and to pre-process it. After having checked the database for consistency, we removed duplicate forum posts. In order to perform a statistical analysis of the terms appearing in the data we had to break all text messages into sentences. Then, we split each sentence into a sequence of words (or tokens). This step is called "tokenization. " In order to avoid as much noise as possible in the data signal, we removed all English common stop-words such as conjunctions (if, then, because…), determinants (the, a), pronouns (he, she…) and common verbs (is, be, have, had, will…). These words appear frequently in any English text but add no useful information to the content or significance of the document. Eventually, we removed all punctuation marks and normalized the text to lowercase.
Topic Modeling Next, we examined the data from the HA forum on Tinnitustalk. com under the lens of a topic model. Topic modeling is a type of statistical learning aimed at unveiling the abstract topics that occur in a collection of documents. The LDA algorithm is the most well-known example of topic modeling. It is a generative probabilistic model that unveils the underlying topics that structure a corpus of texts  [Blei et al. , 2003]  and measure their relative importance. Within the LDA framework, each document (forum post) is modeled as a statistical mixture of different topics. And these topics can be interpreted as a set of words each with a certain probability of occurrence. The words with the highest probability are the representatives of the topics computed by the LDA algorithm. Recently, LDA has been used in different fields such as the classification and generation of political speeches  [Kassarnig, 2016].
We took the Java-implementation of the LDA algorithm from the open-source machine learning library Mallet (http://mallet. cs. umass. edu/). The topic modeling of the Tinnitustalk. com data was conveyed using a set of high level R functions that performed Table  1. Example of a database entry from the discussion forum on hyperacusis on TinnitusTalk. com in JSON format { _id: ObjectId("56ce3f8ca935896388afad30"), "post-id": "163876," url: "https://www. tinnitustalk. com/threads/hyperacusis-tinnitus-help. 13495/," text: "Hi M. sorry you are having these issues atm If I were you I would skip band practise for at least a couple of weeks. You have a unique chance of recovery from this provided you give your ears a good rest Plugs do not cancel out all the harmful frequencies and you still have your whole life in front of you so take good care," user: "087b3d6ead7f45c976560a23f74cdb83," "post-date": "Feb 10, 2016" } JSON, Javascript Object Notation; atm, at the moment.
Assessing Tinnitus/HA Heterogeneity from Machine Learning 5 Audiol Neurotol DOI: 10. 1159/000504741 the following steps: (1) read the data from the Mongo Database (JSON format), (2) put the data into a data frame and (3) run a parallel LDA topic modeling on the whole BOW corpus of "Hyperacusis" forum posts calling the Java LDA model from Mallet. To make the interface between the R program and the MALLET models (in Java), we used the rJava and Mallet-R packages. The output of the whole process is a probabilistic model giving K topics, each topic described as a distribution of words. The words and association of words are characteristic of a topic. The set of topics discovered by the algorithm are to be considered a real-life picture of HA patients' concerns on an Internet forum (i. e. , symptoms and distress) as we shall see in the Results section.
Results The word count of the 50 most frequent words occurring in the text corpus is displayed in Figure  1. In this thread dedicated to HA the 5 words most frequently used are: ear (n = 1,504), HA (n = 1,358), sound (n = 1,172), tinnitus (n = 1,087), and pain (n = 965). Figure  1  also shows some less frequent but highly relevant words with a psychological valence such as anxiety (n = 350), sensitivity (n = 191), feeling (n = 190), sleep (n = 179). The main results of running the LDA model on the BOW dataset on the "Hyperacusis" thread have been summarized in Figure  2  which shows a graphical representation of the 15 topics unravelled by the LDA algorithm  [Sievert and Shirley, 2014]. Each topic is represented by a circle (Fig. 2a ) whose area is proportional to the importance of the topic in the corpus of patients' posts and whose distance with the other circles is proportional to their inter-correlation.
Discussion Even if NLP can be applied to any kind of textual data (i. e. , clinical reports, patients' testimonies, speech to text output) this preliminary study demonstrates that patients' free speeches available on the Internet constitute a valuable dataset for machine learning statistical analysis aimed at categorizing ST/HA complaints. Indeed, by collecting all the posts exchanged among patients of the HA thread on TinnitusTalk. com, we have been able to perform a frequency analysis of these textual data. We found that the top 10 keywords associated with the HA thread are ear, HA, sound, tinnitus, pain, time, loud, noise, hearing and people. Those terms are in accordance with the clinical knowledge about HA highlighting the links between HA and hearing impairment, ST, chronic pain and noise exposure.
1%), hearing (23. 5%), health (20. 7%), sleep (14. 6%), and situational (11. 1%) difficulties. Beyond the simple frequentist point of view, we also analyzed the HA forum data using a topic modeling approach (LDA). Even if the more important Topics 1 and 2 are pretty general, less frequent ones seem to be related to definite subgroups of patients and/or conditions such as the tensor tympani syndrome (Topic 6), Menière's disease (Topic 11) or comorbid psychological factors (Topic 13). Apparently the LDA algorithm has the ability to categorize the patients' complaints into a discrete number of clusters that may be representative of homogeneous subgroups of subjects/patients. Even if this has not been performed in this study for ethical reasons (respect of patients' anonymization) a further step, would be to analyze statistically the contribution of each individual subject to each topic.