dataesr/scientific-paragraphs-categorization huggingface dataset:
Bertotti's loss models Equation (1) calculating the magnetic losses as a function of frequency and peak induction is valid only if B(t) is in sinusoidal form. This formula has been generalized to apply for any waveform by introducing the time derivative dB/dt as expressed in equation (  2 )  [9]. This method, named (M1), has been available in Altair FLUX software for transient magnetic regime calculations. P ¼ 1 T Z k hys B 2 m f þk eddy dB dt 2 þ k exc dB dt 3=2 dt: ð2Þ In order to apply this model, the skin effect must be negligible. Furthermore, in  [19, 20] , the excess coefficient k exc is reported to increase rapidly with the peak induction, however for practical use and especially when implemented in finite element software, it is assumed to be constant. Consequently, although we identify k hys , k eddy and k exc with data obtained at different frequencies and levels of induction, the model (M1) has a low accuracy at high induction peak and high frequency.
Programming of the identification program The identification program is written under MATLAB ® using the function fgoalattain ()  [22]. This function makes it possible to find local minima of multi-objective problems by using the successive quadratic optimization algorithm. By identifying the parameters of Bertotti's original model (M1), it was realized that the relative errors were too high, especially for samples whose iron losses increase rapidly at high induction. In order to ensure the robustness of the identification process, the points of high induction are removed from the database.
B. Deep CNNs Recent developments in computing power have reduced computation time of CNNs  (Schmidhuber, 2014). At the same time, problems such as over-fitting  (Hinton et al. , 2012) , where a CNN gives good accuracy on training images but not when applied to new unseen images, and vanishing gradients  (He et al. , 2015) , where deep networks with many layers do not converge to a solution during training, have been addressed. This progress allows the construction of deeper networks (more layers) using larger images (e. g. ,  He et al. , 2015 He et al. , , 2016;; Zagoruyko and Komodakis, 2016) , and since 2012, the performance of deep CNNs on common evaluation datasets has surpassed engineered features (e. g. morphology)  (Krizhevsky et al. , 2012)  and is approaching human performance  (Russakovsky et al.
, 2019). At CEREGE, we have also been developing deep CNN classification systems for use in our microfossil sorting machine, MISO. The application is two-fold, firstly we wish to identify images so that the machine can separate the particle into different species or morphotypes for further analysis. Secondly, we want to classify images from large downcore datasets to perform species or morphotype counts and abundance calculations. In this study, we detail our method for automated classification foraminifera images, with application to down-core images sets. The method is applicable to other single particle classification tasks. It consists of four steps: • Acquisition of images. • Curation of a training image set. • Pre-processing the images • Selection, training and evalution of a CNN • Application of the CNN to classify a larger down-core image set.
Regions that are too small or too locally concave, and thus not representative of a foraminifera shape, are removed. The remaining candidate regions are thus considered to represent particles. The centre of mass (CoM), and the maximum radius between the CoM and the perimeter are calculated. Each particle is then segmented by cropping a square image centred at the CoM and with side length approximately 2. 2 times the maximum radius (Figure  1d ). This ensures the particles appear at roughly the same relative size in the images, with enough of a buffer to clearly realise the particle edge feature, as well as enable rotation of the particle within the image without going out-of-bounds. Note that if using images from other sources, e. g. the Endless Forams database  (Hsiang et al. , 2019) , any extra regions with non-photographed information, such as added white regions with metadata text, are removed.
B. Labelling With these caveats in mind, rather than trying to create a single universal foraminifera classifier, we create classifiers (and thus training sets), on a per-core or per-site basis. The idea being to ensure that the CNN is trained on the species or morphotypes that are specific to that core, and use images taken with the same acquisition system and camera parameters. As such, a training set is chosen by selecting a images from a few representative samples, or taking a random subset from the larger down-core image set. Images are labelled with the aide of the ParticleTrieur software developed at CEREGE. To begin with, images are labelled and new classes created as different species or morphotypes are identified. As labelling progresses, the number of images in each class is monitored, and low count classed are checked to make sure that the spectrum of morphological variability is covered.
V. TRAINING The CNNs are trained using the SoftMax cross-entropy loss function on the predicted labels. We use Adam (adaptive moment) optimisation  (Kingma and Ba, 2014)  an initial learning rate of 0. 001, as it this been found to be good starting point for other image sets  (Wilson et al. , 2017). A batch size of 64 is used for most CNNs except for larger networks with bigger input image sizes, where it is reduced to 32 or 16 if there are memory constraints. Training is performed via custom Python scripts and uses the Tensorflow library.
The best performing network is "frozen", whereby trainable variables are replaced with constants, and saved in protobuf format. An XML file is created with metadata about the network, such as the input size and class names, so that all the information necessary to be able to use the network for classification is present, and thus the CNN can be readily shared for with other users. An example XML file is [here]. The frozen network is now ready for use in classification. An optional step is to the train the entire image set (both training and validation) on the best performing network. Since there are no validation images the accuracy cannot be measured, however, one would expect that the extra images should improve classification performance on new images.
VII. CLASSIFICATION Finally, the chosen trained network is used to classify the larger down-core image set. Images in the down-core set are arranged into folders by depth. Each is pre-processed as for training (Section V-A), and passed into the CNN to calculate the final layer SoftMax output. The output is a vector of prediction scores, one for each class, ranging from 0 to 1, with all scores adding to 1. We consider a score above a fixed threshold as a positive classification for the class. If no scores are above the threshold, the image is classed as "unsure". The threshold must be chosen from the range (0. 5, 1. 0] as then only one class will be above the value. We use a threshold of 0. 8. These operations are performed using the ParticleTrieur program developed at CEREGE.
VIII. DEMONSTRATION ON ENDLESS FORAMS In this section we demonstrate the performance of the different CNN topologies and their parameters, as well as the training method, with a hyper-parameter search using the large, publicly available "endless forams" planktonic foraminifera image set  (Hsiang et al. , 2019). The image set consists of 27,729 colour images in 35 species classes, ranging from 4 specimens (Globigerinella adamsi) to 5914 specimens (Globigerinoides ruber) in each class. We excluded five classes, Globigerinella adamsi (4), Globigerinita uvula (7), Tenuitella iota (8), Hastigerina pelagica (13), and Globorotalia ungulata (25) because they had less than 40 images, meaning that only 1 to 8 images are available for validation and thus likely not a reliable measurement. Each image was pre-processed to remove the white metadata panel at the bottom of the image and the black border around the particle, so that only the real photographic part of the image remained, then padded to make them square using the method in Section V-A.
The processed images are available for download from https://1drv. ms/u/s! AiQM7sVIv7fah4UKCwkj2wMbG2doUA?e=QnbK5I Both the transfer learning and full network training approaches were investigated using the CNNs described in Section IV. Training was run using Tensoflow 1. 14. 1 and Python 3. 7, on a Windows 10 desktop computer with NVIDIA RTX 2080 Ti GPU, AMD Ryzen 2700X CPU, Sandisk 970 EVO SSD, and 32GB of RAM.
B. Full Network Training We also investigated full network training of the ResNet18 network and our custom base cyclic and ResNet cyclic topologies, for 64 x 64 and 128 x 128 greyscale images. For our networks, the number of filters in the first block was varied between 4 and 16 III). For both 64 x 64 and 128 x 128 images, the base cyclic design with 16 filters gave the best overall accuracy with 87. 5% while the the ResNet cyclic with 16 filters gave the best class-averaged precision and recall. In all cases, using 128 x 128 images gave higher accuracy, up to 90. 3%. Training time increased proportionally with image size and number of filters, and the ResNet-Cyclic network took longer than the base cyclic network, up to almost 3. 5 hours for one configuration. The long training time is why larger image sizes were not investigated further for this dataset, however, it may feasible for smaller sets of just a few thousand images.
C. Comparison The full network training gave better accuracy than the transfer learning methods at the expense of much longer training times. The likely due to the use of augmentation with the full network training. Using augmentation with transfer learning would require calculation of the vector output every time a training image is fed into the network, rather than a single pre-calculation at the start. Since the inference times of the transfer learning networks were around 2 to 10 times longer than longest for the Base-Cyclic and ResNet-Cyclic full networks, we can infer that using augmentation with transfer learning would result in training times much longer than the already long times for the full networks. The short training time for the transfer learning networks is advantageous for quickly investigating the expected accuracy for a particular dataset.
B. Training Set A training set was constructed from 15274 images of foraminifera from seven representative samples from cores MD02-2508 and MD02-2519. The images from MD02-2519 (not the core of interest) were used as the core is from a similar location to MD02-2508 and contained a very similar benthic foraminiferal fauna, and the images had already been acquired. The training images were manually labelled using the ParticleTrieur software into 12 benthic species, (main species according to  Tetard et al. (2017) ), and an "other-benthic" class (grouping the less abundant benthic species), a single catchall planktonic class, a radiolarian class, and then some nonforaminifera classes such as double (specimens in contact with each other) and fragments (Figure  3 ). Images ranged from 188 × 188 to 1502 × 1502 pixels in size, corresponding to particles 0.
As a consequence, the identification and discrimination of these forms can be difficult even for a taxonomist, and the CNN also mistakes these species more often. This can also explain why E. smithi, which was also not present in the core during the Holocene, did not have a strong false positive bias, as few other classes were confused with it during training. We note some species can be discriminated under a stereoscopic microscope by their flatness, (e. g. B. spissa and B. argentea versus B. subadvena and B. seminuda), which helps for manual identification, but this depth information is lost for the 2D images in our automated approach. This bias is thus largely dataset dependant as out of the 8 main species analysed in this study, four belong to the same genus (i. e. Bolivina). Thus the dataset provides a good case study on the perfomance of a CNN classifier; overall the most of the classes were correctly identified.
g. plastic core liner or sediment) or specifics of the acquisition system (e. g. ring light pattern). The example application shows that very large throughput is possible with an automated system. A few hours labelling a well-constructed training set can save months of time manually counting specimens. Furthermore, the CNN obtained can be re-purposed to aide in constructing other training sets by using the predictions to suggest labels. The software program ParticleTrieur which was used to manually label the training set and automatically classify the down-core set is available with tutorial from http:// particle-classification. readthedocs. io, and the python scripts to train the CNNs in this paper are available from http:// www. github. com/microfossil/particle-classification and can be install using the pip Python package.
Biogeochemical Description The biogeochemical modules of C-GEM and pyNuts-Riverstrahler include similar key state variables and processes in spite of a greater level of complexity of Riverstrahler's reaction network (Figure  3 ). Both models include an explicit description of all nutrients (nitrate, ammonium, phosphates, and dissolved silica), dissolved oxygen, phytoplankton and a representation of the carbonate system. C-GEM only simulates diatoms and non-diatoms, while pyNuts-Riverstrahler described biogeochemical cycles including their interactions with biological compartments, including three functional phytoplankton groups (diatoms, Chlorophyceae, and Cyanobacteria), zooplankton (rotifers and microcrustaceans) and heterotrophic bacteria (small autochthonous and large allochthonous) as well as nitrifying bacteria.
The most significant difference between both models regards the benthic compartment which is ignored in the current version of C-GEM while pyNuts-Riverstrahler explicitly takes into account a benthic module (recently updated:  Billen et al. , 2015)  simulating exchanges with the water column for several variables (benthic organic matter, inorganic particulate phosphorus, benthic biogenic silica). The lack of benthos in C-GEM is implicitly compensated by the assumption that, in the estuary, denitrification especially occurs in the turbidity maximum zone within the water column. Both models use invariant parameterization within their respective simulation domain thus assuming homogeneity of the physiological properties of organisms along the river network  (Garnier et al. , 2002)  but use different parametrizations between the riverine and estuarine sections of the Seine.
(2016b). Finally, both models include similar inorganic carbon module, which allows quantifying the estuarine inorganic carbon dynamics. The latter is a recent development of pyNuts-Riverstrahler described in extenso in  Marescaux (2018)  while its implementation in C-GEM is described in  Volta et al. (2014)  FIGURE 3 | Simplified conceptual scheme of the shared state variables and processes between C-GEM and Riverstrahler models. The two models simulate nutrients as N (NO 3 , NH 4 ), P (PO 4 , Total P), Si (DSi, BSi in Riverstrahler only). In the Riverstrahler, both dissolved (DOC) and particulate (POC) carbon are modeled according to three classes of degradability (rapid, slow, and refractory); heterotrophic bacteria (BAC) are described as large and small, and the zooplankton (ZOO) includes micro-crustaceans and rotifers-ciliates.
Hydrodynamics and Transport The performance of the hydrodynamics module was first evaluated through a series of steady state simulations under varying discharge and tidal conditions (Figure  4 ). A set of six increasing tidal amplitudes from 3. 1 to 7. 7 m were applied at the marine boundary. The selected range of tidal amplitudes at the mouth corresponds to six tidal coefficients (C35, C45, C65, C80, C95, and C115) on a scale ranging from 20 to 120. The smallest (C35) and largest tidal coefficients (C115) represent extreme spring and neap tides, respectively. Two values of fresh water discharges were applied at the riverine boundary condition. The lowest value (250 m 3 •s -1 ) is roughly equivalent to summer moderate regime conditions while the highest value (800 m 3 •s -1 ) is equivalent to a moderately high water regime at Poses (based on long term discharge chronicle available in the Banque Hydro database 1 ).
This paper is structured into four main sections. Music corpus. A corpus of 60 melodies was selected from Baroque, Romantic, and Post-tonal repertoire: 30 instrumental melodies (10 for each style) and 30 vocal melodies (10 for each style). Rating experiment. Perceptual data were collected involving 30 listeners, mainly amateur musicians, who were asked to indicate the most important notes while listening to each melody. Music analysis. A music analysis of the melodies was performed in order to identify the musical properties that could be responsible for the accent marks obtained in the listening experiment. Computational modeling. A set of 43 features was defined and models were formulated that predicted the collected perceptual data from the extracted features.
The total number of melodies was 60, equally divided among the three styles (20 in each) and then further divided between vocal and instrumental (10 in each). We limited the selection to 60 examples since we estimated this to be the highest possible number of melodies that could be rated by the participants in two sessions. The complete list is provided in Supplementary Table  S1. Since we are using machine learning methods for the modeling, it is important to have a large database. From a musicological point of view, we are aware that this is a rather modest selection that cannot be considered a representative sample of these three styles. However, we think that it can represent a reasonable sample and provide a rather large melodic variation in relation to perceived accents.
RECORDING For our experimental purpose, only the melodies were extracted from the scores, disregarding any other parts such as accompaniment. Following our definition of perceived immanent accents, the melodies were rendered using only nominal pitch and note values, disregarding any additional score information, such as crescendo, sforzando, and accelerando marks. The melodies were first coded in mscz format with the free notation software MuseScore 2. 1, and then exported as xml and converted into the Director Musices (DM)  (Friberg et al. , 2006)  mus format by means of the Humdrum Toolkit  (Huron, 2002)  (following the sequence: xml, krn, mus). The mus files were then imported in DM and checked for compatibility. Performance tempi were extracted from examples provided by professional performances of the melodies.
The duration of each extract lasted approximately 30 s. A midi output was also created as an input for the experimental interface (see section "Procedure"). The melodies were played by the computer using DM version 3. 1. 1, without any performance variation on a midi-controlled Disklavier Yamaha C3pro grand piano, and then recorded in audio format. Recordings were made by means of professional recording equipment (RME Fireface 800 audio card, two Brüel & Kjaer 4003 microphones mounted 20 cm above the strings, Apple MacBook Pro computer using Audacity 2. 1. 1 software). To cope with calibration inhomogeneity in the Disklavier and due to the room acoustics in relation to microphone positions, we adjusted the dynamics of each tone of the keyboard by ear, until we obtained uniform perceived dynamics along the whole keyboard range (verification was made by looking at the spectrogram representation in Audacity).
Procedure To perform the task, we specifically developed a computer interface using Matlab R2017a, where the audio was aligned to a corresponding midi piano roll display. A blue bar followed the notes, which were displayed as white rectangles on a black background (see Figure  1 ). This interface was introduced to reduce the bias associated with the score, so the only information about the pieces that was provided to the participants regarded the physical content of the signal [i. e. , note onset and duration (on the x-axis) and pitch (on the y-axis)]. Our interface was provided with a scroll-bar and buttons (play, stop, restart, and next), enabling participants to listen to the melodies as many times as they wanted, as well as to stop the music at any time and/or listen to small portions of the piece, and move to the next piece.
Before moving to the next melody, a pop-up window appeared asking the participant to indicate whether the piece was known or not. At the beginning of the experiment, participants signed a consent form informing them about the purpose and content of the study, ensuring that the procedure did not involve any discomfort or risk, and guaranteeing freedom to withdraw at any time and anonymity. Then the experimenter distributed and read instructions for the graphical interface. Before starting with the experiment, participants were provided with two familiar practice trials (Happy birthday to you and Santa Lucia) to get used to the task and were requested to fill in a form concerning some personal data (nickname, nationality, age, gender, expertise, and location). At the end of the task, they received a questionnaire concerning their education, music preferences, familiarity with the selected music styles, and strategies adopted for selection.
Agreement and Overall Measure of Ratings An important purpose of the data collection was to find a representative overall measure of the accent ratings for each note across all raters or across a selection of raters. This overall measure was then used for predicting accents' positions and saliences in the model below. In this study, averages across participants were used as the overall measure. This resulted in a continuous measure in which most notes had a nonzero accent value (see Figure  2 ). Averaging across participants could be considered standard procedure in listening experiments. However, it does not always represent the majority of the listeners since outliers may have a large impact. An alternative method that was tried was to consider only those notes that had m number of raters who marked that note with an accent mark.
Meter Features 40-43 mark four different metrical levels. The meter analysis was based on the notated meter in the original score. The different metrical levels (beat0-beat3) are computed according to Supplementary Table  S2. These features mark each note that appears on one of these metrical levels with the corresponding label. The list in Supplementary Table  S2  covers most of the basic meters, as well as the more unusual ones found in the database. The marking of the metrical levels is an extended version of  Bisesi et al. (2019). However, the specific salience values are not computed and instead left to the machine learning model for optimization. Since the presented melodies lacked the original accompaniment (as specified in the original score), the perceived meter might not correspond to this notated meter for all melodies.
Prediction Methods and Training In the selection of the prediction methods, we chose to use both a simple and a more advanced model. Due to the relatively small number of features (34 after selection) in relation to the total number of cases (4204), Multiple Linear Regression (MLR) could be used as the first method. MLR has the advantage that it is easy to estimate the contribution from each feature. However, obviously, due to its definition, this method does not account for any interaction among features, something that was hypothesized as being important. Therefore, as a second method we applied Support Vector Machine regression (SVR) using the LIBSVM version 3. 22 package for Matlab  (Chang and Lin, 2011). A radial basis function was used as the kernel, and the model parameters C and gamma were optimized using a grid search.