dataesr/scientific-paragraphs-categorization huggingface dataset:
ACKNOWLEDGMENTS We are grateful to  Marine Beinat ,  Julie Rivière ,  Emmanuelle Rebours ,  Jordi Estellé ,  Caroline Morgenthaler  and  Arni Janssen  for participating in the sample collection and organization during the project. We particularly thank  Valentine Ballan  and  Fanny Blanc  who helped to isolate the blood RNA. We also thank  Catherine Philippe  for the SCFA analysis in feces,  Diane Esquerré  who prepared the libraries and performed the MiSeq sequencing at the  GeT-PlaGe genomics core facility (INRAE, Toulouse , https://get. genotoul. fr/)and  Estel Blasi , who created the picture of horses. Lastly, we are grateful to the  INRAE MIGALE bioinformatics facility (MIGALE, INRAE, 2020, Migale bioinformatics facility , doi:  10. 15454/1. 5572390655343293E12 ) for providing computing and storage resources and to all the horse owners, riders and endurance race organizers who participated in the study.
DATA AVAILABILITY STATEMENT The datasets presented in this study can be found in online repositories. This data can be found here: Microarray expression data: Gene Expression Omnibus (GEO) repository under the accession number GSE163767 (https://www. ncbi. nlm. nih. gov/geo/ query/acc. cgi?acc GSE163767) Metabolomic data: NIH Common Fund's Data Repository and Coordinating Center UrqK1489; (http:// dev. metabolomicsworkbench. org:22222/data/DRCCMetadata. php? Mode Study&StudyID ST000945) Gut metagenome 16S rRNA targeted locus data: DDBJ/EMBL/GenBank under the accession KBTQ00000000, version KBTQ00000000. 1; (locus KBTQ01000000). The corresponding BioProject is PRJNA438436, and the accession numbers of the BioSamples included in it were SAMN08715709 to SAMN08715760.
In this article, we propose a new method of sound transformation based on control parameters that are intuitive and relevant for musicians. This method uses a variational autoencoder (VAE) model that is first trained in an unsupervised manner on a large dataset of synthesizer sounds. Then, a perceptual regularization term is added to the loss function to be optimized, and a supervised fine-tuning of the model is carried out using a small subset of perceptually labeled sounds. The labels were obtained from a perceptual test of Verbal Attribute Magnitude Estimation in which listeners rated this training sound dataset along eight perceptual dimensions (French equivalents of metallic, warm, breathy, vibrating, percussive, resonating, evolving, aggressive). These dimensions were identified as relevant for the description of synthesizer sounds in a first Free Verbalization test.
A last complexity of this mapping comes from the non-linear relationship between acoustic and perceptual spaces. After decades of linear models and regressions to correlate both spaces  (Garnier et al. , 2007; Fritz et al. , 2012) , more recent studies have started using machine learning methods, in particular deep neural networks (DNNs) such as autoencoders (AEs) or generative adversarial networks (GANs)  (Goodfellow et al. , 2016) , to model and synthesize audio  (Colonel et al. , 2017; Engel et al. , 2017; Roche et al. , 2019; Donahue et al. , 2019; Engel et al. , 2019). In particular, AE-based methods identify a limited number of latent dimensions underlying the physical space of a training dataset. The extracted latent space is then used as a control space for creating or hybridizing new sounds  (Engel et al.
, 2017). As an alternative to the classic AE, the variational autoencoder (VAE) introduced by  Kingma and Welling (2014)  and  Rezende et al. (2014)  has also been applied to musical sound modeling (e. g. Çakir and Virtanen, 2018; Esling et al. , 2018; Roche et al. , 2019; Girin et al. , 2019). It can be seen as a probabilistic extension of the (deterministic) AE. Specifically, a prior distribution is used to structure (or regularize) the extracted latent coefficients and thus encourage dimensions to be mutually orthogonal. Since however they do not easily relate to perception  (Esling et al. , 2018; Roche et al. , 2019) ,  Esling et al. (2018)  investigated how to force the VAE latent space to match the topology of a perceptual timbre space using a fully-labeled dataset of orchestral instruments and an extra regularization term in the VAE objective function.
This is presented in Section 3. 1. So far, very few previous timbre studies have focused on purely synthetic sounds that do not imitate orchestral instruments  (Lichte 1941; von Bismarck, 1974; Miller and Carterette, 1975; Grey and Moorer, 1977; Samson et al. , 1997; McAdams et al. , 1999; Kendall et al. , 1999; Zacharakis, 2013)  and they were not necessarily interested in their verbal description, or they mostly relied on English labels selected a priori. (ii) Creating a dataset of 80 synthetic sounds rated by human listeners along these main perceptual dimensions (this is presented in Section 3. 2), and using these perceptual ratings to regularize a VAE model and force its latent space to follow the identified perceptual dimensions. To do so, we used an extra term to perceptually regularize our model, in line with  Esling et al.
(2018). However, in contrast to this previous study, our regularization aims at encouraging each individual latent dimension to drive one of the perceptual dimensions. Furthermore, due to the moderate size of our dataset (of purely synthetic sounds), our study relies on a weakly-supervised method whereas  Esling et al. (2018)  used bigger sound datasets (of orchestral instruments coming from some of the above-listed MDS studies), which enabled them to use a fullysupervised method instead. Also, compared to  Esling et al. (2020) , our perceptual regularization relies on continuous values and does not depend on the synthesizer's engine (since sounds remain synthesized directly from the VAE latent space as by  Esling et al. (2018) ). The overall methodology of sound transformation with a VAE model is presented in Sections 2.
In the VAE methodology, it is approximated by a tractable parametric inference model q φ (z|x) which acts as the probabilistic encoder. This model is generally similar in form to the decoder: ( ) ( ) f f f = 2 | ; ( ),diag{ ( )} , q N z x z x x m s ( 3 ) where µ φ (x) ∈ ℝ L and 2 ( ) L f + Î x  s are non-linear functions of x implemented as the output of the so-called encoder network. The encoder network is also a feed-forward DNN, here parameterized by φ. It is illustrated in the left part of Figure  2. The marginal log-likelihood of a data vector log p θ (x) is also intractable. The training of the VAE model, i. e. the estimation of θ and φ, is therefore done by maximizing a tractable lower-bound of log p θ (x) over a large dataset of vectors x. It is shown by  Kingma and Welling (2014)  that this lower bound, called the variational lower bound (VLB), is given by (for an individual vector x): ( )  where D KL (•ǁ•) ≥ 0 denotes the Kullback-Leibler (KL) divergence between two distributions.
2-step learning procedure Since the labeled subset of sounds resulting from our listening test is limited in size (see Section 3), we could not use supervised training. We therefore focused on a semisupervised approach. Hinton and Salakhutdinov (2007)  presented a semi-supervised learning method operating in two steps: the first step is to train the model in an unsupervised manner, using all available data to extract "sensible, high-level features. " The second step consists in refining the model (i. e. fine-tuning) using only the labeled data. Following both  Hinton and Salakhutdinov (2007)  and  Esling et al. (2018) , we investigated the use of a 2-step learning procedure to add perceptual regularization to a VAE model. In a first step, the model is trained in an unsupervised manner, using both the unlabeled dataset U and the labeled dataset L  , and maximizing the weighted VLB of Eq.
(  5 ). Then, the VAE is fine-tuned, using the labeled dataset L  only, and maximizing the regularized VLB of Eq. (  6 ). Thus, the proposed methodology can be summarized as: 1. Unsupervised pre-training: Maximize ( , , , )   x  f q b from Eq. (  5 ) on U L    , 2. Supervised fine-tuning: Maximize ( , , , , ) x  f q b a from Eq. (  6 ) on L .
Stimuli First, a large audio dataset (referred to as the ARTURIA dataset) 2 was created, consisting of 1,233 audio samples generated with varying ARTURIA software applications, 3 having the same pitch (E3, 165 Hz), a similar duration (2 to 2. 5 seconds) and normalized in loudness. Fifty stimuli were selected from the ARTURIA dataset to cover as broadly as possible the range of acoustic variation of these sounds. To this end, a k-means algorithm was applied from the acoustic characterization of the dataset, using 12 classical audio descriptors. 4 The 50 stimuli were then randomly chosen from the 50 clusters obtained. Finally, a subset of 20 stimuli was assigned to each participant, half of which was shared with the preceding listener. Thus, each of the 50 stimuli received a comparable number of evaluations (21.
Stimuli Eighty stimuli were selected from the ARTURIA dataset so as to be as representative as possible of its acoustic space (by applying again a k-means algorithm to its acoustic characterization, as described in Section 3. 1. 1). These stimuli were split into a training subset (10 stimuli) and a main subset (70 stimuli). The 10 training stimuli were always presented at the beginning of the test so that participants could become familiar with the perceptual dimensions. They were followed by 30 additional stimuli, 25 of which were randomly selected from the main subset and five from the training subset (in order to evaluate the intra-listener agreement). On average, the samples of the main subset were evaluated by 26 different participants and samples from the training subset received 36 different evaluations.
Waveform reconstruction was achieved by combining the resulting magnitude spectrogram with the original phase spectrogram, and then applying inverse STFT with overlap-add and optionally the Griffin & Lim algorithm (see Section 2. 1). Our experiments were conducted using the entire ARTURIA dataset described in Section 3. 1. 1 and containing 1,233 synthesizer sound samples. This dataset was split into a training set (80%) and a testing set (20%). The unlabeled dataset U consisted of the normalized Table  2 : Intra-and inter-listener agreement on the eight perceptual dimensions, for the second perceptual test. The first two columns give information on the intra-listener agreement: the average Pearson's coefficient R and the percentage of participants showing an R > 0. 5. The four last columns report the levels of inter-listener agreement observed over the whole group of participants and for each group of participants identified from the HAC analysis (the percentage of participants in each group being reported in brackets).
log-magnitude spectra of the unrated sounds, computed as described above. The labeled dataset L  was composed of the normalized log-magnitude spectra of the 80 sounds rated in the VAME test (see Section 3), associated to the 80 corresponding perceptual score vectors (PSVs) (note that the PSV of a sound is used to label all the successive spectral vectors that compose the normalized log-magnitude spectrogram of that sound). To maximize the use of the labeled data, when dividing the dataset into training and test sets, we ensured that all the 80 labeled sounds were contained in the training set and not in the test set. Following the methodology explained in Section 2. 3, the unsupervised pre-training was made using U L    , composed of 243,861 513-dimensional spectral vectors and the supervised fine-tuning was performed using L  , containing 16,011 vectors.
Regularized VAE implementation Considering the results reported by  Roche et al. (2019)  and other similar experiments that we conducted with the ARTURIA dataset, we focused in this experiment on a VAE model of the form  [513, 128, enc, 128, 513]  (this vector contains the number of neuron units on the successive layers). We investigated different values for the encoding dimension enc ranging from 8 to 100. Note that 8 the size of the perceptual space evidenced in Section 3, so when enc = P = 8 we have z 1:P = z in Eq. (  7 ). We used tanh and linear activation functions for the hidden and output units respectively. β was empirically set to 0. 25 so that the reconstruction term and the conventional KL regularization term were in the same range. Concerning the weighting coefficient α for the proposed perceptual regularization, we tested three values: 0.
01, 0. 1 and 1. All models were implemented in Python using the Keras toolkit 10 with tensorflow backend  (Abadi et al. , 2016). The implementation of the perceptually-regularized VAE is mainly based on the implementation of the VAE provided in the Keras VAE tutorial, for which we added the perceptual regularization loss described in Eq. (  6 ). The training was performed using the Adam optimizer  (Kingma and Ba, 2015)  with a learning rate of 10 -3 and with a batch size of 512. For the unsupervised pre-training phase, 20% of the training set was kept for validation and an early stopping criterion with a patience of 30 epochs was used on the validation loss to avoid overfitting. During the fine-tuning stage, the model was forced to train for 600 epochs.
Organization of the latent space To analyze the organization of the latent space, we first computed the Spearman correlation coefficients (SCC) between the extracted latent dimensions obtained using samples from the labeled dataset. The first column of Figure  5  shows that the classic VAE presents low correlations between the different dimensions extracted. The second column shows that the perceptual regularization significantly affected the eight constrained dimensions, while preserving the others. The bottom chart of Figure  5  shows the SCC for the perceptual score vectors collected during the perceptual test of Section 3. 2. We can see that the structure of the eight constrained VAE latent dimensions (second row) closely matches that of the PSVs, which shows the effectiveness of the perceptual regularization.
Stimuli For this test, we focused on the five perceptual dimensions that are more related to spectral characteristics: métallique, chaud, soufflé, qui vibre and agressif. We did not consider the dimensions percussif, qui résonne and qui évolue because our first attempts at controlling these dimensions with a VAE were not convincing, certainly due to the fact that they were more related to sound temporal dynamics and therefore poorly captured by the static VAE model. For each of these five dimensions, we selected 12 sound samples. Six of these were selected from the labeled dataset (train): the three samples that received the highest mean rating (close to 1) on this perceptual dimension, with the lowest standard deviation (thus considered as very representative of this dimension), and the three samples that received the lowest mean rating (close to -1), with the lowest standard deviation (thus considered as unrepresentative of that dimension).
Six other samples were randomly selected from the test dataset (unlabeled) to evaluate the generalization ability of the model. The 60 selected sound samples (12 stimuli per perceptual dimension × 5 dimensions), were then transformed applying the analysis-modification-synthesis method described in Section 2. 1. The modification consisted in adding a predefined constant offset to the trajectory of the latent coefficient corresponding to the perceptual dimension to be modified. From the results presented in Section 4. 3, we decided to use the perceptually-regularized VAE with a  [513, 128, 64, 128, 513]  architecture, β = 0. 25 and α = 0. 1. Since the modification of the latent coefficients can lead to significant changes in the decoded spectrogram, the waveform signal was reconstructed with the Griffin & Lim algorithm  (Griffin and Lim, 1984).
Conclusions In this study, we proposed a new method to perceptually regularize a variational autoencoder (VAE) model, using a semi-supervised learning procedure. This method allowed  us to obtain perceptually relevant control parameters for the transformation of synthesizer sound timbre. To our knowledge, no clear definition of relevant perceptual dimensions and corresponding verbal descriptors has already been provided in the literature for synthesizer sounds. We therefore conducted a first perceptual test to identify these relevant perceptual dimensions and their associated labels in French: métallique, chaud, soufflé, qui vibre, percussif, qui résonne, qui évolue, agressif. We then conducted a second VAME test to get an entire sound dataset rated along these perceptual dimensions. We then used these ratings as perceptual score vectors for the regularization of our VAE.
Our experiments allowed us to draw several conclusions. As expected, we first observed that the additional regularization slightly degraded the quality of the audio signals generated by the model but that this quality remained acceptable when choosing an appropriate weighting factor α. This issue may be overcome in the future by considering bigger datasets (both labeled and unlabeled L  and U ). We also observed that using this extra regularization increased the interpretability of the constrained dimensions and modified their behavior to relate closely to the perceptual ratings obtained from our listening test. Finally, we conducted a last perceptual test to get a preliminary evaluation of how well the perceptually-regularized model performed sound transformation. This experiment validated the proposed methodology, showing that the model was relatively good at capturing and modifying the acoustic properties of the dimensions agressif and qui vibre.
Notes 1 The choice of the exact data representation to feed the VAE model will be detailed further in Section 4. 1. 2 This dataset is publicly available at http://doi. org/10. 5281/zenodo. 4680486. 3 Sounds were generated from all factory presets of the ARTURIA software applications resulting in singlepitched sounds with a sample rate of 44. 1 kHz. 4 Attack time, attack slope, decay time, jitter, shimmer, MFCC, spectral centroid, spectral bandwidth, spectral contrast, spectral flatness, spectral roll-off and zerocrossing rate; see  Peeters et al. (2011)  for a review of commonly used audio descriptors and their implementations. 5 The participants freely agreed to participate in the three online perceptual tests. No identifying information was asked or stored. 6 Which could be translated to English as metallic, warm and bright.
Flow cytometry analysis of cell populations Cell phenotyping from 6-to 9-week-old mice was performed on thymus, spleen, and bone marrow using the following antibodies: CD4, CD8, CD25, CD28, CD44, CD69, B220, CD19, CD43, and IgM (all from Sony Biotechnology, using respectively PECy7, FITC, PerCPCy5. 5, PE, BV510, APC, PE, PECy7, FITC, APC fluorophores). Intracellular IgM expression in E18. 5 FL cells was performed as previously described  (Abramowski et al. , 2018)  using CD19, B220, CD43, IgM for extracellular staining followed by cell fixation and permeabilization (Invitrogen) and intracellular IgM staining (all antibodies from Sony Biotechnology, using respectively PECy7, BV605, PE, FITC, and APC). Cells were recorded by FACS LSR-Fortessa X-20 and analyses were performed with FlowJo 10 software.
After washing, nuclear staining was achieved by incubation with 4′-6-diamidino-2-phenylindole (DAPI) to quantify apoptosis induction by the detection of pyknotic nuclei  (Roque et al. , 2012). Slides were mounted under Fluoromount (Southern Biotechnologies Associates). Tissues were examined under a fluorescence microscope (50i, Nikon, Japan) with a 10× (NA = 0. 3) objective in three channels (appearing red, green, and gray) as separate files. These images were then stacked with Photoshop software (Adobe).
Statistical analysis Non-parametric Mann-Whitney test or two-tailed Fisher's exact test was performed with α-risk = 0. 05. p-Values were taken to be significant: *significant 0. 05≥p>0. 01; **very significant 0. 01≥p>0. 001; ***highly significant 0. 001≥p>0. 0001; ****highly significant p≤0. 0001. Using G*Power software (https://www. psychologie. hhu. de/ arbeitsgruppen/ allgemeine-psychologie-und-arbeitspsychologie/ gpower), we determined that a minimum of 47 newborns was necessary to conclude to an embryonic lethality of the Xrcc4 M61R -Nhej1-/-double KO with a power of 80% and a p-value < 0. 05.
Acknowledgements This work was supported by institutional grant from  INSERM  and  Université de Paris , state funding from the  Agence National de la Recherche  under ' Investissements d'avenir' program  ( ANR-10-IAHU-01 ) and grants from  Institut National du Cancer  ( PLBIO 16-280 ) et  Ligue Nationale contre le Cancer (Equipe Labellisée ). BR received fellowships from  INSERM ,  ARC , and LNCC.
Protein Preparation The protein structure of Phosphodiesterase 9A was downloaded from the protein databank with Pdb id:4QGE  [24]. PDE9 (Zn/Mg) was used from the above system, PDE9(Mg/Mg) and PDE9(Zn/Zn) developed by replacing Zn with Mg. Protein structures were prepared by correcting the bond orders, adding missing hydrogens, optimizing H-bond with protonation states of residues at pH 7. 0, and restraining minimization for added hydrogens using OPLS2005 forcefield by protein preparation wizard  [25]. We have performed DFT calculations for the ligands to minimize the geometry to get equilibrium bond lengths, bond angles and torsional angles for MD simulation. The charges derived using DFT methods are of high quality compared to other existing autogenerated programs based on low-level semiempirical methods.
Molecular Dynamics Simulations The stability of the complexes was studied by MD simulations using Gromacs2019  [30]. The Amber99sb-ildn force field was used for the protein parameters  [31]. The protein-ligand complexes were solvated explicitly using the TIP3P water model inside the cubic box, and its size extends 0. 1 nm away from the protein on the edges of the box in each direction. The system's overall charge was neutralized by adding a 0. 1 M salt concentration (Na + Cl -). All the simulation was carried out in the GPU enabled Linux clusters. The entire system was minimized until the maximum force was less than 10 kj/mol with the maximum steps of 50,000. The system was then equilibrated for 5 ns under NVT conditions with temperature coupling for two separate groups protein-ligand and water-ions, at 300 K.
Statistics Statistical analyses were performed to assess the changes between the PRE, POST, and 24 and 48 h measurements. Due to the limited sample size of our study and the absence of a normal distribution of the data according to Shapiro-Wilk tests, we performed non-parametric Friedman analysis of variance. Statistical differences between time points were assessed using Dunn's multiple comparisons test. The Friedman test statistic χ 2 and the effect size (Kendall's W test value) are reported. The Kendall's W coefficient value range from 0 (no relationship) to 1 (a perfect relationship). Missing data were imputed using the MissMDA R package Correlations were performed between CMJ height loss at 24 h and changes in CK, Mb, myomiRs and muscle soreness. Indeed, as muscle function (i. e. CMJ height) loss at 24h is considered to be the best indirect marker for evaluating the magnitude of exercise-induced muscle damage  11  , any significant correlation suggests that the associated measure could be useful to approach the magnitude of muscle damage.
The level of association between parameters was assessed using Spearman's rank correlation coefficient (ρ). Significance was defined as p < 0. 05. Analyses were performed using GraphPad Prism (Version 8. 4. 3, GraphPad Software, CA, USA) and R (version 3. 6. 1, R Core Team (2017). R: A Language and Environment for Statistical Computing, available at: https://www. R-project. org/). All data are presented as the means ± standard deviation (SD) throughout the manuscript.