dataesr/scientific-paragraphs-categorization huggingface dataset:
Signal preprocessing and Fourier transform For the procedure described in this paper, the only parameters that are dataset specific are related to low-level signal preprocessing, and require only a limited amount of knowledge about the repertoire of interest to be set properly. They are the sample duration T , the sampling rate fs and the Fast Fourier Transform (FFT) window size NFFT, which Table  2  reports for each dataset. Note that the maximum frequency for Mel filterbanks was always set to the nyquist frequency, and so is defined by the sampling rate implicitly. This section describes how these parameters were set, along with the full procedure that compiles signals into spectrograms to feed to the auto-encoder. A consideration for setting spectrogram parameters is that auto-encoder's decoders typically reconstruct images by successive factor two up-sampling.
Input spectrograms which will be matched with auto-encoder reconstructions thus need to be of dimensions that follow (k f 2 n × k t 2 n ), with k f and k t integers and n the number of factor two up-sampling (in the proposed architecture n = 5, more details are given in the next section). For all datasets in this study, 128 frequency bins and 128 temporal bins appeared to suffice in containing vocalisation details; all spectrograms were thus set to be of size 128x128 (k f = k t = 4). Note however that if more spectral or temporal bins were needed for a new species, the same auto-encoder architecture could allow to manage 256 bins or more. Preprocessing first consists in extracting the signal surrounding the center of the annotation. The sample duration T was fixed for each dataset. In order to fully contain most vocalisations while avoiding hiding details for small ones, it was set at the 3 rd quartile of all vocalisation durations.
In the case of pre-cut vocalisation files with a smaller duration than this fixed value, signals were zero-padded. The sampling rate was fixed to the most common value in the dataset, and vocalisations with a different sampling rate were resampled to match the common value using the scipy python package  [38]. The exception is the humpback whale dataset, for which the sampling rate was reduced to 11,025 Hz in order to reduce the nyquist frequency and increase the frequency resolution for the relatively low frequency vocalisations. Resulting signals are then z-normalised before its frequency decomposition. The choice of Fourier transform parameters define the spectro-temporal resolution of spectrograms, and need to suit spectro-temporal modulation rates of vocalisations. For this study, FFT window sizes were manually set by quick spectrogram inspections or by borrowing from studies publishing the databases.
128 ). The Fourier transform comes with the limitation of having to choose a fixed spectro-temporal resolution, which wavelet based transforms can help alleviate  [39] , allowing a better representation especially for transient signals. However, choosing such approach would imply a significant increase in computation, and additional dataset specific settings (e. g. , choice of wavelet family or hyperparameters). For the sake of simplicity and usability, and as the studied signals are not transient, we chose the general purpose Fourier transform in our experiments. Following the short term Fourier transform, several frequency and dynamic range compressions were tested. Mel filterbanks with 128 filters between 0 and the nyquist frequency were used, and compared to keeping the spectrogram with a linear frequency layout.
Vocal repertoire discreteness The learnt representation, besides its potential agreement with experts' perception of similarity for categorisation, can help us learn the discreteness of vocal repertoires which varies across animal species  [19]. Indeed, the clusterability of a dataset can be measured using the Hopkins statistic  [46]  (Eq 2), which we can apply to vocalisations' latent distributions  [25]. Hopkins ¼ P m i¼1 u i P m i¼1 u i þ P m i¼1 w i : ð2Þ This measure compares the sum of distances u i between m points sampled from a random distribution and their closest neighbour in a dataset, with the sum of distances w i between m points sampled from the dataset and their closest neighbour. Here, m was set to 100 and a normal distribution was estimated using the dataset's moments. The Hopkins statistic computed on UMAP embeddings is reported for each dataset in Fig  3.
Handcrafted features and deep audio embeddings To have a baseline of comparison for the clustering performances of learnt features against handcrafted ones, we ran experiments using two feature sets commonly found in the litterature, i. e. , whole spectrograms and PAFs. Following Sainburg et al. [25] , we computed 32x32 spectrograms for each vocalisation and used the magnitudes of each spectro-temporal bin as independent features. For the PAFs, we used the set of 18 features used in the latter study, extracted using the Biosound package  [24]. To allow the comparison between these feature extraction methods and the auto-encoder's, for all experiments, sample durations, sampling rates and Fourier window size remained unchanged. Additionally, to get a sense of how relevant is the auto-encoder learning framework, we projected vocalisations using third party neural networks of the Holistic Evaluation of Audio Representations (HEAR)  [47].
This section presents them along with the rationales and empirical findings behind the different choices that were made. Spectrogram configuration. As for spectrogram magnitude normalisation and frequency layouts, no configuration performs better in all cases. This is due to the diversity of frequency ranges and signal to noise ratio found across datasets. The Mel frequency layout appears mostly beneficial, but the effect of dynamic range compression (whether none, PCEN, or logarithmic) is dataset specific (S1 Fig). For its good compromise across datasets and species, we suggest using the Mel frequency layout with logarithmic compression as a first choice, and selected it for the following experiments. Dimensionality reduction. An early intuition of this study was that lower dimensional bottlenecks would lead to the elimination of unnecessary information (e.
We used a grid search to systematically tune these parameters for all datasets and found that the best compromise in performances is for a minimum cluster size of 10; minimum samples of 3; epsilon of 0. 1; and the leaf cluster selection algorithm, except for the humpback whale datasets for which the EOM algorithm is beneficial. This exception aside, across all datasets, these 'generic' settings do not reduce the NMI by more than 4% as compared to optimal ones, and thus were kept for the following experiments. For experimenting with new species, we suggest to start of with these parameters and to inspect clustered embeddings in case some tuning is necessary. insight that Fig 5 reveals is that learning features using an auto-encoder will yield better clustering results (in terms of agreement with expert labels) than using PAFs or whole spectrograms.
Comparison with other representations Deep audio embeddings. Deep learning models sometimes yield embeddings that are useful for tasks that they were not specifically trained for. For instance, the OpenL3 model trained on audio / video correspondence from youtube content can inform on the presence of a queen in a beehive  [47]. Fig 6 reports on the relevance of such audio embeddings for the vocalisation clustering task studied here. Among the three models of the HEAR baseline, OpenL3 model gives the best overall performances. Nonetheless, with the exception of the dolphin whistle dataset, the auto-encoder gives better results than all third party task models. In addition to the HEAR challenge baseline models, we give the performance of autoencoders when trained on all datasets except the one it is tested on.
This strongly alleviates the effort needed to employ the auto-encoder approach on new repertoires: a 'generic' encoder can be used and no dataset specific training is required. Learning vocalisation representations with auto-encoders has been studied in the past for zebra finch and mice  [28] , with an important difference made in the loss function used in training: a variational auto-encoder loss is used. We suggest to not include the gaussian constraint on the bottleneck distribution since it seems counter-productive for samples' clusterability. Rowe et al. [30]  have also studied auto-encoders for bird species clustering. The authors employed a MSE loss and their approach does not outperform handcrafted features (MFCC). In this study, the use of the perceptual loss yields equal or better performing embeddings across datasets.
Presumably, by focusing on the reconstruction error of highly salient spectrogram components, the perceptual loss reduces the amount of information related to background noise found in the bottleneck. Despite using the perceptual loss instead of a regular pixel loss, recordings sometimes contain background noises that create an unwanted variability in embeddings of vocalisations from the same type. Often, these background noises are relatively stationnary, and thus can be easily cancelled by substracting the median of each frequency bins of the spectrogram. For the humpback whale complete dataset (as opposed to its 'small' counter part) the NMI between clusters and expert labels remains around 0. 5. This might be explained by the graded nature of humpback whale unit types, which have been shown to evolve both within and across songs  [53]  (songs are are transmitted culturally, changing at every reproductive season  [17] ).
To challenge this hypothesis, we ran the repertoire discovery algorithm on annotations of the secondary analyst of the study that provided the dataset  [37] , which gather 1,800 vocalisations recorded on a single day. The resulting agreement between clusters and expert labels rises around 0. 7 of NMI, suggesting that the intra-type vocalisation variability (lowered when working with a limited time span) is responsible for poorer clustering results. Vocalisation clustering is an important building block for the modelling of non-human animal communication systems. For this study, we chose to use discrete expert labels of vocalisation types as a proxy to measure the relevance of learnt representations. Being aware of the potential limitations of such ground truths, they were necessary to asses the applicability of the method until more data is collected specifically on animal perception  [54].
Conclusion This paper proposes a framework to assist vocal repertoire discovery using deep representation learning with auto-encoders. It takes as input a set of detected vocalisations and automatically suggests categories by acoustic similarity. This demands only three compulsory manual settings that can be defined with very limited knowledge of vocalisation characteristics: the sample rate, the size of the Fourier transform window, and the signal duration suitable to describe each vocalisation. These settings are used to compute vocalisation spectrograms, which an auto-encoder is then trained to compress. The resulting high-level representation is projected with UMAP and clustered with HDBSCAN. This paper gives hints on how to choose settings for the two latter algorithms depending on dataset characteristics.
We experiment this auto-encoder framework on 8 different datasets with varying amounts of vocalisations, repertoire sizes, SNRs, and frequency ranges. Despite these variations, we demonstrate a relatively good agreement between the unsupervised categorisation and expert labels of vocalisation types, consistently superior to the baseline approach of using whole spectrograms. The two methods are compared both with a metric of information theory (NMI) and with indicators of the effort needed to manually annotate the dataset given clusters. Indeed, the method is not trustworthy for the direct transcription of vocal sequences, but can significantly reduce the human effort needed in characterising and annotating unknown vocal repertoires. The procedure is made available as an open source Python package for the community to use it on animal vocal repertoire studies, including functionalities such as auto-encoder training, interactive browsing of embeddings, clustering, and plotting for efficient annotation.
Acknowledgments We are grateful to the researchers who shared their annotated datasets of humpback whale vocalisations ( Franck Malige ,  Divna Djokic , and  Renata Sousa-Lima ). Humpback whale data collection was possible thanks to the logistic support from  Projeto Baleia Jubarte  sponsored by  Petroleo Brasileiro S. A. (PETROBRAS). We are also grateful to the researchers that provided the dolphin signature whistles ( Laela Sayigh  and  Vincent Janik ). We acknowledge their significant contribution to this paper in this regard.
HerveGlotin received the grants  ANR-20-CHIA-0014  and  ANR-21-CE04-0019 , and  Ricard Marxer  received the grant  ANR-20-CE23-0012-01  from  Agence Nationale de la Recherche. These financed salaries and computers to run experiments. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. To collect the humpback whale data,  Renata Sousa-Lima  received grants from  Fundacão O Boticario de Protecão à Natureza / MacArthur Foundation  and the  Society for Marine Mammalogy (Small-Grantsin-Aid of Research). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Furthermore, both compounds were added to A2780 cells at the same time (0/0 h protocol). Further, we asked whether the properties of these cells in response to CisPt, Res, or CisPt+Res treatments change under hypoxia as compared to normoxia. Subsequently, to assess the combinatorial efficacy of CisPt and Res, defined as synergism, additive effect, or antagonism, we generated dose-effect curves, median-effect plots, isobolograms, and combination index (CI) using the CalcuSyn software v. 2. 0 (Fig-  ures  2A-E and S2A-E, Table  S3 ). According to the data summarized in Table  S3  and pre-   and (E) TOV-112 OC cells for three different cell treatment protocols: both compounds were added at the same time (mix 0/0 h), Res given 2 h before CisPt (mix 2/0 h), or given 2 h after CisPt (mix 0/2 h).
Assessment of Apoptosis The PE Annexin V Apoptosis Detection Kit (Becton Dickinson) was used to detect apoptosis in cells treated with CisPt at IC 25 dose (5. 9 µM), Res at IC 25 dose (63 µM), and CisPt+Res (at the same concentrations) in normoxia and hypoxia, as described in Section 4. 4: Establishment of Hypoxic Culture Conditions and Experimental Outline. After 48 h incubation with drugs, cells were harvested and stained according to the manufacturer's protocol. Then, the samples were analyzed with a FACSCalibur flow cytometer (Becton Dickinson), equipped with 355 nm, 405 nm, 488 nm, and 640 nm lasers. Data were analyzed using FCS Express 4 Flow software.
Cell Cycle Analysis A2780 cells were seeded in 6-well plates at 7 × 10 4 per well and then treated with CisPt, Res, and CisPt+Res in normoxia and hypoxia as described above. After a 48 h incubation with drugs under normoxic and hypoxic conditions, the cells were fixed in 70% ethanol, and stored at 4 • C overnight. Analysis of cell cycle was performed by flow cytometry after staining by propidium iodide (Becton Dickinson) for DNA content using a BD FACSCalibur flow cytometer (Becton Dickinson), equipped with 488 nm and 633 nm lasers. Cell fractions in G0/G1, S, and G2/M phases were calculated with Modfit software (Verity Software House, Topsham, ME, USA).
Measurement of Reactive Oxygen Species (ROS) Intracellular ROS levels were quantified using a fluorescent probe of 2 ,7 -dichlorofluorescein diacetate (DCFDA, Sigma-Aldrich). Briefly, A2780 cells were seeded into 6-well plates at the 7 × 10 4 density and were treated with CisPt, Res, and CisPt+Res for 1 h and 48 h in normoxia and hypoxia as described above. Then, cells were harvested and stained with 5 µM DCF-DA for 30 min at 37 • C. Next, they were washed with PBS and the signal was measured using a CytoFLEX flow cytometer (Beckman Coulter, Brea, CA, USA), using the excitation at 488 nm and emission at 780 nm. Analysis was performed using CytExpert software v. 2. 3. 0. 84 (Becton Dickinson). The mean MFI of intracellular ROS was normalized to the normoxia control as 1. Experiments were performed in triplicates.
000; Vector Laboratories, Burlingame, CA, USA) for 1 h at RT. Signals were detected by enhanced chemiluminescence substrate (Santa Cruz Biotechnology) using X-ray films (Carestream, Rochester, NY, USA). The band intensity was quantified using ImageJ software. Band intensities were normalized to the intensities of their corresponding loading controls (Vinculin) and relative fold changes were calculated after normalization to the normoxia control as 1. Experiments were performed in triplicates.
Cell Migration Assay A2780 cells were seeded in a 24-well plate at 4 × 10 3 cells per well and were treated with CisPt, Res, and CisPt+Res in normoxia and hypoxia, as described above. The cell migration was recorded for 48 h at 20 min intervals at 5× magnification using a Zeiss Axio Observer microscope Z1/7-software Zen v. 2. 6 blue edition (Zeiss, Oberkochen, Germany). The velocity of cells was analyzed using the previously described method developed at the Jagiellonian University (Cracow, Poland) by the Cell Biology Department of the Faculty of Biology Biophysics and Biotechnology  [90] [91] [92] [93]. The movement of 50 single non-proliferating cells treated with CisPt or/and Res or the control cells under normoxic and hypoxic conditions was analyzed. The designated time points (5 h, 48 h) represent the average speed of cell movement in 5 h, e.
Data Analysis Data were expressed as means ± SEM (standard error of the mean) for at least three biological experiments. To calculate IC 50 values of CisPt and Res alone or in their combination, the combination index (CI) values from CalcuSyn software (BioSoft, v. 2. 0) were used. Assessment of the normality of the distributions of the data was evaluated using the Shapiro-Wilk test. To estimate the effects of CisPt and Res alone and in combination on changes in IC 50 values, two-tailed Student's t and Mann-Whitney U tests were used. For comparison between normoxia-and hypoxia-treatment groups, GraphPad Prism software v. 9. 0 (GraphPad Software, San Diego, CA, USA) was used, using one-way ANOVA or two-way ANOVA followed by a Tukey's post hoc test (for Gaussian distribution). For non-Gaussian distribution, the non-parametric Mann-Whitney U test or Kruskal-Wallis was used, followed by a Dunn's post hoc test.
This research was funded by subvention of the  Ministry of Education and Sciences  grant no  425/2016  and National Science Center grant no  2016/23/B/NZ1/03211. A. B. , A. F. -D. and K. W. were supported by  European Social Fund : POWER "Next generation sequencing technologies in biomedicine and personalized medicine". During a part of this project, J. Z. K. was supported by the  Polish Ministry of National Defense  with grants "Kościuszko" 571/2016/DA and 508/2017/DA. Institutional Review Board Statement: Not applicable.
Methods We simulate the dynamics of the Enceladean ocean using the pseudo-spectral code PARODY 2 , which solves for the governing equations for thermal convection in a rotating 3-D spherical shell in the Boussinesq approximation. We neglect any topography at the bottom of the ice shell. Equations are solved using finite differences along the radial direction with N r radial levels. At each radial level, fields are expanded in spherical harmonics up to degree and order ℓ max. We run a total of 21 simulations with resolution up to (N r , ℓ max ) = (260, 340) (table  S1 ). We assume no longitudinal symmetries and hyperdiffusion is not employed. We impose no-slip conditions for the velocity at the boundaries. We prescribe a heterogeneous heat flux (figure S1) of mean value q i at the bottom boundary and a fixed temperature at the top.
Acknowledgements Simulations were performed on the  CCIPL facility  ( Centre de Calcul Intensif des Pays de la Loire, Nantes, France). This study acknowledges the financial support from the  French Agence Nationale de Recherche , project  ANR   COLOSSE  (grant  N  •  ANR-2020-CE49-0010 ). F. T-N. is supported by the  Centre National des Études Spatiales (CNES, France).
Statistical analysis Mechanical characterization and ex vivo migration are presented as box plots (min to max value with a line presenting the median). In vitro MSC migration area and tissue volume in % are presented as mean AE standard deviation (SD). For each of the hydrogel groups (THA, GelMA, norHA) the results of the two modifications were compared. All statistical analyses were performed using SPSS (version 28. 0. 1. 0, IBM Corporation, USA). Student t-tests were performed for mechanical characterization, in vitro MSC migration and in vivo tissue formation and two-sided p-values are reported. In case the variances between the two modifications were significantly different, p-value of equal variances not assumed is reported and marked with a *. Mann-Whitney U tests were used to assess statistical significance in the ex vivo migration scores.
Acknowlwedgements This work received financial support from the  AO Foundation  through the  Osteochondral Defect Collaborative Research Program  ( AO-OCD Consortium   TA1711481 :  Osteochondral Bone Repair with Innovative Tissue Engineering and 3D Bioactive Composite Scaffold ). J. Xu  was supported by a  CSC scholarship  (grant number  202006370090 ,  2020 ). The authors like to thank  Dirk Nehrbass  and  Nora Goudsouzian  ( AO Research Institute Davos ) and  Nicole Kops  ( Erasmus MC ) for their support with histological processing and imaging of samples. hMSCs were provided by  Department of Orthopedics and Sports Medicine, Erasmus MC Rotterdam.
PCR and sequencing Samples were tested for circoviruses using nested PCRs targeting the Rep coding gene: first, the PCR developed by Halami was used  (Halami et al. 2008) , and later the one developed by  Li et al. (2010). The PCR products of the second round were sequenced using the Sanger method and the sequences were deposited in the NCBI Nucleotide database under accession numbers OP380746-OP380799.
Phylogenetic analyses The sequences were assembled and translated into amino acid sequences using Geneious. To facilitate proper phylogenetic placement, where available, the full Rep amino acid sequences of the reference strains were used and aligned with the partial coding sequences to infer the tree. The strains were first typed in a preliminary phylogenetic tree reconstruction: the entire class Arfiviricetes was represented, and here all the virus strains we analysed were clustered into the order Cirlivirales. Thus, the second analysis focused only on this order. The reference sequences were derived from the ICTV Virus Metadata Resource v37. 2 database and were supplemented with CRESS1-5 sequences based on the publication of  Kazlauskas et al. (2019). An additional 195 bat-associated cirlivirus strains were included in the analyses  (Dhandapani et al.
2. The initial methionine amino acid of the Cap protein was replaced with serine in strain GT757B, so, to compare Cap structures, the complete protein was modeled from both strain GT757B and strain Acheng30 (ASU92176), both of which belong to species Bat associated circovirus 10, using Alpha-Fold2  (Jumper et al. 2021). For each protein, five models were generated and scored using the protein quality assignment module of AlphaFold2, and the best one was used for visualization (Schrödinger 2021). Molecular graphics were generated using VMD version 1. 9. 3  (Humphrey et al. 1996).