dataesr/scientific-paragraphs-categorization huggingface dataset:
Filtering and clustering using Seurat Single-nuclei RNAseq analyses were performed using Seurat v4. 1. 0  (Butler et al. , 2018; Stuart et al. , 2019)  and Rstudio v1. 4. 1717. Aligned snRNAseq datasets were filtered to retain only nuclei expressing between 200 et 5000 genes and expressing less than 2% of mitochondrial genes and 1. 5% of ribosomal genes. Contamination from myogenic cells were removed from the analyses. After filtering, we obtained 1378 nuclei from uninjured periosteum, 1634 from day 3 post-fracture, 2089 from d5 post-fracture and 1112 from day 7 post-fracture. The replicates of the uninjured dataset were integrated using Seurat. The integrated dataset was regressed on cell cycle, mitochondrial and ribosomal content and clustering was performed using the first 15 principal components and a resolution of 0.
5. SSPC/fibroblast and osteogenic cells were isolated and reclustered using the first 10 principal components and a resolution of 0. 2. Uninjured, d3, d5 and d7 datasets were integrated using Seurat. The integrated dataset was regressed on cell cycle, mitochondrial and ribosomal content. Clustering was performed using the first 20 principal components and a resolution of 1. 3. SSPC, IIFC, chondrogenic and osteogenic clusters from the integration were isolated to perform subset analysis. The subset was reclustered using the first.
Acknowledgements This work was supported by  ANR-18-CE14-0033 ,  ANR-21-CE18-007-01and NIAMS R01 AR072707  to C. C. and  Ted Miclau. S. Perrin  was supported by a  PhD fellowship  from  Paris Cité University. We thank  O. Pellé  from the  Flow Cytometry platform at Imagine Institute  and all the staff from the  Imagine genomic core facility at Imagine Institute.
unique trajectory emerging from Sca1+ SSPCs. Periosteal Sca1+ SSPCs become IIFCs, a state marked by a decreased expression of stemness markers and a strong expression of extracellular matrix genes, such as Postn and Aspn. Following this unique fibrogenic step, IIFC undergo either osteogenesis or chondrogenesis. This newly identified transient IIFC state represents the crossroad of bone regeneration and SSPC differentiation. In depth analyses of the IIFC phase and the mechanisms regulating cell fate decision are needed as IIFC might represent the ideal target to enhance bone healing and potentially treat bone repair defects. Overall, our study provides a complete dataset of the early steps of bone regeneration. Further analyses will establish the mechanisms regulating pSSPCs cell fate decisions and the role other cell populations play in the fracture environment.
Diagnostic assessment All SPECT images from 30 s to 10 min post-injection were spatially normalized to the MNI (Montreal National Institute) space using a dedicated template. This template was created using the 14 frames of 30 s: they were (i) co-registered, to correct for potential movement artefacts, (ii) averaged, and (iii) spatially normalized to the MNI template provided by the SPM12 software (Wellcome Department of Cognitive Neurology, University College, London, UK). All SPECT images were also normalized in intensity using proportional scaling and a Gaussian post-filter of full width at half maximum (FWHM) of 12 mm was applied at the end of the preprocessing. Seed based correlation analyses  (Lee et al. , 2008)  were performed to identify within the set of SPECT images the well-known resting-state networks: default mode, executive, salience, sensory-motor and visual  (Malpetti et al.
To illustrate this point, a comparison of our identified resting-state networks with BOLD-based resting-state networks obtained from a large database  (Schaefer et al. , 2018)  is provided in Supplementary Figure  1. From a methodological point of view, an original scheme of administration combining a bolus followed by a constant infusion was recently proposed for fPET with 18 F-FDG  (Jamadar et al. , 2022). Given the short time for accumulation in the brain, fSPECT with HMPAO should also be tested with a different radiotracer administration scheme. It is also known that the normalization process is a crucial step when dynamic images are used to derive single subject molecular connectivity and can lead to different results depending on the choice of normalization  (Wehrl et al. , 2013; Schaefer et al.
2  Some common suggestions on these forums include using image retrieval to source artworks (from websites like Google Images or DeviantArt) or using design tips and image editing software to produce DIY album art. However, these strategies risk copyright infringement and are limited by musicians' design skills and access to relevant software. To this end, we propose the use of state-of-the-art image generation and styling techniques in computer vision and deep learning, and we add an interface aimed to be fast, convenient, and intuitive for users. We describe a system architecture that relies on user-submitted music and lyrics as input for two models: a generative adversarial network (GAN)  [4]  text-to-image model, which generates seed images from each line of the lyrics, and a style transfer model, which uses information about the dominant moods in the audio file to pull related "emotional" images from a previously assembled database and layer them onto the seed images.
Hepburn et al. addressed the limitation above by using a DC-GAN  [9]  model. This model was trained on the One Million Audio Cover Images for Research (OMACIR)  4 dataset to automatically synthesize album covers  [10]. They also experimented with the AC-GAN  [11]  model to generate album arts for a specific genre. Visualyre adopts a similar approach to synthesize album covers. However, instead of using musical genres as input, it uses a set of lyrics and an audio file from a user-provided song. A GAN model synthesizes an image from the lyrics, which is further enhanced with the moods of an audio file. Moreover, the authors only showcased the results of their model, while our work incorporates a user interface that allows users to generate album covers in an interactive and personalized way.
System description Given that album artwork could serve to preliminary communicate the themes and sentiments of a song (or collection of songs) to a would-be listener, the most helpful image-generation tool for independent musicians would do more than produce attractive images-it would account for both the semantic meanings of the lyrics and the emotional moods of the instrumentation. As such, Visualyre uses the lyrics (text) and music (audio) of the input song as features for the generation of representative images. To do this, it deploys three modules that pair the information from the audio files and the lyrics: (1) a text-toimage GAN model to synthesize images from lyrics, (2) an audio analyzer to evaluate the moods of a song file based on its audio features, and (3) a style transfer model to adapt the image from the first module to a particular mood.
Image generation In the first module, a text-encoder extracts features from each line in the input lyrics and uses these features to conditionate the synthesis of images. This is done by employing a text-to-image generation model. Multiple approaches have been proposed over the years to improve these models, such as using a stacked approach to refine the images  [12] , employing attention to synthesize fine-grained details  [13] , and inferring semantic information to guide the image generation  [14]. Visualyre uses a DM-GAN to visualize the lyrics, generating an image for each sentence in the input lyrics. DM-GAN  [15]  makes use of dynamic memory to refine the content of the synthesized images, even if the quality of the previous images is not good. This generative model is trained using the MS COCO dataset  [16] , which contains a total of 123,287 images (82,783 for training and 40,504 for validation) with five captions per image.
Style transfer To visually represent the moods obtained from the audio analysis described above, we use a technique called style transfer. Style transfer is the process of changing the style of an image to match the style of another image while still preserving the original image's content  [19]. In previous work, style transfer models were only capable to change the style to a specific style, as these models used a different trained model for each style image  [5, 20]. However, Ghaidi et al. [21]  introduced a style prediction network that can predict the style embeddings of an arbitrary style image, effectively enabling stylization using any pair of content and style images. Their network is trained using content images from the ImageNet dataset  5  and the Kaggle Painter By Numbers (PBN) dataset,  6  which consists of 79,433 paintings.
For Visualyre, we utilized Ghaidi et al. style transfer model by preparing a small dataset of "emotional" images called "Style Bank" by querying Unsplash  7  for images that reflect the four different moods assessed by our binary classifiers (Sect. 3. 2 anger, happiness, sadness, and relaxation. We used the moods and words with similar meanings (e. g. , "fury," "joy," "calm," and "sorrow") as search queries to obtain images relevant to a particular mood. Our intention was to use style transfer to apply the "mood" from the images with the images generated by the GAN model (Sect. 3. 1). To determine which "mood" image should be combined with the GAN-generated image, we relied on the mood probability scores derived from the audio analysis. After sampling a mood, the system selects an image representative of the chosen mood from the Style Bank, which serves as the style image.
How effective are the methods of text-to-image generation and style transfer for visualizing lyrics? Visualyre combines the methods of text-to-image generation and style transfer to synthesize the resulting images. To evaluate the first method, we asked participants to rate whether "The application displayed at least one image that matches the theme of the lyrics" (Q5). The mean score for this question was 4. 26 (SD = 1. 46). To evaluate the second method, we asked participants to rate whether "The application displayed at least one style that matches the tonality of the song" (Q6). The mean score for this question was 4. 66 (SD = 1. 08). Participants considered style transfer to be a more effective visualization method than text-to-image generation which may be due to the limitation of the MS COCO dataset used in training the current text-to-image model and the difficulty involved in representing abstract concepts through this dataset.
As such, one improvement could be to expand the training dataset used to include images trained on artistic or abstract concepts. Secondly, the range of supported moods in the audio analysis was limited, and the reference images in the Style Bank were determined subjectively. Future iterations could include a more comprehensive mood detection model for audio analysis and crowdsourced annotations for a more consistent estimation of reference images' emotionality. Thirdly, Visualyre only supports lyrics written in English. Thus, another improvement would involve adding multilingual support for the application, perhaps by training GAN models with multilingual datasets or by adding machine translation capabilities within Visualyre's architecture. Finally, we note that the downloadable images have low resolution.
Acknowledgements The authors would like to thank  Marcus 李俊緯  for consenting to the use of his track "What can I give to you" in the figures. We would also like to express our gratitude towards  Rebecca Buening  for her assistance and insights in the qualitative evaluation of Visualyre. This work was supported by  JSPS KAKENHI  Grant Number  JP19H01118 ,  Japan  and collaborative research fund from  Yahoo Japan Corporation.
Stochastic Gradient Descent for DGP In Section 2 we mentioned that most of the standard methods for static word representations solve an optimization problem similar to (1) by employing SGD (Stochastic Gradient Descent). It turns out that problem (12) is a particular case of (1) and thus it raises the question of whether it is advantageous to use DG-based methods instead of simply applying SGD to  (12). To answer this question, we report some numerical experiments on synthetic datasets to illustrate the difference in performance between GBU and SGD. We generate random datasets varying the number of points n, the dimension K and the noise level η, following the procedure described in Algorithm 3 (see Appendix A). After generating n random points V 1 ,. V n in R K , corresponding to columns of a matrix V , we build the true Gramian G true = V ⊤ V and perturb its entries G true ij by a random noise ϵ ij ∼ N (0, η × G true ij ) to obtain G ij = G true ij + ϵ ij.
A symbol "*" means that a method fails due to numerical instability. As one can see, GBU systematically performs better than SGD when the data are exact, both in time and score. However, GBU ran into numerical problems for K = 200 on instances with noise. We observed that the matrix R, from the QR decomposition of A (see Section 3. 2), happens to be near singular in these cases. A possible explanation is that the volume of the convex hull of the m = K + 1 reconstructed points v 1 ,. , v m in R K is close to zero. Thus, the main advantage of SGD over GBU for these problems (with m = K + 1) is its resistance to noise and numerical stability. Nevertheless, by increasing the number of references m, it is more likely that the volume of the convex hull of v 1 ,. , v m is sufficiently positive or, in other words, that the matrix R is not so ill-conditioned.
( 17 ) The solution of (  17 ) is constructed from the top K eigenpairs of G as discussed in Section 3. We refer to this approach as "PMI-eigs". Due to the sparsity of G ∈ R n×n , such eigenpairs are usually computed by an Implicitly Restarted Lanczos method (IRLM)  [37]  as implemented, for example, in the Matlab routine eigs. Its cost per iteration is given by qγn + (6K + 9)qn + 4q 2 n + 2K 2 n + O((K + q) 3 ), where q is the number of shifts and γ is the average number of nonzero elements of rows of G. See  [38]  for details. If we denote m = K + q, then the cost can be written as qγn + nO(K 2 ) + O( m3 ). Therefore, considering γq ≪ K 2 , the computation of the top K eigenpairs of the PMI matrix requires O((nK 2 + m3 )n iter ) operations, where n iter is the number of IRLM iterations. GBU.
fastText. In  [39] , an extension of SGNS is proposed. This extension takes into account the morphology of words: a vector representation is associated to each character s-gram and words are represented as the sum of these vectors. The complexity of the co-occurrence based methods discussed above are summed up in Table  3. Recall that n = |V| is the size of the vocabulary and K the dimension. Concerning DC and PMI-eigs, it should be noted that usually n ′ iter ≪ n iter hence PMIeigs complexity is not necessarily lower than that of DC. The number of shifts q, q i is  GBU O(m 3 + (n -m)K 2 ) DC O( P i=1 (n i K 2 + m3 i )n ′(i) iter + (K + 1) 3 (P -1)) GloVe O(p 0. 8 n epochs K) internally set in the implementation of eigs in Matlab and it is difficult to estimate, even though it is likely that K ≤ q ≪ n and K ≤ q i ≪ n i.
Protein sequence embeddings An interesting application in bioinformatics is the determination of vector representations for protein sequences in order to infer physical or biological properties of unseen sequences through machine-learning models  [9]. Differently from natural language processing, pre-trained representations for protein sequences are not commonly available and thus we need to train representations from the scratch. Here, following the study in  [9] , a sequence of amino acids is divided into k lists of non-overlapping k-mers. For example, for k = 3, the amino acid sequence "ADTIVAVET" gives rise to 3 lists of 3-mers: "ADT, IVA, VET", "DTI,VAV" and "TIV, AVE". In this way, each amino acid sequence is viewed as a document whose phrases are the lists and words are the k-mers. In  [9] , doc2vec  [40]  embedding models were trained on 524,529 protein sequences from UniProt database  [41]  and used to infer encondings of sequences for input to machine-learning algorithms for the end-task.
Embeddings for the task sequences (not present in the training) were obtained by averaging the embeddings for the k lists of k-mers. We consider two regression tasks named in  [9]  as "Localization" and "T50" (see  [9]  and references therein for details). For these tasks we used Gaussian process (GP) regression models  [42]  trained on the sequence embeddings given by doc2vec and compare its performance against the same GP model (using Matérn kernels with parameter ν = 5/2) trained on embeddings obtained by the proposed DG-based methods. We have used the same training and test sets 2  as in  [9]. The context windows are defined by w k-mers before and after the central k-mer. We have used doc2vec from the Python library Gensim  [43] , with 25 epochs. In Table  4  we evaluate the predictions based on the coefficient of determination (R 2 ) and the mean absolute error (MAE).
The word vectors for PMI-eigs, GBU and DC were generated using Matlab [v. 2018. b]. GloVe 3  and fastText  4  were compiled using GCC Apple LLVM version 10. 0. 0 (clang-1000. 10. 44. 4) with the default parameters from the official code and trained in the corpus described above. We also compare these word representations with a baseline of random word vectors whose components are drawn from a standard Gaussian. Two experimental evaluations of word vectors were considered. First, an intrinsic evaluation, using QVEC  [44] , which was shown to have good correlation with the performance of the word vectors on semantic evaluation tasks, based on alignment of features extracted from lexical resources. These evaluations are reported in Table  5. Second, we evaluate the quality of these representations over three text classification tasks.
Our implementation includes 3 datasets: WebKb (Multiclass), Subjectivity (Binary) and Amazon (Binary). Results are reported in Table  6. For our classification experiments, we use an implementation of a Convolutional neural network (CNN)  [45]  using Tensorflow library  [46]  version 1. 12. Overall, the performance in terms of F1-score between our word vectors, GloVe and fastText are very similar. A noticeable result is the performance of random embeddings on the WebKB dataset, which perform as well as the other representations. The reason for this behavior is that WebKB is made up of documents sharing lots of words in common when they belong to the same class. Therefore, the quality of the embeddings have less impact than the other tasks we consider. We also provide some practical computing times, in the line of our complexity study.
Conclusion and perspectives We proposed a formulation and methods based on Distance Geometry (DG) to generate vector representations of words from co-occurrence data. The resulting Geometric Build-up and Divide and Conquer algorithms are considerably faster than popular word embedding algorithms, as GloVe and fastText. Although the word vectors obtained by DG methods have performance inferior than the benchmarks in our intrinsic evaluation (QVEC), when combined with a convolutional neural network, DG word vectors lead to F1 scores among the best in three text classification tasks, but demanding about half to a quarter of the computing time, depending on the dataset. Concerning the regression tasks on protein sequences, the results obtained by DG-based methods were slightly better than those of doc2vec whereas the training time was extremely reduced.
In our experiments, we considered a vertex order obtained by two heuristics: ranking the frequency of words in decreasing order for the GBU algorithm, and an order based on K-cores for the DC algorithm. The study of other vertex orders and their impact in the results of the GBU and DC algorithms are subject of future studies. Another question we would like to investigate is if the use of DG methods can also be extended for contextual embeddings. Our approach based on distance geometry seems suitable for representation learning in low resources languages, or tasks in bioinformatics where pre-trained embeddings are not publicly available. Since the employed DG algorithms to obtain word vector representations scale well with the vocabulary size and dimension, they are adapted when working with limited computing resources.
We noted the date and number of tadpoles hatched per clutch and took a picture of the tadpoles on the day of hatching to estimate their size, measured from the snout to the tip of the tail. We then placed each tadpole individually in a cup filled with 250 ml of rainwater with some dead leaves. They were fed three times a week ad libitum with a mixture of spirulina and nettle powder, until froglets metamorphosed. We then transferred the froglets to a separate terrarium. We noted the date and the number of froglets that emerged. For pairs that laid multiple clutches over time, we recorded the clutch order. Fig. 2  The location of a our study areas and of the Amazonian Experimental Ecology Platform (PLEEA) within French Guiana, and b within South America (red square). Data were either collected for breeding experiments (blue dots), to obtain field observations (orange dot), or for both purposes (red dots).
Breeding experiments We initially used generalized linear models with a Poisson distribution and a log link to investigate (1) the number of clutches produced per pair (n = 45 pairs; all pairs were housed for approximately the same amount of time), including population and maternal SVL as predictor variables (Table  S2 ). We then used generalized linear mixed models (GLMM) of the R package 'lme4'  (Bates et al. 2015)  to evaluate population differences in (2) egg laying frequency, measured as the number of days between clutch deposition (only possible for pairs that produced more than one clutch, n = 242 clutches), (3) number of eggs per clutch (clutch size, n = 290), (4) number of tadpoles hatched per clutch (n = 299 clutches), (  5 ) time to hatching (n = 217 clutches), defined as the number of days from egg deposition till tadpole hatching, (6) tadpole size (total length from snout to the tip of the tail) at hatching (n = 738 tadpoles), which was used as proxy for offspring size, (7) time to metamorphosis (n = 485 froglets), defined as the number of days from egg deposition to metamorphosis of froglets, and (  8 ) survival (n = 273 clutches), defined as the proportion of eggs surviving from laying to metamorphosis of froglets (Table  S2 ).
Model selection For all analyses, we used a set of candidate models including all possible combinations of the fixed effects, including the above-mentioned two-way interaction. There was no collinearity (Pearson's r < 0. 6 and variance inflation factors < 3) between independent variables within the same model  (Zuur et al. 2010). We scaled and centered all numeric fixed effects to avoid convergence issues and to be able to compare the relative effect sizes. Model selection was based on Akaike's information criterion corrected for small sample size (AICc)  (Burnham et al. 2011 ), using the r package 'MuMIn'  (Barton 2020) , selecting the model with the lowest AICc value. Parameters that included zero within their 95% CI were considered uninformative  (Arnold 2010). For all analyses where population was retained in the most parsimonious model, we conducted post-hoc pairwise comparisons (as described above) to detect differences among populations.