dataesr/scientific-paragraphs-categorization huggingface dataset:
We extracted from the Pfam  [70]  database (last update June 2021), all the kinases detected in Drosophila by selecting the CL0016 clan. Next, we employed the UniprotKB database to distinguish the tyrosine kinases (family: PF07714) from other kinases. In total 56 tyrosine kinases and 251 other kinases are present in the network. We also precalculated the mean and the standard deviation of the Resnik semantic similarity of each regulated node in the network against each other.
Seed selection and network diffusion We applied the random-walk-with-restart-based algorithm  [32]  for the following DV clusters: D (1), L (2), V (5), DL  (6) , DV  (9)  and LV  (12) , once each for the ED score and once for the deviation values, and each for the protein and phosphoproteomic datasets. In the case of the deviation values, we used only those proteins or phosphosites within the interquartile range. We assigned as seed value the reciprocal of the absolute value from both the ED Score or the Log2 Fold Change Deviations. Note that if multiple phosphosites are assigned to the same protein we selected the median of the ED scores or Log2 Deviations. We then partition the seed set in tyrosine kinases, remaining kinases and other proteins and perform the random walk with restart (RWR) from each of the three partitions separately.
Ontology analyses of extreme deviating proteins and phosphoproteins using PANTHER protein class We filtered the proteins and phosphosites with an absolute Log2 deviation value larger than the 95 th percentile (prctitle Matlab function) of the distribution of absolute log2 deviation values. Because deviation values were calculated separately for each ventralized genotype (Toll 10B /def or spn27A ex /def), we obtained two independent lists of proteins and phosphosites with deviation values exceeding the 95 th percentile. From these lists, we selected the proteins and phosphosites whose log2 deviations exceeded the 95 th percentile threshold with both ventralized genotypes. For phosphosites, we used the host phosphoprotein for the ontology analyses. When two or more phosphosites with large deviations were hosted by the same phosphoprotein, we counted the phosphoprotein only once.
Image processing and quantitation For quantitation of nuclear position, single z-slice H2Av-mRFP1 images were converted into tif format, blurred with a Gaussian filter (σ=2), and segmented with CellPose (v2. 2) in 2D using a custom-pretrained nuclear model tailored for each side of the embryo based on manual correction on segmentation generated by a default nuclei model with nuclear diameter set as 20 pixels. The stitch mode was used with a stitch threshold of 0. 4 to generate 3D nuclear segments. Nuclear segments were filtered by size (1000-5000 pixels) and height (>10 μm), while those located at the edge of the imaging area were excluded for data processing. The regionprops function implemented in the Skimage Python library was used to define the bounding box of each nuclear segment, from which the middle Z coordinate of the bounding box was designated as the nuclear position.
For time alignment, t0 (the onset of gastrulation) was defined as the time point, at which apical constriction in ventral furrow produces a 2. 5 μm gap between the cell apex and vitelline membrane for the datasets acquired on the ventral side. Using this t0 designation (from water-injected embryos imaged on the ventral side), the pre-injection cellularisation depths were fitted to a linear function based on the assumption that cellularisation depth is linear with time during midcellularisation. For datasets acquired on the lateral and dorsal sides, the pre-injection timing relative to the onset of gastrulation was derived by plugging in the pre-injection cellularisation depth, from which the t0 frame of the dataset was derived. Data processing and plotting were performed with custom-made Python codes using Numpy, Pandas, Matplotlib, and Seaborn libraries.
For en face membrane visualisation, 3xmScarlet-CaaX images were deconvolved using the Huygens Software (Scientific Volume Imaging) with the deconvolution algorithm Classic MLE using custom parameter sets. A) Top: colour-coded schematic of the cell populations along the dorso-ventral axis of a Drosophila embryo during gastrulation: blue, 'dorsal': ectoderm and amnioserosa; magenta, 'lateral': neuroectoderm ; green, mesectoderm ; yellow 'ventral': mesoderm. Middle: examples of dorso-ventral fate determinants of the domains shown in top panel. Bottom: dorso-ventral and anterior-posterior axes for reference in panel B. B) RNA in situ hybridisations using probes for genes expressed in dorsal (dpp), lateral (sog) and ventral (snail) cell populations in wild type (w 1118 ) and embryos derived from mothers mutant for dorso-ventral patterning genes (gd 9 , dorsalized, Tl rm9 / Tl rm10 lateralized, Toll 10B /def and spn27A ex /def both ventralized).
Acknowledgements We thank  N. H. Brown ,  N. Bulgakova ,  S. Huelsmann ,  N. Perrimon ,  V. Riechmann ,  A. Stathopoulos ,  D. Stein ,  S. Roth  and  A. Wodarz  for reagents and fly stocks,  N. Lawrence  and the  Gurdon Institute Imaging Facility  for help with 3D-SIM imaging,  EMBL Advanced Light Microscopy Facility (ALMF)  and  CECAD Cologne Imaging Facility  for continuous support. Flybase was used throughout the project and is gratefully acknowledged. We also thank  Theresa Haunold  for 2-photon imaging of myosin in DV patterning mutants,  Siegfried Roth  for critical discussions, and all members of the  Leptin lab  for discussions throughout the work. This work was supported by funding from the  European Molecular Biology Organisation (EMBO) , the  University of Cologne  and the  Deutsche Forschungsgemeinschaft  (grant  DFG LE 546/12 ).
It is a very popular technique in computer vision  [8]  and has become inseparable from deep learning applications when rich training sets are not available. Data generation is also used in the case of missing modalities for multimodal image segmentation  [9]. As a result, the model can be trained to generalize images with better quality and avoid overfitting. In addition, some deep learning frameworks, including PyTorch  [10] , allow for on-the-fly data augmentation during training, rather than physically expanding the training dataset. Basic data augmentation operations include random rotations, cropping, flipping, or noise injection. However, these simple operations are not sufficient when dealing with complex data such as medical images. Several studies have been conducted to propose data augmentation schemes more suitable for the medical domain.
Different approaches have been proposed to solve this generative learning trilemma of quality sampling, fast sampling, and diversity  [21]. In this paper, we review the state of the art of deep learning architectures for data augmentation, focusing on three types of deep generative models for medical image augmentation: VAEs, GANs, and DMs. To provide an accurate review, we harvested a large number of publications via the PubMed and Google Scholar search engines. We selected only publications dating from at least 2017 using various keywords related to data augmentation in medical imaging. Following this, a second manual filtering was performed to eliminate all cases of false positives (publications not related to the medical field and/or data augmentation). In conclusion, 72 publications have been kept, mainly from journals such as IEEE Transactions In Medical Imaging or Medical Image Analysis and conferences such as Medical Image Computing and Computer Assisted Intervention and IEEE International Symposium on Biomedical Imaging.
Some publications will be described in more detail in Section 3; these have been selected according to two criteria: date of publication and number of citations. Nevertheless, all the articles are available in descriptive tables as well as other information such as datasets used to perform training. These different papers were organized according to the deep generative model employed and the main downstream tasks targeted by the generated data (i. e. , classification, segmentation, and cross-modal translation). Knowing the dominance of GANs for data augmentation in the medical imaging domain, the objective of this article is to highlight other generative models. To the best of our knowledge, this is the first review article that compares different deep generative models for data augmentation in medical imaging and does not focus exclusively on GANs  [14] [15] [16] , nor traditional data augmentation methods  [22, 23].
Custom-made GAN variants have also been proposed in the current state of the art (see Table  1 ), some of which could be explored further. Notably, conditional synthesis has proven to be particularly useful for tasks such as segmentation and cross-modal translation, as seen with the ACGAN and Pix2Pix, resulting in an improved classification performance. Additionally, two-stage pipeline approaches have been proposed for generating synthetic images conditioned on segmentation masks. To further illustrate the use of GANs for medical image augmentation, we present a summary of the relevant studies in Table  1. This table includes information about the dataset, imaging modality, and evaluation metrics used in each study, as well as the specific type of GAN architecture employed. A further discussion will be presented in Section 4.
In contrast, the standard CVAE encodes the data and conditioning variables in a shared embedding space. The authors evaluate the ICVAE on classification and segmentation tasks using transverse ultrasound images of the spine and FLAIR MRI images of the brain, respectively. The results demonstrate an improvement of 8. 0 ± 1. 0% in classification accuracy and 4. 5 ± 0. 5% in the Dice score compared to the model trained on real images only. The ICVAE model is able to generate more realistic MRI images by encoding appearance features independently of the structures in its latent space. The authors demonstrate the generation of synthetic MRI and ultrasound images using the ICVAE architecture, which are conditioned on a tumor segmentation mask and a label indicating the center-line of the spine, respectively.
The CVAE architecture is also shown for comparison. Chadebec et al. [82]  introduce a novel Geometry-aware VAE for high dimensional data augmentation in low sample size settings. This model combines Riemannian metric learning with normalizing flows to improve the expressiveness of the posterior distribution and learn meaningful latent representations of the data. Additionally, the authors propose a new non-prior sampling scheme based on Hamiltonian Monte Carlo, since the standard procedure utilizing the prior distribution is highly dependent upon the data, especially for small datasets. As a result, the generated samples are remarkably more realistic than those generated by a conventional VAE, and the model is more resilient to the lack of data. An evaluation of the synthetic data on a classification task shows an improvement in accuracy from 66.
Diffusion Models In their study, Pinaya et al. [97]  introduce a new approach for generating highresolution 3D MR images using a latent diffusion model (LDM)  [98]. LDMs are a type of generative model that combine autoencoders and diffusion models to synthesize new data. The autoencoder component of the LDM compresses the input data into a lowerdimensional latent representation, while the diffusion model component generates new data samples based on this latent representation. The LDM in this work was trained on data from the UK Biobank dataset and conditioned on clinical variables such as age and sex. The authors compare the performance of their LDM to VAE-GAN  [32]  and LSGAN  [99] , using the Fréchet inception distance  [100]  as the evaluation metric. The results show that the LDM outperforms the other models, with an FID of 0.
0076 compared to 0. 1567 for VAE-GAN and 0. 0231 for LSGAN (where a lower FID score indicates a better performance). Even when conditioned on specific variables, the synthetic MRIs generated by this model demonstrate its ability to produce diverse and realistic brain MRI samples based on the ventricular volume and brain volume. As a valuable contribution to the scientific community, the authors also created a dataset of 100,000 synthetic MRIs that was made openly available for further research. Fernandez et al. [101]  introduce a generative model, named brainSPADE, for synthesizing labeled brain MRI images that can be used for training segmentation models. The model combines a diffusion model with a VAE-GAN, with the GAN component particularly utilizing SPADE normalization to incorporate the segmentation mask.
The model consists of two components: a segmentation map generator and an image generator. The segmentation map generator is a VAE that takes as input a segmentation map, then encodes and builds a latent space from it. To focus on semantic information and disregard insignificant details, the latent code is then diffused and denoised using LDMs. This creates an efficient latent space that emphasizes meaningful information while filtering out noise and other unimportant details. A VAE decoder then generates an artificial segmentation map from this latent space. The image generator is a SPADE model that builds a style latent space from an arbitrary style and combines it with the artificial segmentation map to decode the final output image. The performance of the brainSPADE model is evaluated on a segmentation task using nnU-Net  [102] , and the results show that the model performs comparably when trained on synthetic data compared to when it is trained on real data, and that using a combination of both significantly improves the model's performance.
Lyu and Wang  [103]  conducted a study that investigated the use of diffusion models for image translation in medical imaging, specifically the conversion of MRI to CT scans. In their study, the authors utilized two diffusion-based approaches: the conditional DDPM and conditional score-based model which utilizes stochastic differential equations  [104]. These methods involved conditioning the reverse process on T2-weighted MRI images. To evaluate the performance of these diffusion models in comparison to other methods (conditional WGAN and U-Net), the authors conducted experiments on the Gold Atlas male pelvis dataset  [105]  using three novel sampling methods and compared the results to those obtained using GAN-and CNN-based approaches. The results indicated that the diffusion models outperformed both the GAN-and CNN-based methods in terms of structural similarity index (SSIM) and peak signal-to-noise ratio (PNSR).
We present a summary of the relevant studies utilizing diffusion models for medical image augmentation in Table  3. This table includes details about the dataset, imaging modality, and evaluation metrics used in each study, as well as the specific diffusion model employed. Upon examining this table, we notice that all the studies included are relatively recent, with the earliest study dating back to 2022. This suggests that diffusion models have gained increasing attention in the field of medical image augmentation and synthesis in recent years. Additionally, we see that in 2022, diffusion models received more attention for these tasks compared to GANs and VAEs, highlighting their growing popularity and potential for use in various scenarios.
The use of GANs in medical imaging, as seen in Table  1 , has been widespread and applied to a variety of modalities and datasets, demonstrating their versatility and potential for various applications within the field. When it comes to classification, DCGAN and WGAN have been the most-commonly used architectures and are considered safe bets in this domain. For example, Zhuang et al. [51]  demonstrated a 3% accuracy improvement in generating fMRIs using an improved WGAN. These architectures, with their capacity for high-quality generation and good mode coverage, offer significant potential for the generation of synthetic images for medical imaging classification. In the case of segmentation and translation, the architectures that have shown the most promise include Pix2Pix, CycleGAN, and SPADE, all of which have proven their potential for conditional generation and cross-modal translation.
For our discussion on medical image augmentation, we have also compiled two summary tables to provide a comprehensive overview of the datasets and metrics used in the reviewed studies. Table  4  presents a summary of the datasets used in the reviewed studies. This table includes information about the title of the dataset, a reference, and a link to the public repository if available, as well as the studied modality and anatomy. From examining this table, we see that MRI is the most-commonly used modality, followed by CT. In terms of anatomy, brain studies dominate, with lung studies coming in second. It is worth noting that the BraTS dataset is widely used across multiple studies, highlighting its importance in the field. Additionally, we notice the presence of private datasets in this table, which is not surprising given that many medical studies are associated with specific medical centers and may not be publicly available.
Abbrv. Reference Metric Name Description Dice  [143]  Sørensen-Dice coefficient A measure of the similarity between two sets of data, calculated as twice the size of the intersection of the two sets divided by the sum of the sizes of the two sets Hausdorff  [144]  Hausdorff distance A measure of the similarity between two sets of points in a metric space FID  [100]  Fréchet inception distance A measure of the distance between the distributions of features extracted from real and generated images, based on the activation patterns of a pretrained inception model IS  [142]  Inception score A measure of the quality and diversity of generated images, based on the activation patterns of a pretrained Inception model MMD  [145]  Maximum mean discrepancy A measure of the difference between two probability distributions, defined as the maximum value of the difference between the two means 1-NN  [146]  1-nearest neighbor score A method for classification or regression that involves finding the data point in a dataset that is most similar to a given query point (MS-)SSIM  [139]  (Multi-scale) structural similarity A measure of the similarity between two images based on their structural information, taking into account luminance, contrast, and structure.
VAEs have not gained as much attention in the medical imaging field, due to their tendency to produce blurry and hazy generated images. However, some studies have used conditional VAEs or hybrid architectures to address this issue and improve the quality of the samples produced. Researchers are therefore exploring the use of hybrid models that combine the strengths of multiple generative models, as well as improved VAE variations that offer enhanced image quality. Hybrid architectures, such as VAE-GANs  [32] , have demonstrated the potential to partially address the issues of both VAEs and GANs, allowing a better-quality generation and good mode coverage. Interestingly, recent research has even combined all three generative models into a single pipeline  [101]. This study has shown comparable results on a segmentation task when using a fully synthetic dataset compared to using the real dataset.
These promising results suggest that hybrid architectures could open up new possibilities. However, these models can be complex and challenging to train, and more research is needed to fully realize their potential. In fact, many VAEs used in medical imaging are hybrid architectures, as they offer a good balance between the strengths and weaknesses of both VAEs and GANs  [85, 101]. It is important to note that VAEs have an advantage over GANs in operating better with smaller datasets due to the presence of an encoder  [158] , which can extract relevant features from the input images and significantly reduce the search space required for generating new images through the process of reconstruction. This feature also makes VAEs a form of dimensionality reduction, and the representation obtained by the encoder can provide a better starting point for the decoder to approximate the real data distribution more accurately.
In contrast, GANs have a wider search space, which may lead to challenges in learning features effectively. For instance, we show in Figure  6  a comparison between synthesized MRIs using vanilla VAE  [18]  and the Hamiltonian VAE  [159]. In addition to the advantage of operating better with smaller datasets, VAEs also offer a disentangled, interpretable, and editable latent space. This means that the encoded representation of an input image can be separated into independent and interpretable features, allowing for better understanding and manipulation of the underlying data. Another option is the use of improved variants of VAEs, which have been proposed to generate high-quality images. There has been limited exploration of improved VAE variants such as VQ-VAE2  [36] , IAF-VAE  [34] , or Hamiltonian VAE  [159]  in the medical imaging field, but these variants have shown promise in generating high-quality images in other domains.
Conclusions In this review, we examine the use of deep generative models for medical image augmentation. The limited availability of training data remains a major challenge in medical image analysis with deep learning approaches, which can be addressed by data augmentation techniques. However, traditional techniques still produce limited and unconvincing results. We focus on three types of deep generative models for medical image augmentation, VAEs, GANs, and DMs, and provide an overview of the current state of the art in each of these models. While deep generative models offer several advantages over traditional data augmentation techniques, including the ability to generate realistic new images that capture the underlying distribution of the training dataset, they also have some limitations.
Moreover, the indentation (c. 5 nm) applied was below 1% of the sample height, meeting the rule for the Hertz model  (Zdunek & Kurenda, 2013). Since AFM mechanical values highly depend on the experimental setups, they are more dedicated to precisely monitor heterogeneity and variations induced by treatment or, in our case, development. Several acquisitions per sample were done on 1 lm-thick cross-sections and at the surface of isolated cutin samples with at least two images per developmental stage processed with GWYDDION software (http://gwyddion. net; Ne cas & Klapetek, 2012). Topographic images were leveled by the least square method, and mechanical measurements such as profiles or sampling in CPM areas were performed on filtered elastic modulus maps to remove outliers, that is, elastic modulus higher than the upper limit of the AFM probes were discarded.
Correlative analysis To associate mechanical properties with chemical information, Raman or OPTIR maps were first subjected to pixel coregistration to spatially colocalize chemical data with mechanical data from elastic modulus maps. The white light images of the area recorded in Raman or OPTIR were superimposed onto AFM topographic maps using at least 10 visual features as anchoring points. These linear transformations were performed in MOUNTAINS MAP 9 (Digital Surf, Besanc ßon, France) software, and newly spatialized mechanical maps were extracted to be further processed in R software. As AFM and Raman or OPTIR imaging have distinct spatial resolution, degradation of AFM resolution by averaging was performed with RASTER (https://cran. r-project. org/web/packages/raster/) and TERRA (https://cran.