dataesr/scientific-paragraphs-categorization huggingface dataset:
IMPLEMENTATIoN AND VALIDATIoN oF 3DVL@ES Our prototype is implemented in JavaScript and Blender 3D. The prototype can help learners experiment with practical work using a suitable room according to its discipline. It also provides learners with multimedia communication facilities and even Peer-To-Peer (P2P) sharing of documents. It enables a learner conducting a scientific experiment to collaborate with other learners, which enhances the learners' experiences and reduces the time of carrying out the practical work. In addition, the prototype is highly flexible for the specification and design process of 3D web applications. It offers easy virtual access and allows collective practical work and knowledge sharing. Several development tools and script languages have been used in developing the 3DVL@ES laboratory.
The main contributions of this article are: • HistoStarGAN, the first model able to simultaneously perform diverse stain transfer, stain normalisation and stain invariant segmentation. • The proposed model generalises stain transfer across many stainings, including unseen stainings. • The model learns using limited annotations from one staining and generalises to a wide range of unseen stainings without requiring additional annotations. • KindeyArtPathology -the first artificially created fully-annotated histological dataset, illustrating the potential and the risks of generating synthetic training data targeting staining methods relevant in renal pathology. • Pre-trained HistoStarGAN, which can be used for numerous offline applications, e. g. augmentation of smallsize private datasets.
Related work Virtual staining approaches are greatly inspired by style transfer literature  [14]  where GAN-based approaches achieve state-of-the-art results in many application areas. Therefore, some authors try to bypass the physical staining process, which introduces most of the variation  [23, 32]  while others try to normalise slide appearance after staining  [36]. Nevertheless, the methods are usually based on image-to-image translations approaches, where one domain (stain) is translated to look like a reference staining. Some approaches  [5, 8, 33]  simulate paired datasets by discarding colour information (e. g. by using a greyscale version of the image or extracting a haematoxylin channel from the image) and learning paired image-to-image translations models, such as pix2pix  [13].
Model Description HistoStarGAN architecture is presented in Figure  1. The model is composed of five modules: generator (G), discriminator (D), mapping network (F ), style encoder (E) and segmentation network (S). The models G, D, F and E are elements of StarGANv2's architecture. The mapping network F generates a stain-specific style by transforming the random latent code z into the target stain's style. The style is injected into the generator G during translation, which enables diverse generations as different latent codes result in different stain-specific styles. In order to ensure that the generator uses the injected style information, the model is constrained with a style reconstruction loss, i. e. the style encoder E extracts the style from the generated image, and the difference between that style and the style provided to the generator during translation is minimised (blue arrows in the Figure  1 ).
Dataset The dataset contains tissue samples collected from a cohort of 10 patients who underwent tumour nephrectomy due to renal carcinoma. The kidney tissue was selected as distant as possible from the tumours to display largely normal renal glomeruli; some samples included variable degrees of pathological changes such as full or partial replacement of the functional tissue by fibrotic changes ("sclerosis") reflecting normal age-related changes or the renal consequences of general cardiovascular comorbidity (e. g. cardiac arrhythmia, hypertension, arteriosclerosis). The paraffin-embedded samples were cut into 3µm thick sections and stained with either Jones' H&E basement membrane stain (Jones), PAS or Sirius Red, in addition to two immunohistochemistry markers (CD34, CD68), using an automated staining instrument (Ventana Benchmark Ultra).
Whole slide images were acquired using an Aperio AT2 scanner at 40× magnification (a resolution of 0. 253 µm / pixel). All the glomeruli in each WSI were annotated and validated by pathology experts by outlining them using Cytomine  [28]. The dataset was divided into 4 training, 2 validation, and 4 test patients. Training the HistoStarGAN model's segmentation branch is supervised, and requires the segmentation masks for all images in the dataset. However, in Computational Pathology this assumption is unrealistic. It is more reasonable to assume that annotations exist for limited examples, e. g. as is common in the field, only for one staining  [10, 42]. Thus, the PAS staining is considered to be annotated (the source stain), while the other stainings (Jones H&E, Sirius Red, CD34, CD68) are considered to be unannotated.
To overcome the lack of annotations in target stainings, the CycleGAN's deterministic nature and limited capacity to perform geometrical changes are used to artificially generate a fully-annotated dataset. The CycleGAN model is trained in an unsupervised manner, using randomly extracted patches from a given pair of stainings. Separate CycleGAN models are trained for each pair of PAS-target stains. When trained, the CycleGAN models are applied to the source (annotated) dataset in order to generate annotated samples in the target stainings.
The segmentation branch's architecture is the same as the generator's decoder, without the adaptive instance normalisation layer. The segmentation branch is trained using the Adam optimiser with a learning rate 10 -5. As for the other networks, exponential moving averages over parameters  [16]  is applied during training to obtain the final segmentation network, as experimentally, it gives better results than the best model saved based on validation performance. The HistoStarGAN model's segmentation branch is trained using a balanced dataset, mainly containing artificial histological images produced by both CycleGAN and HistoStarGAN. However, tasks in Computational Pathology are usually concerned with sparse structures, and using an imbalanced dataset to account for the tissue diversity is ben-eficial for learning  [20, 42].
Moreover, CycleGAN-based translations can be noisy  [44] , which could affect the stain invariant properties of the segmentation module. Thus, after training HistoStarGAN as a whole, the segmentation module is fine-tuned for one epoch using real unbalanced PAS-stained images while the rest of the model is fixed (the choice of the number of fine-tuning epochs is discussed in Section 5). This dataset contains all PAS glomeruli (662 extracted from the training patients) and seven times more negative patches (4634) to account for tissue variability. The Adam optimiser is used with a batch size of 8 and a learning rate of 0. 0001. The same augmentation as in Lampert et al. [20]  is applied during fine-tuning.
Generalisation of stain transfer: Since HistoStarGAN is trained on a variety of stains, the generator's encoder is stain invariant, which for the first time enables virtual staining of unseen stains. Examples of which are presented in Figure  5 , in which a new stain modality named H&E in addition to three double-stainings CD3-CD68, CD3-CD163 and CD3-CD206 are translated to stainings seen during training. Moreover, in Figure  6  the model is applied to the AIDPATH dataset  [4]   Figure  2 : Diverse HistoStarGAN translations of a PAS glomeruli patch to target stains (including PAS) with corresponding segmentations. Fake images in each row are generated using the same random vector transformed into a stain-specific style by the Mapping network. The differences between translations are in microscopic structures (e.
Generalisation of stain invariance: To test the generalisation of HistoStarGAN, the model is applied to new stainings not seen during training. Using the annotations of four whole-slide images from each of the new stainings (H&E, CD3, CD3-CD68, CD3-CD163 and CD3-CD206), the averages of three independent training repetitions with corresponding standard deviations are presented in Table  2. Overall, HistoStarGAN generalises better and is more robust compared to UDA-GAN. A potential cause of UDA-GAN failure in some stains can be its learning process where stain invariance is forced only on the pixel level. Although the HistoStarGAN uses the identical CyleGAN translations, the framework extracts stain-invariant features, which are better transferred to unseen stains. Moreover, the segmentation branch of HistoStarGAN is trained on a dataset with more variety since HistoStarGAN translations are also included (see Figure  1 ).
Ablation Studies HistoStarGAN model builds upon StarGANv2 in several aspects -the first extension is attaching a segmentation module to the generator; the second is using pre-trained CycleGANs to create the training dataset, and the third is fine-tuning the segmentation module using only source data. In the following, each of these aspects will be discussed via an ablation study, in a bottom-to-top direction as illustrated in Figure  7.
Fine-Tuning Table  3  demonstrates that fine-tuning the segmentation branch using the fixed generator's encoder increases segmentation performance. In practice, fine-tuning for just a single epoch on an imbalanced source dataset increases overall performance on virtually seen stains (test patient WSIs) by around 6% in overall F 1 -score. However, longer finetuning, although potentially beneficial for particular stainings, does not offer any benefits. Also, the performance on the source's validation set does not correlate with the obtained improvements, i. e. the fine-tuned model with the lowest validation loss does not bring the best overall results on unseen stains. Thus, fine-tuning for one epoch is chosen.
Dataset Characteristics A balanced and (virtually) fully annotated dataset is used to train HistoStarGAN. A labelled dataset is required since the segmentation branch is trained in a supervised manner, and thus alternative data sampling strategies such as random data sampling  [10, 42]  are not suitable. However, the fully annotated dataset does not need to be balanced. Thus, using the findings of Lampert et al. [20] , an imbalanced source (PAS) dataset is formed where all glomeruli in the training patient images are extracted (662) and seven times as many negative patches (4634). This dataset is translated using CycleGAN to all target stains (CycleGAN models are always trained in an unsupervised way, on random patches). Thus, a fully annotated and highly imbalanced dataset is created on top of which the HistoStarGAN model is trained.
The results obtained in this setting are presented in Figure  8 , in which one glomeruli patch from the PAS stain is translated into multiple stainings using different latent codes. The obtained translations for one stain pair differ from each other in terms of glomeruli texture and surrounding structures. The model is able to change the size of glomeruli by changing the size of the Bowman's capsule (white space), in addition to varying the appearance of stain-specific markers (i. e. macrophages in CD68 stain). Nevertheless, the segmentation branch successfully recognises all of glomeruli variations. Compared to training with a balanced dataset, this setting offers more translation variability related to the surrounding tissue and structure of the glomeruli themselves. However, although globally these translations look realistic, the internal structure inside the glomeruli are not.
Since in HistoStarGAN these translations must also be successfully segmented, this leads to an increase in the false positive rate. For example, it is evident that the produced translations often contain 'artificial looking patterns' such as a tendency to group microscopic structures (e. g. nuclei) into diagonal, horizontal or vertical stripes (in Figure  8  this is mostly visible in translations to Sirius Red and Jones H&E). Therefore, the proposed model uses a balanced dataset since it greatly reduces such possibilities.
Segmentation Branch Without the segmentation branch, HistoStarGAN is reduced to StarGANv2 (see Figure  7 ). Moreover, removing the segmentation module removes the requirement for the dataset to be fully annotated. Thus, such a model can be trained CycleGAN-Generated Balanced Dataset A StarGANv2 model trained on a balanced dataset produced by Cycle-GAN translations of the annotated stain, can also result in diverse multi-domain translations between multiple stainings, as illustrated in Figure  9. Usually, the glomeruli structures are visually preserved. However, when measuring the quality of obtained translation using the performances of pre-trained segmentation models for each staining (baseline models trained in a fully-supervised setting), the conclusion can be different. Figure  10  presents a visual comparisons of HistoStarGAN and StarGANv2, where translations obtained by StarGANv2 are segmented using pre-trained models from each staining.
Since both models can obtain diverse translations, Figure  10  represent the average translation and average segmentation over 50 random latent codes. It can be seen that HistoStarGAN gives a more accurate segmentations, especially in difficult cases, e. g. sclerotic glomeruli, rows 5 and 7. Nevertheless, in the absence of a segmentation branch, the dataset composition itself cannot be a strong guarantee that glomeruli structures will be preserved. Thus, an explicit requirement, such as attaching the proposed segmentation branch, is beneficial to ensure the correctness of the translation and robust segmentation. CycleGAN-Generated Imbalanced Dataset If the dataset used to train the StarGANv2 is imbalanced, the model can no longer preserve the structure of interest. Some examples of stain transfers obtained with this model are presented in Figure  11.
HistoStarGAN Seg. Random Dataset: StarGANv2 can also be trained on a dataset composed of random patches extracted from each stain (PAS, Jones H&E, Sirius Red, CD68 and CD34). Thus, 5000 random patches from the training patients are extracted uniformly, and the total dataset containing 25 000 images is used to train the StarGANv2 model. As in the CycleGAN-generated imbalanced data setting, this model cannot preserve the structure of interest, as demonstrated in Figure  12. Moreover, it is prone to bigger alterations of microscopic structures, which limits the usefulness of such translations and confirms the benefits of the proposed HistoStarGAN model.
KidneyArtPathology Dataset Here we describe a new dataset released to encourage the progress of deep learning-based solutions in the field of renal pathology. The dataset contains 5000 images from five stainings, in a resolution of 512 × 512 pixels, fully annotated for the task of glomeruli segmentation. To achieve this, a HistoStarGAN model is trained using a dataset that is as representative as possible, i. e. two immunohistochemical stainings (CD68 and CD34) in addition to three histochemical stainings (PAS, Jones H&E and Sirius Red). Thus, the model can generate various translations in Moreover, the associated pre-trained HistoStarGAN model (also publicly available) can be further used to augment private datasets by augmenting the number of stainings to those used during HistStarGAN's training.
There are several benefits of such a dataset, which fall under three categories: • Pathological -Non-invasive Pathology Training, the diverse appearance of glomeruli can be helpful in the early stages of a pathologist's training. • Benchmarks -the absence of publicly available real-world datasets poses huge challenges for rigorous comparison in the literature. Thus, such a large collection of annotated patches can serve as a benchmark. • Domain adaptation -The data can be used in addition to a private dataset (which can contain limited data) to build more robust models, e. g. as an augmentation or domain adaptation strategy. New histopathological images: Since the style of a stain is encoded by the mapping network, it is possible to generate new patches in each training staining by providing a random image, rather than a source histopathological patch, to be translated to a given stain.
In addition to a fully-annotated glomeruli dataset, new histopathological images in different stains can be generated. Some generated examples are provided in Figure  13. Alternatively, by providing non-histopathological image, the HistoStarGAN is able to convert it to a histopathological image, examples of which are provided in Figure  14  (although medical use of this is admittedly most likely limited to non-existent, it is an interesting side property of HistoStarGAN).
Although from a style-transfer perspective, such a solution is interesting, changing internal structures in this way is not biologically justifiable. However, according to a recent study  [47] , medically-irrelevant augmentation can increase model robustness; thus, this represents another exciting direction for the work ahead. Nevertheless, the model has great potential to be used for tackling the lack of annotations in histopathological datasets. Depending on the availability of labels in a given target domain, diverse HistoStarGAN translations with corresponding segmentation masks can help training from unsupervised to fully supervised settings, using one or more source (annotated) domains. Finally, it is crucial to note that translations obtained by the HistoStarGAN model should not be used for diagnostic purposes.
Conclusions The HistoStarGAN model represents the first end-to-end trainable solution for simultaneous stain normalisation, stain transfer and stain invariant segmentation. For the first time, obtaining highly plausible stain transfer from unseen stainings is possible without any additional change to the model (e. g. fine-tuning). Moreover, the model achieves new state-of-the-art results in stain invariant segmentation, successfully generalising to six unseen stainings. The proposed solution is general and extendible to new stainings or use cases. Being able to generate diverse translations for a given input, the proposed solution paved the way to generate Kidne-yArtPathology, an artificially created and fully annotated data set illustrating a broad spectrum of synthetic image data ranging from biologically usable results, to visually plausible but biologically useless output.
However, the obtained virtual staining can realistically change the appearance of microscopic structures, such as the number of visible nuclei, size of Bowman's capsule or membrane thickness. Thus, it is ill-advised to use these translations for applications that rely on the assumption that the translation process preserves all structures in the original image. Moreover, such virtually stained images must not be used for diagnostic purposes since diagnostic markers can be altered. As such, this study goes towards saving resources and time for annotating the high-quality datasets required for developing deep learning based models, not to replace the process of physical staining. Nevertheless, it remains an open question why the model's performance varies across different stainings. One hypothesis is that the model, being based on CycleGAN translations, replicates some of their limitations.
Acknowledgements This work was supported by:  ERACoSysMed  and e:Med initiatives by the  German Ministry of Research and Education (BMBF) ;  SysMIFTA  (project management  PTJ ,  FKZ 031L-0085A ;  Agence National de la Recherche ,  ANR , project number  ANR-15-CMED-0004 );  SYSIMIT  (project management DLR,  FKZ 01ZX1608A );  ArtIC  project " Artificial Intelligence for Care"  (grant  ANR-20-THIA-0006-01 ) and co funded by  Région Grand Est ,  Inria Nancy -Grand Est ,  IHU of Strasbourg ,  University of Strasbourg  and  University of Haute-Alsace ; and the  French Government  through co-tutelle PhD funding. We thank the  Nvidia Corporation , the  Centre de Calcul de l'Université de Strasbourg , and HPC resources of  IDRIS  under the allocation 2020- A0091011872  made by  GENCI. We also thank the  MHH team  for providing high-quality images and annotations, specifically  Nicole Kroenke  for excellent technical assistance,  Nadine Schaadt  for image management and quality control, and  Valery Volk  and  Jessica Schmitz  for annotations under the supervision of domain experts.
The rest of the paper is organised as follows. The HistoStarGAN model is explained in more detail in Section 3. 1; visual and quantitative results are given in Section 4. The HistoStarGAN architecture and training parameters are analysed in an ablation study presented in Section 5. KindeyArtPathology dataset is explained in more detail Section 6. The KidneyArtPathology dataset and pre-trained models can be available upon request. Demonstrative examples are provided online1. The limitations and opportunities of the proposed solution are presented in Section 7; conclusions of the paper are presented in Section 8.
Data and Methods The sub-projects of climXtreme employ different methodological approaches, tailored to the corresponding research questions ranging from better understanding of the selected events drivers to sectoral impact assessments. This section summarizes these approaches and provides a basis for the study and analysis of the selected case studies separated in temperature-precipitation and precipitation-wind CEs. The temperature-precipitation storyline includes analysis on drivers of the hot summer 2018, detection of extreme events and spatial patterns, assessment of the impact of the 2018 European drought on soil moisture and groundwater as well as sectoral impacts on agriculture and forestry. The storyline is complemented with an assessment of.
Agricultural and hydrological drought of the year 2018 Lack of sufficient soil moisture, resulting from shortage of precipitation and excess evapotranspiration leads to agricultural drought. Lack of run-off and surface water result in hydrological drought (streamflow deficits)  (Seneviratne et al. , 2021). To estimate the severity of agricultural and hydrological droughts across Europe during summer 2018, we employed the nitrogen version of the vegetation, crop, and hydrology model LPJmL  (Schaphoff, et al. , 2018; von Bloh et al. , 2018; Lutz et al. , 2019; Herzfeld et al. , 2021). The analysis is based on 69 years  obtained from model simulations driven with daily temperature, precipitation, and radiation data from the GSWP-W5E5 dataset  (Kim, 2017; Cucchi et al. , 2020; Lange et al. , 2022)  at 0.
Impact on the agricultural production of 2018 In comparison with the past three decades, the year 2018 was identified as a year with severe winter wheat yield losses estimated using a compilation of LOESS (locally estimated scatterplot smoothing; to take into account improvement of agricultural practises  (Zampieri et al. , 2017) ) detrended and gap-filled yield data at county level aggregated from a variety of sources including the Regionaldatenbank Deutschland (Statistische Ämter des Bundes und der Länder, 2021) and the Statistical Offices of the federal states of Germany  (Ellsäßer and Xoplaki 2022abc; Ellsäßer and Xoplaki, 2023). The resulting annual gridded yield data was evaluated using the Standardized Yield Anomaly Index (SYAI) that expresses yield anomalies in terms of standard deviation from a 30-year time series.