dataesr/scientific-paragraphs-categorization huggingface dataset:
The development is carried out in a collaborative and coordinated framework among the technical group with guidance from the management board. EIDA is not only a technical development infrastructure but also a coordination group within ORFEUS that can provide informed proposals and opinions within the FDSN reducing fragmentation. Recent successful examples of coordinated efforts within the FDSN are the guidelines for DOI, jointly prepared and updated; the registry of data centres at the FSDN and the integration of the routing information; the StationXML documentation and various ongoing discussions that will evolve in proposals for FDSN standards (e. g. Large-N and Distributed Acoustic Sensing (DAS) data management, new authentication method following an approach similar to the one in place for EIDA).
Following valuable feedback from the User Advisory Group (UAG) of ORFEUS, quality metrics and data and metadata harmonization across EIDA are currently in focus. For the former, EIDA is exploring the possibility of aligning quality metrics from WFCatalog  (Trani et al, 2017)  with those produced by MUSTANG  (Casey et al. , 2018)  to achieve standardization within FDSN. Harmonization in terms of enhancing best practices within EIDA for data suppliers including temporary deployments is among the priorities of the group as well. EIDA continues to grow not only in terms of archive volume, but also in other areas: the number of nodes is increasing; new datasets are becoming relevant for seismologists, such as DAS or cheap seismic sensors  (Quinteros et al. , 2021) ; users demand rapid access to massive data volumes.
Acknowledgments EIDA has been built over a decade, involving contributions from many people over time. First of all, EIDA would like to thank the networks who entrust data to EIDA nodes, and the scientific users for their trust and continuous feedback. We thank the various people involved in the governance of ORFEUS and in particular its Secretary General Carlo Cauzzi and the  EPOS Seismology Secretary Florian Haslinger  for their coordination efforts and continuous support. We would also like to thank the many previous participants in EIDA and the EIDA nodes, without whom the present data distribution system would not have existed. Seismological data distribution in Europe started well before EIDA; all of the EIDA team is grateful for the legacy of the initial pioneers of the  Jarek Bienkowski -jarek.
ethz. ch/fdsnws/. • Fdsnws_fetch and fdsnws2sds: https://geofon. gfz-potsdam. de/software/fdsnws_scripts/. • The ORFEUS WedDC3 portal can be accessed at http://orfeus-eu. org/webdc3/ where also additional documentation is available. • The Issue tracker of the user feedback repository is accessible at: https://github. com/EIDA/userfeedback/issues. • Documentation for data centre operators: https://orfeus-eu. readthedocs. io/en/latest/. • ObsPy's routing client: https://docs. obspy. org/packages/autogen/obspy. clients. fdsn. routing. routing_clien t. html. • The EPOS ICS portal for integrated access to earth science data is accessible at: https://www. ics-c. epos-eu. org/.
The biggest fear when deploying machine learning models to the real world is their ability to handle the new data. This problem is significant especially in medicine, where models trained on rich high-quality data extracted from large hospitals do not scale to small regional hospitals. One of the clinical challenges addressed in this work is magnetic resonance image generalization for improved visualization and diagnosis of hip abnormalities such as femoroacetabular impingement and dysplasia. Domain Generalization (DG) is a field in machine learning that tries to solve the model's dependency on the training data by leveraging many related but different data sources. We present a new method for DG that is both efficient and fast, unlike the most current state of art methods, which add a substantial computational burden making it hard to fine-tune.
I. INTRODUCTION Deep learning (DL) and Convolutional neural networks (CNN) empowered the computer vision field to be used in many situations efficiently and provide very promising results. Nowadays, all of our smart phones use facial recognition as an option for authentication with Federated Learning  [1] , and all new self-driven cars  [2]  are based mainly on a combination of deep CNNs for road image processing. This massive adoption raises the bar for computer vision systems to be more robust to edge cases and generalizes well The associate editor coordinating the review of this manuscript and approving it for publication was Bilal Alatas. in unforeseen situations. As useful as DL techniques are, deploying them and using them on real-world data brings some problems that we don't commonly see while working on toy datasets or training data in general  [3] , even if it was taken from previous users of the system.
As powerful as they are, Deep Convolutional Networks showed a huge dependency problem on the data set they were trained on, commonly known as over-fitting  [4]. This problem (called domain-shift  [5]  or concept drift  [6] ) is mainly due to the fact that the training data set (Source domain) comes from a different distribution than the deployment data (target dataset), resulting in a decrease in the performance of the model  [7] , largely due to the fact that the latent distribution extracted by the encoders for both domain don't overlap, this can also be confirmed by using several Manifold Learning  [8] ,  [9]  techniques as Bekkouch et al. showed  [10]  by reducing the dimensions of the rich latent space into a lower dimensionality and visualizing the distributions of both domains. Manifold Learning and domain generalization (deep learning in general) are both similar on many levels since they both reduce the input shape and learn an underlying structure in high dimensional data.
The same can be found in Fig  2  where a horse painting was misclassified as a macaw parrot because of the resemblance between their colors. Such problems are unavoidable in real datasets, which created a new field in transfer learning named Domain Generalization (DG). DG can be also seen as a generalized case of the over-fitting problem, in the sense that the model is learning the data and not the task, even though in DG cases the model performs very well on the source test data, unlike traditional over-fitting scenarios. Domain Generalization (DG)  [13]  is a sub-field of Transfer Learning (TL)  [14]  that aims to solve the aforementioned problem by combining multiple data sources to train a more resilient model in hopes of generalizing to unseen domains. DG assumes the existence of multiple sources of data D s i (e.
g. Photo, Art Paintings, and Cartoon) that are used for the same task T s i (e. g. classifying images of animals), and a target domain D t (e. g. Sketches of the same classes of animals) that is harder to work with (harder to label or to collect). Most DG methods provide an extension to a closely related field, Domain Adaptation (DA)  [10] ,  [15]  which often uses one source domain and one target domain to solve the domain shift problem. At the time of training, DA assumes the availability of target domain data but can be classified according to the presence of labels in the target domain in three key ways: Supervised  [16] , Unsupervised  [10] , and Semi-Supervised DA  [17]. DG differs from DA in the fact that we do not have access to the target data nor its labels at training phase. Therefore, DG aims at building a model that can generalize well to unseen domains rather than generalizing to a single known domain.
Researchers have approached the problem of domain gaps and their consequences in many ways. One traditional yet very commonly used technique is to treat this problem as an over-fitting problem and use regularisation techniques to help the model (parametric models) generalize well  [18] ,  [19]. Many techniques have proven to be useful in the case of deep neural networks such as learning rate decay, dropout  [18] , batch normalisation  [19] , L 1 , L 2 regularisation  [20]  and Shakeout  [21]. Although these techniques were proven effective to help the model generalize well within the same data set and achieve higher test accuracy, however, it is not the most effective method for DG. Hence, we need to develop new methods that are both effective for over-fitting and for DG problems. Recent approaches for DG are commonly neural-networkbased and are separated into two main types: one-for-all and one-for-each.
C. ADVERSARIAL & RECONSTRUCTION LOSSES Using Convolutional Auto Encoders while Pre-Training CNN classifiers is considered one of the best practices when the dataset is too small or when the labels are too sparse  [27]. In order to assist with the absence of labeled data, this task leverages the availability of unsupervised data under reconstruction loss. They are also used widely used for outlier detection  [28] -  [30] , novelty detection  [31] , auto-drawing for with RNNs  [32]  and Open-Set Recognition  [33]. Reconstruction loss is also used for domain adaptation by jointly learning a shared encoding representation for: i) supervised classification ii) unsupervised reconstruction of unlabeled data  [34] , this way the encoder learns to extract information from the target dataset too making it more familiar with it.
Thus, we characterize the model's ability for generalizing to unseen datasets by its ability to forget the data's peculiarities, symbolizing how much of the input has been overlooked or neglected by the encoder. We illustrate the Encoder's ability to sustain low-level image information despite the fact that the only loss we used for the training was the classification loss. Figure  4  explains the amount of information the Encoder preserves even after applying extreme input alterations. After training an Encoder plus a Classifier setup on MNIST, the images were reconstructed based on a frozen Encoder and newly trained Decoder. These findings on the test dataset support our hypothesis that even though we train the encoder for classification only, it retains numerous input features from its source data.
A. DOMAIN GENERALIZATION As with all DG methods, our technique requires S source datasets (domains) and at least one target dataset (domain). N i is used to represent the ith source dataset's sample size, such that X s i = {(x s i,j , y s i,j )} N i j=1 , where x s i,j references the jth sample of the ith source dataset and y s i,j is its corresponding label. Moreover, we denote M as the target domain's sample size with X t = {(x t j , y t j )} M j=1 , where x t j is the jth sample from the target dataset and y t j is its label, the t is used to distinguish between source and target domains. The three main components of our model are: Encoder, Decoder, and a classifier, as shown in Figure  3. The central part of the model and our point of focus is the Encoder E(. ) with its weights θ E , which maps the input samples x into the latent embedding space z.
2) EXTENSION TO OVER-FITTING Over-fitting arrives when a model has learned the training data too well. It is very common with strong models such as neural networks and decision trees. A number of techniques for combating over-fitting in neural networks exist such as reducing the model size, reducing the input data's dimensions, regularization (L1, L2), dropouts, and batch normalization, yet most of them constrain the model from actually learning category informative features. Our technique although made for DG, can be easily applied in the case of single source datasets and contrarily to other over-fitting techniques, ours allows the model to learn as deep as possible without letting it over-fit on the style of the training data. Our method is not exclusive with other techniques, but it should be used along the side of most of the previously mentioned techniques since they are considered to be the best practice for the training process.
IV. ANALYSIS Our Adversarial Reconstruction Loss method provided outstanding performances compared to other states of the art methods on several experiments using different datasets. This section is split into four main parts; the first one is the dataset, where we present the five primary datasets and their 13 subdatasets. The second part is the main results section, where we compare our model against several Domain Generalization methods on four benchmarks. The third and last parts are related to unsupervised domain adaptation and over-fitting results.
A. DATASETS To explore our Method's effect on the domain generalization problem and its related issues (UDA, overfitting), we analyze five datasets extensively chosen in the field. The first one is MNISTR; the Rotated MNIST dataset is an alteration to the popular digits classification dataset MNIST. The different domains of RMNIST are created via rotating images by 15 degree increments: 0, 15, 30, 45, 60, and 75 (referred to as M 0 ,. , M 75 ). We employ a leave-one-out situation at the training phase, signifying that we will have five source domains and one remaining for the target. Nevertheless, the data has an identical test/train split as the primary MNIST; therefore, there is no overlap between train and test samples of the different domains. Next, we use the MNIST-SVHN-USPS Street View House Numbers (SVHN) which is a real-world image dataset for digit recognition commonly used with MNIST for domain adaptation tasks.
SVHN is obtained from house numbers in Google Street View images and is a little bit more challenging because of many side artifacts in it and the inclusion of color. US Post Office Zip Code Data (USPS) Handwritten Digits has 7291 train and 2007 test images. The images are 16*16 grayscale pixels which make them similar to MNIST but less complex. This combination of datasets is used both for Domain Generalization and Unsupervised Domain Adaptation. PACS dataset is a new benchmark challenge dataset for object classification which covers seven object classes (person, elephant, dog, house, giraffe, horse, and guitar) spread across four different domains (Photo, Art Paintings, Cartoon, and Sketches), producing a more tough predicament for our models. Hence, we start with a pre-trained imagenet model, namely AlexNet.
VLCS dataset is commonly used in Domain Generalization settings as a benchmark for performance evaluation on multi-class object recognition tasks. VLCS is an abbreviation of the four datasets that make it up: PASCAL Visual Object Classes 2007 (V)  [42] , LabelMe (L)  [43] , Caltech (C)  [44] , and SUN09 (S)  [45]. It was created by combining the five common classes between its sub-datasets, which are: Birds, Cars, Dogs, Chairs, Person. For evaluation purposes, we use the same setup as the previous works  [13] ,  [22] ,  [46] ,  [47]  by using pre-extracted DeCAF6 features (4096-dimensional vector) and performing a leave-one-domain-out validation by randomly splitting each domain into 70% training and 30% testing. We also use a two fully connected layer neural network inputting to two fully connected layers with sizes of 1024 and 128 respectively with ReLU activation.
IXMAS is a cross view action recognition dataset containing eleven different human actions that are recorded by five cameras in different positions. We aim to build an action detector that works regardless of the angle of view. We follow the same experimental setup as  [23] ,  [47] ,  [48]  by using the same Dense trajectory input features and excluding the irregularly performed actions by only keeping the first five actions (check watch, cross arms, scratch head, sit down, get up) performed by the six actors (Alba, Andreas, Daniel, Hedlena, Julien, Nicolas). Each camera position is treated as a separate data domain named (0,1,2,3,4). Following the previous works, we generate a 4-source domain generalization task (leave-one-domain-out). Skin lesion dataset is a combination of 7 public datasets for skin lesion detection collected from different equipments.
The main dataset is HAM10000  [49]  which is used as part of the source data of all experiments following the setup of  [50] ,  [51]. The other datasets are Dermofit (DMF)  [52] , Derm7pt (D7P)  [53] , MSK  [54] , PH2  [55] , SONIC (SON)  [54] , and UDA  [54]. All the datasets contain 7 common lesions which are melanoma (mel), melanocytic nevus (nv), dermatofibroma (df), basal cell carcinoma (bcc), vascular lesion (vasc), benign keratosis (bkl), and actinic keratosis (akiec). Following  [51]  we split the data into training (50%), validation (20%) and testing set (30%) in a stratified manner. In each experiment we choose one of the secondary datasets as a target domain and keep HAM10000 and the other dataset for the source domains. We use a pretrained Resnet18 as the backbone of our model for fair comparison with the other methods.
Hip MR scan Landmark detection HML dataset is a 3D dataset of 423 3D Magnetic resonance scans of the hip area for 114 patients  [56]. The dataset contains 12 landmarks annotated by doctors for diagnosis of several pathologies such as Hip dysplasia and impingement syndrome. The dataset contains three domains which are the different MRI sequences (T1 weighted, T2 weighted and PD weighted). All three modalities are needed for correct indentification of early signs of hip abnormalities. However there is no guarantee that all three of them will be available at a specific hospital. The challenge is therefore to mitigate the problem of missing sequences and ensure higher rates of abnormality defections. For each experiment we use two source domains and the remaining one is the target. We split the data into 80% training 5% validation (For hyper-parameter tuning) and 15% testing in a stratified manner according to the pathologies for the patients and the patient IDs don't overlap between sets.
1) DIGIT CLASSIFICATION: RMNIST For the task of digit classification, we assessed our model's performance versus numerous state of the art deep learning methods in domain generalization which are: MTAE  [57] , CAE  [58] , BSF  [26] , UDS  [46] , PSSO  [59] , AFLAC  [60]. We were inspired to pursue this method after conducting experiments on the MNIST dataset to understand domain dependency better. Therefore, our model performs significantly better on this dataset than all the current state of the art, as Table  1  clearly shows our model's performance exceeds all the other models on average and is ranked at least first or second in each experiment. The reported results are the averaged over 20 runs of the model with the learning rate set to 0. 003, generalizaing_epochs = 50, pretraining_epochs = 100, and the balancing factor set to β = 0.
1. Our method outperformed all other methods on average providing more consistent results than others especially on the extreme case of 75 degrees, where we had 1. 33% accuracy increase over the second best method AFLAC. We trained our model on a Tesla V100 SXM2 32 GB with a server with 64 cores and 80G of ram, for a total of 5 hours and 46 min. The time needed to train the models for classification only without our loss is 2 hours and 18 mins. In order to fully understand what our technique achieves we regenerated the experiment from Fig 4 but with adversarial reconstruction loss used for the training of the model. So our experiment goes as follows, We train the Encoder by the adversarial reconstruction loss and the classification loss as described in Algorithm 1 and after convergence, we re-train a new decoder on the latent space of the MNIST dataset without changing the encoder weights.
3) OBJECT RECOGNITION 2: PACS We followed the same protocol as the previous deep learning papers, by using the same train/test/validation splits for a fair comparison and the same model sizes and pre-trained weights. Our learning rate is α = 0. 003, generalizaing_epochs = 150, pretraining_epochs = 500, and the balancing factor is set to β = 0. 25. PACS object recognition dataset provides a much more challenging setting due to its big image resolution, small sample size, and the notable variation among the domains. Nevertheless, our method outperformed most of the state of the art as table  3  shows. Furthermore, it gave near-perfect results on the Photo target domain being the best at this experiment. Overall, our model performed very well and was ranked 2nd after JigSaw with similar performances as the MetaReg model.
4) ACTION RECOGNITION:IXMAS IXMAS is a human action dataset with 5 actions and 5 different domains. We train on 4 domains and test on the last one. We report the average accuracy of over 20 runs. We use one hidden layer network with 2000 hidden neurons VOLUME 9, 2021 TABLE 5. Skin Lesion results for Domain Generalization. The bolded experiment is the best and underlined in the second. as the previous works did. Our learning rate is α = 0. 01, generalizaing_epochs = 50, pretraining_epochs = 150, and the balancing factor is set to β = 0. 1. We trained our model on a Tesla V100 SXM2 32 GB with a server with 64 cores and 80G of ram, for a total of 6 hours and 08 min. The time needed to train the models for classification only without our loss is 3 hours and 02 mins. From our experimental results in Table  4  we see that our model is very competitive with the state of the art having the best average accuracy and if most experiments either being the best or the 2nd best.
5) SKIN LESION Skin Lesion dataset is an image classification dataset used to benchmark the knowledge transfer abilities of several models. It contains 7 classes and 7 domains (1 primary and 6 secondary). For each of the experiments of Table  5  we use one of the 6 secondary dataset as the target data and the rest as the source. We report the results of the average of 5 runs on each experiment and take the results as mentioned in their original papers. We use the same experimental setup as the state of the art methods used for comparison, by training a Resnet 18 as our base classifier and using mirror of their encoder as our decoder component. Our learning rate is α = 0. 003, generalizaing_epochs = 75, pretraining_epochs = 150, and the balancing factor is set to β = 0. 2. We trained our model on a Tesla V100 SXM2 32 GB with a server with 64 cores and 80G of ram, for a total of 7 hours.
The time needed to train the models for classification only without our loss is 4 hours. As the results on Table  5  show, all the DG techniques can outperform the DeepAll methods (which trains on all the source domains using only the classification loss) which is the expected behaviour. The best method on average is ours with a significant marge of 1. 58%. Our paper provides the best results on 3 out of 6 experiments followed by LDDG and MASF. We can see that the results of the different methods are overall consistent with the difficulty of the domain gaps, where they provide good results on datasets such as PH2 and SON, and fail on datasets such as DMF.
6) PELVIC LANDMARK DETECTION The pelvic Landmark Detection  [56]  dataset is a 3D MR scans dataset manually annotated by expert doctors. It contains 12 landmarks with 423 3D scans of size ranging from 350*350*42 to 370*370*128 for the x, y, and z axes respectively. The dataset contains 3 different domains which represent the different MR squences used for each scan: T1, T2, and PD. We report the results of the average of 20 runs on each experiment. We use a Resnet 18 as our base classifier and using mirror of their encoder as our decoder component. Our learning rate is α = 0. 01, generalizaing_epochs = 50, pretraining_epochs = 150, and the balancing factor is set to β = 0. 03. We trained our model on a Tesla V100 SXM2 32 GB with a server with 64 cores and 80G of ram, for a total of 12 hours.