dataesr/scientific-paragraphs-categorization huggingface dataset:
Input data nature (2D/projected 3D/3D) For texture analysis, the application is usually performed on 2D images. Indeed,  Wu et al. (2016)  were only interested in investigating some layers of the dermis with the strongest collagen intensity. In  Cicchi et al. (2009) , the investigation was also limited to 2D SHG images of human dermis. In the case of a skeletonization, the authors of  Koch et al. (2014)  used two 2D images (one of the fiber skeleton and one of the enhanced image) to determine the fiber radii. In addition, to estimate the fiber length, they used the skeleton of the fibers. Sugita and Matsumoto (2017)  focused also on the fiber length and used 2D SHG images since fiber centers were determined in the 2D plane. It is also possible to extract the fiber network using the CT-FIRE algorithm and determine the fiber length and diameter  (Drifka et al.
). In turn, Pylith allows for the computation of geomechanical equilibrium under static, quasi-static and dynamic conditions, in domains including discontinuities at different scales. This open-source, finite-element code embeds several constitutive laws for faults, and provides templates to implement additional bulk and fault models. While TOUGH2 is written in Fortran77, Pylith is written in C++ and Python. Pylith allows for parallel computing and TOUGH2 runs in serial; however, a recent version, TOUGH3, can be run in parallel  (Jung et al. , 2017). Similarly to TOUGH-FLAC  (Rutqvist, 2011; Blanco-Martín et al. , 2017) , TOUGH-Pylith is based on the fixed-stress split sequential method to couple flow and geomechanics  (Kim et al. , 2009). After an initial version of the simulator  (Miah et al.
To study fault failure more accurately, the fault zone includes a surface of discontinuity. We perform sequential, two-way THM coupling using TOUGH-Pylith, a simulator based on the non-isothermal, multiphase and multicomponent flow simulator TOUGH2, and the geomechanics, open-source code Pylith. The two simulators are successfully coupled for the first time in a THM framework, and TOUGH-Pylith takes advantage of the strengths of the two codes, namely, a wide range of equations-of-state to account for the thermodynamic behavior of pure fluids and mixtures, and the implementation of fault surfaces as discontinuities whose behavior is more representative of observed fault frictional strength, with quasistatic and dynamic ruptures. 2D and 3D simulations can be performed with TOUGH-Pylith. The coupling of the two codes is verified by means of comparison of numerical results with available analytical solutions.
On the other hand, active-learning inference focuses on the most probable region during inference and only trains the NDEs optimized in this local region, so it can effectively save the cost for simulations for inference with data from only one observation. However, this "training during inference" process should be repeated for inference with a new set of observational data, so it is computationally expensive to implement postvalidations with many mock observations, each using activelearning inference. While the active-learning inference is a valid option in our code package 21cmDELFI-PS, we choose to use the amortized inference in this paper for the purpose of posterior post-validations. Our code 21cmDELFI-PS employs the PYDELFI 7  package for the DELFI implementation. We choose the MAFs (see Z22 for a detailed description) as the NDEs with fixed architectures that we find are flexible enough to handle all levels of data sets in 21cmDELFI-PS.
Database We use the Latin Hypercube Sampling to scan the reionization parameter space. For the mock observations with only a cosmological 21 cm power spectrum, we generate 18,000 samples with different reionization parameters used for training the NDEs, and 300 additional samples for testing or validating the DELFI. For the mock observations with noise and foreground cut, we generate 9000 samples with different reionization parameters, and make 10 realizations of total noise and foreground cut for each such sample-because the noise is random-so a total of 90,000 samples are used for training the NDEs, and 300 additional samples are used for testing the DELFI. We discuss the effect of the sample size on the inference performance in Appendix B. The initial conditions for all samples (with different reionization parameters) were independently generated by sampling spatially correlated Gaussian random fields with the matter power spectrum given by linear theory.
Fast soil moisture content (SMC) mapping is necessary to support water resource management and to understand crops' growth, quality and yield. Thereby, Earth Observation (EO) plays a key role due to its ability of almost real-time monitoring of large areas at a low cost. This study aimed to explore the possibility of taking advantage of freely available Sentinel-1 (S1) and Sentinel-2 (S2) EO data for the simultaneous prediction of SMC with cycle-consistent adversarial network (cycleGAN) for time-series gap filling. The proposed methodology, first, learns latent low-dimensional representation of the satellite images, then learns a simple machine learning model on top of these representations. To evaluate the methodology, a series of vineyards, located in South Australia's Eden valley are chosen.
Specifically, we presented an efficient framework for extracting latent features from S1 and S2 imagery. We showed how one could use S1 to S2 feature translation based on Cycle-GAN using S1&S2 time series when there are missing images acquired over an area of interest. The resulting data in our study is then used to fill gaps in time series data. We used the resulting latent representations to predict SMC with various ML tools. In the experiments, cycleGAN and the autoencoders were trained with data randomly chosen around the site of interest, so we could augment the existing dataset. The best performance was demonstrated with random forest algorithm, whereas linear regression model demonstrated significant overfitting. The experiments demonstrate that the proposed methodology outperforms the compared state-of-the-art methods if there are missing optical and synthetic-aperture radar (SAR) images.
The autoencoders are specifically learned through a reconstruction problem with 2 -loss for building such representations. These representations are then concatenated and fed into a machine learning model for predicting soil moisture. A technical difficulty arises then from the fact that some SM measurements might have corresponding images from either S1 or S2, and so we end up having missing data. To leverage this technical issue, we make use of the cycleGAN model to recover the missing data through an unsupervised manner. The cycleGAN method is an unsupervised deep learning model which consists in learning two mappings F : X → Y and G : Y → X in order to translate images from a domain X to a domain Y and vice-versa. Denoting by p X and p Y the probability distributions of the domains X and Y respectively, the cycleGAN objective function  [29]  is given by:.
III. STUDY AREA AND PROBLEM DEFINITION A series of Sentinel-1/2 images were used from early 2017 to April 2020 to cover SM measurements for 200 acres of vineyards located in the Upper Hunter Valley region in Australia. Ground Range Detected (GRD) Sentinel-1 backscattering data at HH and HV polarizations were collected during both ascending and descending orbits. 10 m and 20 m spatial resolution Sentinel-2 bands were downloaded in Level 2A, which provides a shadow and cloud mask and top of canopy reflectance. In addition to the backscattering and spectral reflectance values, backscattering ratio and vegetation index, namely Normalized Difference Vegetation Index (NDVI) were formed. All these continuous different resolution variables were resampled to 10 × 10 m resolution according to the modelling process.
IV. EXPERIMENTAL RESULTS In order to estimate SMC over vineyards, the proposed architecture is applied in three steps. Firstly, the training strategy of cycleGAN for image translation is introduced for recovering missing S1 or S2 images. Secondly, low-dimensional representations of the data are extracted through pre-trained autoencoders. With this low-dimensional representation data, SMC estimation is finally performed using various prediction models which are presented with their respective accuracy assessments.
This model is a meta estimator that fits a number of regression decision trees on various subsamples of the same dataset and uses averaging to improve the predictive accuracy and control over-fitting. 4) SVM Regressor (SVR): Support vector machine (SVM) regression is a non-parametric machine learning model, which relies on kernel functions. SVR approach depends on some subset of the training data, called the support vectors. Radial basis kernel function was used for the SVR. 5) Ridge Regression and Kernel Ridge Regression: Finally, we train ridge regression and kernel ridge regression models. Ridge regression model differs from simple linear regression by the addition of regularisation, given by the l2-norm. Kernels are used to calculate the inner product of two vectors in a feature space.
We add kernels to ridge regression to add nonlinearity. This increases the accuracy of the model on our dataset. The RF performs better than the other algorithms, neural networks and kernel regression give slightly similar results and then comes Ridge regression, whereas the linear regression model demonstrated significant overfitting, while the other methods do not perform well at all. In order to justify the usage of such a complex we have run a series of experiments with GANs and CNN, avoiding dimensionality reduction. However, one of the main drawbacks of CNNs is that they need a lot of data to perform well enough. In this case, the resulting amount of images was not enough to successfully train a CNN to predict SMC. To overcome the issue of not having enough data, we proposed to the inference part into two different steps: dimensionality reduction followed by ML classifiers, thereby offering the possibility to work with a limited amount of data.
In generative modelling, we are given a dataset of samples x drawn from some unknown probability distribution p r (x), where r stands for 'real'. We use the samples x to derive the unknown real data distribution p r (x). A generative model G encodes a distribution over new samples, p g (x), where g stands for 'generated'. The aim is that we find a generative distribution such that pg(x) ≈ pr(x) according to some metric. The most popular metric, used so far in generative models, is inception score (  [38]. The Inception Score (IS) is a metric for automatically evaluating the quality of image generative models  [37]. It uses Inception-V3 model  [42] ), pre-trained on a large dataset of general-purpose images such as ImageNet  [43]. The IS calculates a statistic of the network's outputs when applied to generated images, in words the pretrained Inception-V3 model is used to classify generated images and the quality of images is assessed on the accuracy of the predicted class.
Formally, the IS can be written as following: IS(G) = exp(E x∼pg D KL (p(y|X) p(y))), (3) where X ∼ p g indicates that x is an image sampled from p g , D KL (p q) is the KL-divergence between the distributions p and q, p(y|X) is the conditional class distribution, p(y) = X p(y|X)p g (X) is the marginal class distribution  [37]. IS metric is widely applied in literature, however, it was recently found that applying the Inception Score to generative models trained on datasets other than ImageNet gives misleading results. Barratt and Sharma  [38]  showed that the results of IS performance even on a dataset, close to ImageNet, such as CIFAR-10, can be not good since the classes in ImageNet and CIFAR-10 do not line up identically. Therefore, this metric cannot be, since satellite imagery classification is very far from ImageNet in terms of both classes it is trained on and band resolution of images (ImageNet has 3 bands (RGB), whereas Sentinel imagery has 12).
Even when we use only the True color version of the Sentinel-2 imagery, the classification for any possible class will be far away from ImageNet. The only option to this metric is to train Inception-V3 on satellite imagery for some related classification task, this is not possible without a large dataset for satellite imagery classification. Another metric, used to evaluate GANs, was developed recently, called Fréchet Inception Distance (FID)  [39]. It was proposed as an improvement over Inception Score. Similarly to IS, FID uses the Inception-V3 network as part of its calculation. However, instead of using the classification labels of the Inception-v3 network, it uses the output from a feature layer. Research has shown that deep convolutional neural networks trained on difficult tasks, like classifying many classes, build increasingly sophisticated representations of features going deeper into the network: the first few layers may learn to detect different kinds of edges and curves, also colour and texture, whereas the later layers respond to increasingly more complex stimuli, including parts of objects that they were trained to recognise.
The feature vector for a given image is computed by feeding it to a pre-trained VGG-16 classifier  [41] , this method accessible for the reasons described above. To perform similar classification, we needed to train our own classifier. We used a classifier network that we trained on the dataset, obtained in one of our previous studies. The model was trained on a large dataset of 7000 vineyard blocks in the Australian region. However, the results for this analysis were significantly lower than the state of the art results for GAN models, trained on large datasets. We believe that the reason for such a performance is that the features that were learned by our classifier (vineyard detection) differ from the ones that are required for soil moisture estimation. To summarise, the best way to estimate the performance of the proposed GAN is to visually estimate the fidelity of the image.
VI. CONCLUSIONS In this paper, cycleGAN methodology is proposed for monitoring any biophysical parameters using S1 and S2 data. In particular, the limitations and potentials of SMC estimation using S1 and S2 data, for the first time, with the GAN-based architecture is explored. To our knowledge, this is the first time autoencoders and cycleGANs were used together to obtain features from Sentinel-1 and Sentinel-2 imagery. This research demonstrated the feasibility of the proposed approach and allows us to fuse two different sources of information for efficient prediction of soil moisture content. Being one of the major horticulture crops consumed worldwide, grapevine is chosen to assess SMC estimations in horticulture from space. In-situ measurements conducted simultaneously with satellite acquisitions provide a discussion on the feasibility of Sentinel-1/-2 data to retrieve soil properties of vineyard, specifically SMC.
The results of the experiments on the data set demonstrate that the proposed methodology is effective in regression based biophysical parameter estimation in agriculture studies, in which the possibility of having simultaneous radar and optical image acquisitions is critical due to their complementary information. Although the proposed architecture (cycleGAN coupled with autoconders) in this paper is promising for SMC estimation in vineyards, the similar performance could be achieved with simpler ML model for different crops with different conditions (e. g. , crop specific irrigation, planting patterns, the presence of dense in-situ data and etc. ). Furthermore, the regression based biophysical parameter estimation using EO data is widely accepted in plantation agriculture. However, when it comes to orchard crops, the complex interaction among microwave signal, canopy morphology and soil makes the radar based studies more complicated on woody crops compared to herbaceous crops.
Research in context Evidence before this study We searched PubMed from database inception to Jan 18, 2022, using the search terms ((machine learning) OR (artificial intelligence)) AND ((primary aldosteronism) OR (pheochromocytoma) OR (paraganglioma) OR (Cushing's syndrome) OR (endocrine hypertension) OR (hypertension)) AND (Multiomics), with no language restrictions. Our search yielded 10 results. Three articles reviewed the use of artificial intelligence in cardiovascular medicine and hepatology. Although one similar review article focussed on primary hypertension, however, it did not include any multi-omics based study on hypertension. Five research articles used machine learning in conjunction with multi-omics for major heart diseases, intraocular pressure and metabolic syndrome. One of the articles studied heart failure in rats.
Plasma miRNA Levels of 179 human miRNAs were measured in plasma after extraction of total RNA from 200 µL EDTA-plasma using the miRNeasy Mini kit (QIAGEN, Manchester, UK). 4 µL of undiluted RNA was then reverse-transcribed to cDNA in a 20 µL reaction volume using the Universal cDNA synthesis kit II (Exiqon, Vedbaek, Denmark). Selected plasma miRNAs were quantified using Serum/Plasma Focus microRNA PCR Panels (384-well, V4. M, Exiqon) according to their standard protocol, in combination with ExiLENT SYBR Ò Green master mix (Exiqon) and ROX solution (Thermo Fisher, Renfrew, UK) on a Quantstudio 12K Flex Real-time PCR System (Thermo Fisher). Raw data generated by the QuantStudio System were analysed using GenEx software (v. 6, MultiD Analyses, Vedbaek, Denmark). Quality controls all along the procedure were performed using different spike-in RNAs and cDNAs.
Generation of the MOmics dataset All the omics data was catalogued in RDMP  33  for systematic access. Although biosamples for 487 patients were included in the study, after quality controls omics measurements for all five omics were available for 408 (CS = 30, PA = 100, PPGL = 69, PHT = 108, and NV = 101) patients (Appendix Table  3 ). In total 409 MOmics features (PmiRNA: 173, PMetas: 4, PSteroids: 16, USteroids: 27, and PSmallMB: 189) along with age and sex were exported and used to conduct supervised ML experiments (Table  1 ). The names of features were prefixed with 'O1_', 'O2_', 'O3_', 'O4_', 'O5_', and 'O6_' for PmiRNA, PMetas, PSteroids, USteroids, and PSmallMB respectively. The complete list of all MOmics features and their distribution is included in the Appendix Table  2. The pairwise relationship amongst all features was evaluated by clustering Pearson correlation coefficients (Appendix Figure  1 ).
Training, validation and test dataset The supervised machine learning used non-overlapping training, validation and test datasets. These can be defined as: (a) training dataset: The sample of data used to fit the model, (b) validation dataset: The sample of data used to validate the trained models during the 100 random repeats and (c) test dataset: The independent samples of 'unseen' data used to test the final trained model. The MOmics and mono-omics datasets were once randomly split into training (»80%) and testing (»20%) set (Appendix Table  4 ). The training set was then further split into training-validation (80À20%) split for 100 Random Repeats (RR) using random seed for each iteration to ensure reproducibility. Finally, the test set was only used for independent validation of the final trained model and not used during any model training.
Disease combinations The five different disease combinations used for classification were: ALL-ALL (PPGL vs PA vs CS vs PHT), EHT (PPGL+PA+CS)-PHT, and each individual endocrine hypertension (i. e. , PPGL/PA/CS)-PHT. These combinations did not include NV since the key question addressed was: How can hypertensive patients be stratified amongst themselves? Omics data from NV was used to compare individual biomarkers with patients of different hypertension types. This data was not used for any model development or testing.
Classifiers, feature selection and classification performance metrics An assorted set of 8 different classifiers: Decision Trees (J48),  34  Na€ ıve Bayes (NB),  35  K-nearest neighbours (IBk),  36  LogitBoost (LB),  37  Logistic Model Tree (LMT),  38  Simple Logistic (SL),  39  Random Forest (RF),  40  and Sequential Minimal Optimisation (SMO)  41  were used (Stage 1 of Figure  1 ) with given hyperparameter settings (Appendix Table  5 ). The classification was implemented using the caret  42  and RWeka 43 in R  44  and software code was released. 45 For feature selection, wrapper (Boruta)  46  and filter (Correlation-based feature selection À CFS)  47  methods were compared (Stage 2 of Figure  1 ). The classifications were evaluated over 100 RR using performance metrics: balanced accuracy (to adjust for the class imbalance problem), sensitivity, specificity, AUC, F1, and Kappa score.
Patients and omics measurements The study included 487 patients with PA, PPGL, CS and PHT as well as normotensive volunteers (NV) (PA = 113, PPGL = 88, CS = 41, PHT = 112, and NV =133), who were recruited by reference centres for adrenal disorders of the ENS@T-HT Horizon2020 consortium. 19 Diagnosis was based on current guidelines for each disease in each expert centre. Omics studies involved measurements of plasma miRNA (PmiRNA: 173), plasma catechol O-methylated metabolites (PMetas: 4), plasma steroids (PSteroids: 16), urinary steroid metabolites (USteroids: 27), and plasma small metabolites (PSmallMB: 189) in biosamples collected from each patient within 24 h. After completion of omics measurements, quality controls and data cleaning, 408 patients had complete omics sets for further analysis (Appendix Table  3 ).
Performance metrics The MOmics classifier outperformed mono-omics classifiers when considering balanced accuracy, AUC (except CS-PHT), F1, and Kappa score (Figure  2a ). Using MOmics data for ALLÀALL combination, RF classifier (trained with balanced dataset) provided better classification performance on test set (»92% balanced accuracy, 0. 95 AUC with 88% sensitivity, and 96% specificity) when compared to other 5 mono-omics. The corresponding decision value (prediction probability) for each test sample was evaluated (Figure  2b ). High decision values highlighted the confidence of the classifier in predicting the test sample. Many correctly classified samples had high decision values, which emphasise the fact that MOmics classifier provided better performance in comparison to others. For ALL-ALL, MOmics classifier had 7 incorrectly classified samples (Figure  2b  À top row, left).
Also, PmiRNA-based LB classifier provided 0. 99 AUC with 81% specificity. Moreover, for the CS-PHT combination, the MOmicsbased SL classifier provided 100% specificity and »92% balanced accuracy, but with a lower AUC of 0. 93. In contrast, mono-omics based classifiers using PSteroids and USteroids achieved higher AUC of 0. 98 and 0. 97 respectively. The probability values for the test set showed the difference of confidence amongst classifiers (Figure  2b  À bottom row). These classifiers were also tested on the training dataset to understand the effect of overfitting (Appendix Table  6 -10). Amongst the three classifiers (LB, RF and SL), evidently RF provided superior classification results when tested on the training set in comparison to the testing set. This highlighted the overfitted training of the RF classifier irrespective of whether the training data was balanced or not.