dataesr/scientific-paragraphs-categorization huggingface dataset:
Performance comparison between GestaltMatcher and DeepGestalt To validate the GestaltMatcher approach, we first worked with the 323 images of patients with 91 syndromes from the London Medical Database (LMD)  20  that were already used for benchmarking the performance of DeepGestalt 12. When using the frequent gallery, which contains syndromes that DeepGestalt currently supports, GestaltMatcher achieved 64. 30% and 86. 59% accuracy within the top-10 and top-30 ranks, respectively, which was lower than the 81. 28% top-10 accuracy and 88. 34% top-30 accuracy achieved by DeepGestalt with a Enc-F2G softmax approach (Supplementary Table  2  and 3 ). However, when we used the gallery of all 1,115 syndromes for GestaltMatcher (frequent + rare) which is a search space that is roughly four times larger, the top-10 and top-30 dropped by only 2.
However, a more systematic analysis of pairwise distances still revealed considerably smaller distances between subjects with de novo mutations and their family members than between these subjects and unrelated individuals (Supplementary Figure  9 ). This reflects similarities in the nonclinical features of the face, which is also higher within the same ethnicity and is a known confounding factor for the GestaltMatcher approach. However, it is a bias that can be attenuated 23 and will also diminish over time when more diverse training data become available  24.
We analyzed the correlation of the mean of the distinctiveness score from human experts with the top-10 accuracy that GestaltMatcher achieves for these syndromes without having been trained on them (Figure  5a , Supplementary Table  6 ). The Spearman's rank correlation coefficient was 0. 400 (P = 0. 004), indicating a clear positive correlation between distinctiveness score and top-10 accuracy. Syndromes with a higher average score tended to perform better, with Schuurs-Hoeijmakers syndrome being amongst the best-performing syndromes in GestaltMatcher. The analysis on 20 selected syndromes from the GMDB dataset also showed a positive correlation between distinctiveness score and top-5 accuracy (Supplementary Figure  10  and Supplementary Table  7 ). The correlation for GestaltMatcher accuracy and disease prevalence was not significant (P = 0.
For our cluster analysis, we sampled subjects in our database with subtypes of four large phenotypic series and found high intersyndrome separability in addition to considerable intrasyndrome substructure in Noonan syndrome, CdLS, Kabuki syndrome, and mucopolysaccharidosis. A t-SNE  26  projection of the FPDs into two dimensions yielded the best visualization results (Supplementary Figure  11 ). Although any projection into a smaller dimensionality might cause a loss of information, the clusters are still clearly visible for the 743 subjects sampled from these four phenotypic series. This observation provides further evidence that characteristic phenotypic features are encoded in the FPDs. To demonstrate the separability of syndromes with facial dysmorphism, we also used t-SNE to project 4,353 images of the ten syndromes from the frequent set with the largest number of subjects and 872 images of ten non-distinct syndromes (syndromes without facial dysmorphism) into 2D space.
In addition, we calculated the Silhouette index  27  for both of these datasets. The FPDs of the frequent syndromes showed ten clear clusters of subjects (Supplementary Figure  12 ), but the t-SNE projection of subjects with non-distinct syndromes created no clear clusters. Moreover, the Silhouette index of the frequent syndromes (0. 11) was higher than that of the nondistinct syndromes (-0. 005); the negative Silhouette index indicates poor separation of the non-distinct syndromes.
GestaltMatcher's framework also allows us to abstract the encoding of a dataset away from the classification task. For example, one can evaluate both phenotypic series and pleiotropic genes within a single CFPS, or obtain the most-similar patients for each of the matched syndromes, with minor computational cost (i. e. , in real time). Furthermore, the GestaltMatcher framework computes the similarity between each of the test set images across the entire dataset of images. This similarity can be computed using different metrics, e. g. , cosine or Euclidean distance. The results are then aggregated according to the chosen configuration. For example, image similarity can be aggregated at the patient level or the syndrome level. Furthermore, the dataset can be filtered according to different parameters (such as ethnicity, disease-causing genes, or age) to further customize the evaluation.
We matched 40 of 79 different families in 15 GeneMatcher publications by top-30 rank (Table  2  and Supplementary Figure  8 ), and 11 candidate genes are currently under evaluation. This result shows the power and potential of GestaltMatcher to identify novel syndromes. Although the number of individuals and the diversity of their phenotypes will affect the performance, cases with a high syndromic similarity will remain matchable due to the high dimensionality of the CFPS. We therefore hope that GestaltMatcher will be readily integrated into other matching platforms to aid in determining which phenotypes should be grouped together into a syndrome or phenotypic series, as well as linking individual patients to a molecular diagnosis. We collected images of subjects with clinically or molecularly confirmed diagnoses from the Face2Gene database (https://www.
face2gene. com). Extracted, deidentified data were used to remove poor-quality or duplicated images from the dataset without viewing the photos. After removing images of insufficient quality, the dataset consisted of 26,152 images from 17,560 subjects with a total of 1,115 syndromes (Supplementary Table  9 ). GestaltMatcher was designed to distinguish syndromes with different properties. We separated syndromes by the number of affected subjects and whether they had already been learned by the DeepGestalt model. Supplementary Figure  14  provides an overview of how the dataset was divided. The current DeepGestalt approach requires at least seven subjects to learn a novel syndrome. We first used this threshold to separate the syndromes into "frequent" and "rare" syndromes. The objective of our study was to improve phenotypic decision support for "rare disorders".
Of the 1,115 syndromes in the entire dataset, 299 were frequent and 816 were rare. DeepGestalt cannot yet be applied to rare syndromes. We further divided each of these two datasets into a gallery and a test set. The gallery is the set of subjects that we intend to match, given a subject from the test set. First, 90% of subjects with each frequent syndrome were used to train the models, and the remaining 10% of subjects were used to validate the DeepGestalt training; the 90% then became the frequent gallery and the 10% were assigned to the frequent test set. For the rare dataset, we performed 10-fold cross-validation. In each syndrome, 90% and 10% of subjects were assigned to the gallery and test set, respectively. The test sets were designed to have the same distribution of distinctiveness as the training sets.
GMDB dataset We collected images of subjects with clinically or molecularly confirmed diagnoses from publications and individuals that gave appropriate informed consent for the purpose of this study. This dataset can be used as a public training and test set for benchmarking and is available at GestaltMatcher Database (https://gestaltmatcher. gene-talk. de). At the time of the data freeze on 9 June 2021, the dataset consisted of 4,306 images of 3,693 subjects with a total of 257 syndromes from 902 publications (Supplementary Table  9 ). Six of the 3,693 subjects have not yet been published, but appropriate consent has been obtained. For a fair comparison with the Face2Gene dataset, we performed the data separation in the same way. The dataset was first split by the same threshold (seven subjects) into frequent and rare datasets, giving 139 syndromes in the frequent dataset and 118 syndromes in the rare set.
Both datasets were also later separated into gallery and test sets. The data split is shown in Supplementary Figure  15. Of the 3,693 subjects in GMDB, 963 are also in Face2Gene dataset. To use the GMDB rare set as the test set for both the GMDB frequent set and the Face2Gene frequent set, we made sure that there is no syndrome that is in both the GMDB rare set and Face2Gene frequent set (Supplementary Figure  16 ).
London Medical Dataset validation analysis We compiled 323 images of patients diagnosed with 91 frequent syndromes from the LMD  19  and used this as the validation set for frequent syndromes. We first evaluated the validation set using softmax, which is a DeepGestalt method. To compare the performance with that of GestaltMatcher, we evaluated the performance of GestaltMatcher on two different galleries: a gallery of frequent syndromes consisting of 19,950 images of patients with 299 syndromes, and a unified gallery consisting of 22,298 images of patients with 1,115 syndromes. We then reported the top-k accuracy and compared the results of these three settings (DeepGestalt with softmax, GestaltMatcher with frequent gallery, and GestaltMatcher with unified gallery).
Matching undiagnosed patients from unrelated families We selected 15 articles published from 2015 to 2019 in which GeneMatcher was used to establish an association of a gene with a novel phenotype with facial dysmorphism from unrelated families. In total, these studies contained 108 photos of 91 subjects from 79 families. The details are shown in Table  2. The 15 genes were not among the Face2Gene frequent syndromes, so we can consider them each as a novel phenotype to the model. We performed leave-one-out cross-validation on this dataset; that is, we kept one photo as the test set, and we assigned the rest of the photos to a gallery of 3,533 photos with 816 rare syndromes to simulate the distribution of patients with unknown diagnosis. We then evaluated the performance by top-1 to top-30 rank.
We calculated Spearman rank correlation coefficients for the following two pairs of data: between top-10 accuracy and the syndrome's distinctiveness score, and between top-10 accuracy and the prevalence of syndromes collected from Orphanet. The same analysis was also performed on the GMDB dataset. We selected 20 syndromes from GMDB frequent instead of 50 syndromes because the GMDB dataset is smaller than the Face2Gene dataset, and we trained the Enc-GMDB-exclude-20 on the remaining 119 frequent syndromes. The details of the 20 selected syndromes and the results are reported in Supplementary Table  7. Please note that we report the top-5 accuracy in the GMDB dataset instead of top-10 accuracy because of the smaller number of syndromes in the gallery.
For each setting (number of subjects, number of syndromes), we had five models. We then encoded the photos separately with each model and tested them on the rare syndromes, which had not been seen by the models. In the end, we averaged the performance by the five models and report the top-10 accuracy for each setting in Figure  3. We also used the models described above to encode the GMDB dataset, tested them with the GMDB rare set, and report the results in Supplementary Figure  2. Because the GMDB dataset is smaller than Face2Gene dataset, we were not able to use the same number of subjects and syndromes to perform the analysis. For the GMDB dataset, we used 10, 20, 40 for the number of subjects, and the syndrome intervals of [1, 10], [11, 40], and [41, 80]. The results of training on GMDB and testing of the GMDB rare set are shown in Supplementary Figure  3.
DeepGestalt was first trained on images of healthy individuals from CASIA-WebFace 19 , and later fine-tuned on a dataset with patient images (Face2Gene or GMDB). The encoder without fine-tuning on patient images was called Enc-healthy. The encoder later trained on 299 frequent syndromes in the Face2Gene dataset was named Enc-F2G. The encoder trained on 139 frequent syndromes in GMDB was named Enc-GMDB. In the following sections, we have several encoders trained on different subsets of the Face2Gene and GMDB datasets. The summary of all the encoders used in this study is shown in Supplementary Table.
Accurate seasonal forecasting of cereal yields is an important decision support tool for countries, such as Morocco, that are not self-sufficient in order to predict, as early as possible, importation needs. This study aims to develop an early forecasting model of cereal yields (soft wheat, barley and durum wheat) at the scale of the agricultural province considering the 15 most productive over 2000-2017 (i. e. , 15 × 18 = 270 yields values). To this objective, we built on previous works that showed a tight linkage between cereal yields and various datasets including weather data (rainfall and air temperature), regional climate indices (North Atlantic Oscillation in particular), and drought indices derived from satellite observations in different wavelengths. The combination of the latter three data sets is assessed to predict cereal yields using linear (Multiple Linear Regression, MLR) and non-linear (Support Vector Machine, SVM; Random Forest, RF, and eXtreme Gradient Boost, XGBoost) machine learning algorithms.
The calibration of the algorithmic parameters of the different approaches are carried out using a 5-fold cross validation technique and a leave-one-out method is implemented for model validation. The statistical metrics of the models are first analyzed as a function of the input datasets that are used, and as a function of the lead times, from 4 months to 2 months before harvest. The results show that combining data from multiple sources outperformed models based on one dataset only. In addition, the satellite drought indices are a major source of information for cereal prediction when the forecasting is carried out close to harvest (2 months before), while weather data and, to a lesser extent, climate indices, are key variables for earlier predictions. The best models can accurately predict yield in January (4 months before harvest) with an R 2 = 0.
Instead of using a single data source as predictors of crop yield, several studies have combined multi-source data to predict crop yield. For example, Cai and Sharma,  [46]  and Balaghi et al. [12]  combined remote sensing data (NDVI) and weather data (rainfall and temperature) as predictors of rice and wheat yield in India and Morocco, respectively. Most of the models developed in the previously cited studies are based on the classical Multiple Linear Regression (MLR) while the linkages between yields and potential predictors are likely to be non-linear. For this reason, non-linear machine learning algorithms have been employed to improve crop yield prediction  [47]. Recently, several studies have examined the performance of machine learning algorithms such as Support Vector Machine (SVM), Random Forest (RF), eXtreme Gradient Boost (XGBoost), Artificial Neural Network (ANN) and Long-Short Term Memory (LSTM) for yield forecasting at county or province scales.
75). For Morocco, the existing literature on seasonal yield forecasting is limited. Balaghi et al. [12]  proposed empirical linear regression models to forecast wheat yields up to 2 months before the harvest at a provincial and national scale. NDVI from AVHRR, rainfall sums and average monthly air temperatures were used. More recently, Lehmann et al. [45]  have used SST and causal precursors from geopotential height anomalies at 500 hPa to forecast wheat yield anomalies at the country scale. Several studies have used the combination of multi-source datasets and machine learning algorithms to forecast crop yield including cereals  [48] [49] [50] [51] [52] [53] [54]. However, the combination between remote sensing drought indices, weather data and climate indices as predictors of cereals yield has not been assessed yet.
Materials and Methods In this study, the target variable is cereal yield. The potential predictors are the satellite drought indices, weather data (rainfall and temperature) and climate indices derived from atmospheric and oceanic variables. In order to limit the number of agricultural provinces, a threshold of 90% of the national production was set: the 15 selected provinces corresponding to the most productive are displayed in Figure  1. The forecasting models are then built using the extensively used multi-linear regression approach and three nonlinear machine learning methods. An overview of the methodology is represented in the flowchart of Figure  2. Table  1  lists all the raw datasets with their sources. Table  2  displays the predictor variables derived from these raw datasets together with the time span of the year considered in the model based on biophysically sound linkages highlighted by previous studies.
Choice of Input Data sets In order to identify the best combination of input data among the satellite-based drought indices, the weather data and the climate indices, the seasonal forecasting models of cereal yields were developed using the different combinations of input data within the season, about 1 month prior to harvest in April, by considering all available predictors from October to April (Table  2 ). All the provinces are considered to build a so-called global model. The statistical metrics for the different combinations of input datasets and for the different methods are reported in Table  3. The results presented in Table  3  show that the.
Yield Data Soft wheat, barley and durum wheat are the main types of cereals cropped in Morocco. Data on cereal crop productions and harvested areas over the study period 2000-2017 at the administrative provincial scale were gathered by the Economic Services of the Ministry of Agriculture in Morocco (https://www. agriculture. gov. ma/, accessed on 31 July 2021). The yearly cereal yield used in this study for each province was calculated as the ratio of the total crop production by the total harvest areas. The average cereal yield over the study period ranged from 0. 7 t. ha -1 to 2. 2 t. ha -1 depending on the province. The 15 selected provinces were classified into four groups with similar cereal yield interannual variability using a k-means algorithm based on the correlative distance  [58].
Machine Learning Methods for Cereal Yield Forecasting In order to build the seasonal forecasting models, we relied on Multiple Linear Regression, and three non-linear machine learning algorithms extensively used for crop yield prediction  [47] , which are: Support Vector Machine (SVM), Random Forest (RF) and eXtreme Gradient Boost (XGBoost). The scikit-learn package in Python 3. 7  [68]  was used in this study.
Random Forest (RF) The RF algorithm was introduced by Breiman (2001) and is used for both classification and regression. RF for regression is an ensemble of multiple decision trees regression model; each tree provides its prediction and the optimal prediction of RF obtained by averaging the prediction of all decision trees regression in RF  [70]. The RF-based model follows three steps to provide the optimal predictions. In the first step, the dataset is split into data subdivisions. In the second step, each data subdivision is used to develop a single decision tree representing a sub-regression model that gives its prediction. In the last step, predictions of all decision trees are averaged to provide the final prediction. The hyper-parameters that need to be tuned in the RF algorithm are the number of trees or the number of regression trees, the number of features to consider when looking for the best split, and the maximum depth of the tree.
Four experiments were then designed. The first experiment was constructed to identify the best combinations of input datasets among the satellite-based drought indices, the weather data and the climate indices that will reach the high performance in forecasting final cereal yield in Morocco. For this reason, all machine learning algorithms were applied using the different combinations of available input data collected from October to April (see Table  2 ): (i) Satellite-based drought indices only; (ii) Satellite-based drought indices and weather data; and (iii) Satellite-based drought indices, weather and climate data. The second experiment was conducted in order to assess the performance of the models as a function of the lead time before harvest from 4 to 2 months. In this experiment, we used the best combination of inputs data, determined from the first experiment, and the input data were collected from October to January, October to February and from October to March, based on Table  2 , to build the forecasting model in January, February and March, respectively.
Then, the performance of machine learning models was evaluated in March, February and January which corresponds to 2, 3 and 4 months before the harvest. In addition, the importance of each input data point was computed using the best machine learning algorithm in order to assess the contribution of each input data point for each lead time of prediction. The third experiment was designed to test the practical performance of the developed models. For this reason, the predictions are performed using a "leave-one year-out" approach consisting in predicting the yield value of one year using all the other years data to train the model (for instance, yield in 2017 is predicted based on a model trained using the 2000-2016 database). Finally, the last experiment was designed to assess the performance of using specific models developed separately for each group of provinces with regards to one global model used in the previous experiments.
Choice of Input Data Sets In order to identify the best combination of input data among the satellite-based drought indices, the weather data and the climate indices, the seasonal forecasting models of cereal yields were developed using the different combinations of input data within the season, about 1 month prior to harvest in April, by considering all available predictors from October to April (Table  2 ). All the provinces are considered to build a so-called global model. The statistical metrics for the different combinations of input datasets and for the different methods are reported in Table  3. The results presented in Table  3  show that the statistical metrics of the model improve with the increase of the number of datasets used for prediction. In addition, all statistical metrics are improved when adding a dataset and this is also true for all the tested methods.
The results showed that the yield variability is reasonably explained with satellite-based drought indices only, with R 2 values ranging from 0. 67 (for MLR) to 0. 81 (for XGBoost) and RMSE from 0. 66 t. ha -1 (for MLR) to 0. 44 t. ha -1 (for XGBoost). By combining satellite-based drought indices and weather data, the performances of all models are improved by 2-7% for R 2 and by 25-30% for RMSE. The best statistical metrics are obtained by combining the three datasets with a further improvement of the statistical metrics by about 11-45% for RMSE and 4-10% for R 2 depending on the used method. This means that climate indices such as the Northern Hemisphere Teleconnection Patterns (NAO and SCA) and the main modes of SST variability in the Atlantic contributes to improving the model performances.
Model Performance as a Function of Lead Time before Harvest In this section, the performance of the forecasting models using the three datasets are evaluated as a function of the leading time prior to harvest from January to March (from 4 to 2 months before harvest). The RMSEs and R 2 of the models are plotted as a function of the lead time in Figure  3  to investigate the prediction accuracy. In addition, the relative importance of each dataset is reported in Figure  4  using the XGBoost algorithm as the method providing the best statistical metrics for all lead times. from January to March at Figure  3. The best method whatever the lead time is XGBoost as already shown, followed closely by RF based approaches. The models based on XGBoost explain 88%, 92% and 96% of yield variability (RMSE of 0.