dataesr/scientific-paragraphs-categorization huggingface dataset:
This external set was collected by Avdeef  [29]  and is made up of four publicly available sets with intrinsic solubility values reported. To further evaluate the workflow automatization, an external dataset II was also selected. To create this database, a search was carried out with the following keywords (new molecules, experimental solubility, µM, and log S). The final external set II eventually consisted of 30 new molecules reported in the last three years  [30, 31]. For these external evaluations and with the aim of increasing the methodology prediction performance on the external datasets, the CLEAN and RELIABLE solubility subsets were concatenated, and the model was retrained in this richer dataset to increase the structural diversity of the training set. The supporting information includes a detailed list of the molecules belonging to each set as well as their solubility values (Tables  S1, S2 , S3 and S4, supplementary data).
Model building and evaluation One of the main problems in estimating solubility is the fact that solubility values can vary over a wide range (several orders of magnitude). To develop local regression models based on two sections of the solubility range of our data set, we implemented a consensus model based on a combination of classification and regression techniques. The molecules belonging to CLEANED data were split into two distinct tagged classes according to log S values. Molecules with log S  -2 (n = 4395) were tagged as soluble and highly soluble (Class 1), while molecules with log S < -2 (n = 7918) were tagged as slightly soluble and insoluble (Class 2). According to this rule, a Gradient Boosted Tree model was trained for classification. Two distinct regression models were developed to predict aqueous solubility for molecules belonging to separate Class 1 and Class 2 of the CLEANED data.
To demonstrate that the model can incorporate the information represented by the descriptors and relate it to solubility in the training set, the RELIABLE dataset was used as a test set. In order to increase the chemical diversity of the training set and obtain a better prediction of the external sets, the procedure described above was performed using the CONCATENATED data (RELIABLE + CLEANED) as the training set. Details on the consensus model are summarized in Figure  4. Additionally, a classification model for aqueous solubility was carried out starting from the regression model by thresholding its output prediction (see Table  5 ). Considering that there isn't a global consensus to select the cut-off value to be used in a binary classification of solubility, molecules with log S < -4 were regarded as insoluble in this study, while those with log S  -4 were regarded as soluble.
Analysis of the new database of aqueous solubility A new large and diverse database of aqueous solubility was generated. Once the database was carefully curated and tested, its solubility values ranged from -16 to + 2. 1 logs units. However, the quality of this type of database with a wide range of solubility has been questioned  [34]. According to Bergström and Larsson, the determination of concentrations of molecules with log S < -12 (picomolar) requires very sensitive analytical methods, while molecules with log S > + 1. 7 (55, molar) means that these molecules are more soluble than the water in water, so the use of this type of data in QSPR models may affect their accuracy. In this sense, the final database of log S between -12 to +1. 7 was composed of 12674 molecules. The statistical distribution on the log S scale is close to normal (skewness = -0.
57 and excess kurtosis = 0. 19) and contains a wide chemical diversity. An analysis of the dataset on the distribution of properties in the chemical space defined by an extended rule of five (eRo5) shows its broad diversity and its potential use to develop models of solubility in silico with application during the processes of drug discovery and development. More than 99. 5 % of all molecules are within the property range: molecular weight (M W  1250 g/mol); topological polar surface area (TPSA  250); rotatable bonds (RBN  25); hydrogen bond donors (HBD  10) and hydrogen bond acceptors (HBA  15), and compounds such as antibiotics, antifungals, vitamins and cardiac glycosides that are initially outside the classical "rule of five" range are included in this dataset. The physicochemical properties of the database form a reasonable oral druggable space  [35].
These results are justified because it is a composite variable related to steric and H-bonding interactions  [36] , which are quite important in the solubility of liquids and solid crystalline molecules. The database was also analysed in terms of the quality of the solubility data. In this sense, 1839 molecules had reliable experimental data (Solubility STD  1) and it was called the RELIABLE dataset. Of the remaining molecules, with high variability data (Solubility STD > 1), a recursive cleaning procedure was able to identify 250 molecules with unreliable solubility data (in average after running statistics of five iterations on the whole process) and 10592 molecules formed the CLEANED dataset. Both datasets were CONCATENATED (12431 molecules) and were used to develop the regression and classification models of aqueous solubility for external datasets.
Similarly, a new recursive procedure was developed to clean the database of compounds with high and sometimes unreliable experimental aqueous solubility values  [28]. The application of this recursive partitioning process resulted in a higher quality database of 10592 compounds, which was used to develop the solubility models and 243 molecules were identified as failed. After completing the statistics of five iterations, 615 molecules were highlighted as failed. Among these molecules, 57 were tagged as failed in all iterations. The results obtained with the use of this method, support its relevance, since we can rule out compounds potentially with extreme and variable solubility that affect the final prediction of the model.
Predictive models for aqueous solubility In order to find a way to improve the predictive accuracy of aqueous solubility models in silico, a new protocol was developed based on the combination of regression and classification models. To increase the performance of the individual models, a final consensus model was applied. A clear description of the modeling protocol is shown in Figure  4. The predictive performance of the consensus model is summarized in Table  2. As can be seen in this table, the training set formed by the CLEANED dataset was able to adequately predict the solubility values of the test set (RELIABLE dataset). The r 2 = 0. 87 in the test set is a good indicator of the model's ability to successfully predict new external compounds. For the four external sets (TS1-TS4), the predictions were quite different.
The prediction of the external series with the model obtained with the CLEANED dataset was adequate for the TS1 and TS4 series (r 2 = 0. 78-0. 83, RMSE = 0. 81-1. 0) but not for TS2 and TS3 (r 2 = 0. 40-0. 47, RMSE = 0. 98-1. 02). However, once the model was trained with the CONCATENATED training set (RELIABLE + CLEANED), the statistics increased for all the series and especially for TS2 and TS3 (r 2 = 0. 47-0. 56, RMSE = 0. 92-0. 93). Eight drugs (4. 4 % of the external set I) with standardized residues greater than 2 or less than -2 were identified as response outliers. They were folic acid (TS2 and TS3), antipyrine (TS1), amiodarone (TS4), cisapride (TS3), enalapril (TS3), rifabutin (TS4), ritonavir (TS3) and mifepristone (TS4). From these compounds, only five were identified as outliers in all iterations (see Figure  6 ).
Compounds such as antipyrine and mifepristone have been also erroneously predicted by other authors  [15, 29]. It is described that these drugs have conflicting solubility data reported in the literature, and in the case of antipyrine, multiple and different experimental solubility values (-0. 66 to 0. 55) have been reported  [29, 37, 38]. The wrong prediction of compounds such as amiodarone and mifepristone has been attributed to extreme solubility values which made their detection more difficult with sensible analytical methods and sometimes, to obtain the complete solubilization, requires the use of co-solvents  [38]. In the case of compounds with very high solubility (antipyrine) and very low solubility (amiodarone and mifepristone), their solubility prediction could be influenced by a poor representation of similar compounds in the chemical space of the database  [39, 40].
Folic acid has been poorly predicted by other authors  [29, 40] , which has conflicting solubility data reported in the literature attributed to drug degradation. The low representation of high solubility compounds within the database (7 %) could influence the large prediction error for the Enalapril. However, the solubility values reported for external set I were intrinsic solubility values, while the consensus model was developed with molecules with aqueous solubility. This suggests, that in the first case the value corresponds to the solubility of a compound in its free acid or base form at a pH where it is completely non-ionized, and in the second case the solubility depends on the pH used to perform the measurements and therefore can be different for ionizable compounds. In general, the aqueous solubility is greater than or equal to the intrinsic solubility  [2].
72-0. 87, RMSE = 0. 65-0. 79), and the lowest prediction was for set TS3 (r 2 = 0. 51, RMSE = 0. 85). The prediction of aqueous solubility for all datasets of the external set I was also evaluated using Pipeline Pilot software  [43]  (see Table  4 ). The aqueous solubility Pipeline Pilot algorithm is built-in and provided as a "black box" node for prediction only, with no possibility of training or parameterization. The algorithm is based on Tetko et al. method  [44]. The results obtained demonstrate the difficulty of predicting these data sets. The predictions of our consensus model (see Table  2 ) are satisfactory in comparison with these results, which reinforces the potential of the methodology developed and the predictive capacity of the final model obtained. Chevillad et al. [8]  showed the performance of solubility predictions for different commercial software on five data sets.
Automatic workflow for the prediction of aqueous solubility The final workflow integrated two main sections (see Figure  8 ). The first one was related to the development of the QSPR protocol and included the preprocessing of the data (data curation, parametrization of the molecular structure, etc. ), the merging of the data sources (combination of different datasets) and the development of regression and consensus models (recursive selection of variables and data cleaning, data partition, building, training and validation of the consensus model). The second section was designed to automate the prediction of a new external set based on the consensus model obtained in the preliminary development section. In order to validate the functionality of this workflow, a new dataset made of 30 new molecules from the last three years was assembled.
Comparison with previous in silico solubility models Several QSPR models have been previously published to predict aqueous solubility. All models published in the last ten years and their performances are clearly described in Table  9. Among regression models, only one was derived using large datasets (over 8000 compounds). Cui et al. [12]  developed a deep neural network (DNN). However, their prediction of the external set was low in terms of r squared but good for RMSE (r 2 = 0. 39, RMSE= 0. 68). As far as we know, this study generated one of the largest aqueous solubility data with more than 12000 compounds. The overall prediction for all compounds belonging to the external set I (ALL) is good (r 2 = 0. 69, RMSE= 0. 92), but once the intrinsic solubility values of the five outliers were replaced by the aqueous solubility values, the results improved (r 2 = 0.
73, RMSE = 0. 80). For our classification model, the overall results showed a better performance in predicting the soluble and insoluble classes compared to other published models. One of the most widely used external test sets was 32 molecules of the first solubility challenge. In this dataset our model had a balanced accuracy of 86 %, with a specificity value of 71 % and sensitivity value of 100 %, highlighting a better performance of our proposal.
Conclusions This study reported the first automatic workflow, developed on the KNIME Analytics Platform, to predict aqueous solubility of compounds. All steps of the QSPR were automated, focusing on data integration and curation procedures to obtain an extensive and structurally diverse database of aqueous solubility. Based on the large set of solubility data, the relationships between four simple molecular properties and solubility have been studied. By combining two recursive machine learning approaches for data cleaning and variable selection, and two regression and classification algorithms in a consensus model, good regression and classification statistics were obtained to predict the aqueous solubility of molecules. However, these results could be improved by increasing the chemical space of our dataset with more drugs and more reliable samples.
This allows us to track every eect the repair has on the model and provide guidance on how a developer can propagate the performed change through aected parts of the model. The result is not a single repair but rather alternative sequences of changes. We evaluated the performance and usability of our approach on 18 case studies consisting of inconsistent models. Two of those models were taken from GitHub. Among the 176 changes, we were able to reach a consistent model state within ve repairs on average for every developer change. We further conducted an experiment with 22 students. The experiment helped us to further assess the usefulness of our consistent change propagation approach, by investigating if our approach can reproduce the repair sequences the students have applied to the model.
To distinguish those two transitions in the text we will annotate them with t1  [playOrStop]  for the transition from state stopped to playing, and t2  [playOrStop]  for the transition from state playing to stopped. In this paper we use the Object Constraint Language (OCL)  [OMG(2014) ], a declarative language based on rst order logic, to dene our consistency rules for UML models. Table  1  shows three examples of consistency rules (CRs). Consistency rules dene specic constraints that must hold in software models. These constraints express relations among model elements that can range from well-formedness to very domain-specic ones for non-functional properties such as maintainability or usability  [Reder and Egyed(2013) ]. CR1 ensures that all messages in a lifeline have an equally named operation in their corresponding class.
Our approach assumes that the initial developer change was performed with intent (i. e. , the software engineer wants to evolve the state machine diagram in this manner). The resulting inconsistencies need to be repaired with respect to this intent. It is easy to see that these inconsistencies imply incomplete changes. Splitting playOrStop requires additional changes in the sequence and class diagrams. Without automated support, the developer may be unaware of this problem. Even if the developer knows about the inconsistencies then she may not know which further changes are needed to resolve the inconsistencies. It is important to note that there are many alternatives to propagate a change throughout the whole model. A repair mechanism that does not understand the developer's intent would for instance, suggest to repair I1 by either 1) renaming transition play to playOrStop, 2) adding an operation with name play to the model or 3) renaming the existing operation playOrStop.
To x those two inconsistencies we can add operation playOrStop to class Streamer, which would result in the same consistent model state as shown above (i. e. , presence of operations play and playOrStop). This is an alternative sequence of propagating the change, but it needs one additional action to x the new inconsistencies. As another example after play has been propagated, the software engineer changes the transition t2  [playOrStop]   Streamer or we rename operation playOrStop. Renaming the already existing play should not be done here, since it has been propagated beforehand, and would undo this propagation. We can rename operation playOrStop to stop and then rename message 2:playOrStop also to stop. Thus, class Streamer then only contains two operations play and stop and the model is consistent.
Evaluation We evaluate our approach by assessing its correctness, scalability and usefulness. For the evaluation we applied 20 consistency rules to 18 models taken from three dierent sources: academia (VOD, Curriculum Planer, Course System, Hotel Management, Calndarium), industry (eBullition, MVC, Inventory, Tele, Vacation System, Home Control System, DESI, Micro, iTalks, Dice, dSpace) and GitHub (Pro11, fullAdder)  [Hebig et al(2016)   Hebig, Quang, Chaudron, Robles, and Fernandez] , and one model used in an experiment for this paper. The domains of the models range from control of a micro wave oven to a model view controller of software and an inventory storage management system. Two of these models from GitHub have two versions each, where version one had inconsistencies that had been xed in version two by a developer.
This further allowed us to assess the quality of our approach and the relevance of our repairs, i. e. whether the manually applied repairs by the developers could be replicated by our approach. The model sizes ranged from 300 to 8800 model elements, the number of applied changes from 1 to 29 and the sum of all repaired inconsistencies during the propagation of all changes from 4 to 99. Table  3  shows those details per model. Note that our implementation has a compilation module integrated to check the syntactical correctness of the OCL consistency rules. The data set is archived in the FigShare platform 1  to be used for reproducibility and for comparison purposes. We executed the evaluation on a Windows 10 PC with a Core i7 3. 4GHz and 32GB RAM.
However, our approach uses negative side eects to propagate a change, which is helpful in cases where negative side eects are unavoidable. Martinez et al. ,  propose Astor a framework for automated program repair  [Martinez and Monperrus(2019) ]. It denes extension points to which users can attach already existing approaches for code transformations, search space navigation and validation candidate solutions. Astor generates program variants, and generates patches/xes for the original program. In this context, many approaches exists that compute patches for program failures, such as  [Le et al(2016)   Le, Lo, and Le Goues,Mechtaev et al(2016)Mechtaev, Yi, and Roychoudhury, Long et al(2017)   Long, Amidon, and Rinard, Saha et al(2017)   Saha, Lyu, Yoshida, and Prasad, Xin and Reiss(2017) ] The patches are then validated if they are indeed able to x their corresponding bug.
Acknowledgements This research was supported by the  Austrian Science Fund FWF P 31989 , the  LIT Secure and Correct Systems Lab , and  Pro2Future , a  COMET K1-Centre  of the  Austrian Research Promotion Agency (FFG) , grant no. 854184  co-funded by the  Provinces of Upper Austria and Styria , and by the  Natural Sciences and Engineering Research Council of Canada (NSERC)  grant  RGPIN-2017-0542.
Tree-ring archives are one of the main sources of information to reconstruct climate variations over the last millennium with annual resolution. The links between treering proxies and climate have usually been estimated using statistical approaches, assuming linear and stationary relationships. Both assumptions may be inadequate, but this issue can be overcome by ecophysiological modelling based on mechanistic understanding. In this respect, the model MAIDEN (Modeling and Analysis In DENdroecology) simulating tree-ring growth from daily temperature and precipitation, considering carbon assimilation and allocation in forest stands, may constitute a valuable tool. However, the lack of local meteorological data and the limited characterization of tree species traits can complicate the calibration and validation of such a complex model, which may hamper palaeoclimate applications.
The goal of this study is to test the applicability of the MAIDEN model in a palaeoclimate context using as a test case tree-ring observations covering the 20th century from 21 Eastern Canadian taiga sites and 3 European sites. More specifically, we investigate the model sensitivity to parameter calibration and to the quality of climatic inputs, and we evaluate the model performance using a validation procedure. We also examine the added value of using MAIDEN in palaeoclimate applications compared to a simpler tree-growth model, i. e. VS-Lite. A Bayesian calibration of the most sensitive model parameters provides good results at most of the selected sites with high correlations between simulated and observed tree growth. Although MAIDEN is found to be sensitive to the quality of the climatic inputs, simple bias correction and downscaling techniques of these data improve significantly the performance of the model.
The split-sample validation of MAIDEN gives encouraging results but requires long tree ring and meteorological series to give robust results. We also highlight a risk of overfitting in the calibration of model parameters that increases with short series. Finally, MAIDEN has shown higher calibration and validation correlations in most cases compared to VS-Lite. Nevertheless, this latter model turns out to be more stable over calibration and validation periods. Our results provide a protocol for the application of MAIDEN to potentially any site with tree-ring width data in the extratropical region.
For a specific study site, local meteorological data and measurements of several ecophysiological variables allow for a precise calibration of many individual processes included in the model. However, this is a rare case and likely one of the main limitations in the application of the model to a wide range of sites and soil conditions or when driven by climate model results that have known biases  (Flato et al. , 2013). We first present in Sect. 2. 1 the two dendroclimatic models that are compared in this study, namely the complex model MAIDEN and the more simple model VS-Clim. Past, 16, 1043 Past, 16, -1059 Past, 16, , 2020  https://doi. org/10. 5194/cp-16-1043-2020 Lite. MAIDEN and VS-Lite are applied to selected sites of the Northern Hemisphere (described in Sect. 2. 2), covering a range of environmental conditions and tree species.
A first set of data consists of a large number of sites from the same region with similar environmental conditions but with low in situ replication, while a second set only contains a few sites but with good replication. In this way, we test the applicability of MAIDEN to two datasets contrasted in terms of site documentation. This allows us to evaluate the extent to which MAIDEN can be applied. We compare the calibration methods adopted for VS-Lite  (Tolwinski-Ward et al. , 2013)  and MAIDEN  (Hartig et al. , 2019)  in Sect. 2. 3. Different strategies to select the value for the most sensitive parameters of the MAIDEN model as well as the sensitivity of parameter calibration to the quality of climatic inputs are tested in Sect. 3. 1, 3. 2 and 3. 3. Finally, we compare calibration and validation statistics of both models and discuss their applicability to a wide range of sites and species in Sect.