BACKGROUND
an ideal lead candidate for an anticancer drug is one that is non-toxic to the host, is well absorbed and so can be administered orally, and is effective at inhibiting cancer cell growth.
data on safety, pharmacokinetics, and cytotoxicity are expensive to generate in the laboratory, however, and there is need for developing reliable in-silico predictive models.
one aspect of developing reliable models is to make efficient use of all available training data.
for example, if training data are available for an additional task that is related to the primary task of interest, such data could be useful in constructing a more reliable model.
this paper explores the use of a multitask model for oral clearance, where bioavailability is the second task.
it is among the first papers to report results for a human oral clearance model.
multitask models are most useful when data is limited, as is the case with the oral clearance model.
in some cases, however, multitask models can also be useful with larger data sets.
for the cytotoxicity models constructed here, use of a multitask model did not affect accuracy, but did reduce computation time .
another aspect of developing reliable models is to base the models on random samples from well-defined populations.
moreover, the training populations should be very similar or identical to the population of compounds that one wishes to make predictions for.
this is often difficult to achieve in practice, and new solutions are not proposed in this paper.
this important topic is addressed more fully in the discussion section, but the reader should be aware that while the accuracy results presented here are valid for the training and testing sets, they may or may not be valid for predictions on other sets.
nevertheless, predictions are made here on other compounds to demonstrate the approach of using multiple qsar models to screen a large compound library.
predicted results for any particular compound passing the screen would need to be verified in the laboratory.
such an approach has been used by boik et al.
<cit>  in a small study.
predictive models of safety, pharmacokinetics, and cytotoxicity could be designed and used for a variety of purposes.
keeping in mind the model's limitations, the intended purpose in this paper was to screen a large library of natural products for those that might be suitable for preclinical study as components of anticancer drug mixtures.
the criteria for suitability was that a compound be predicted to:
• inhibit multiple cancer cell lines in vitro at modest to low concentrations ,
• be of low systemic toxicity , and
• exhibit a low to modest oral clearance .
note that if the goal were to identify promising compounds for study as individual drugs, as opposed to components of mixtures, different criteria would likely be used.
for example, more potent cytotoxic agents might be desired.
in addition, only novel compounds might be of interest.
three qsar classification models were constructed.
qsar models identify statistical relationships between a response  and molecular features of a compound, such as molecular weight, logp, and functional group counts.
as noted above, the three proposed models are based on data for human oral clearance, rat ld <dig>  and in-vitro cytotoxicity.
oral clearance is a measure of the rate of drug removal from the body after oral administration, and ld <dig> refers to the expected dose needed to kill  <dig> percent of an animal population.
the three models were applied to a set of over  <dig>  natural compounds and hundreds were predicted to be cytotoxic, of low systemic toxicity, and of low to modest oral clearance.
the data modeled here were challenging.
correlations between single features and responses were very weak and many of the features were highly correlated with one another.
in addition, a large number of features  were employed, which in the case of the oral clearance model was greater than the number of records.
lastly, the processes modeled were biochemically complex and observed responses were noisy.
for example, measurements of oral clearance commonly exhibit a within-study coefficient of variation of  <dig> to  <dig> percent  <cit> .
in a  <dig> report, ld <dig> values were observed to vary by as much as 3- to 11-fold between different laboratories  <cit> .
in-vitro cytotoxicity data were also noisy, typical of high-throughput screening experiments.
with regards to classification, noisy measurements are particularly problematic when they occur at the thresholds used to demarcate active from inactive drugs.
in the cytotoxicity data modeled here, compounds were concentrated near these thresholds.
models were constructed using kernel multitask latent analysis , an algorithm developed by xiang and bennett  <cit>  based on earlier work by momma and bennett  <cit>  and used here with minor changes.
kmla is closely related to partial least squares , an algorithm that is commonly used in qsar and microarray studies when features are highly correlated and the number of records is small compared to the number of features  <cit> .
pls algorithms were originated by wold in the 1970s and were later refined by a number of researchers  <cit> .
briefly, pls algorithms use a series of linear projections to create a small set of orthogonal "latent" data columns from the original data so that the covariance between the latent features and response is maximized.
in this way, the dimensions of the original data are greatly reduced, problems with correlated features are eliminated, and maximal information related to the response is retained.
whereas pls produces linear models, kmla can produce nonlinear models, with the degree of nonlinearity determined by the choice of kernel function and kernel parameters.
kmla is designed for  multitask learning.
multitask learning can be useful when the number of records is small relative to the number of features  <cit> , as is the case for the oral clearance data.
multitask learning models have been proposed by several authors  <cit> , although their use in qsar is still rare.
in kmla, collective learning is ensured by forcing all problems to use a shared set of latent features.
this type of collective learning has been referred to as common feature mapping.
in the common latent feature space each task is independently treated as a single-task learning problem  <cit> .
because each task is independently modeled, tasks need not share common records and multiple types of models can be used .
the common set of latent features is obtained by minimizing loss functions across modeled tasks.
RESULTS
oral clearance
oral clearance can be calculated as clearance divided by bioavailability, where clearance refers to the systemic clearance after intravenous administration  and bioavailability is a fraction less than or equal to  <dig> .
because there is an upper limit to the systemic clearance , very high oral clearance values are associated with very low bioavailability.
high oral clearance is, in general, not a desirable characteristic for compounds that are being developed as oral drugs.
the complexities of physical events that impact oral clearance make accurate prediction of oral clearance difficult to achieve.
not surprisingly, the accuracy of predictions made by the oral clearance model was not high under any model variation tested here.
accuracy improved, however, when multitask learning was used, with bioavailability as an additional task.
by definition, bioavailability is related to oral clearance and the relationship should be particularly strong for drugs with high oral clearance values.
results are summarized in table  <dig> for models that employed three latent features.
the five oral clearance models are given the abbreviations oc.
<dig> through oc.
<dig> .
models were replicated  <dig> times and standard deviations of results are given in parentheses.
* significant differences by anova : oc.
<dig> vs. oc.2
** significant differences by anova : oc.
<dig> vs. oc.
<dig>  oc.
<dig>  and oc.
<dig> when testing oc.
<dig> through oc.5
*** significant differences by anova : oc.
<dig> vs. oc.
<dig> when testing oc.
<dig>  oc.
<dig>  and oc.4
the last two columns of the table provide test set precision for positive and negative labels averaged over all replications.
precision on the positive labels is defined as the number of records that are true positives and predicted to be positive, divided by the total number of true positives.
precision on the negative labels is defined as the number of records that are true negatives and predicted to be negative, divided by the total number of true negatives.
confusion matrices can be constructed from the data given in the table  <dig>  the two column labels of the confusion matrix are true positives and true negatives, respectively.
the two row labels are predicted positive and predicted negative, respectively.
the diagonal elements of the confusion matrix, expressed as fractions, are given in the last two columns of table  <dig>  using model oc.
<dig> as an example, the diagonal of the confusion matrix would be  <dig>  and  <dig> .
the lower left cell would be  <dig> – <dig>  and the upper right cell would be  <dig> – <dig> .
there were  <dig> oral clearance records in total and about  <dig> percent  were used for testing in any given replication.
of these, about  <dig> percent  were true positives and about  <dig> percent  were true negatives .
thus for example, the diagonals of the confusion matrix for oc.
<dig> would be about  and , respectively, if expressed in the units of records.
single task models and linear models tended to preform worse than multitask models and gaussian kernel models.
when precision on negative labels was compared for all models, precision for oc.
<dig>  was significantly lower than that for all others .
when precision on negative labels was compared only for gaussian models, precision for oc.
<dig>  was significantly higher than that for oc.
<dig> .
when comparing only the single task models, precision for the gaussian model was significantly higher than that for the linear one.
the training algorithm was designed to maximize total precision and minimize the difference in precision between the positive and negative labels.
the percentages of positive and negative labels in the complete data set were unequal, however, which generally makes balanced precision more difficult to achieve.
indeed, test set precision for the single task models  was lower for the negative labels than for the positive labels .
precision for the multitask models was more balanced, however.
for all model variations tested, use of model averaging tended to improve precision.
table s. <dig> in additional file  <dig> lists results using oc.
<dig> and oc.
<dig> as examples.
although the trend was consistent across all models, differences were not significant .
to further investigate model validity, model oc.
<dig> was estimated again using scrambled response values.
average test-set precision for positive and negative labels was  <dig>   and  <dig>  , respectively, which was significantly different from the means of  <dig>  and  <dig>  obtained for oc.
<dig> when labels were not scrambled .
the expected results for scrambled records would have been  <dig>  for both positive and negative labels if model averaging had not been used.
with model averaging, there was a small bias towards negative predictions and this bias had a relatively larger impact on the precision for the negative labels .
to verify that the kmla algorithm provided some benefit over a standard partial least squares algorithm, an oral clearance model was estimated using the generalized partial least squares  library of r  <cit> .
to make the comparison as fair as possible, kmla was used in single-task mode, paired training and verification sets were used between approaches, three latent variables were used for each approach, and hyper-parameter selection  was done in the same manner for each approach.
eight replicate models were constructed and results were assessed using the one-sided paired t-test.
in comparing the kmla and gpls approaches for single task models, kmla has the advantages that model averaging and cost-sensitive learning can be used.
in addition, nonlinear responses can be modeled .
without the use of model averaging, precision on the negative labels  was higher for the kmla algorithm  but the difference was not statistically significant.
when model averaging was used, precision was again higher for the kmla algorithm and the difference was statistically significant .
differences in precision for the positive labels were not statistically significant.
in summary, use of a multitask model modestly but significantly improved precision on the negative labels.
multitask models also tended to exhibit more balanced precision between positive and negative labels.
use of model averaging also tended to improve precision, although differences were not statistically significant.
the kmla algorithm preformed slightly better on an oral clearance task than did a standard partial least squares algorithm.
rat ld50
the number of records in the rat ld <dig> data set  was more than twice as large as the number of features, yet it was small enough that all records could be modeled in a reasonable amount of time.
results are summarized in table  <dig> for models that employed four latent features.
the two models are referred to as ld.
<dig> and ld.
<dig> .
models were replicated  <dig> times and standard deviations are given in parentheses.
this was a lower number of replicates than used for the oral clearance model, as the ld <dig> data set was larger and modeling results were less variable.
the ld <dig> data set contained only one response and so kmla was used in the single-task mode.
* significant differences by anova : model  <dig> vs. model 2
model ld.
<dig>  which used a gaussian kernel, performed significantly better on the negative labels  than the model using a linear kernel .
model averaging tended to improve precision but differences were not significant.
without model averaging, average test-set precision for the positive and negative labels of ld.
<dig> was  <dig>  and  <dig> , respectively.
confusion matrices can be constructed from the data given in table  <dig>  there were  <dig>  records in total, about  <dig> percent of which  were used for testing in any given replication.
of these, about  <dig> percent  were true positives and about  <dig> percent  were true negatives .
cytotoxicity
the number of records in the cytotoxicity data set  was more than five times as large as the number of features.
it was not practical to model more than about  <dig>  records, however, and a subset of records was used.
only about five percent of records had a positive label, and so all of these were retained for modeling along with a selected group of records with negative labels.
multitask models were constructed using lc <dig> and tgi  responses; models were desired for both responses and it was expected that the biochemical processes involved in both were related.
results are summarized in table  <dig> for models that employed three latent features.
the eight models are referred to as c. <dig> through c. <dig> .
models were replicated  <dig> times and standard deviations are given in parentheses.
testing sets contained about  <dig>  records.
* significant differences by anova : c. <dig> vs. c. <dig> and c.2
** significant differences by anova : c. <dig> vs. c. <dig> and c.3
the gaussian kernels performed significantly better than the linear kernels for the h <dig> cell line for both positive and negative labels .
based on these results, only gaussian kernels were tested for the other cell lines.
the multitask models did not perform significantly better than the single-task models, as expected given the large number of training records.
multitask models did reduce training time, however.
in particular, the time needed to select records for inclusion in training sets was cut in half.
confusion matrices can be constructed from the data given in table  <dig>  the number of records in each cytotoxicity data set is given in table s. <dig> in additional file  <dig>
model averaging consistently tended to improve precision, although differences were not significant.
for example, without model averaging, average test-set precision for positive and negative labels for c. <dig>  were  <dig>  and  <dig> , respectively, as compared to  <dig>  and  <dig> , respectively, with model averaging.
the models listed in table  <dig> were based on a subset of records, where the subset was chosen using an algorithm described in the methods section.
as an alternative, records with negative labels could be randomly chosen.
using model c. <dig>  as an example, the randomization method reduced the precision for negative labels from  <dig>  to  <dig> .
the precision for positive labels was not greatly changed , which was expected because the same set of positive labels was used in both models.
the inferior performance of the model with random negative labels is due to the fact that some labels useful for classification were left out.
in contrast, selection of negative labels by the algorithm helped assure that useful records were retained.
precision of a final cytotoxicity model was investigated in three ways, using model c. <dig>  as an example.
first, the precision was determined for the  <dig>  compounds that were contained in both the nci data set and the set of  <dig>  natural compounds.
some of these duplicate compounds would have been included in the training set for c. <dig> and some  would have been excluded from it.
of the  <dig>  compounds,  <dig> had a positive label and  <dig>  had a negative one.
the precision was  <dig>  and  <dig> , respectively.
second, the precision was determined for a smaller set of  <dig> compounds that were contained in the natural compound set and for which additional nci cytotoxicity data were available.
most of these were compounds that were added to the nci database after  <dig>  none of them was used in training the model.
there were  <dig> compounds with positive labels and  <dig> with negative ones.
precision was  <dig>  and  <dig> , respectively.
given that the testing set precision on the positive labels was much higher , and that the test sets contained far more compounds , it seems likely that the lower precision on the new data may have been due to differences in sample composition.
neither the set of  <dig> nor the set of  <dig>  mentioned above was a random sample from the natural compounds set.
third, precision was determined for the set of  <dig> compounds as above, only using a model that was constructed with scrambled training responses.
average precision on the positive and negative labels was  <dig>  and  <dig> , respectively .
the low precision on the positive labels was due to the low percentage of positive labels in the training set.
in summary, a model with gaussian kernel performed significantly better in cross-validation than one with a linear kernel for the h <dig> cell line, and use of a multitask model reduced the time needed for record selection.
model averaging tended to improve precision, and the algorithm for choosing negative training records was more useful than random selection.
predictions for natural compounds
models described above were used to make predictions for a set of more than  <dig>  natural compounds.
gaussian kernels were employed, and multitask models were used for cytotoxicity and oral clearance.
the number and fraction of natural compounds passing various screening criteria are listed in table  <dig>  compounds passed a given screen if their predictions were all positive for the selected criterion.
screens  <dig> to  <dig> emphasize the h <dig> model for illustrative purposes .
the purpose of the first, most rigorous screen is to identify compounds that are likely to have low acute toxicity, possess cytotoxic activity, and be suitable for oral administration.
these compounds would be a priority for preclinical study, where their properties could be confirmed.
comparing results from screens  <dig>   <dig>  and  <dig>  compounds that were predicted to be active in the h <dig> cytotoxicity models also tended to be predicted active in the other two cell lines.
comparing screens  <dig> and  <dig>  most of the compounds that were predicted to be active against the h <dig> cell line also were predicted to be toxic to rats.
indeed, halle  <cit>  reported that in-vitro cytotoxicity can be used to predict rat ld <dig> values.
not surprisingly, given a prediction of cytotoxicity the criterion of passing the ld <dig> model was more restrictive than that for passing the oral clearance model .
the diversity of molecules passing the more restrictive screens was considerably lower than the diversity of molecules in the entire natural compounds data set.
for molecules passing the more restrictive screens, two groups were heavily represented: anthraquinones and flavonoids.
a typical molecule from the first group was aloe-emodin, and ones from the second group were quercetin glycosides.
in summary, compounds that were predicted to be cytotoxic in one cell line were usually predicted to be cytotoxic in the other cell lines, as well as systemically toxic to rats.
as would be expected, use of the cytotoxicity and ld <dig> screens together resulted in far fewer passing compounds compared to use of the cytotoxicity screen alone or use of cytotoxicity and oral clearance screens.
many of the compounds that passed the more restrictive screens were either anthraquinones or flavonoids.
discussion
multitask learning
qsar models in biology often suffer from a lack of available records for training and testing.
this was the case with the oral clearance model and the results presented here suggest that for this model, multitask learning can modestly but significantly improve precision .
multitask learning did not significantly improve precision of the cytotoxicity models , but this result was expected due to the large number of records available for training.
however, multitask learning did reduce computation time for the combined lc <dig> and tgi models .
multitask learning has not yet become popular for qsar modeling, and these results suggest that it could play a larger role.
for the tasks modeled here, the results also show that nonlinear models can in some cases perform significantly better than linear ones.
model averaging also tended to improved accuracy, but differences were not significant.
comparison with published models
the models developed here seem to be of comparable accuracy to ones previously published in the literature, however such comparisons are difficult to make because each published study used different data and a different modeling approach.
ralaivola et al.
<cit>  and swamidass et al.
<cit>  used graph kernels to construct qsar models of cytotoxicity, also based on the nci data set.
their approach differed in that they modeled gi <dig> rather than lc <dig> or tgi values.
gi <dig> is a measure of the growth inhibitory power of a compound, tgi is a measure of cytostatic effect, and lc <dig> is a measure of cytotoxic effect.
by design, gi <dig> < tgi < lc <dig>  the record selection methods and threshold values they used resulted in training sets that were nearly balanced between positive and negative labels.
in comparison, the average fraction of positive labels in the nci data modeled here was only  <dig> .
one could expect better predictive accuracy under more balanced conditions.
even so, results presented in table  <dig> were comparable.
for the three cell lines modeled here, ralaivola et al.
<cit>  reported an average precision of  <dig> , whereas average precision for tgi here was  <dig> .
by modeling lc <dig> and tgi rather than gi <dig>  the models developed here are designed to identify compounds with higher average potency.
several qsar models of human oral absorption  <cit>  and bioavailability  <cit>  have been published for heterogenous sets of compounds, but papers on oral clearance are rare.
in comparing the complexity of the physiological events involved in absorption, bioavailability, and oral clearance, one would expect that oral absorption models would be the most accurate of the three and oral clearance models would be the least accurate.
for example, first-pass metabolism is not accounted for when measuring absorption, and systemic clearance by the liver is not accounted for when measuring bioavailability.
indeed, hou et al.
<cit>  reported that human bioavailability was much more difficult to predict than oral absorption.
in a multi-label classification study of bioavailability based on  <dig> drugs, pintore et al.
<cit>  reported that average test set accuracy was  <dig>   over all classes.
yoshida and topliss  <cit>  published a multi-label classification study on bioavailability of  <dig> drugs and obtained a somewhat lower average test set accuracy .
this result for bioavailability is similar to the one reported here for oral clearance .
only one qsar study on oral clearance could be found.
wajima et al.
<cit>  published a regression model for oral clearance that used  <dig> drugs and produced a cross-validation q <dig> correlation coefficient of  <dig> .
some of the features were generated from animal pharmacokinetic experiments, however, and not from chemical structure alone as done here.
it can be expected that human oral clearance would be more correlated with animal pharmacokinetic data than with molecular descriptors.
numerous qsar models of rat ld <dig> have been published, but almost all of these used smaller, homogenous sets of compounds  <cit> .
commercial qsar ld <dig> models  also tend to use multiple qsars on smaller, homogenous sets of data.
in a comparison of several commercial qsar models, tunkel et al.
<cit>  reported that  <dig> percent of multi-label predictions by topkat were correct , and  <dig> percent of predictions by mcase were correct.
neither model was able to classify all  <dig> compounds because some were outside the training sets.
while the ld <dig> model developed in this paper is not multi-label, it did correctly classify  <dig> percent of compounds in the test sets.
of the three models constructed here, the oral clearance model was the least accurate .
nevertheless, the model is still of interest for two reasons.
first, this is one of the few published attempts at modeling oral clearance and better results have not been reported for comparable data.
second, the model variants that were constructed suggest that use of multitask learning may impart a small but significant learning advantage.
model generalizability
in this paper, three classification models were constructed for oral clearance, cytotoxicity, and rat ld <dig> data, respectively, and predictions were made for a large set of natural compounds.
in all three models, the largest publically available data sets were employed .
each set of compounds used was a sample from some larger population of compounds, but it seems highly unlikely that they were random samples.
moreover, the set of natural compounds is unlikely to represent a random sample from the population of all natural compounds that could pass the inclusion criteria.
furthermore, the training sets were unlikely to be random samples from the natural compounds population.
the populations that the training sets were drawn from are essentially unknowable, and therefore true random samples from those populations cannot be collected.
for example, oral clearance values were taken from the literature and may have been subject to publication bias.
one might expect published results to be rich in compounds that have some significance to pharmacology or toxicology, and that have low to modest oral clearance.
the cytotoxicity data provide another example.
nci selected particular compounds for screening based on a variety of concerns, which likely included expected activity, chemical structure, past results, and the submissions made to the nci program.
in spite of the uncertainties of the populations, these are the best available public data on which to base models and so are used here.
there is some assurance that the models will generalize to the hold-out test sets, as demonstrated by cross-validation, but there is little assurance that the models will generalize to any new set of compounds collected by nci or others, or to the set of natural compounds on which predictions were made.
one would hope, however, that the size of the data sets, particularly the nci and ld <dig> data sets, might increase the generalizability of the models.
the best way to test generalizability would be to know the populations from which the training sets were taken and then to randomly sample compounds from that population for additional testing.
if these populations were known, a sizable sample of compounds, perhaps many hundreds, would need to be tested in the laboratory.
such an undertaking is beyond the scope of this project.
furthermore, this still would not address the issue that the prediction set of natural compounds may be different from the training sets.
if the only population of interest were the particular set of natural compounds used here, a random sample from that set could be tested in the laboratory.
but again the sample would need to be large.
in spite of the uncertain generalizability of the models to new compounds, such models may still be worthy of investigation.
the kernel-based multitask modeling approach itself should be of interest to investigators, and furthermore it is possible that the models could be of use in drug discovery.
in a small application of the models, boik et al.
<cit>  used them to help identify several dozen compounds that were predicted to be active in-vitro against the three nci cell lines, were predicted to have low rat ld <dig> values, and were commercially available.
of these,  <dig> were tested in-vitro and  <dig> were sufficiently water-soluble and cytotoxic in a specific 48-hour assay to allow their use in the study.
CONCLUSIONS
the results shown here suggest that in some cases, multitask learning can be useful for constructing qsar models.
depending upon the multi-task model, precision was improved over single-task models and computation time was reduced.
when applied to a large natural compound library, the models developed here for cytotoxicity, ld <dig>  and oral clearance identified an active set of about  <dig> compounds that was rich in flavonoids and anthraquinones.
this is the first published report of an oral clearance qsar model that used only chemical information as explanatory variables.

 an ideal lead candidate for an anticancer drug is one that is non-toxic to the host, is well absorbed and so can be administered orally, and is effective at inhibiting cancer cell growth. data on safety, pharmacokinetics, and cytotoxicity are expensive to generate in the laboratory, however, and there is need for developing reliable in-silico predictive models. one aspect of developing reliable models is to make efficient use of all available training data. for example, if training data are available for an additional task that is related to the primary task of interest, such data could be useful in constructing a more reliable model. this paper explores the use of a multitask model for oral clearance, where bioavailability is the second task. it is among the first papers to report results for a human oral clearance model. multitask models are most useful when data is limited, as is the case with the oral clearance model. in some cases, however, multitask models can also be useful with larger data sets. for the cytotoxicity models constructed here, use of a multitask model did not affect accuracy, but did reduce computation time. another aspect of developing reliable models is to base the models on random samples from well-defined populations. moreover, the training populations should be very similar or identical to the population of compounds that one wishes to make predictions for. this is often difficult to achieve in practice, and new solutions are not proposed in this paper. this important topic is addressed more fully in the discussion section, but the reader should be aware that while the accuracy results presented here are valid for the training and testing sets, they may or may not be valid for predictions on other sets. nevertheless, predictions are made here on other compounds to demonstrate the approach of using multiple qsar models to screen a large compound library. predicted results for any particular compound passing the screen would need to be verified in the laboratory. such an approach has been used by boik et al. in a small study. predictive models of safety, pharmacokinetics, and cytotoxicity could be designed and used for a variety of purposes. keeping in mind the model's limitations, the intended purpose in this paper was to screen a large library of natural products for those that might be suitable for preclinical study as components of anticancer drug mixtures. the criteria for suitability was that a compound be predicted to: • inhibit multiple cancer cell lines in vitro at modest to low concentrations, • be of low systemic toxicity, and • exhibit a low to modest oral clearance. note that if the goal were to identify promising compounds for study as individual drugs, as opposed to components of mixtures, different criteria would likely be used. for example, more potent cytotoxic agents might be desired. in addition, only novel compounds might be of interest. three qsar classification models were constructed. qsar models identify statistical relationships between a response and molecular features of a compound, such as molecular weight, logp, and functional group counts. as noted above, the three proposed models are based on data for human oral clearance, rat ld and in-vitro cytotoxicity. oral clearance is a measure of the rate of drug removal from the body after oral administration, and ld refers to the expected dose needed to kill percent of an animal population. the three models were applied to a set of over natural compounds and hundreds were predicted to be cytotoxic, of low systemic toxicity, and of low to modest oral clearance. the data modeled here were challenging. correlations between single features and responses were very weak and many of the features were highly correlated with one another. in addition, a large number of features were employed, which in the case of the oral clearance model was greater than the number of records. lastly, the processes modeled were biochemically complex and observed responses were noisy. for example, measurements of oral clearance commonly exhibit a within-study coefficient of variation of to percent. in a report, ld values were observed to vary by as much as 3- to 11-fold between different laboratories. in-vitro cytotoxicity data were also noisy, typical of high-throughput screening experiments. with regards to classification, noisy measurements are particularly problematic when they occur at the thresholds used to demarcate active from inactive drugs. in the cytotoxicity data modeled here, compounds were concentrated near these thresholds. models were constructed using kernel multitask latent analysis, an algorithm developed by xiang and bennett based on earlier work by momma and bennett and used here with minor changes. kmla is closely related to partial least squares, an algorithm that is commonly used in qsar and microarray studies when features are highly correlated and the number of records is small compared to the number of features. pls algorithms were originated by wold in the 1970s and were later refined by a number of researchers. briefly, pls algorithms use a series of linear projections to create a small set of orthogonal "latent" data columns from the original data so that the covariance between the latent features and response is maximized. in this way, the dimensions of the original data are greatly reduced, problems with correlated features are eliminated, and maximal information related to the response is retained. whereas pls produces linear models, kmla can produce nonlinear models, with the degree of nonlinearity determined by the choice of kernel function and kernel parameters. kmla is designed for multitask learning. multitask learning can be useful when the number of records is small relative to the number of features, as is the case for the oral clearance data. multitask learning models have been proposed by several authors, although their use in qsar is still rare. in kmla, collective learning is ensured by forcing all problems to use a shared set of latent features. this type of collective learning has been referred to as common feature mapping. in the common latent feature space each task is independently treated as a single-task learning problem. because each task is independently modeled, tasks need not share common records and multiple types of models can be used. the common set of latent features is obtained by minimizing loss functions across modeled tasks.  oral clearance oral clearance can be calculated as clearance divided by bioavailability, where clearance refers to the systemic clearance after intravenous administration and bioavailability is a fraction less than or equal to. because there is an upper limit to the systemic clearance, very high oral clearance values are associated with very low bioavailability. high oral clearance is, in general, not a desirable characteristic for compounds that are being developed as oral drugs. the complexities of physical events that impact oral clearance make accurate prediction of oral clearance difficult to achieve. not surprisingly, the accuracy of predictions made by the oral clearance model was not high under any model variation tested here. accuracy improved, however, when multitask learning was used, with bioavailability as an additional task. by definition, bioavailability is related to oral clearance and the relationship should be particularly strong for drugs with high oral clearance values. results are summarized in table for models that employed three latent features. the five oral clearance models are given the abbreviations oc. through oc.. models were replicated times and standard deviations of results are given in parentheses. * significant differences by anova: oc. vs. oc.2 ** significant differences by anova: oc. vs. oc. oc. and oc. when testing oc. through oc.5 *** significant differences by anova: oc. vs. oc. when testing oc. oc. and oc.4 the last two columns of the table provide test set precision for positive and negative labels averaged over all replications. precision on the positive labels is defined as the number of records that are true positives and predicted to be positive, divided by the total number of true positives. precision on the negative labels is defined as the number of records that are true negatives and predicted to be negative, divided by the total number of true negatives. confusion matrices can be constructed from the data given in the table the two column labels of the confusion matrix are true positives and true negatives, respectively. the two row labels are predicted positive and predicted negative, respectively. the diagonal elements of the confusion matrix, expressed as fractions, are given in the last two columns of table using model oc. as an example, the diagonal of the confusion matrix would be and. the lower left cell would be – and the upper right cell would be –. there were oral clearance records in total and about percent were used for testing in any given replication. of these, about percent were true positives and about percent were true negatives. thus for example, the diagonals of the confusion matrix for oc. would be about and, respectively, if expressed in the units of records. single task models and linear models tended to preform worse than multitask models and gaussian kernel models. when precision on negative labels was compared for all models, precision for oc. was significantly lower than that for all others. when precision on negative labels was compared only for gaussian models, precision for oc. was significantly higher than that for oc.. when comparing only the single task models, precision for the gaussian model was significantly higher than that for the linear one. the training algorithm was designed to maximize total precision and minimize the difference in precision between the positive and negative labels. the percentages of positive and negative labels in the complete data set were unequal, however, which generally makes balanced precision more difficult to achieve. indeed, test set precision for the single task models was lower for the negative labels than for the positive labels. precision for the multitask models was more balanced, however. for all model variations tested, use of model averaging tended to improve precision. table s. in additional file lists results using oc. and oc. as examples. although the trend was consistent across all models, differences were not significant. to further investigate model validity, model oc. was estimated again using scrambled response values. average test-set precision for positive and negative labels was and, respectively, which was significantly different from the means of and obtained for oc. when labels were not scrambled. the expected results for scrambled records would have been for both positive and negative labels if model averaging had not been used. with model averaging, there was a small bias towards negative predictions and this bias had a relatively larger impact on the precision for the negative labels. to verify that the kmla algorithm provided some benefit over a standard partial least squares algorithm, an oral clearance model was estimated using the generalized partial least squares library of r. to make the comparison as fair as possible, kmla was used in single-task mode, paired training and verification sets were used between approaches, three latent variables were used for each approach, and hyper-parameter selection was done in the same manner for each approach. eight replicate models were constructed and results were assessed using the one-sided paired t-test. in comparing the kmla and gpls approaches for single task models, kmla has the advantages that model averaging and cost-sensitive learning can be used. in addition, nonlinear responses can be modeled. without the use of model averaging, precision on the negative labels was higher for the kmla algorithm but the difference was not statistically significant. when model averaging was used, precision was again higher for the kmla algorithm and the difference was statistically significant. differences in precision for the positive labels were not statistically significant. in summary, use of a multitask model modestly but significantly improved precision on the negative labels. multitask models also tended to exhibit more balanced precision between positive and negative labels. use of model averaging also tended to improve precision, although differences were not statistically significant. the kmla algorithm preformed slightly better on an oral clearance task than did a standard partial least squares algorithm. rat ld50 the number of records in the rat ld data set was more than twice as large as the number of features, yet it was small enough that all records could be modeled in a reasonable amount of time. results are summarized in table for models that employed four latent features. the two models are referred to as ld. and ld.. models were replicated times and standard deviations are given in parentheses. this was a lower number of replicates than used for the oral clearance model, as the ld data set was larger and modeling results were less variable. the ld data set contained only one response and so kmla was used in the single-task mode. * significant differences by anova: model vs. model 2 model ld. which used a gaussian kernel, performed significantly better on the negative labels than the model using a linear kernel. model averaging tended to improve precision but differences were not significant. without model averaging, average test-set precision for the positive and negative labels of ld. was and, respectively. confusion matrices can be constructed from the data given in table there were records in total, about percent of which were used for testing in any given replication. of these, about percent were true positives and about percent were true negatives. cytotoxicity the number of records in the cytotoxicity data set was more than five times as large as the number of features. it was not practical to model more than about records, however, and a subset of records was used. only about five percent of records had a positive label, and so all of these were retained for modeling along with a selected group of records with negative labels. multitask models were constructed using lc and tgi responses; models were desired for both responses and it was expected that the biochemical processes involved in both were related. results are summarized in table for models that employed three latent features. the eight models are referred to as c. through c.. models were replicated times and standard deviations are given in parentheses. testing sets contained about records. * significant differences by anova: c. vs. c. and c.2 ** significant differences by anova: c. vs. c. and c.3 the gaussian kernels performed significantly better than the linear kernels for the h cell line for both positive and negative labels. based on these results, only gaussian kernels were tested for the other cell lines. the multitask models did not perform significantly better than the single-task models, as expected given the large number of training records. multitask models did reduce training time, however. in particular, the time needed to select records for inclusion in training sets was cut in half. confusion matrices can be constructed from the data given in table the number of records in each cytotoxicity data set is given in table s. in additional file model averaging consistently tended to improve precision, although differences were not significant. for example, without model averaging, average test-set precision for positive and negative labels for c. were and, respectively, as compared to and, respectively, with model averaging. the models listed in table were based on a subset of records, where the subset was chosen using an algorithm described in the methods section. as an alternative, records with negative labels could be randomly chosen. using model c. as an example, the randomization method reduced the precision for negative labels from to. the precision for positive labels was not greatly changed, which was expected because the same set of positive labels was used in both models. the inferior performance of the model with random negative labels is due to the fact that some labels useful for classification were left out. in contrast, selection of negative labels by the algorithm helped assure that useful records were retained. precision of a final cytotoxicity model was investigated in three ways, using model c. as an example. first, the precision was determined for the compounds that were contained in both the nci data set and the set of natural compounds. some of these duplicate compounds would have been included in the training set for c. and some would have been excluded from it. of the compounds, had a positive label and had a negative one. the precision was and, respectively. second, the precision was determined for a smaller set of compounds that were contained in the natural compound set and for which additional nci cytotoxicity data were available. most of these were compounds that were added to the nci database after none of them was used in training the model. there were compounds with positive labels and with negative ones. precision was and, respectively. given that the testing set precision on the positive labels was much higher, and that the test sets contained far more compounds, it seems likely that the lower precision on the new data may have been due to differences in sample composition. neither the set of nor the set of mentioned above was a random sample from the natural compounds set. third, precision was determined for the set of compounds as above, only using a model that was constructed with scrambled training responses. average precision on the positive and negative labels was and, respectively. the low precision on the positive labels was due to the low percentage of positive labels in the training set. in summary, a model with gaussian kernel performed significantly better in cross-validation than one with a linear kernel for the h cell line, and use of a multitask model reduced the time needed for record selection. model averaging tended to improve precision, and the algorithm for choosing negative training records was more useful than random selection. predictions for natural compounds models described above were used to make predictions for a set of more than natural compounds. gaussian kernels were employed, and multitask models were used for cytotoxicity and oral clearance. the number and fraction of natural compounds passing various screening criteria are listed in table compounds passed a given screen if their predictions were all positive for the selected criterion. screens to emphasize the h model for illustrative purposes. the purpose of the first, most rigorous screen is to identify compounds that are likely to have low acute toxicity, possess cytotoxic activity, and be suitable for oral administration. these compounds would be a priority for preclinical study, where their properties could be confirmed. comparing results from screens and compounds that were predicted to be active in the h cytotoxicity models also tended to be predicted active in the other two cell lines. comparing screens and most of the compounds that were predicted to be active against the h cell line also were predicted to be toxic to rats. indeed, halle reported that in-vitro cytotoxicity can be used to predict rat ld values. not surprisingly, given a prediction of cytotoxicity the criterion of passing the ld model was more restrictive than that for passing the oral clearance model. the diversity of molecules passing the more restrictive screens was considerably lower than the diversity of molecules in the entire natural compounds data set. for molecules passing the more restrictive screens, two groups were heavily represented: anthraquinones and flavonoids. a typical molecule from the first group was aloe-emodin, and ones from the second group were quercetin glycosides. in summary, compounds that were predicted to be cytotoxic in one cell line were usually predicted to be cytotoxic in the other cell lines, as well as systemically toxic to rats. as would be expected, use of the cytotoxicity and ld screens together resulted in far fewer passing compounds compared to use of the cytotoxicity screen alone or use of cytotoxicity and oral clearance screens. many of the compounds that passed the more restrictive screens were either anthraquinones or flavonoids. discussion multitask learning qsar models in biology often suffer from a lack of available records for training and testing. this was the case with the oral clearance model and the results presented here suggest that for this model, multitask learning can modestly but significantly improve precision. multitask learning did not significantly improve precision of the cytotoxicity models, but this result was expected due to the large number of records available for training. however, multitask learning did reduce computation time for the combined lc and tgi models. multitask learning has not yet become popular for qsar modeling, and these results suggest that it could play a larger role. for the tasks modeled here, the results also show that nonlinear models can in some cases perform significantly better than linear ones. model averaging also tended to improved accuracy, but differences were not significant. comparison with published models the models developed here seem to be of comparable accuracy to ones previously published in the literature, however such comparisons are difficult to make because each published study used different data and a different modeling approach. ralaivola et al. and swamidass et al. used graph kernels to construct qsar models of cytotoxicity, also based on the nci data set. their approach differed in that they modeled gi rather than lc or tgi values. gi is a measure of the growth inhibitory power of a compound, tgi is a measure of cytostatic effect, and lc is a measure of cytotoxic effect. by design, gi < tgi < lc the record selection methods and threshold values they used resulted in training sets that were nearly balanced between positive and negative labels. in comparison, the average fraction of positive labels in the nci data modeled here was only. one could expect better predictive accuracy under more balanced conditions. even so, results presented in table were comparable. for the three cell lines modeled here, ralaivola et al. reported an average precision of, whereas average precision for tgi here was. by modeling lc and tgi rather than gi the models developed here are designed to identify compounds with higher average potency. several qsar models of human oral absorption and bioavailability have been published for heterogenous sets of compounds, but papers on oral clearance are rare. in comparing the complexity of the physiological events involved in absorption, bioavailability, and oral clearance, one would expect that oral absorption models would be the most accurate of the three and oral clearance models would be the least accurate. for example, first-pass metabolism is not accounted for when measuring absorption, and systemic clearance by the liver is not accounted for when measuring bioavailability. indeed, hou et al. reported that human bioavailability was much more difficult to predict than oral absorption. in a multi-label classification study of bioavailability based on drugs, pintore et al. reported that average test set accuracy was over all classes. yoshida and topliss published a multi-label classification study on bioavailability of drugs and obtained a somewhat lower average test set accuracy. this result for bioavailability is similar to the one reported here for oral clearance. only one qsar study on oral clearance could be found. wajima et al. published a regression model for oral clearance that used drugs and produced a cross-validation q correlation coefficient of. some of the features were generated from animal pharmacokinetic experiments, however, and not from chemical structure alone as done here. it can be expected that human oral clearance would be more correlated with animal pharmacokinetic data than with molecular descriptors. numerous qsar models of rat ld have been published, but almost all of these used smaller, homogenous sets of compounds. commercial qsar ld models also tend to use multiple qsars on smaller, homogenous sets of data. in a comparison of several commercial qsar models, tunkel et al. reported that percent of multi-label predictions by topkat were correct, and percent of predictions by mcase were correct. neither model was able to classify all compounds because some were outside the training sets. while the ld model developed in this paper is not multi-label, it did correctly classify percent of compounds in the test sets. of the three models constructed here, the oral clearance model was the least accurate. nevertheless, the model is still of interest for two reasons. first, this is one of the few published attempts at modeling oral clearance and better results have not been reported for comparable data. second, the model variants that were constructed suggest that use of multitask learning may impart a small but significant learning advantage. model generalizability in this paper, three classification models were constructed for oral clearance, cytotoxicity, and rat ld data, respectively, and predictions were made for a large set of natural compounds. in all three models, the largest publically available data sets were employed. each set of compounds used was a sample from some larger population of compounds, but it seems highly unlikely that they were random samples. moreover, the set of natural compounds is unlikely to represent a random sample from the population of all natural compounds that could pass the inclusion criteria. furthermore, the training sets were unlikely to be random samples from the natural compounds population. the populations that the training sets were drawn from are essentially unknowable, and therefore true random samples from those populations cannot be collected. for example, oral clearance values were taken from the literature and may have been subject to publication bias. one might expect published results to be rich in compounds that have some significance to pharmacology or toxicology, and that have low to modest oral clearance. the cytotoxicity data provide another example. nci selected particular compounds for screening based on a variety of concerns, which likely included expected activity, chemical structure, past results, and the submissions made to the nci program. in spite of the uncertainties of the populations, these are the best available public data on which to base models and so are used here. there is some assurance that the models will generalize to the hold-out test sets, as demonstrated by cross-validation, but there is little assurance that the models will generalize to any new set of compounds collected by nci or others, or to the set of natural compounds on which predictions were made. one would hope, however, that the size of the data sets, particularly the nci and ld data sets, might increase the generalizability of the models. the best way to test generalizability would be to know the populations from which the training sets were taken and then to randomly sample compounds from that population for additional testing. if these populations were known, a sizable sample of compounds, perhaps many hundreds, would need to be tested in the laboratory. such an undertaking is beyond the scope of this project. furthermore, this still would not address the issue that the prediction set of natural compounds may be different from the training sets. if the only population of interest were the particular set of natural compounds used here, a random sample from that set could be tested in the laboratory. but again the sample would need to be large. in spite of the uncertain generalizability of the models to new compounds, such models may still be worthy of investigation. the kernel-based multitask modeling approach itself should be of interest to investigators, and furthermore it is possible that the models could be of use in drug discovery. in a small application of the models, boik et al. used them to help identify several dozen compounds that were predicted to be active in-vitro against the three nci cell lines, were predicted to have low rat ld values, and were commercially available. of these, were tested in-vitro and were sufficiently water-soluble and cytotoxic in a specific 48-hour assay to allow their use in the study.  the results shown here suggest that in some cases, multitask learning can be useful for constructing qsar models. depending upon the multi-task model, precision was improved over single-task models and computation time was reduced. when applied to a large natural compound library, the models developed here for cytotoxicity, ld and oral clearance identified an active set of about compounds that was rich in flavonoids and anthraquinones. this is the first published report of an oral clearance qsar model that used only chemical information as explanatory variables. 
few qsar studies have investigated multitask learning, however. multitask learning is an approach that allows distinct but related data sets to be used in training. models were constructed using kernel multitask latent analysis, an approach that can effectively handle a large number of correlated data features, nonlinear relationships between features and responses, and multitask learning. multitask learning is particularly useful when the number of available training records is small relative to the number of features, as was the case with the oral clearance data. multitask learning modestly but significantly improved the classification precision for the oral clearance model. for the cytotoxicity model, which was constructed using a large number of records, multitask learning did not affect precision but did reduce computation time. multitask learning can be useful in some qsar models. the models developed here were used to predict activities for natural compounds. hundreds of natural compounds, particularly in the anthraquinone and flavonoids groups, were predicted to be cytotoxic, have high ld values, and have low to moderate oral clearance. a suite of qsar models was constructed and used to screen a large drug library for compounds likely to be cytotoxic to multiple cancer cell lines in vitro, have low systemic toxicity in rats, and have favorable pharmacokinetic properties in humans. in this paper, a suite of three qsar models is developed to identify compounds that are likely to exhibit cytotoxic behavior against cancer cells, exhibit high rat ld values, and exhibit low to modest human oral clearance.
 quantitative structure-activity relationship models have become popular tools to help identify promising lead compounds in anticancer drug development. few qsar studies have investigated multitask learning, however. multitask learning is an approach that allows distinct but related data sets to be used in training. in this paper, a suite of three qsar models is developed to identify compounds that are likely to exhibit cytotoxic behavior against cancer cells, exhibit high rat ld values, and exhibit low to modest human oral clearance. models were constructed using kernel multitask latent analysis, an approach that can effectively handle a large number of correlated data features, nonlinear relationships between features and responses, and multitask learning. multitask learning is particularly useful when the number of available training records is small relative to the number of features, as was the case with the oral clearance data.  multitask learning modestly but significantly improved the classification precision for the oral clearance model. for the cytotoxicity model, which was constructed using a large number of records, multitask learning did not affect precision but did reduce computation time. the models developed here were used to predict activities for natural compounds. hundreds of natural compounds, particularly in the anthraquinone and flavonoids groups, were predicted to be cytotoxic, have high ld values, and have low to moderate oral clearance.  multitask learning can be useful in some qsar models. a suite of qsar models was constructed and used to screen a large drug library for compounds likely to be cytotoxic to multiple cancer cell lines in vitro, have low systemic toxicity in rats, and have favorable pharmacokinetic properties in humans. 