Kortagere et al. the in vitro assay was not analyzed. Because radioligand binding is not a viable option for high-throughput screening for PXR and addi- tionally would probably not give information on actual overall activity, there may be differ- ences between docking and functional assays, a situation likely not unique to PXR. All compounds from the ToxCast data set were docked to the five hPXR crystal struc- tures and scored using GoldScore and hybrid scoring schemes. When molecular docking is used as a tool for classification studies, the question arises as to what docking scores should be considered as a cutoff. Because there are no standard rules governing the choice of cutoff scores, in the past we have designed hybrid scoring schemes that used similar- ity scores derived from molecular shapes as a weighting factor (Kortagere et al. 2009). The goal of the hybrid scoring schemes is to increase the gap between hPXR agonists and hPXR nonagonists and thus ease classification of compounds. In this study we derived cutoff scores either by using the docking scores of the cocrystal structure ligands or by averaging the docking or weighted scores of all the ligands binding to the five crystal structures (Table 1). Predictions from the complete ToxCast data set performed marginally better than the sam- ple data set, thereby emphasizing the utility of the approach to classify large data sets prior to in vitro testing. The hybrid scoring scheme had a sensitivity of approximately 69%, speci- ficity of approximately 56%, overall predic- tion accuracy of approximately 59%, and a Matthews correlation coefficient of 0.2 for classifying all ToxCast compounds based on the NCGC data (Judson et al. 2010), whereas the GoldScore-based scheme provided the best sensitivity of approximately 74% (Table 3). Classification studies also benefit from the availability of multiple crystal structures of the protein in complex with a variety of ligands. Although all the cocrystallized ligands bind to the same binding site and the crystal structures superimpose within approximately 1 Å root mean squared deviation, the size, nature, and chemical composition of these ligands are very different (Ekins et al. 2009). In addition, the availability of multiple crystal structures of promiscuous proteins such as PXR helps improve sampling of the docking mode of these compounds. This is evident Table 3. Statistical parameters for the hPXR con- sensus docking results from the sample data set (n = 28) and complete (all; n = 308) ToxCast data set. Data set Sample data set, gs Sample data set, hs All ToxCast, gs All ToxCast, hs SE 66.67 58.33 73.85 69.23 SP 46.67 50.00 51.22 55.69 Q 55.55 53.57 55.95 58.52 C 0.13 0.08 0.20 0.20 Abbreviations: C, Matthews correlation coefficient; gs, GoldScore; hs, hybrid scoring schemes; Q, overall pre- diction accuracy; SE, sensitivity; SP, specificity. 1416 in the range of docking scores we obtained for each compound across each of the five crystal structures [see Supplemental Material, Table 2 (doi:10.1289/ehp.1001930)], which also emphasizes that averaging methods to get a single score should not be used in these cases. Instead, we classified each docked com- plex using the cutoff scores listed in Table 1 and then used a majority vote (3 of 5) method for consensus classification for both GoldScore and hybrid scoring schemes. The ToxCast data set also consists of com- pounds that have diverse structure, size, and chemical composition, albeit with similar functionality important for interactions with PXR [see Supplemental Material, Figure 1 (doi:10.1289/ehp.1001930)]. Thus, classify- ing this diverse data set by similarity-weighted scoring schemes with approximately 69% sen- sitivity for predicting agonists is encouraging (considering the promiscuity of the receptor), and the statistics are comparable with or better than our previous data from studies using the GoldScore and hybrid scoring schemes (Ekins et al. 2009; Kortagere et al. 2009). These data should improve in the future as docking tools develop; however, the current approaches may be more accurate with other less-promiscuous nuclear receptors such as the estrogen and androgen receptors. The hybrid scoring scheme performed better for the ToxCast data (sen- sitivity and Matthews correlation coefficient were consistently higher; Table 3) compared with a series of 119 steroidal molecules we pre- viously used with average sensitivity of 52%, specificity of 50.34%, accuracy of 50.76%, and Matthews correlation coefficient of 0.02% (Kortagere et al. 2009). This suggests that there may be differences in docking utility depend- ing on compound class and therefore value in evaluating molecules beyond drugs to gain a broader insight into potential PXR agonists among industrial chemicals, pollutants, natural products, and so on. We also compared docking with a ligand- based QSAR method based on steroidal com- pounds to provide a further benchmark, and in this case its performance was poor, pos- sibly for several reasons. First, many groups have illustrated the importance of chemical space coverage and the applicability domain of ligand-based models (Chekmarev et al. 2008, 2009; Dimitrov et al. 2005; Ekins et al. 2006; Kortagere et al. 2008, 2009; Sheridan et al. 2004; Tetko et al. 2006, 2008). When we analyzed the molecular space covered by the steroids in the ligand-based Bayesian model compared with the ToxCast compounds, we found that they can be clearly separated [see Supplemental Material, Figure 2 (doi:10.1289/ ehp.1001930)], which is indicative of little overlap. This would suggest it may be difficult for this local hPXR model to reliably predict compounds that are not steroidal. Thus, it may be important in future work to develop a separate Bayesian model with the ToxCast compounds to predict hPXR agonists added to later versions of the database. Our ligand- based data also confirm that to predict diverse compounds with likely hPXR agonist activity, methods that are more generic or global in nature are required that can capture some of the flexibility of the ligand-binding domain (Ekins et al. 2009). Although ToxCast is a major U.S. EPA initiative for prioritizing toxicity testing of large numbers of pesticides and other chemicals (Dix et al. 2007; Houck et al. 2009; Judson et al. 2009, 2010; Knight et al. 2009), we suggest that it also represents a unique opportunity to evaluate various predic- tive computational approaches used for toxi- cology end points prospectively in addition to the many ways to mine the data retrospec- tively. In the present study we have addressed only a tiny fraction of the data produced to date and perhaps raised the question that focusing on one cell line for a single nuclear receptor, such as hPXR, may be too simplistic. We still can learn a great deal from the efficient combination of in vitro and in silico approaches, such that multiple iterations of prediction may yield a more cost-effective route to selecting compounds for testing from a large database. The ToxCast data set there- fore represents an important and evolving basis for evaluating computational methods used in toxicological assessments of compounds important for environmental and health applications. Although we did not identify any PXR antagonists or allosteric antagonists, based on the few samples tested, the com- plete ToxCast data set could be more exhaus- tively studied with both computational and in vitro methods in the future (Ekins et al. 2008a). Current opinion suggests that classi- cal competitive antagonists for PXR that bind in the ligand-binding pocket may be difficult to identify (Xue et al. 2007a) compared with allosteric antagonists that bind elsewhere on the protein surface (Ekins et al. 2007, 2008a). It is therefore important to evaluate whether a compound may be a more selective allosteric antagonist, because this could outweigh any potential PXR agonist activity in vivo. Ref Rencs 26:2216–2224. 21:1304–1314. Bachmann K, Patel H, Batayneh Z, Slama J, White D, Posey J, et al. 2004. PXR and the regulation of apoA1 and HDL- cholesterol in rodents. Pharmacol Res 50(3):237–246. Biswas A, Mani S, Redinbo MR, Krasowski MD, Li H, Ekins S. 2009. Elucidating the ‘Jekyll and Hyde’ nature of PXR: the case for discovering antagonists. Pharm Res 26:1807–1815. Chekmarev D, Kholodovych V, Kortagere S, Welsh WJ, Ekins S. 2009. Predicting Inhibitors of acetylcholinesterase by regression and classification machine learning approaches with combinations of molecular descriptors. Pharm Res Chekmarev DS, Kholodovych V, Balakin KV, Ivanenkov Y, Ekins S, Welsh WJ. 2008. Shape signatures: new descrip- tors for predicting cardiotoxicity in silico. Chem Res Toxicol ovlume 118 | number 10 | October 2010 • Environmental Health Perspectives