Article Outcome success ligand never ranks well ligand fails to dock automatically Table 1. Docking Successes with DOCK Blastera Astex-85 Gold-114 DUD-38 PDB-9050 29 44 2 27 67 20 15 21 2 1398 6357 1295 a Number of targets where fully automatic docking with DOCK blaster is successful, as judged by both good pose fidelity (< 2.0 A˚ RMSD) and rank (top 5%) versus about 100 property-matched decoys. Table 2. Docking Success Depends on Docking Parametersa parameter choice DOCK score ScreenScore FlexX PLP PMF all any 18 29 18 36 16 33 18 38 10 22 a Success is defined as ligand within 2 A˚ RMSD of the crystal pose and rank in the top 5% of about 100 property-matched decoys. Astex-85 benchmark. For instance, whereas 29 targets succeeded for pose fidelity and rank using any of four parameter sets and the standard DOCK scoring function, only 18 of these succeeded with all parameter sets. (Supporting Information Table S2 , and http://data.docking. org/2009/AstexTables.doc). Some parametrization schemes were more successful than others. The best was “finer/ polarized” with 49 successes, followed by “finer/normal” with 46, “coarser/polarized” with 37, and the least successful being “coarser/normal” with 34 successes. The “finer” schemes having more spheres generally outperformed the “coarser” schemes, although this was not always true. The polarized dipoles in the electrostatics of “polarized” were slightly more successful than the “normal” scheme having standard AMBER94 charges on the protein. For the 51 protein-ligand complexes docked within 2 A˚ rmsd of the crystal pose, we then asked how well the redocked crystallographic ligand ranked compared to around 100 property-matched decoys (Tables 1 and 2 and http://data.docking.org/2009/AstexTables.doc). The well- posed ligand also ranked in the top 5% of the property matched decoys in 29 cases, suggesting that these are suitable for automatic prospective docking. Just beyond the 2 A˚ cutoff, there were 11 cases in which the ligand posed close, within 3 A˚ , and a further 20 cases where the ligand never got even as close as 3 A˚ rmsd to the crystallographic ligand. For 18 targets, the results were excellent regardless of which docking parameter set was chosen; we could find no common theme among the targets to which to attribute this good fortune (Figure 4). There were three nuclear receptors (1M2Z, 1N46, 1SQN), six kinases (1KE5, 1OPK, 1YWR, 1V4S, 1PMN, 1T46), and seven other enzymes (1HWI, 1OF6, 1LPZ, 1Q4G, 1JLA, 1R9O, 1VCJ). There was no obvious pattern in the physical or chemical properties of the ligands to explain why these targets worked so reliably. We did notice some patterns among the less successful docking calculations, however. Five targets achieved good poses with all parametriza- tions, yet the ligand never ranked well (Figure 5 and http:// data.docking.org/2009/AstexTables.doc, class 4): 1IA1, 1J3J, 1TZ8, 1XM6, and 1XOQ. These cases are interesting because pose fidelity is often used as a metric for docking success. Poor ranks despite good poses can arise when either the ligand scores unexpectedly poorly or the decoys score unexpectedly well. For instance, many decoys for 1IA1, dihydrofolate reductase (DHFR), contained guanidinium Journal of Medicinal Chemistry, 2009, Vol. 52, No. 18 5715 and other groups that might well bind to DHFR and in turn received excellent scores. Because the ligands looked compe- titive to us, we attribute the failure to achieve a good rank as being due to highly competitive decoys rather than a failure of the ligand itself, as it also received a good score. Future work will investigate whether the decoy selection procedure might be improved to remove viable-looking ligands that are not currently eliminated by our fingerprint-based dissimilar- ity metric. Failure to achieve even a good pose against perhaps 20-30% of targets is common in docking studies, so we were not much surprised to note 31 targets where the ligand did not dock even close (within 3 A˚ rmsd) to a satisfactory pose (Figure 6 and http://data.docking.org/2009/AstexTa- bles.doc, class 6). Many of these pose fidelity failures could be attributed to one of four common causes: (a) insufficient conformational sampling of the ligand, particularly of ali- phatic ring puckering, (b) symmetric or pseudosymmetric molecule, (c) critical missing water molecules, and (d) ligand pose dominated by electronic (orbital) effects. These issues are common to all docking methods and protocols and are therefore expected for this study. In the scripted, automatic context of DOCK Blaster, some of these problems could be addressed, for instance by using a rescoring method to allow for relaxation of ring puckering or an approach that samples explicit water molecule positions.50 Other failures, such as the plausible reversed docking pose of a pseudosymmetrical molecule in 1YV3, could be better addressed using a density- based pose-assessment method.51 Some failures are probably simply beyond the capabilities of an orbital-na ı¨ve method like DOCK, such as 1P2Y, a P450, where the ligand pose appears to be under electronic rather than steric control. Whereas some of these failures may be addressed by im- provements in the protocol, we worry about a tendency to build in too much “expertise” that leads to over fitting. Some of these problem targets will likely remain beyond the capability of full automation, at least in the near term. With four parameter sets to choose from, we wondered whether one of them was more successful at achieving good pose fidelity than the others. To answer this, we plotted the frequency and cumulative frequency of pose fidelity for the best parameter set for each case (Figure 7) and also for each parameter set separately (Supporting Information Figure S1). We found that all parameter sets achieved a pose within 2 A˚ rmsd at least 50% of the time. The “finer” schemes performed slightly better at 60%. These graphs confirm that all parameter sets are roughly comparable at achieving good ligand poses, that each is best some of the time, and none of the parame- trizations is either much better or much worse than average. We wondered whether better pose fidelity could be used to predict rank. If it were, good pose fidelity as measured by rmsd could be used to select the best docking parameters. We plotted pose fidelity (A˚ , rmsd) versus rank using the para- meters that led to the best pose fidelity in each case (Figure 8) and also for each parameter set separately (Supporting Information Figure S2). Poor pose fidelity was generally associated with poorly ranked compounds, although there was one notable exception, beta II tryptase (2BM2), where the ligand docked backward but still managed to get a good score compared to its decoys. However, good pose fidelity was entirely uncorrelated with rank, regardless of which parameter set was used. It would thus appear that pose fidelity cannot be used to predict rank and thus whether the model is suitable for prospective docking.