Great meaningless questions in urology: which is better, open, laparoscopic, or robotic radical prostatectomy?

Alberto Contador won the 2010 Tour de France riding a Specialized SL3 racing bike. The US rider Chris Horner, who came in 12 minutes behind, rode a Trek Madone. The top rookie Daniel Lloyd, rode a Cervelo S3, but was pretty far down the field, over 4 hours behind the leaders. We presume that no self-respecting urologist would use this information to claim that the Trek is a faster bike than the Cervelo, or that Daniel Lloyd could have won, if only he had ridden a Specialized. About radical prostatectomy claims, however, it seems that urologists are considerably less discriminating. A good number of studies have recently purported to address the question of whether the open, pure laparoscopic or robotic approach to radical prostatectomy leads to superior results. In a typical paper, that of Ficarra et al.1, urinary and erectile outcomes were documented for patients treated by one of six surgeons, four open and two robotic. The authors reported a 9% absolute higher probability of continence at one year for robotic surgery, and concluded that the robotic approach leads to improved urinary function. To know whether this conclusion is reasonable, we have to consider what sort of variation we might expect between individual surgeons. For example, if all surgeons had continence rates within a few percent of one another, it would be difficult to explain the 9% difference in terms of the four open surgeons, just by chance, being less skilled than the two robotic surgeons. There is now overwhelming evidence that the results of individual surgeons vary by a good deal more than a few percentage points. Begg and colleagues reported that complication rates among high volume surgeons ranged from less than 5% to greater than 50%2. With respect to functional outcomes, our group found 30 – 40% absolute differences in erectile and urinary function between a small number of surgeons at a single tertiary care center3. Differences between surgeons therefore dwarf differences between surgical approaches. Accordingly, just as an analysis of Tour de France results tells us more about the riders than the bicycles, a study of a handful of surgeons concluding that one approach is better than another most likely reflects the relative skill of the surgeons practicing each approach. That said, at least patients in the Ficarra paper were part of the same cohort. Several papers have compared results reported for open and robotic case series published in the literature4, 5. Alternatively, authors have reported their results for robotic or laparoscopic surgery and compared these to published open series6. It is just about impossible to tell whether patients and outcomes are comparable in these sorts of studies: pathologists may grade and score surgical margins differently; patients may differ in terms of baseline function; assessments of function and surgical complications are unlikely to be identical. These studies are the equivalent of asking Contador, Lloyd and Horner to leave home for a 100 mile ride and then announcing that the best bike is the one ridden by the first cyclist home, ignoring any differences in terrain or weather. One of the reasons why we would want to avoid concluding that one bike is better than another on the basis of finishing places is that some riders are more experienced than others: Contador was a prior Tour winner, whereas Lloyd was a rookie. Few urologists would deny that radical prostatectomy is similarly associated with a learning curve. Indeed, our group has documented dramatic learning curves for two independent groups of surgeons, open7 and laparoscopic working without robotic assistance8. A comparison of the two learning curves is shown in the figure. There are extremely large reductions in recurrence rates as a surgeon gains experience with radical prostatectomy. Figure 1 Learning curves for open (grey) and laparoscopic (black) radical prostatectomy This has obvious implications for the question of whether the open, pure laparoscopic or robotic approach is better. When we ask this question, are we referring to the recently trained surgeon, the highly experienced surgeon or some kind of “average” surgeon? The figure shows that both highly experienced surgeons, as well as those who are just starting their career, likely have similar results on open and non-robotic laparoscopic surgery. However, the latter is harder to learn. There are currently few comparable data on the learning curve for robotic radical prostatectomy. My guess is that robotic surgery does have a learning curve and that, when practiced by high volume surgeons, this compares favorably with the learning curves for open and laparoscopic. Either way, it should be obvious that “what is better, open, robotic or pure laparoscopic?” is meaningless outside the context of the learning curve. The presence of surgeon heterogeneity raises further questions about the “average” effect of different procedures. As a simple example, imagine that we compared two groups of 10 orthopedic surgeons, one of which used what we will call the “dependable” technique and the other a novel, but “difficult” approach for refractory back pain. Assume that all 10 surgeons in the “dependable” group reported close to 35% of their patients experiencing good pain relief. Further assume that, in the “difficult” group, success rates amongst 9 of the surgeons varied widely between 5% and 50%, with an average of 30%. However, the 10th surgeon had managed to master the approach with 95% of patients reporting less pain, giving an overall success rate of 36.5%. On average, the “difficult” technique led to a higher chance of pain relief than the “dependable” technique, but we would likely prefer all surgeons to get reasonable results than there to be huge variation, with a limited number of experts obtaining remarkable results. This is particularly because the proportion of surgeons who would be able to master the “difficult” technique would likely not be well estimated. If results of different surgeons differ dramatically, then it seems more meaningful to ask about how results vary within a particular approach than about average results. Lance Armstrong famously stated that “It’s not about the bike”; some urologists seem to say, “No, but it is all about the robot”. I work with robotic surgeons and have observed several perform a radical prostatectomy. I have been impressed by their skill and I am confident that their results are better than many open and pure laparoscopic surgeons. But we need to get beyond our obsession with technology and do the hard work of figuring out why it is that some surgeons have so much better results than others, a question largely irrespective of surgical approach. Empirical studies of surgeon technique will require a far greater investment of time, resources and scientific ingenuity than retrospective analyses of surgical databases, but they are far more likely to produce discoveries that will lead to surgical progress.