Comparative Effectiveness: Response to Gould et al.

To the Editor We thank Gould et al. for their interest in our work. We acknowledge, as we did in our article, that limitations exist in all methodologies, including retrospective analysis using large clinical databases. However, we do not agree with the Gould et al. statement, “we now have two studies using different research methods that have arrived at conflicting results. Each of these advanced therapies has been shown to be efficacious for wounds not responding to evidence-based standard care. ” Zelen et al. reported that 85% of dehydrated amnion/chorion membrane (dHACM) treated patients achieved complete healing after 4 weeks and that 95% of dHACM patients achieved complete healing after 6 weeks. The median time to healing with dHACM was reported to be 13 days. Such healing rates are dramatically different than any other product in the published wound healing literature. Small, underpowered clinical trials (only 20 patients were treated with dHACM), even when randomized, need to be confirmed, and promising treatments identified in small trials warrant further confirmation in larger studies. Zelen et al. investigation would be considered a small phase II clinical trial, even by wound healing standards. We have all seen significant results in phase II studies that could not be repeated in larger phase III clinical trials, a result that occurs twothirds of the time. We were not able to confirm the high healing rates for dHACM in this retrospective large database study. We found wound closure of 9% at 6 weeks, 28% by 12 weeks, and 47% by 24 weeks, with a median time to healing of 182 days. We did find the results for Bioengineered Living Cellular Construct (BLCC) consistent with 15 years’ worth of clinical trials and clinical practice experience. The reasons for this discrepancy are unclear. Designs flaws may be at play. As an example, BLCC-treated patients received a mean of 6.2 applications in a company-designed and -sponsored efficacy trial, while the Food and Drug Administration (FDA)–approved product labeling states that the safety and effectiveness of BLCC have not been established for patients receiving greater than five device applications. It is unclear why a study protocol evaluating the outcomes and cost-effectiveness of a product would permit usage contrary to product labeling. The more likely cause for the discrepancy is the limited number of observations in the randomized clinical trial. An example of the consequences of limited observations relates to the locations of the wounds treated in each study. Gould et al. suggest that the differences in wound location for dHACM compared to BLCC in our study were concerning, as 30% of the dHACM wounds were on the heel compared with only 20.9% in the BLCC group. We controlled for this significant difference in our analysis. In the randomized clinical trial, however, the authors reported that there were no significant differences between wound locations, yet 25% of the BLCC treated wounds were on the heel compared to 5% in the dHACM group. The difference was not statistically significant due to the low power of the study to detect these differences. Gould et al. raise two additional issues we wish to clarify. First, Gould et al. suggest the reason for the differing results is that, “when a new product becomes available, the tendency is to use it on the wounds that have failed everything else. ” If this statement is true, it suggests that product use in clinical practice would differ significantly from the population studied in the randomized clinical trial. Small clinical trials may study only a small, homogeneous subset of the wounds encountered in “real world” practice, and thus it would not be expected that these results should be reproduced in a large, retrospective database study. If Zelen et al. findings can be confirmed, it would be very useful to define the specific segment of the Diabetic Foot Ulcer (DFU) population that experienced the excellent results in the clinical trial. Confirmation of these results in a larger study would then identify a patient population expected to benefit highly from this adjuvant therapy, hence avoiding use of the agent in wounds that are not likely to benefit while maximizing cost-efficacy. Concern was elicited about our pragmatic definition of wound closure. One advantage of effectiveness trials is that endpoints need not be the same as required by the FDA in efficacy trials. We had previously reasoned that a patient going from having a large to a very small wound size (i.e., 0.25 cm) and not having any additional follow-up visits, most likely had healed. This is a published endpoint used for venous leg ulcer research, a wound type without callus, which was another concern expressed by Gould et al. In the this study, this outcome was defined a priori and was not biased (i.e., to favor BLCC over dHACM). In theory, more wounds might be considered (virtually) healed rather than actually healed. In this case, the 28% 12 week healing seen in dHACM might actually be lower by the definition Gould et al. suggest. We agree that issues raised about research methodologies are critically important. However, they must not cloud the issue that more robust, well-designed studies are needed to replicate the results of Zelen et al. We welcome seeing those studies, at which time it would be appropriate to ask “how to select which therapy is best” for a particular patient. We agree with Gould et al. that the complications of unhealed diabetic foot ulcers are profound. These wounds are detrimental, associated with morbidity and mortality, and the patients affected are too important for research to go unquestioned without validation. The time is right for the wound healing community to seize this opportunity to demand better research, both in randomized studies and in retrospective studies, and to comprehensively evaluate all