Improving on a Coin Toss To Predict Patient Adherence to Medications

According to the father of medicine, Hippocrates, The physician must not only be prepared to do what is right himself, but also to make the patient cooperate (1). Although Hippocrates' paternalistic tone might affront modern physicians and patients, the challenge of promoting patient adherence to effective treatments still confounds providers 23 centuries later. Adherence has become an increasingly central aspect of patient care because of the burgeoning array of effective treatments for many chronic diseases. In the face of remarkable medical advances, it is especially distressing that evaluation of patient adherence (also called compliance or pill taking) remains an imperfect science at best. The article by Liu and colleagues in this issue [2] contributes to the field of adherence measurement by demonstrating an improvement in predictive validity after incorporation of data from several adherence measures when data from an electronic monitor were missing. In this study of 108 community-based patients starting highly active antiretroviral therapy, three measures were used: 1) electronic monitoring by a special pill bottle cap with a microchip that records each time the bottle is opened (Medication Event Monitoring System [MEMS]], 2) pill count, and 3) patient interview. The authors used data from these three measures to create yet another measure that they termed a composite adherence score (CAS). This additional measure should really have been called supplemented MEMS. When MEMS data were missing or inaccurate, the investigators replaced the data points first with pill count and then, if needed, with interview data on adherence. In an innovative move, the authors did not directly use the values from the pill count and interview measures, which seemed to overestimate adherence, to supplement the MEMS data. Instead, they calibrated pill count and interview data to adjust for this overestimate before incorporating these values when MEMS data were not evaluable. Therefore, their paper offers several interesting options for investigators to consider in improving assessment of adherence. Studies of medication adherence often fail to offer an adequate description of their measures and methods to allow replication by others (3). However, in this area of research, the devil is in the details. The study by Liu and colleagues offers scant information about such issues as how patients were instructed in the use of MEMS caps, approaches used to reduce pill dumping before pill counts, and who asked questions about adherence. In this study, adherence was calculated as the percentage of prescribed doses taken in the past 4 weeks for the MEMS and pill count measures and as the percentage of prescribed doses taken in the past week for the patient interview. Since CAS relies primarily on the MEMS adherence measure, it largely reflects adherence over 4 weeks. Such variations in the time frames covered by adherence measures are common but make comparisons of results across measures more difficult. The predictive validity of all four adherence measures was assessed by examining their associations with undetectable HIV RNA levels 8 and 24 weeks after starting treatment. The use of an outcome measure such as viral load suppression makes this study more clinically relevant than those that simply report adherence levels. Although the success of antiretroviral treatment in achieving an undetectable viral load represents a complex interaction of patient-related, virologic, and medication-related factors, the study by Liu and colleagues adds further evidence that excellent adherence plays a central role in suppressing HIV viral load (4, 5). Of all four measures, CAS (supplemented MEMS) was the most strongly predictive of viral load; the receiver-operating characteristic (ROC) curve statistic was 0.84, where a value of 1.0 would be a perfect prediction of an outcome and a value of 0.5 would indicate a 50:50 chance of predicting the outcome. However, the ROC curves for pill count and MEMS were only slightly inferior (0.77 and 0.76, respectively). The improvement in the ROC statistic for CAS compared with the MEMS-only measure shows that missing electronic monitor data may be informative. Because the mean CAS was higher than the MEMS-only adherence value (76% vs. 63%, respectively), the missing data points seem to represent times when patient adherence was better, strengthening concerns that the unsupplemented MEMS measure underestimates adherence. Despite several innovations in the analysis of data, Liu and colleagues' study is subject to several limitations that plague community-based research. First, 40% of eligible patients declined to participate and only 80% of participants had data at just 8 weeks into the study. Thus, problems with recruitment and retention are evident. Second, although the authors measured adherence at 4-week intervals for 48 weeks, only 666 data points, not 1296, were available for the 108 participants because of dropout and other unspecified reasons. Third, substantial data were unavailable for each of the measures. Of the 666 data points, MEMS data were missing or unreliable for 16% and pill count data were unavailable for 10%. More worrisome, the authors collected interview data only at 3 of a possible 12 time points. The data from the third time point (48 weeks) were not analyzed, and, for the remaining 2 time points, more than 40% of the participants had missing data. This significant amount of missing data for patient interview makes the results in relation to this measure virtually uninterpretable. Liu and colleagues' use of electronic monitors as the backbone of their enhanced adherence measure is supported by studies of other diseases that report strong associations between adherence measured by this tool and clinical outcomes (6, 7). However, Liu and colleagues' study does not capitalize on the unique potential of electronic monitors to assess intervals between doses (8). Consideration of dose intervals may be particularly important when medications are taken several times per day (9). Yet use of electronic monitors to assess adherence is not a panacea. First, patients who are accustomed to using pillboxes and other reminder systems might decline to participate in a study that requires them to stop using these devices. Such concerns may explain the high refusal rate for participation in the study by Liu and colleagues. Second, improper use of electronic monitors by patients has resulted in the loss of substantial amounts of data (10). Bangsberg and coworkers (4) found that almost one third of patients ignored study instructions and admitted to taking out an extra pocket dose of medication when they left home. Such behaviors may explain why MEMS underestimates adherence. Fourth, as with any electronic device, adherence monitors can be damaged or lost. Fifth, the cost of these monitors obviates their use in clinical practice without special funding sources. Last, the steps required to obtain accurate data from electronic monitoring devices may act as a type of intervention that would change adherence behavior. Given the limitations of electronic monitors, what other measures should clinicians and researchers consider using? To their credit, Liu and colleagues did not even try to obtain physicians' predictions of patients' adherence levels. Clinicians would do better to toss a coin than to try to predict nonadherence (11). On the other hand, pill counts seemed to perform almost as well as the MEMS-only measure in the study by Liu and colleagues. A supplemented pill count measure could be explored, as suggested by these authors, in which missing or inaccurate data are replaced by data from patient interviews. Pill counts have been used for decades to address adherence (12), and researchers have been complaining about the logistics of obtaining this measure for nearly as long. Fletcher and associates (13) listed problems with this approach, including patients' forgetting to bring their bottles, patients' taking offense at having their pills counted, difficulties in determining the start of the current medication prescription, and patients' combining several refills in one bottle. Although pill counts without prior warning have been performed in previous studies to avoid such inaccuracies as pill dumping (14), many patients may be hesitant to agree to this approach. Despite efforts to improve the reliability of pill counts, this method still overestimates adherence (15). Liu and colleagues cursorily dismissed the use of pharmacy medication refill data to evaluate persistence with pill taking. Nonetheless, several studies have reported significant associations between HIV viral load suppression and adherence measures using pharmacy data (16, 17). This approach requires access to complete pharmacy data that have been checked for accuracy. Pharmacy-based assessments offer several advantages, including the possibility of conducting population-based analyses and avoiding a possible Hawthorne effect because patients are aware of being evaluated. Should we really give up on patient interview to evaluate adherence? Definitely not. Several studies of HIV-infected patients have shown significant associations between HIV viral load suppression and self-reported adherence (4, 18). Admittedly, careful attention must be given to a preamble before adherence questions to help the patient understand that he or she is not being judged and that honest answers are sought. Ideally, the questioner should not be a health care provider. Patient recall may limit questions about adherence during periods longer than 1 week. Although patient interview overestimates adherence, patients who admit to poor adherence are generally also nonadherent by other measures (14). This is an important group to identify because self-acknowledged nonadherent patients may be more responsive to interventions to improve adherence than those who deny any a