Adult patients attending one of four outpatient primary care clinics were recruited in two distinct cities (Shreveport, Louisiana; Chicago, Illinois). One clinic in each city was a general medicine academic practice; a second clinic was a safety net community health center. Subject recruitment took place between June and August 2007. Patients were considered eligible for the study if they were 18 years of age or older, and ineligible if the clinic nurse or study research assistant identified a patient as having one or more of the following conditions: (1) severely impaired vision; (2) hearing problems; (3) too ill to participate in the survey; 4) non-English speaking. Institutional Review Boards for all locations approved the study. A total of 562 patients were approached in the order they arrived at the clinics and prior to the medical encounter; 530 consented to the study. Thirteen patients were excluded based on self-reported impairments with hearing (n=3) or vision (n=10). Ten patients were excluded due to limited English proficiency, and seven others were excluded based on incomplete information. In all, 500 patients participated in the study; the sample was evenly split across the two study locations (n=250 per city) and practice setting (academic, community; n=125 within each study location). A response rate was determined following the American Association for Public Opinion Research standards, estimating 92.8 percent of approached eligible patients participated in the study.13 Intervention The intervention was the use of 'enhanced' auxiliary prescription drug warning labels, with or without patient-centered icons to support comprehension. Previously, nine of the most commonly-used drug warnings and/or precautions placed directly on prescription medication bottles were revised with patient feedback and pilot tested.12 Specifically, text was rewritten using more clear, concise, and explicit language. The icons supporting comprehension of the text message were altered to more accurately reflect the mental representations patients identified with each warning (see Table 1). Guidelines established by the International Organization for Standardization for the development and testing of universal icons were followed.14 Other design elements were also implemented, including removal of the use of color and optimizing font size for clarity.

Structured Interview A structured "cognitive" interview protocol was developed to assess patient understanding of the drug labels; a process previously used by our research team.1-4,15 After patients consented to the study, a trained research assistant administered the structured interview that included self-report of sociodemographic information (age, gender, race/ethnicity, education, number of prescription medications currently taken daily). Actual prescription pill bottle containers with drug warning labels attached were then shown to all of the patients for review. Once the patient provided their interpretations, the research assistant administered the Rapid Estimate of Adult Literacy in Medicine (REALM), a reading recognition test comprised of 66 health-related words. The REALM is the most commonly used test of patient literacy in medical settings.16 It is highly correlated with standardized reading tests and the Test of Functional Health Literacy in Adults.17, 18 Assignment Nine drug warnings were used in this study. Each warning had three versions (standard, enhanced text, enhanced text + icon) for a combined total of 27 labels under evaluation. Within each label version, warnings were randomly organized into groups of three labels and then placed on the back of prescription bottles. We viewed three warnings on a bottle to be a realistic portrayal of the number of warnings commonly found on pill bottles; many pharmacies allot space for as many as five per bottle (see Figure). In total, nine bottles were created (three bottles per label version).

The initial order of the warnings on the back of the bottle (e.g. first, second, or third position) was also randomly assigned and replicated across all label versions (see Figure). The bottles were then randomly grouped into three sets; every set containing one bottle from each label version and all of the nine warnings (Table 1). Patients thereby were exposed to all of the drug warnings and each label type, only seeing one version of each warning message. Consecutive patients scheduled for medical appointments at each of the clinics that consented to the study were subsequently assigned, in a systematic rotating order, to one of the three regimens to review.

Outcomes Patient attendance to and correct interpretation of the nine prescription drug warnings placed on container vials were evaluated. A trained research assistant at each location would direct patients to the back of one of three prescription vials, and ask "In your own words, what do these mean to you?" The patient's verbatim responses were documented on a separate form. All patient responses to each of nine drug warnings were then independently rated as either correct or incorrect by three general internal medicine attending physicians from three different academic medical centers. If subjects did not attempt to interpret one of the warning labels in their responses, this was coded as 'non-attendance'.

Blinding and Coding Each physician rater was blinded to all patient information and was trained to follow stringent coding guidelines agreed upon previously by the research team. Specifically, correct scores were to be given only if the patient's response included all aspects of the label's message. Responses were given an incorrect score if they were inaccurate, or if they did not contain all aspects of the warning.

Inter-rater reliability between the three physicians coding the patient responses was high (Kappa = 0.87). The 380 responses (8.4%) that received discordant ratings between the three reviewers were sent to an expert panel that included a primary care physician and clinical psychologist and health services researcher with expertise in health literacy for further review. Each panel member, also blinded to patient information, independently reviewed and coded the responses as correct or incorrect. For 86.1 percent (n=327) of the 380 responses, a consensus ruling was achieved among the expert panel for a final ruling on the coding of those responses. For the remaining 53 patient responses, a majority rule was imposed and the rating by a minimum of two panel members was used to determine the scores.