# Presentation: 100772

## Evaluation of Written Prescription Information Provided in Community Pharmacies: A National Study  

- Bonnie L. Svarstad, Ph.D.
- Jeanine K. Mount, Ph.D. and R.Ph.
- University of Wisconsin - Madison

## Study done in cooperation with...

- FDA
- National Association of Boards of Pharmacy
- National expert panel

## National Expert Panel

- 16 experts
- Nominated by seven pharmacy organizations
- Pharmacy practitioners and experts in pharmacotherapy and communications
- Faculty at 9 pharmacy colleges and universities

## Panelists

- Mary Amato
- Heidi Anderson-Harper
- Robert Beardsley
- C. A. Bond
- Marie Gardner
- Betty Dong
- Carole Kimberlin
- Duane Kirking
- Sharlea Leatherwood

- Matt Osterhaus
- Anthony Provenzano
- Mary Pubentz
- Betsy Sleath
- Jenene Spencer
- Judith Sommers Hanson
- Gayle Dichter
- Bonnie Svarstad, Chair

## Past Studies

- Distribution of patient information increased from 16% in 1982 to 74% in 1998
- 1999 study in 8 states found 87% given information but quality was variable 		[Svarstad & Bultman, 1999]

## How does this study differ? 

- Pharmacies sampled from national list
- Professional shoppers visited pharmacies
- Experts and consumers rated information
- Additional analyses performed

## Primary Aims of Study 

- What percent of patients are given written prescription information?
- How do experts rate this information?
- How do consumers rate this information?
- How well does information adhere to criteria for useful medication information?

## Secondary Aims of Study

- How do expert/consumer ratings compare?
- Do ratings vary by consumer background?
- Do ratings vary by pharmacy type?
- Do ratings vary by leaflet characteristics?

## Objectives for Today  

- Review study design and procedures
- Review evaluation criteria and forms
- Present results:						- Leaflet distribution and ratings 			- Factors that might influence ratings

## Study Design 

- Shoppers acting as patients presented four prescriptions at each pharmacy
- Patient information leaflets mailed to UW
- Expert panelists rated leaflets
- Consumer panelists rated leaflets
- Rating forms mailed to UW for analysis

##  Sampling of Pharmacies

- Excluded: hospital, clinic, LTC, mail order, IV infusion, dispensing physician, government settings, Alaska, Hawaii, Puerto Rico, states prohibiting filling of prescriptions for research
- Selected from 57,157 community pharmacies
- Simple random sample of 384 pharmacies
- 35% independent, 65% chain
- Data collected in 44 states

## Observer Protocol

- Shoppers hired by professional shopper firm
- 72% of visits by females
- 66% of visits by persons 45 years or older
- Mean age = 50

## Standard Scenario  

- Presented new prescriptions for atenolol, glyburide, atorvastatin, nitroglycerin SL
- Did not ask questions or initiate talk
- If asked, patient-observer reported:			- new diagnoses: heart disease, diabetes	- no prior use of study drugs

##  Observer Protocol (cont.) 

- Shopper mailed materials to shopper firm
- Firm removed pharmacy/pharmacist names
- All leaflets, brochures, other information items mailed to researchers
- All items referred to as  “leaflets”

## Expert Evaluation Forms

- Each form included 8 general criteria and 62-63 sub-criteria
- Eight criteria from 1996 Action Plan for “useful” information
- Sub-criteria based on approved labeling
- Forms revised until all panelists approved

## Eight Criteria from 1996 Action Plan for Useful Information  

- 1. Drug names and indications
- 2. Contraindications and what to do before using
- 3. Specific directions about how to use,     monitor, get most benefit
- 4. Specific precautions and how to avoid harm

## Eight Criteria from 1996 Action Plan for Useful Information

- 5. Serious and frequent adverse reactions, what to do
- 6. General information, encouraged to ask questions
- 7. Scientifically accurate, unbiased, up-to-date
- 8. Readily comprehensible and legible

## Scoring Method for Expert Forms  

- Each criterion rated by 4-10 sub-criteria
- Each sub-criterion rated by adherence level:  full (2), partial (1), no adherence (0)
- Computer calculated % points obtained
- Scale ranged from 0 to 100% (best)

## Scoring Categories 

**Level of Adherence****	****	****	****Point Range %**
- 5					80-100
- 4					60-79
- 3					40-59
- 2					20-39
- 1					0-19

## Inter-rater Reliability 

- Experts assigned to one of four drug groups
- Independently rated same sub-set of leaflets
- Good inter-rater reliability obtained
- Pearson r for overall adherence = 0.90-0.97

## Expert Rating Process 

- Each expert assigned to one of four drugs
- Each leaflet rated by one expert
- Experts rated 1,367 pharmacy-generated leaflets and 31 manufacturer-generated leaflets (mean = 87 per expert)

## Consumer Evaluation Form  

- 1-page form with 12 items
- Based on 1996 Action Plan and Krass et al
- Items scored 1 (poor) to 5 (good)
- Overall scale ranged from 0 - 100% (best)

## Scoring Categories for Consumer Form 

**Level of Adherence****	****	****	****Point Range %**
- 5					80-100
- 4					60-79
- 3					40-59
- 2					20-39
- 1					0-19

## Three Legibility Items

- Poor-good print size
- Poor-good print quality
- Poor-good spacing between lines

## Six Comprehensibility Items

- Poorly-well organized
- Poor-good length
- Unclear-clear
- Unhelpful-helpful
- Incomplete-complete
- Hard-easy to find important information

## Three Summary Items 

- Hard-easy to read
- Hard-easy to understand
- Not useful-useful

## Test-Retest Reliability

- Nine consumers independently rated 18 leaflets at two sessions
- Good test-retest reliability obtained
- Pearson r for overall score = 0.82

## Consumer Rating Process 

- Recruited 154 consumer raters in 11 states
- Recruited at senior centers, clinics, work and social organizations
- Facilitator met with 8-15 raters per session
- Each rater independently rated ~ 10 leaflets

## Rater Characteristics

- Mean age = 61 yrs (20-89 yrs)
- 68% female, 89% white
- 77% used medication daily
- 8% not completed high school, 38%  completed high school, 54% beyond h.s.

## % Patient-Observers Given Any Written Information

- atenolol			89.6
- glyburide			88.8
- atorvastatin		89.3
- nitroglycerin		88.3

##  Expert Ratings, All Criteria (n=1,367 leaflets)

## Expert Ratings by Criterion

- Highest Ratings: 	#7: accuracy, unbiased
- Moderate Ratings:  	#1: names, indication	#3: directions

- Low Ratings:		#5: ADRs, what to do	#6: general information
- Lowest Ratings:		#2: contraindications	#4: precautions		#8: legibility/comprehens.

## Highest Expert Ratings - Criterion 7 Scientifically Accurate, Unbiased, Up-to-date 

**<1**

**<1**

## Moderate Expert Ratings - Criteria 1 & 3

## Low Expert Ratings - Criteria 5 & 6

## Lowest Expert Ratings - Criteria 2 & 4

## Lowest Expert Ratings - Criterion 8Legibility/Comprehensibility

##  Consumer Ratings, All Items (%)  

## Consumer Scores by Item 

- Scores varied by item
- Lowest scores: print size, print quality, spacing,  overall readability
- Moderate-high scores: easy to understand, useful
- 36% of leaflets given low ratings on readability

## Consumer Scores on Readability &  Understandability Items 

## Consumer Scores on Usefulness Item  

## Expert Versus Consumer Ratings

- Low correlation between total scores
- Expert rating of usefulness was related to consumer usefulness item
- Expert rating of usefulness was unrelated to consumer readability item

## Expert Rating of Usefulness by Consumer Rating of Usefulness Item (n=339 atenolol leaflets)

**Consumer Score ****	****	****Mean expert (%)**
- 1 (poor)			40.3
- 2				46.3
- 3				50.0
- 4				55.0
- 5 (good)			55.0 (p<.001)

## Conclusions -- Part I

- 89% given some information
- Ratings varied by criterion
- Experts most critical of information on contraindications and precautions and legibility/comprehensibility
- Consumers most critical of readability

## Results -- Part IIWhat factors might influence leaflet ratings? 

- Consumer characteristics?
- Pharmacy type?
- Leaflet characteristics?
- Changes made by pharmacy organization or pharmacy integrator?

## Leaflet Ratings by Consumer Characteristics 

- Ratings unrelated to gender, age, education, current drug use
- White raters gave higher scores than non-white raters for nitroglycerin leaflets
- Race unrelated to other ratings

## Leaflet Distribution and Ratings by Pharmacy Type

- P<.001 all tests

##  Expert Ratings by Leaflet Length 

- P<.001 all tests

## Consumer Ratings by Leaflet Length 

- P<.001 all tests

## Consumer Ratings by Font Size 

- P<.001 all tests

## Consumer Ratings of Readability  by Font Size 

- P<.001 all tests

## Data Vendors Examined  

- Vendor - not ascertainable	13.5% (46)
- Vendor 1 - partial message          5.0% (17)
- Vendor 1 - full message 	     	81.5% (277)
- Vendor 2* (comparison leaflets,institutional)

## Leaflet Length by Vendor

**Vendor****	****	****	****% leaflets <5.6 inches**
- Vendor - not ascertainable	82.6%
- Vendor 1- partial message	94.1%
- Vendor 1- full message		27.1%
- Vendor 2*			 	  0.0%

- P<.001

## Results By Data Vendor

- Substantial variability by data vendor
- Highest: Vendor 1 and Vendor 2
- Lowest:  Partial messages and unidentified vendors

##  Expert Ratings by Vendor

- P<.001 all tests

## Ratings of Atenolol Leaflets by Vendor 

- .

- P<.001 all tests

## Ratings of Atenolol Leaflets by Vendor 

- P<.001 all tests except Criterion 7 (p>.05)

## Consumer Ratings by Vendor 

- P<.001 all tests

## Consumer Ratings of Atenolol Leaflets by Vendor 

- P<.001 all tests

## Data Vendor by Pharmacy Type

- P<.001

## Analysis of Vendor 1 Leaflets

- Analyzed full leaflets used by 16 organizations
- 5 organizations used Version 1 (33 leaflets)
- 5 organizations used Version 2 (50 leaflets)
- 6 organizations used Version 3 (72 leaflets)
- Compared leaflets with prototypes

## Results 

- Ratings varied by leaflet version
- No prototype fully met criteria
- Some organizations added and deleted information from prototype

## Expert Ratings by Leaflet Version  

- P<.001 all tests

## Expert Ratings by Leaflet Version  

- P_<_.001 for all criteria

## Additions/Deletions from Prototypes 

**V1:** One organization deleted publisher and disclaimer information; two used leaflet+label
**V2:** One deleted additional information and added label
**V3: **Five deleted warning box and overdose sections; four deleted drug names and notes; five added disclaimer; 1 added publisher section

## Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 1

## Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 2

## Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 3

## Conclusions (1) 

- Highest ratings for scientific accuracy and being nonpromotional
- Lowest ratings for contraindications and precautions

## Conclusions (2) 

-  Lowest ratings for leaflets:				-  < 5.5 inches long or 				- font size < 10 point
- Lowest ratings for leaflets from: 			- independent pharmacies				- unidentified vendors

## Conclusions (3)

- No prototype fully met criteria
- Experts, consumers both critical of legibility
- Pharmacy organizations can influence ratings by
- selecting vendor and leaflet version
- modifying leaflets themselves

## Examples

- Leaflets with low versus high ratings
- Leaflets from different vendors
- Changes made by pharmacy organizations or integrators