Patient Viewpoint:A Website for Patient-Reported Outcomes Assessment

Claire Snyder, PhD

Albert Wu, MD, MPH

Johns Hopkins 

Schools of Medicine and Public Health

Presentation Objectives
        - To present the results from a literature review and qualitative research study supporting the development of a prototype website for patient-reported outcomes assessment, including:

aspects of patient reported health that are useful to assess

website features that patients and providers desire 

barriers that need to be addressed 
        - Prototype demonstration

Background (1)
        - 2004 NIH Roadmap RFA:  Meetings and Networks for Methodological Development in Interdisciplinary Research

“applications for meetings or networks to develop methodologies that will facilitate interdisciplinary health research among behavioral or social scientists and investigators in the biomedical, mathematical/computational, physical sciences, and/or engineering”

Background (2)
        - Creation of the Quality of Life in Clinical Practice Interest Group within the International Society for Quality of Life Research 
        - Goals of using quality of life in clinical practice: 

To improve clinician-patient communication

To facilitate individual patient management

To assist with medical decision-making

Research Proposal

Creation of a network of internationally known researchers and practitioners to develop a centrally accessible website for patient-reported outcomes assessment that links to the electronic medical record

Goals of the Grant
        - Determine what aspects of patient reported health would be most useful for evaluating the effectiveness and quality of outpatient oncology practice
        - Explore desirable features for a web-based system to administer standardized questionnaires to patients
        - Explore potential technical, cultural, ethical, and legal barriers to the use of such a system
        - Develop a prototype website for use by outpatient oncology providers to collect patient-reported data in the clinical setting

*

Research Network Members
        - Albert W. Wu, MD, MPH (Johns Hopkins University)
        - Neil Aaronson, PhD (Vrije Universiteit, the Netherlands)
        - Michael Brundage, MD, MSc (Queens University, Canada)
        - Carolyn Gotay, PhD (University of Hawaii)
        - James Hodge, JD (Johns Hopkins University)
        - Denise Hynes, RN, MPH, PhD (Veterans Administration)
        - Claire Snyder, MHS, PhD (Johns Hopkins University)
        - John Wasson, MD (Dartmouth University)
        - Susan Yount, PhD (Evanston Northwestern Healthcare/Northwestern Univ.)
        - Brad Zebrack, PhD, MSW, MPH (University of Southern California)

*

Ably Assisted By:
        - J.B. Jones, MBA, PhD(c)
        - S. Orion Courtin, MHS
        - Roxanne Jensen
        - Web Development Team

Methods
        - Literature Review
        - First Research Network Meeting (Oct ’05)
        - Develop Website Specifications
        - Second Research Network Meeting (Apr ’06)
        - Website Design
        - Qualitative Research
        - Third Research Network Meeting (Feb ’07)
        - Website Development
        - Usability Testing

Literature Review
        - Rationale for web-based PRO assessment
        - Existing electronic and web-based platforms for collecting PROs
        - Desirable features from clinician’s and patient’s viewpoint
        - Potential ethical and legal issues

Jones et al. Quality of Life Research 2007;16:1407-1417

Advantages of Web-Based PRO Assessment
        - Patient can complete the assessment anytime from anyplace with Internet access
        - Reminders and decision support can be built in
        - Scoring is automatic
        - Comparison scores immediately available
        - Data is easily accessible (privacy and security issues)
        - Summary reports can be generated at the patient level or practice level

Jones et al. Quality of Life Research 2007;16:1407-1417

Desirable Features
        - Efficiency and Ease of Use

Will the website make efficiency better or worse?

Will the website fit into clinical workflow?

Will it add to my workload?

Will patients have an easy time using it?
        - Results Presentation and Interpretation

What do the results mean?

What should the clinician/patient do about them?

Jones et al. Quality of Life Research 2007;16:1407-1417

Ethical and Legal Issues
        - Privacy Rule:

Protected health information can only be disclosed with written authorization unless certain criteria for exemption are met

Exemptions include public health research, minimal risk research that cannot be practicably done without a waiver
        - Security Rule:

Defines the technical, administrative, and physical safeguards that must be put in place to protect the confidentiality, integrity, and availability of protected health information

Jones et al. Quality of Life Research 2007;16:1407-1417

Patient Viewpoint
        - A web-based system with a database back end for management and storage of patient responses 

Developed using Microsoft ASP.Net and SQL database
        - Linked to the electronic medical record
        - Three main functions provided by the system:

Clinicians select questionnaires to administer to patients and how often 

Patients enter responses, submit comments, and view the results of their response

Clinicians have a text and graphical view of patients’ responses and scores over time 
        - Security

Database and programming code located on separate servers

Servers backed up regularly and backups stored at a remote location

Secure Sockets Layer secure the site as patients enter their responses and physicians view the results

Access to the system is controlled by requiring username and password

Qualitative Research
        - Interviews 

20 Cancer Patients

3 Oncologists

4 Research Nurses
        - Semi-structured in-depth interviews

Followed structure of the prototype website

Probed for opinions, possible improvements, and innovations

Findings-Patients (1)

Keep surveys short and simple“I always have questions for him about my well-being, but if it becomes too long or too complicated, I’ll give up.”

Response: Worked with physicians to choose a concise package of measures to lower respondent burden

Findings-Patients (2)

Present multiple questions per survey page, but avoid long scrolling

““I would [prefer multiple questions], as long as there’s not too much.”

Response: Will present questions with same response type together on a page

Findings-Patients (3)

Some patients are interested in their survey results, others just want to comply with the request of their care provider

“Somebody is going to find it interesting, but I’m not, you know personally, once I’ve done the survey and made my additional feedback, I’m basically done.”

Response: Provide simple feedback for patients who are interested

Findings-Patients (4)

Patients interested more in changes over time than survey values

“It will be more meaningful just to see the individual change. I want to be able to look back six months and say, where was I six months ago.”

Response: Will generate simple graphs showing change over time

Findings-Patients (5)

Patients suggest making direction of improvement consistent across types of measures

“I’d want them to run in the same direction, with high being bad and low being good. That’s like these tests they give us, and you need to be consistent.”

Response: Explore options for making results easy to interpret while keeping in mind issues with violating conventions

Findings-Patients (6)

Patients want direct communication, want their comments to inform the next visit, worried it will not be read

“You need to go to your doctor with questions in writing, and it seems that this would be a vehicle to get those questions there.”

“I am concerned where you say that you won’t read it until the next appointment,..., what am I chopped liver or something?”

Response: Will keep the free text response and integrate it into the clinician results

Findings-Patients (7)

Patients perceive the site’s purpose as valuable, concerned it will become burdensome

“It would be useful from their point of view, so I’d be willing to take five minutes to fill out the thing.”

Response: Will clearly state the purpose of the site, keep surveys short and continue to improve usability

Findings-Clinicians (1)

Clinicians need guidance for interpreting the meaning of values

“Of course I have no idea if this is a good score or a bad score.”

Response: Will present results on commonly understood dimensions for their patients on standard scales

Findings-Clinicians (2)

Clinicians are most interested in meaningful individual change over time

“The individual patient values would be more useful, I want to know about this individual patient, some patients will always score low, this doesn’t represent change over time, you can’t always make a miserable person better, is there something that I can fix?”

Response: Focusing results presentation on individual changes over time, exploring ways to determine what constitutes a significant change for each measure

Findings-Clinicians (3)

Need ways to emphasize the key findings to focus on during the visit

“We’re all taught to fix the one thing that the patient needs today, is there one parameter that is tipping the scale, or are they generally having trouble.” 

Response: Added a question after each survey asking what a patient would most like their care provider to address at their next visit

Findings-Clinicians (4)

The website could be useful in clinical practice, but it needs to be easy to integrate into current routines with little administrative burden

“I would want someone to print the information out to be reviewed with the chart, or linked directly in the EPR. I need something to hit me over the head to indicate that there’s something I need to look at.”

Response: Adding a batch print feature to print results for patients based on scheduling

Response: Working with Electronic Patient Record personnel to integrate with their system

Findings-Clinicians (5)

Initially we envisioned clinicians choosing from a menu of surveys for each patient, but clinicians were concerned about administrative burden and lack of familiarity with different surveys

Response: We conducted research to narrow the choice of measures, then met with physicians to choose the ones most appropriate to their practice

Summary of Qualitative Study Findings
        - Respondents optimistic about potential value but concerned about practicality and integration into current practice

Use short surveys and a straightforward interface for patients

Integrate results into the current clinical routine through the Electronic Patient Record or print for the paper chart
        - In results presentation, both patients and clinicians were most interested in individual changes over time

Demonstration of Patient Website
        - Patient visits site and logs in

Demonstration of Patient Website
        - Patient receives welcome screen

Demonstration of Patient Website
        - Patient is presented with disclaimer

Demonstration of Patient Website
        - Patient is presented with survey(s) to complete

Demonstration of Patient Website
        - Patient survey – sample question

Demonstration of Patient Website
        - Patient survey – sample question

Demonstration of Patient Website
        - Patient survey – sample question

Demonstration of Patient Website
        - Patient gets a last chance to answer skipped questions

Demonstration of Patient Website
        - Patient completes survey and can ask their Doctor a question or view results

Demonstration of Patient Website
        - Patient views results

Demonstration of Clinician Website
        - Clinician logs in and can search for patient

Demonstration of Clinician Website
        - Clinician can search for patient by survey

Demonstration of Clinician Website
        - Clinician can view patients results

Demonstration of Clinician Website
        - Clinician can search for another clinician

Demonstration of Clinician Website
        - Clinician can add or edit a patient

Demonstration of Clinician Website
        - Clinician can schedule a patient a survey

Demonstration of Clinician Website
        - Clinician can add/edit another care team clinician

Next Steps
        - Usability testing is ongoing
        - R21 submitted to conduct initial pilot test

Use

Usefulness

Acceptability

Questions?