The accuracy of patient‐reported measures for GI symptoms: a comparison of real time and retrospective reports

Self-report is one of the most common, inexpensive, and efficient methods for collecting patient information in both clinical and research settings. Clinically, patients’ reports of their symptoms inform clinical decision-making particularly for functional gastrointestinal (GI) disorders (e.g., IBS) whose symptoms lack a reliable biomarker that gauges illness severity. The day to day burden of many common GI disorders is oftentimes best explained by subjective variables (e.g., pain, fatigue, quality of life) that can only be assessed through self-report or concepts that are not readily observed (e.g., stool consistency or frequency). In research settings, patient reported outcomes are increasingly acknowledged as the primary source of data for documenting therapeutic benefit (improvement in symptoms or functional capacity) that are detectable by the patient. To this end, the FDA (2) issued its Guidance on Patient Reported Outcomes or PROs in 2009. The Guidance provides a road map for developing and testing endpoints that focus on concepts that are most important and meaningful to patients. While the FDAs Guidance underscores the importance of subjective data, it acknowledges problems inherent to self-report. When patients are asked to recall their symptoms over a discrete time period, such as the last week, their responses are subject to memory failure and biases that arise from mental shortcuts (heuristics) used to make judgments quickly and efficiently about a stream of symptom experiences on varying dimensions (e.g., duration, intensity, severity, etc). For example, rather than mentally keeping an exact running total of their experience, people may rely on how they felt most recently and at their most extreme to summarize symptom levels (3–5). Recall heuristics streamline information processing of complex mental tasks but come at the cost of potential reduced accuracy such that what symptoms patients experience in real time can differ from their memory of the symptom experience they reconstruct and convey to a physician or researcher. In spite of the potential problems inherent in a source of data that are critical to clinical and research activities, efforts to document the accuracy of retrospective recall of GI symptoms are limited (6, 7). No known study has validated recall of GI symptoms that have occurred over the past week, which is the time frame most commonly used in research and clinical care (1, 8, 9). In the present study, we estimated the accuracy of recalled GI symptom data over 7 days relative to data collected in real time using electronic diaries during each of those 7 days. The daily diary data capture the symptom experience close to the time of experience, thereby limiting distortion from forgetting and recall bias, and yielding reasonably accurate data. We used electronic diaries because of known data quality and compliance difficulties with paper diary methods. Studies have shown, for example, that patients do not reliably complete paper diaries in real time but instead frequently rely on “back filling” and even “forward filling” of the diary. In one study, researchers documented that respondents’ compliance with paper diaries was as little as 11%. -- in other words, 89% of paper diaries are not completed as prescribed (10). The electronic diary methods used in the present study addressed such problems by time stamping when responses were entered and preventing entry of responses after a specified time period. The electronic diary data were aggregated across seven days and served as a gold standard against which to compare end-of-period self-reported recall of symptoms over the seven day time frame. We validated the accuracy of GI symptom recall at both the (aggregate) group level as well as the individual level. Group level accuracy refers to whether a group mean level of symptom severity based on recall accurately reflects the true mean level of symptom severity for that group of individuals as reflected by a “gold standard.” It is of interest, for example, when evaluating the overall effects of a treatment on a population of individuals enrolled in a clinical trial. Individual level accuracy refers to whether the level of symptom severity recalled by an individual accurately reflects the true level of symptom severity experienced by that individual. It is of interest in clinical settings when a patient reports symptoms to a physician and in research that seeks to identify correlates of symptom severity. Although not commonly recognized, recall can be accurate at the group level but not at the individual level. For example, if some individuals overestimate symptom severity relative to their true symptoms and other individuals underestimate it, when aggregated, the overestimations cancel the underestimations and the group level (mean) recall will show good agreement despite (potentially substantial) error at the individual level. The unique perspectives that this information reveals about how persons -- both as members of a comparison group of a clinical trial and as individual patients treated by a gastroenterologist in an office setting -- remember GI symptoms has important implications for clinicians and researchers in carrying out their respective tasks.