The objectives of this report are to review the assessment of patient-focused outcomes in pediatric orthopedic surgery, to describe a framework for identifying appropriate sets of measures, and to illustrate an application of the framework to a challenging orthopedic problem.A detailed framework of study design and measurement factors is described. The factors are important for selecting appropriate instruments to measure health status and health-related quality of life (HRQL) in a particular context. A study to evaluate treatment alternatives for patients with neurofibromatosis type 1 and congenital tibial dysplasia (NF1-CTD) provides a rich illustration of the application of the framework. The application involves great variability in the instrument selection factors. Furthermore, these patients and their supportive caregivers face numerous complex health challenges with long-term implications for HRQL.Detailed summaries of important generic preference-based multi-attribute measurement systems, pediatric health profile instruments, and pediatric orthopedic-specific instruments are presented. Age-appropriate generic and specific measures are identified for study of NF1-CTD patients. Selected measures include the Activities Scale for Children, Gillette Functional Assessment Questionnaire Walking Scale, Health Utilities Index, and Pediatric Inventory of Quality of Life.Reliable and valid measures for application to pediatric orthopedics are available. There are important differences among measures. The selected measures complement each other. The framework in this report provides a guide for selecting appropriate measures. Application of appropriate sets of measures will enhance the ability to describe the morbidity of pediatric orthopedic patients and to assess the effectiveness of alternative clinical interventions. The framework for measurement of health status and HRQL from a patient perspective has relevance to many other areas of orthopedic practice. Assessments of patient-focused health status and health-related quality of life (HRQL) are being recognized increasingly by clinicians, patient advocates, regulatory authorities, administrators and policy makers as primary measures of the need, efficacy, effectiveness and efficiency associated with health care services. These types of measures are commonly reported in the published literature of many health care disciplines although there are few published reports concerning orthopedic surgery. However, the orthopedic community is becoming interested in using evidence reported by patients and patients' family members to determine what is best for patients.Patient-focused evidence  refers to results from functional health status (FHS) and HRQL studies of orthopedic outcomes with measurements obtained from the perspective of patients or their non-clinician caregivers . The field of orthopedics is concerned with all musculo-skeletal problems from the top to the bottom of the human body. This broad anatomical range is associated with a wide range of functional problems. Some orthopedic problems are highly focused in regards to anatomy and associated with highly effective therapies. However, other orthopedic problems are of a diverse nature, the etiology is often poorly understood, and effective treatment strategies are sometimes elusive.The breadth and depth of some particularly challenging orthopedic problems has stimulated an interest in patient-based perspectives of their own FHS and HRQL. These types of measures will be referred to as "health measures" for the purposes of this paper. Health measures can be used to assess the burden of morbidity associated with orthopedic diagnoses, and to assess the effectiveness and efficiency of therapies. The diversity of issues also led the American Academy of Orthopedic Surgeons and the Pediatric Orthopedic Society of North America to develop the Pediatric Outcomes Data Collection Instrument (PODCI), and led other researchers to develop function-specific instruments such as the Activities Scale for Kids (ASK). However, there are a large variety of potentially appropriate health measures and there are limits to the number of these measures that can be used in any given study. The most appropriate set of health measures should be selected on the basis of underlying study objectives and design criteria .a priori specification of the comprehensive measure of health and a credible assessment viewpoint for the purposes of primary analysis of study results. Secondary study objectives may involve use of data collected from other health measures or from other assessment perspectives.Objectives and designs vary greatly across orthopedic studies. In general, orthopedic procedures and programs are undertaken to improve the health of patients. The health of patients can be measured in different ways. Conventional clinical or physiological measures are generally very useful for diagnostic and therapeutic purposes. Conventional clinical measures may, however, provide an incomplete assessment of health. For example, clinical examination and gait measures have been used to describe the quality and quantity of limitations in walking ability associated with hip flexion contracture but these types of assessments provide little information about the importance of this contracture relative to that of other serious health problems. In addition, various measures may provide conflicting results: some indicating improvement while others suggest decline in patients' health. Therefore, in the interest of scientific rigor, protocols to assess the overall effectiveness of therapy should include FHS and HRQL measures are important for a variety of reasons that complement conventional clinical measures . FHS mea"Health-related quality of life is the value assigned to duration of life as modified by the impairments, functional states, perceptions, and social opportunities that are influenced by disease, injury, treatment or policy" .It is generally accepted that a goal of therapy is to make patients feel better  and thisA series of studies from a randomized controlled clinical trial (RCT) and prospective cohort study of elective total hip arthroplasty are illustrative of these purposes. These examples, while not from a pediatric setting, are orthopedic and demonstrate how data can be used for a variety of purposes. In the RCT, Rorabeck and colleagues  undertooThere are many factors to consider when designing studies using patient-focused health measures for orthopedic patients. Among the most important factors are the study objectives, types of patients, the dimensions and FHS constructs associated with the health problem under study, the appropriateness of the questionnaire for the age range of the patients, the viewpoint of the assessors answering the questionnaire, the method of collecting questionnaire information (self-completion or interviewer-administration), the period of time respondents are asked to consider when answering the questionnaire , and measurement properties of assessment tools. Measurement properties include content and construct validity, test-retest and inter-rater reliability, responsiveness, and practical limitations of data collection. It takes many years, and often decades, for credible and relevant evidence to be accumulated about the measurement properties of instruments. Instruments with well-established measurement properties should be used whenever possible. Two prime issues for assessments of children are the developmental stage of the subjects and the most appropriate respondent for questionnaires.The methods section of the paper presents a framework of important factors for consideration in designing studies to assess FHS and HRQL for orthopedic patients. The results were generated by applying the framework in the context of developing a proposal for a comprehensive international study of patients with neurofibromatosis type 1 (NF1) and congenital tibial dysplasia (CTD). The NF1-CTD protocol provides a rich illustration because it encompasses great variability in the instrument selection factors, and these patients with their supportive caregivers face numerous complex health status challenges with long-term implications for HRQL. Furthermore, there is little information in the published literature about the comprehensive health status and HRQL of patients with NF1-CTD. The rationale and approach to studying FHS and HRQL from a patient perspective have relevance to many other areas of orthopedics.This section identifies specific study design and measurement instrument factors that should be considered in selecting measures for use in studies, and presents some background information for illustrating the application of these factors.Study design factors are specified in detailed protocols that clearly define the study objectives and conditions. Well-defined study objectives identify the study subjects, and type and number of measurements required. Study conditions describe practical issues including data collection sites, and budgets for time and resources.A highly focused, single objective study may require relatively few instruments while a broad-based, multi-faceted study will require numerous measures. For example, a RCT of efficacy for an experimental technique to achieve union of bone compared to a conventional technique might specify patients' reports from a single well-established walking ability scale as the primary outcome measure. However, an economic evaluation of the cost-effectiveness and cost-utility of the experimental technique would require at least two instruments: the single well-established walking ability scale to estimate effectiveness for the numerator in the cost-effectiveness ratio; and a utility-based scale of overall HRQL to estimate quality-adjusted life years for the numerator in the cost-utility ratio.The age of study subjects affects the choice of measurement instruments and the method of collecting data. Each instrument is valid for a specific age range and the age range is quite small for many pediatric instruments. For example, developmental changes during childhood make it difficult for single instruments to be valid from infancy through adolescence. Many instruments are valid for one or more of the following age categories: infants, pre-school children, primary-school children, adolescents, young / middle-age adults, seniors and elderly. Inability to read well , or see well , or concentrate well  inhibits use of self-complete questions. Well-designed interviewer-administered questionnaires pose clear, short, well-focused questions with readily understood and easily remembered response options .It is becoming widely accepted in many disciplines that studies of health care programs and technologies should include measures of patients' perceptions of their FHS and HRQL . MeasureThe locations and cultural characteristics of the study population will determine the language and sometimes the choice of assessors. Relatively few instruments have been carefully translated and culturally-adapted to facilitate use in a large variety of communities. Some cultures accept a variety of assessment viewpoints  while other cultures recognize only a physician viewpoint for health assessments.The literature suggests that there may be important effects due to mode of data collection  and, theFHS questionnaires should have well-specified assessment periods to help ensure that the subjects and researchers know the period of time covered by the responses. Assessment recall period refers to the period of time the assessor is asked to consider when answering the questionnaire. The period should match the objectives of the study. For instance, if a new surgical technique is thought to reduce the peri-operative burden of morbidity then frequent assessments with a brief recall period (24 hours) might be suitable. Standard assessment periods include the past 24 hours, the past 1 week, the past 2 weeks and the past 4 weeks. Short assessment periods should be used in studies of patients whose FHS varies over time and in studies involving serial assessments to ensure that there is no overlap in assessment times. Relatively long recall assessment durations may be used when it can be assumed patients' health status is fairly stable. For example, a four-week recall assessment period was used to measure the HRQL of patients in a randomized clinical trial and economic evaluation of two alternative treatment strategies for patients with knee osteoarthritis ,16.There are limits to the number and type of measures that respondents can be expected to complete without getting tired or frustrated, and/or that the study budget can afford in regards to both data collection and analysis. The evidence about the limits of respondent burden is sparse. All else being equal, studies involving serial assessments should expect to collect fewer measurements per assessment than single-assessment cross-sectional surveys. To maximize efficiency, instruments should be selected to provide complementary rather than overlapping information.There are numerous definitions of FHS and HRQL. For the purposes of the paper, a FHS measure is defined as being descriptive in terms of functional ability and HRQL is defined as involving some form of valuation of that health status.One published taxonomy  suggestsEach multi-attribute system includes a descriptive classification scheme to describe and assess health status, and a preference-based valuation system. HRQL scores for health states defined by multi-attribute systems are calculated from models fitted from directly measured preference measurements (see below). Multi-attribute systems provide descriptive information about comprehensive health status, and interval-scale preference scores of overall HRQL from a community perspective on a scale where 0.00 is the score for being dead and 1.00 is the score for being in perfect health. Several multi-attribute systems define negative scores of overall HRQL to represent preferences for states considered worse than being dead. A few systems include single-attribute preference-based scales of morbidity. Single-attribute morbidity scales are defined such that the least desirable level within an attribute  has a score of 0.00 (blind) and the most desirable level has a score of 1.00. The community perspective is most widely recommended for technology assessment and reference case economic evaluation analyses -29. InteA valid measure is "sound and sufficient" . There aA measure is reliable if it is sound and dependable . ReliabiResponsiveness is also referred to as sensitivity to change. It is an important feature for determining a measure's ability to detect effects of treatments or natural changes over time . Husted and colleagues reviewed the literature and defined two major types of responsiveness: internal and external responsiveness . InternaCeiling and floor effects are undesirable properties that reduce the validity, reliability, and responsiveness of measures. A ceiling effect may occur when a large proportion of measurement observations are close to the upper bound of the measurement scale. A ceiling effect results in a positively skewed distribution of measurements, limited ability of the measure to discriminate among subjects at the upper end of the scale, and attenuated responsiveness to improvements in health in longitudinal studies. A floor effect may occur when a large proportion of measurement observations are close to the lower bound of the measurement scale. Floor effects create a negatively skewed distribution of measurements, limited ability of the measure to discriminate among subjects at the lower end of the scale, and decreased responsiveness to decrements in health in longitudinal studies. Many generic and specific measures of HRQL may be subject to ceiling effect problems in that they may not be able to describe patients or subjects with above average  health. Some measures are subject to floor effect problems. Some are subject to both. Typically floor effect problems are more serious in clinical studies (which often involve patients with disabilities) and ceiling effect problems may be more problematic in general population studies.The limits of respondent burden depend upon many factors including the number of questions presented, how the questions are presented, the complexity of questions, the sophistication of respondents, and the respondents' interest in the questions. In general, the allowable length of questionnaire is shorter for mail and phone administration than for face-to-face interviewer administration . One setApplications of FHS and HRQL measures are greatly facilitated by expert advice, detailed instructions and other services designed to support users of a measure. Supporting documentation is usually protected by copyright and should not be used without written permission of the original developers. Documentation obtained from third-party sources should be considered suspect because it is frequently invalid. Licensing fees are used to fund high quality, readily accessible service centers. Permission to use copyright materials is typically granted one study at a time. Support services may also include consultation about the most appropriate versions of questionnaires for use in a specific study. Application packages may include data collection instruments such as questionnaires, procedure manuals, coding algorithms and scoring systems, as well as background information about the conceptual and measurement properties of the instrument.Recently there has been interest in using measures of FHS and HRQL to evaluate treatment alternatives for NF1-CTD. NF1 is one of the most common genetic disorders in childhood . It is eCTD is rare in the general population, approximately 1 per 140,000 . It has Conventional clinical measures of CTD include the Crawford classification system . These mSurveys of the published literature, experts in the fields, web sites and other sources of information were conducted to determine the dimensions of health that are affected by NF1-CTD, the types of FHS and HRQL measures that have been used, which measures should be considered as potentially useful for studies of NF1-CTD, the measurement properties of potentially useful measures, and the relative merits of various measures. A review of the on-line Quality of Life Instruments Database (QOLID) developed by Dr. Marcello Tamburini and the MAPI Research Institute , and corThere are five important research objectives of an NF1-CTD study that provide a context for applying the framework described in the Methods section:1) to document long-term health outcomes associated with the disease and its treatment;2) to measure the burden of disease and treatment during active therapy;3) to investigate the hypothesis that improved HRQL is associated with initial amputation compared with multiple limb-saving procedures;4) to determine relationships of FHS and HRQL with conventional clinical variables used in diagnosis and management; and5) to assess the measurement properties  of selected FHS and HRQL measures in NF1-CTD patients.These detailed objectives require the identification and assessment of leading FHS and HRQL measures for use in both cross-sectional and prospective longitudinal surveys.The prevalence of NF1-CTD is relatively low. Patients will need to be recruited from numerous clinical centers in North America to generate precise estimates of FHS and HRQL. Questionnaires should be available in at least 3 major languages: English, French and Spanish. The survey population ranges in age from newborn into adulthood and linking results across the study objectives requires that at least some of the assessment tools be in common across the age range of study patients. To avoid potential confounding effects, data collection techniques should be consistent across subjects and measures.The patient-focus will be represented by collecting measurements from all patients old enough to provide self-assessments, and from parents acting as proxy assessors for all children and adolescents. Self-complete questionnaires requiring minimal supervision should be used to eliminate the need for interviewers at each clinical center, to facilitate use of mail-out surveys, and to avoid potential "interviewer" effects. The number and type of measures per assessment, and the number of serial assessments per patient, should be sufficient to address all the study objectives within the limits of study resources and assessor burden. Measures of morbidity associated with NF1-CTD should be comparable with data on norms from surveys of general populations and other patient groups, and be useful for assessing the effectiveness and efficiency of health care services.The HRQL measure should be comprehensive and preference-based, to facilitate a broad variety of comparisons. A pediatric health profile measure and other specific measures will be selected to complement the selected preference-based HRQL measure. FHS measures may be focused on one or more of the following: the population of interest ; the major underlying disease ; the major human function of most interest ; the medically-defined health problem of most interest ; the medical speciality most involved with treatment of the health problem .There are six major generic preference-based HRQL utility systems , presentThe major population-specific health profiles include the Child Health Questionnaire (CHQ), Pediatric Inventory of Quality of Life (PedsQL), Pediatric Evaluation of Disability Inventory (PEI) and TNO-AZL Pre-School Children Quality of Life questionnaire (TAPQOL). The PEI is limited to children age 0.5 – 7 years of age and requires a structured parent interview or clinician observation . TAPQOL The major pediatric disease-/function-/specialty-specific instruments include the PODCI , ASK, Gillette Functional Assessment Questionnaire Walking Scale , and Wee-FIM. Wee-FIM , a populA summary of the major pediatric generic health profiles appears in Tables Table The choice of existing measures is based on a process of elimination considering the relative strengths of each instrument and the complementarities among measures. Neither the SF-36 nor the EQ-5D is valid for use in adolescent patients with orthopedic problems . A revieIn summary, there are few measures available for assessing subjects less than 5 years of age and even fewer for subjects less than 2 years of age. Most relevant measures are available in self-complete format only. Preliminary recommendations for the NF1-CTD study were that PedsQL be used as the generic health profile, HUI be the multi-attribute preference-based measure of HRQL utility scores for children age 5 years and older and that CHSCS-PS be the measure for children age 2 through 4 years, ASK be the measure of activity limitation, FAQ walking scale be the measure of walking ability, and that a small feasibility study of these instruments be completed with a convenience sample.A pilot feasibility study surveyed 8 NF1 patients using HUI and FAQ walking scale measures. Questionnaires were completed by 6 NF1 patients and 3 parents. The combined HUI and FAQ walking scale questions took respondents an average of 13 minutes  to complete. The patients were 11 to 50+ years old and had health problems ranging from mild to severe. One patient with tibial dysplasia and 2 patients with scoliosis were included.HUI data were collected from 5 patients, 2 parents and both the patient and parents in one case. Health problems were reported in 7 of the 8 HUI3 attributes . The attributes associated with the most morbidity, as assessed using HUI3 single-attribute utility scores , were paFAQ walking scale data were collected from 4 patients (1 of the 5 survey patients did not answer the question), 2 parents and both the patient and parents in one case. Five patients were reported to be at Level 10 , one patient to be at Level 8 , and one patient at Level 6 .In summary, the feasibility study showed that the HUI and FAQ walking scale questions were acceptable to patients' families and that results, especially for HUI, reflected the large variability in HRQL of the sample of patients.No single measure will provide sufficient data to address all the important study objectives. A set of measures is required. The set of measures should provide complementary data of health status and preference-based scores of HRQL. Redundancy in measurement is reduced, and efficiency of measurement is increased, by selecting the most comprehensive generic measures and then supplementing these measures with the most appropriate set of specific measures.It is recommended that HUI be selected as the comprehensive generic measure for ten reasons:a) it includes both generic health profile and preference-based scoring systems;b) the preference-based scoring systems are well-validated;c) it is the most comprehensive, compact and efficient of these types of systems;d) it includes many of the most important domains in the context of NF1-CTD;e) it is applicable for all people age 5 years and older;f) well-developed data collection questionnaires are available to match the study design criteria;g) HUI results facilitate integrating effects of morbidity and mortality, and cost-utility economic evaluations;h) it has been used successfully in a variety of studies of musculoskeletal problems;i) population norm data are available; andj) a closely-related health status system, the CHSCS-PS, is available to assess children 2 through 4 years of age.The HUI will provide a broad set of measures for comparisons with other populations and for estimating HRQL on a general scale such that dead = 0.00 and perfect health = 1.00. As a generic measure, HUI also has the ability to capture side effects and the effects of co-morbidities. However, these broad measures may not be responsive to small but important changes in health status. Therefore, HUI should be complemented by a set of instruments focused on pediatric, orthopedic and walking issues.The PedsQL 4.0, a pediatric generic health profile, should also be included in the set of measures because:a) it includes domains, social and school function, which complement HUI and CHSCS-PS domains;b) it is appropriate for children ages 2 through 18 years;c) it is not overly burdensome in terms of data collection;d) patient and parent assessment questionnaires are available; ande) it can be interviewer-administered to facilitate data collection by telephone, if necessary.Two specific measures should also be part of the set of instrumentation: the ASK and the FAQ walking scale. ASK is an orthopedic-specific instrument which has been shown to cover the most important domains in the context of musculoskeletal disorders, including the impact of limb lengthening surgery experienced by many children with tibial dysplasia. ASK is also attractive because it provides overall summary scores for both performance and capability measures, and is only moderately burdensome to complete. Walking ability is one of the most important aspects of health that is frequently compromised in NF1-CTD patients, and the FAQ walking scale is the most complete scale of functional walking ability currently available and it only requires asking one question.CHQ is not recommended because it does not add much to the set of recommended measures and it is burdensome to complete. PODCI is not recommended because it is very burdensome to complete, it has been reported to have major problems with "missing data", and the system for collapsing questionnaire responses into summary scores is not well validated.Children ages 2 to 5 years of age should be assessed by their parents using three questionnaires: the CHSCS-PS (12 questions), the PedsQL (23 questions); and the FAQ walking scale (1 question). It is expected that all three of these questionnaires can be completed in an average of 15 minutes.Children and adolescents ages 5 to 17 years of age should be assessed by their parents using four questionnaires: the HUI (15 questions), the PedsQL (23 questions), the FAQ walking scale (1 question), and the ASK (30 questions). These four questionnaires are expected to be completed in an average of 20 to 30 minutes.Children and adolescents older than 11 years should provide self-assessments using four questionnaires: the HUI (15 questions), the PedsQL (23 questions), the FAQ walking scale (1 question), and the ASK (30 questions). On average, it is expected that respondents will complete all four questionnaires in 20 to 30 minutes.This paper highlights reasons why patient-focused measures of FHS and HRQL should be considered important tools in the field of orthopedic surgery. It has also noted that there is increasing competition for scarce health-care resources, that allocation decisions about these resources are being informed by evidence based on patient-focused health measures, and that these measures are being under-utilized by the orthopedic surgery community.The orthopedic community faces numerous obstacles in utilizing FHS and HRQL measures. One major obstacle is that the multitude of existing measures makes it difficult to decide which measures may be appropriate for a specific application. A second obstacle is that most of the information about FHS and HRQL measures is not reported in the orthopedic literature. A third obstacle is that usually no one measure can capture all the important aspects associated with a specific orthopedic issue. The framework outlined in the paper provides guidance for selecting appropriate FHS and HRQL measures. The framework guides orthopedic investigators to combine their basic study criteria, including objectives and clinical context, with key criteria for FHS and HRQL measures from the published literature.The results in this paper identify some major sources of information about health measures, identify some of the most widely used measures of FHS and HRQL, and provide summaries of key characteristics for selected measures in three major taxonomical classes: generic preference-based multi-attribute systems; generic pediatric health profile systems; and orthopedic-specific systems. It is clear that there are many important differences among measures both within and across taxonomical classes. All measures are not equal. There are sound factors for making judgements about which measures are most appropriate for a given application. A process of appraisal and elimination was used to select one measure from each taxonomical class for inclusion in the NF1-CTD study illustrative example, and a pilot study of the most readily available selected measures confirmed the feasibility of their use in a small sample of NF1-CTD patients.The paper shows that a set of relevant, valid, reliable, responsive and practical patient-focused health measures for use in an orthopedic study can be readily identified and selected from the published literature and information available on the worldwide web. We encourage orthopedic researchers to use the framework to identify and select appropriate patient-focused health measures in their future studies.®) instrumentation and provides methodological advice on the use of HUI.W. Furlong and D. Feeny have a proprietary interest in Health Utilities Inc. which distributes copyright Health Utilities Index (HUIAll authors were involved in critical review of drafts for intellectual content and have given final approval to the version submitted for publication. W Furlong was responsible for much of the overall design, acquisition of data, and initial drafting. R Barr and D Feeny made important contributions to the analysis and interpretation of results. S Yandow conceived the idea of presenting the information in a published manuscript.


Since the mid eighties, responsiveness is considered to be a separate property of health status questionnaires distinct from reliability and validity. The aim of the study was to assess the strength of the relationship between internal consistency reliability, referring to an instrument's sensitivity to differences in health status among subjects at one point in time, and responsiveness referring to sensitivity to health status changes over time.We used three different datasets comprising the scores of patients on the Barthel, the SIP and the GO-QoL instruments at two points in time. The internal consistency was reduced stepwise by removing the item that contributed most to a scale's reliability. We calculated the responsiveness expressed by the Standardized Response Mean (SRM) on each set of remaining items. The strength of the relationship between the thus obtained internal consistency coefficients and SRMs was quantified by Spearman rank correlation coefficients.Strong to perfect correlations (0.90 – 1.00) was found between internal consistency coefficients and SRMs for all instruments indicating, that the two can be used interchangeably.The results contradict the conviction that responsiveness is a separate psychometric property. The internal consistency coefficient adequately reflects an instrument's potential sensitivity to changes over time. Responsiveness, a concept introduced in the mid-eighties by bio-medical researchers, is considered to be an essential measurement property of health status questionnaires, distinct from reliability and validity . HoweverWe used three datasets comprising the scores of patients on three widely used health status instruments on two moments in time in order to assess health changes. The first dataset was from a randomized clinical trial investigating the effects of arm and leg rehabilitation training on the functional recovery of stroke survivors using the 10-item Barthel (basic) activities of daily living scale . PatientOnly subjects with no missing values at baseline or follow-up were included in the analysis. The Barthel dataset had no missing values, the SIP datasets had 13 % (16/120), 28% (64/227) and 0.7% (1/141) missing values respectively and the GO-QOL had 0.6% (1/164) missing values. For the SIP datasets, there was a mean deterioration in health (1 point), for the Barthel and GO-QOL scales patients were improved at follow-up  reflecting sensitivity to differences in health status among patients , and theFigures Our results contradict the conviction that responsiveness is a separate psychometric property of health scales. Internal consistency reliability, reflecting a scale's sensitivity to cross-sectional differences in health, closely coincided with the instruments' sensitivity to change as measured with the standardized response mean. Our results also reflect what is already known within the framework of classical test theory. A test score cannot correlate more highly with any other variable than its own true score . This imWe used the SRM effect size that uses the standard deviation the change scores and therefore includes all information about the changes on the selected instruments. The results can not generalized to alternative effect sizes such as Cohen's effect size or Guyatt's responsiveness statistic  because In a frequently cited paper, Guyatt et al.  made theDuring the past 20 years, clinimetric research has resulted in about 25 definitions and 30 measures of instrument responsiveness, sometimes referred to as sensitivity to change or longitudinal validity . MoreoveRobert Lindeboom conceived the idea for the manuscript, Mirjam Sprangers and Robert Lindeboom wrote the manuscript, Koos Zwinderman provided the mathematics and rewrote parts of the manuscript in earlier draftsAppendix 1: Is it possible to have one high reliable scale with low 'responsiveness', and a low reliable scale with high 'responsiveness' measuring the same construct?Click here for file


To investigate the interchangeability of measures of disability and health-related quality of life (HRQL) by comparing their associations patterns with disease-related impairment measures in patients with a variety of conditions.z effect size (ES(z)). Weighted averages were reported as pooled ES(z) with 95% confidence intervals (CI).A systematic literature search of MEDLINE, EMBASE, Web of Science and a hand search of reference lists through January 2006. Studies were included if they reported associations patterns between impairment and disability and between impairment and HRQL. Correlation coefficients were transformed to Fisher's z) = 0.69; 95% CI, 0.66 – 0.72) than between impairment and HRQL (pooled ES(z) = 0.38; 95% CI, 0.36 – 0.41). The physical component score (pooled ES(z) = 0.43; 95% CI, 0.39 – 0.47) and disease-specific HRQL (pooled ES(z) = 0.46; 95% CI, 0.40 – 0.51) were stronger associated with impairments than the mental component score (pooled ES(z) = 0.28; 95% CI, 0.20 – 0.36) and generic HRQL (pooled ES(z) = 0.36; 95% CI, 0.33 – 0.39).The relationship between impairment and disability was stronger (pooled ES(This study shows measures of disability and different HRQL domains were not equally related to impairment. Patient's impairments are better reflected in disability measures, than in HRQL instruments. There are many outcomes of interest and precisely defining them and measuring them will improve assessing the impact of new interventions. Choosing an outcome measure for use in clinical research is a complex process. If the outcomes are chosen inappropriately, a study may provide unreliable results . FrequenBesides biological measures other levels of clinical measurement can be considered in clinical studies: impairments, disability and health-related quality of life (HRQL). Impairments are the direct organic manifestations of the disease such as consciousness and paresis ,3. DisabRecently, patient-relevant outcomes in terms of disability and HRQL have become more important . AlthougImpairment or Body function (Publication date from 2001) or Body structure (Publication date from 2001) and Disability  or Activity or Disabled Persons or Activities of Daily Living and Health-related quality of life combined with Association or Evaluation Studies or Comparative Study or Validation Studies was used. Studies from English, German and French literature were included. The electronic search was supplemented by hand searching the investigators files, and retrieval of references cited in available literature. The search strategy was composed by one of the authors (NW) in consultation with a clinical librarian.We conducted a systematic literature search of MEDLINE from 1966 through January 2006, EMBASE from 1980 through January 2006 and Web of Science from 1988 through January 2006 to identify studies addressing the relation between outcome measures. A search strategy using Medical Subject Heading, text words and Publication Types A set of explicit criteria, composed by three investigators , were used for selection of the literature. Articles were included if they focused on methodological or metric aspects of patient-based outcomes .Studies were included in which both the association between impairment and disability, and between impairment and HRQL were calculated by means of correlation coefficients or other association measures, regardless of the disease with the exception of psychiatric disorders. Hence, to improve the comparability between the correlation patterns between the different health concepts, only studies were included in which the three types of health outcomes were assessed in the same patient population. The measurement instruments were questionnaires or observation lists. When a multi-scale questionnaire focuses on more than one concept we only included the sub-scale for the health domain in question. In case of double publication we selected the first publication in time. Excluded were studies which valuated HRQL in terms of utilities  or compoInternational Classification of Impairments, Disabilities, and Handicap version ; r = 0.27; r2 = 7%) [The pooled ES(2 = 12%) ,37-42,452 = 18%) ,46. Impa2 = 17%) -40,45 thp > 0.50). The fail-safe calculation by Rosenthal's method retrieved for both analyses a large number of studies (n = 27077/n = 9755) needed to be added in order to change the results of the meta-analysis.Review of normal quantile plots showed no clear indications for publication bias in both the analyses for the relationship impairment and disability and for impairment and HRQL Figure  and 3. IIn this study we investigated the impact of impairments on patients' functional health in a spectrum of diseases. The results of the meta-analysis show that impairments explain 36% of the variance of the disability scores and 13% of the variance of HRQL scores. Impairment scales account for 17% of the physical dimension of HRQL, but can explain no more than 7% of the psychological aspects of HRQL .Due to the great variability in disease-related impairment scales and disability and HRQL instruments used in patient groups with a wide range of disorders, we expected heterogeneous data sets. Interestingly, after removal of one study which deviated from normality, homogeneity could be demonstrated in all analyses, except when we investigated the relationship between impairment and the mental component of HRQL. Possibly, this type of HRQL is less clearly defined compared to the other types of functional outcomes.Treatments, such as new drugs or new surgical or radiological interventions, are aimed at reducing mortality and morbidity. If these treatments reduce impairments, the beneficial effects on functional health will be far more better detectable in the patient's level of disability than in his or her level of HRQL. Moreover, if the outcome is assessed in terms of HRQL, treatment effects will be more visible in the physical component than in the mental components, and will be better reflected in the disease-specific measures than in the generic HRQL measures. Therefore, outcome measures at the level of functioning of patients should be selected carefully when clinical studies are designed. It clearly does matter on which level the functional outcome measurement is chosen.No doubt, HRQL scales represent an important measure from a patient's point of view, but have serious disadvantages in many clinical efficacy studies. The extent to which patients fulfill social roles and participate in their environment cannot be observed directly and depends not only on their functional ability, but also on personal traits, social circumstances and societal barriers. In particular, HRQL scores, which reflect the patient's perception of the consequences of disease, depend on numerous additional, usually psychosocial, factors other than the disease itself. This explains why the correlation between impairments and HRQL scores is weak, especially when emphasis is laid on subjective psychological scores of these scales. The subjective HRQL scores therefore are often beyond the influence of the physician. In the primary assessment of new drugs or new surgical procedures, there are many outcomes of interest and precisely defining them and measuring them will improve assessing the impact of new interventions. In contrast, disability  seem to be a more appropriate primary end point for assessing the functional health status of patients. The ability to perform basic and complex activities of daily life is, as we showed, not only better related to the disease process itself, but is also observable and predictive of dependence , and forA shortcoming of this meta-analysis might be that there are probably more studies than the 31 we found. Our search strategy was based on the ICIDH and ICF terms and few researchers distinguish between the terms impairment, disability and HRQL. If they distinguished between these terms correlations between impairment and disability or between impairment and HRQL were given rarely for all three domains together. Another point of discussion may be the concern for publication bias. However, the normal quantile plots, Spearman's rank correlation coefficient method and fail-safe calculation method do not give an indication for this type of bias. Finally and unfortunately, we were not able to perform subgroup analysis based on the methodological quality of the studies. This because the psychometric and clinimetric articles analyzed used a hotchpotch of research designs which made it impossible to formulate unequivocal methodological criteria in terms of, for example, appropriateness of the design chosen, sample size consideration, blind assessment of outcome parameters, statistical adjustment for confounders, or loss to follow-up .New drugs or new surgical or radiological treatments are developed from a pathophysiologic perspective and are primarily directed at reducing impairments. Traditionally, biological and impairment measures are preferred. Increasingly, it becomes evident that this type of outcome is not always relevant for patients ,51. Our The author(s) declare that they have no competing interests.NW, RdH and MV initiated the project. NW and RdH screened, extracted and analysed the data. All authors participated in the discussion of the results, in the writing of the paper, and read and approved the final manuscript.The pre-publication history for this paper can be accessed here:


Responsiveness, or sensitivity to clinical change, is an important consideration in selection of a health-related quality of life (HRQL) measure for trials or clinical applications. Many approaches can be used to assess responsiveness, which may affect the interpretation of study results. We compared the relative responsiveness of generic and heart failure specific HRQL instruments, as measured both by common psychometric indices and by external clinical criteria.We analyzed data collected at baseline and 6-weeks in 298 subjects with heart failure on the following HRQL measures: EQ-5D , Kansas City Cardiomyopathy Questionnaire (KCCQ) , and RAND12 . Three external indicators of clinical change were used to classify subjects as improved, deteriorated, or unchanged: 6-minute walk test, New York Heart Association (NYHA) class, and physician global rating of change. Four responsiveness statistics  were used to evaluate the responsiveness of the select measures. The median rank of each HRQL measure across responsiveness indices and clinical criteria was then determined.Average age of subjects was 60 years, 75 percent were male, and had moderate to severe heart failure symptoms. Overall, the KCCQ Summary Scores had the highest relative ranking, irrespective of the responsiveness index or external criterion used. Importantly, we observed that the relative ranking of responsiveness of the generic measures  was influenced by both the responsive indices and external criterion used.The disease specific KCCQ was the most responsive HRQL measure assessing change over a 6-week period, although generic measures provide information for which the KCCQ is not suitable. The responsiveness of generic HRQL measures may be affected by the index used, as well as the external criterion to classify patients who have clinically change or remained stable. In chronic medical conditions, small changes in clinical status must be readily identifiable to monitor patients' progress and to modify treatment strategies, if necessary . In hearHRQL measures have been commonly used in the clinical trial setting in the evaluation of new treatment strategies, and more recently are even used as the primary outcome assessment for clinical trials ,8. When true underlying change in the patients' health status over time [Several disease and generic measures of HRQL have been used in both clinical practice and in clinical trials of patients with heart failure. To accurately capture these changes in health status over time, the HRQL measure must have evidence of longitudinal validity or 'responsiveness' . Responsver time . Two appSeveral factors may therefore influence the responsiveness of a HRQL measure, including, but not limited to, the content of the measure , the validity of the measure, the error associated with the HRQL scores, the indices used to determine the responsiveness  and the external criterion or 'gold standard' used to identify subjects as changed or not changed. When considering evidence of responsiveness, it is important to consider the extent to which these factors affect reported estimates of responsiveness .To provide empirical evidence to support this issue, and to assist in the selection and interpretation of health status measures for heart failure, we analyzed longitudinal HRQL data in patients with heart failure to evaluate the relative responsiveness of selected disease-specific and generic HRQL measures. We explicitly compared their relative performance as measured by common responsiveness indices and by different external clinical criteria for change.Patients with heart failure were recruited through the Cardiovascular Outcomes Research Consortium across 14 medical center outpatient departments in the United States and Canada . All subPatients completed several HRQL questionnaires at baseline, including the RAND12, EQ-5D, and the Kansas City Cardiomyopathy Questionnaire (KCCQ).The RAND12 is a short, well-validated, generic measure of health status -12. The The EQ-5D is a 5-item self-administered utility measure . In addiThe KCCQ is a 23-item heart failure specific questionnaire. The KCCQ has domains on physical limitations, heart failure specific symptoms , quality of life, social impact of the disease, and patients' assessments of their disease knowledge or self-efficacy . The psyIn addition to the HRQL questionnaires, several clinical assessments were also completed by the residing cardiologist at baseline. Specifically, the patients baseline New York Heart Association (NYHA) classification was assessed and patients completed a six-minute walk (6 MW) test according to a standardized protocol . All patThere is no universally accepted gold standard for identification of clinical change in patients with heart failure. As a result, change in the patients' clinical status was assessed using several criteria. First, cardiologist's assessment of the patients' NYHA classification at baseline and at the 6-week follow-up was used. Subjects were classified as improving two NYHA classes , improving one NYHA class, no change in NYHA class, and deteriorating one NYHA class from the baseline to 6-week follow-up. No subjects deteriorated by two NYHA classes during the 6-week period.Second, the resident cardiologist, blinded to the subjects self-reported HRQL and the 6 MW test results, completed a previously validated global rating of change assessment from the baseline to the 6-week follow-up visit ,18. ThisFinally, the difference from baseline to 6-weeks in distanced traveled in the 6 MW test was recorded. This difference was categorized into 7 mutually exclusive categories of clinical change according to previous research : substanMean change scores in the HRQL measures were calculated by subtracting the baseline score from the 6-week follow-up data. Responsiveness indices, including T-statistic , effect size (ES) (mean change divided by the standard deviation of the baseline score), the Guyatt's responsiveness statistic (GRS) (mean change divided by the standard deviation of change in subjects who remained unchanged), and the standardized response mean (SRM) (mean change divided by the standard deviation of the change score) , were caTo facilitate comparison, the median rank of each HRQL measure was determined across each of the four responsiveness indices . In ordeA total of 476 subjects were enrolled in the study and provided baseline and 6-week follow-up . Of thesUpon follow-up (average 6 ± 2 weeks), 52 (17%) subjects were classified as improved according to the NYHA class Table , 60 20%, and 101Overall, the magnitude of change in the HRQL scores were larger for subjects who improved compared to subjects who deteriorated over the 6-week period, with the exception of the 6 MW data, which showed the opposite trend Table . ImportaSimilar to the raw HRQL change scores, the magnitude of the responsiveness indices were influenced by the direction of clinical change Tables , 7, 8. OIn general, the relative ranking of the HRQL measure within each responsiveness index was similar, regardless of the responsiveness indices used Tables , 7, 8. IAlthough small differences existed, the relative ranking of the generic HRQL measures across the responsiveness indices were similar and generally differed by only one rank position. For example, the EQ-5D–US Scoring system for subjects who improved +2 NYHA classes had a relative ranking of '3' using the T-statistic, ES and GRS and a rank of '4' using the SRM. Thus, the relative responsiveness of a disease-specific versus generic HRQL measure was not substantively influenced by the choice of responsiveness index. Differences did exist, however, in the relative ranking of generic HRQL measures according to the external clinical criterion of change used Tables , 2, 3. Ttrue underlying change can have a major influence on the perceived responsiveness of a HRQL measure [In this study, we found that the disease specific KCCQ measure was more responsive to underlying clinical change than either the generic EQ-5D or RAND12 measures. The greater responsiveness of the KCCQ was consistent across all four responsiveness indices used and across the three external clinical criteria of change. Importantly, we also found that the methods and definitions used to define  measure ,20, part measure .true underlying change in patients with heart failure. Clinicians are interested in determining if the difference in HRQL scores over time signifies a trivial, small but clinically important, moderate, or a large change in HRQL [It is important to understand and interpret how changes in the HRQL scores over time reflect  in HRQL ,21. ThisThe high responsiveness of the KCCQ may have been expected as specific measures, by design, typically have very strong content validity for a specific disease or population. It is generally accepted that when true change occurs in the setting of a clinical trial, disease-specific measures are more responsive to this change as compared to generic measures of HRQL . They arThe KCCQ, for example, specifically focuses on the impact of dyspnea, a prominent complaint for people with heart failure. In the RAND12, dyspnea could be captured, but only in much broader terms under the domain of physical functioning. Thus, disease-specific measures may be more sensitive and responsive to within-patient change as compared to generic measures . This chWhile our data would suggest that the use of a disease-specific measure, like the KCCQ, may provide the best opportunity to capture small but highly relevant clinical changes in heart failure patients, not all changes in HRQL may be captured with the use of a disease-specific measure, as often the overall effects of a new treatment may not be fully known. For example, we were intrigued by the relative performance of the physical and mental health summary scores of the RAND12. Although heart failure is often perceived largely as a physical disease, the RAND12 PCS Score performed very poorly compared to the other HRQL measures. The reason for the lower responsiveness of the RAND12 PCS is not known but may be related to the underlying health status of the study population. RAND12 PCS scores were, on average, 1.5 standard deviations below the standardized US population mean at baseline indicating that the subjects had significant physical deficits (Table Also of note was that the relative responsiveness of the HRQL measures depended on the direction of clinical change. Overall, the HRQL measures were more responsive to improved clinical status as compared to deteriorating clinical status. This may be related to the fact that heart failure patients in this study were quite severely affected by their disease and had substantial deficits in their HRQL. As a result, HRQL measures may be susceptible to 'floor effects' in this population, which may limit their ability to capture deterioration in clinical status. The observation that no patient deteriorated by 2 NYHA classes further supports this hypothesis. Thus, not only can the responsiveness indices and external criterion standards influence instrument responsiveness, but also it is important to consider the population studied.Irrespective of the scope of the HRQL measure or the direction of clinical change, the responsiveness of a particular measure may also be influenced by the responsiveness index used. Although the responsiveness indices used in this study provided similar rankings, differences did exist among the responsiveness indices when the generic HRQL measures were concerned. In this study, four commonly utilized responsiveness indices were used ,27. TerwAlthough most indices use the mean change in HRQL over time, there are significant differences in how the standard deviations or variability in the data is used in the calculation. For example, the GRS was calculated using the standard deviation of the change scores among subjects who are clinically stable, whereas the SRM uses the standard deviation of the change scores. It is therefore possible that significant differences could exist in the variability in the selected subgroups, resulting in differences in the perceived responsiveness of the HRQL measure depending upon the responsiveness index chosen.true clinical change in this population [Our results and interpretation should be considered in light of several potential limitations. First, as discussed, the ability to identify patients who truly changed during the follow-up period was subjective, as no 'gold standard' exists for patients with heart failure. We did apply, however, the NYHA classification system, physician global rating of change assessment, and 6 MW tests, which are well validated and common methods to identify change in clinical status in patients with heart failure ,17,18. Fpulation . Second,pulation .With these limitations in mind, we believe this study highlights the importance of considering the measurement properties and instrument content in selecting HRQL measures for clinical trials. It is important to have a measure capable of detecting small but highly relevant and important changes in HRQL, especially when the HRQL outcome of interest represents the primary outcome or main secondary outcome of the trial. Furthermore, in the design of a clinical trial, researchers must be confident that the HRQL measure is responsive to the minimal importance difference that was hypothesized during the study design. The sample size required and power of the study will be directly related to the minimal importance difference that the researchers wish to detect. However, while disease specific measure may be considered as the first choice for a primary HRQL outcome, a combination with a generic HRQL measure may still be desirable to fully assess HRQL outcomes in clinical trial settings. Further, if the therapy under consideration will be evaluated under a cost-effectiveness framework, it would be desirable to include a measure compatible with that framework, such as a utility-based measure .We found the disease specific measure, the KCCQ, was the most responsive HRQL measure assessing change over a 6-week period, although generic measures provide information for which the KCCQ is not suitable. We noted that the responsiveness of generic HRQL measure may be affected by the responsiveness index used, as well as the selection of the external criterion to identify patients who have clinically change or remained stable.Dr. Spertus owns the copyright to the KCCQ. While he assisted in the review of the manuscript and was responsible for the original study, he did not design these analyses nor dictate the interpretation of the results. Otherwise, the authors have no conflicts of interest, including specific financial interests, non-financial interests, relationships, and affiliations relevant to the subject matter or materials discussed in the manuscript.DTE and JAJ participated in the conception and design of the study, analysis, and interpretation of the data, drafting of the manuscript, and revising it critically for important intellectual content. KJL participated in the analysis and interpretation of the data and revising the manuscript for critically important intellectual content. JAS participated in the conception and design of the study, acquisition of data, interpretation of the data, and revising the manuscript for critically important intellectual content. All authors read and approved the final manuscript.


Whether responsiveness represents a measurement property of health-related quality of life (HRQL) instruments that is distinct from reliability and validity is an issue of debate. We addressed the claims of a recent study, which suggested that investigators could rely on internal consistency to reflect instrument responsiveness.516 patients with chronic obstructive pulmonary disease or knee injury participating in four longitudinal studies completed generic and disease-specific HRQL questionnaires before and after an intervention that impacted on HRQL. We used Pearson correlation coefficients and linear regression to assess the relationship between internal consistency reliability , instrument type (generic and disease-specific) and responsiveness .2 = 0.01) whereas the type of instrument was a strong predictor of the SRM . In multivariable models applied to individual studies Cronbach's alpha consistently failed to predict SRMs  whereas the type of instrument did predict SRMs .Mean Cronbach's alpha was 0.83 (SD 0.08) and mean SRM was 0.59 (SD 0.33). The correlation between Cronbach's alpha and SRMs was 0.10 (95% CI -0.12 to 0.32) across all studies. Cronbach's alpha alone did not explain variability in SRMs (p = 0.59, rInvestigators must look to data other than internal consistency reliability to select a responsive instrument for use as an outcome in clinical trials. Health-related quality of life (HRQL) instruments should demonstrate adequate test-retest reliability, cross-sectional and longitudinal validity before investigators use them to assess outcomes in research studies. Whether responsiveness, the ability of an instrument to detect change in HRQL when change occurs, is a measurement property distinct from reliability and validity remains, however, controversial -4.Lindeboom et al. purportedly tested the assumption that responsiveness is not a distinct measurement property, but is embodied in internal consistency reliability . To inveThe first conceptual problem with the approach Lindeboom et al. chose is that they looked at the correlation of internal consistency reliability and responsiveness within single studies and instruments only. However, this approach does not take into account that responsiveness depends on the type of an intervention while internal consistency reliability does not. Most HRQL measures may be very reliable, but internal consistency reliability has nothing to do with the therapy that is producing the change. In contrast, if an intervention targets aspects of HRQL that are specifically covered by a disease-specific instrument, for example, responsiveness is likely to be high. If the effect of another intervention targeting aspects other than those covered by the instrument, responsiveness will be lower. Thus the within study approach does not take into account that responsiveness is not a fixed measurement property.Another important issue to consider is the influence of other determinants of an instrument's responsiveness such as the type of instrument, generic or disease-specific. There is ample evidence that responsiveness depends on the type of instrument. -9 LindebFinally, if the within instrument approach with step-wise deconstruction of domains is used, one would expect step-wise decreases of internal consistency reliability, responsiveness and other measurement properties such as cross-sectional validity for the following reasons. Internal consistency reliability is reduced when the items contributing most to internal consistency reliability are removed because the error term in the denominator increases. For the same reason, responsiveness deteriorates if the number of items is decreased. Thus itHaving considered the methodological challenges and constraints above, we analysed the relationship between internal consistency reliability and responsiveness of entire domains across different instruments and studies using data from several of our previous studies.A priori we defined the following eligibility criteria to ensure an unbiased selection of datasets as possible and to ensure that it was theoretically possible to detect a correlation between internal consistency reliability and responsiveness if one existed. We applied the following criteria:1. Studies must have longitudinal follow-up with a baseline assessment and at least one follow-up assessment completed by the CLARITY research group  within the last five years.2. Studies must have investigated an intervention of established effectiveness that induces changes in HRQL.3. Studies must include ≥ 2 multi-item HRQL instruments that allow calculation of Cronbach's alpha and instruments within a study must have different degrees of responsiveness (e.g. generic versus disease-specific) to ensure variability in responsiveness. We expected variability in Cronbach's alpha to be limited to values ≥ 0.60 because only those are generally accepted to represent sufficient internal consistency reliability [We calculated Cronbach's alpha using baseline scores for each domain of each HRQL instrument or for the total instrument if domains did not exist. Similarly, for each domain or for a total score we calculated SRMs (change score divided by standard deviation of change score).We calculated the correlation between Cronbach's alpha and the corresponding SRM using Pearson correlation coefficients across all studies and for each study separately. We then built linear regression models with the SRM as the dependent variable and Cronbach's alpha as the independent variable. Since the type of instrument (generic or disease-specific) affects the SRM -9, we inThe following four studies met the eligibility criteria:This prospective study measured HRQL in 85 patients with chronic obstructive pulmonary disease (COPD) before and after participation in Canadian inpatient respiratory rehabilitation programs similar to many inpatient programs worldwide . All patThis was a prospective randomised study of 177 patients with COPD before and after respiratory rehabilitation in Canada and the United States. We randomised patients to complete either the interviewer or self-administered CRQ ,15. All This prospective study enrolled 71 patients with COPD following a respiratory rehabilitation program at four cites in Switzerland, Germany and Austria. We also randomised patients to complete either the interviewer or self-administered CRQ as in study 2 ,15 and aThis prospective study enrolled patients undergoing anterior crucial ligament reconstruction  and knee arthroscopy  to determine their ability to recall pre-operative quality of life and functional status. Patients completed the disease-specific Anterior Crucial Ligament Quality Of Life questionnaire (ACL-QOL)  . In contrast, an analysis that included the type of instrument showed that the generic versus specific categorisation predicted responsiveness . Analysing the studies separately showed similar results  and the consistency of results across these studies and populations enhances the generalizability of our study. Replication in other populations would further strengthen our conclusions.Our study demonstrates that internal consistency reliability is a poor predictor of responsiveness and that both conceptual and statistical evidence exists to support the argument that they are distinct measurement properties of evaluative instruments.MAP, DB, GHG and HJS designed the study and wrote the study protocol. MAP, GHG, HJS and DB collected the data. MAP, DB and DHA performed the statistical analysis. MAP drafted and DB, GHG, DHA and HJS critically revised the manuscript. All authors read and approved the final manuscript.


This article deals with the problem of interpreting health-related quality of life (HRQL) outcomes in clinical trials. First, we will briefly describe how dichotomization and item response theory can facilitate interpretation. Based on examples from the medical literature for the interpretation of HRQL scores we will show that dichotomies may help clinicians understand information provided by HRQL instruments in RCTs. They can choose thresholds to calculate proportions of patients benefiting based on absolute scores or change scores. For example, clinicians interpreting clinical trial results could consider the difference in the proportion of patients who achieve a mean score of 50 before and after an intervention on a scale from 1 to 100. For the change score approach, they could consider the proportion of patients who have changed by a score of 5 or more. Finally, they can calculate the proportion of patients benefiting and transform these numbers into a number needed to treat or natural frequencies. Second, we will describe in more detail an approach to the interpretation of HRQL scores based on the minimal important difference (MID) and proportions. The MID is the smallest difference in score in the outcome of interest that informed patients or informed proxies perceive as important, either beneficial or harmful, and that would lead the patient or clinician to consider a change in the management. Any change in management will depend on the downsides, including cost and inconvenience, associated with the intervention. Investigators can help with the interpretation of HRQL scores by determining the MID of an HRQL instrument and provide mean differences in relation to the MID. For instance, for an MID of 0.5 on a seven point scale investigators could provide the mean change on the instrument as well as the proportion of patients with scores greater than the MID. Thus, there are several steps investigators can take to facilitate this process to help bringing HRQL information closer to the bedside. This article deals with the problem of interpreting health-related quality of life (HRQL) outcomes from clinical trials. The alternative titles that we had in mind for this paper exemplify the problem: "The great merit of simple-minded dichotomies for clinicians" or even more provocative, "The great merit of dichotomies for simple-minded clinicians." These possible titles reflect the fact that clinicians have difficulties understanding quality-of-life measures because of the numerous available instruments and their diversity in items, response options, lack of familiar units, and approaches to aggregation. We propose that dichotomies can help with understanding HRQL instruments.In this article we will offer some possible strategies based on examples. First, we will briefly describe how dichotomization and Rasch analysis, a particular form of analysis according to item response theory (IRT), can facilitate interpretation ,2. We wiSecond, we will suggest an approach to the interpretation of HRQL scores based on the minimal important difference (MID) -6. The MTo highlight the problem, we describe an example of the failure to present HRQL in a transparent and understandable way to clinicians. In a trial including 553 patients with psoriasis, a group of investigators evaluated the impact on HRQL of alefacept, a fusion protein that inhibits T-cell activation and promotes apoptosis of CD2+ T cells that play a role in psoriasis, on HRQL . They usIn the authors' opinion, the data from the SF-36 survey confirmed that alefacept had no negative impact on general quality of life, because the SF-36 did not show important changes. The authors then reached the conclusion that alefacept improved quality of life in patients with chronic plaque psoriasis and maintained this benefit for at least 12 weeks following cessation of treatment. This presentation of HRQL information is quite typical for trials of this sort . Unfortunately, most clinicians will have difficulty understanding what an improvement of 4.4 vs 1.8 or 3.4 vs 1.4 means to their patients. Does it mean that patients do feel substantially better or have noticed an important change, or does it mean that the improvement was small, not noticeable for patients, and statistically significant only because of the large sample size? The answer to this question is not clear because the authors do not provide satisfactory guidance as to how important these score changes are. However, the example should clarify one of the fundamental problems of HRQL data: the failure to present the data in interpretable and transparent manner. In the following paragraphs we will present possible solutions to these problems.Possible solutions to the problem of clinicians' lack of familiarity with HRQL scores exist. To focus on what is called the content-based interpretation of results proposed by Ware and Keller, let us take a look at the distribution of the SF-36 scores of the Physical Function scale based on the Medical Outcomes Study, a large study in the U.S. . Figure Figure Another example of the use of content-based interpretation of HRQL measures is the use of the visual function 14 index (VF14) . This inFigure Thus, the important contribution of IRT to the interpretation of HRQL scores is that it provides clinicians and patients with guidance what they can expect from respondents based on the entire score on a multi-item instrument. However, the IRT approach is restricted to instruments with a clear gradient of the level of difficulty across the instruments' items. It cannot be applied to situations when the response options there is no clear gradient in the severity of limitation or impairment associated with the items. The VF-14 works well in this regard because each item is more challenging than all lower items. Being able to drive at night, for instance, requires much better visual acuity than recognizing people even when they are close. The IRT approach to interpretability would not work for instruments in which this clear ordering does not exist.Different methods to determine the MID exist. Anchor-based methods rely on examining the associations between scores on the instrument that is under investigation and an anchor, an independent measure of HRQL that clinicians can easily interpret ,12. For However, two questions come to mind. First, does a group mean change of 0.6 in response to a treatment on the 7-point scale for the CRQ, AQLQ, or CHQ mean that all patients benefit? Second, does a group mean change of 0.3 mean that no patient benefits? In the following sections we will provide examples that will help answer these questions.Let us examine the Rankin stroke scale, an instrument widely used to measure dysfunction in patients who experienced a stroke . The RanAnother example is that of the use of neurolytic coeliac-plexus block (NCPB) versus systemic analgesic therapy (SAT) alone in unresectable pancreatic cancer . The autWe will now examine examples of how clinicians can interpret change score differences. Figure Figure The vertical line shows the hypothetical MID for patients participating in this trial. A certain proportion of patients in the control group [labeled P(Imp|C)] show improvement greater than the MID. In the treatment group, a larger proportion [P(Imp|T)] of patients show improvement greater than the MID. The difference between these two proportions [P(Imp|T) - P(Imp|C)] is the proportion of patients who improved above the MID after accounting for placebo or control group effects.From this proportion one can easily calculate a risk differences or NNT. Norman et al showed a consistent relation between effect size and NNTs across a wide range of plausible specifications of the MID . Table 1Figure As we described above, the MID for the CRQ on each of the four HRQL domains is 0.5. This example shows results from a randomized controlled trial comparing intensive respiratory rehabilitation to conventional care in patients with moderate to severe chronic respiratory disease (COPD) . The linThese examples of the interpretation of HRQL scores demonstrate that dichotomies may help the understanding of information provided by HRQL instruments in RCTs. There are several steps investigators can take to facilitate this process. They can choose thresholds and dichotomize responses on HRQL based on absolute scores or change scores to facilitate interpretation. Second, they can determine the MID of an HRQL instrument and provide mean differences and relate these to the MID. Finally, they can use the dichotomized responses and the MID to calculate the proportion of patients benefiting and transform these numbers into NNTs and natural frequencies. Simple dichotomies may bring HRQL information closer to the bedside.HJS and GG are authors of the CRQ. McMaster University and a research account used by HJS and GG receive licensing fees from the use of the CRQ.HJS prepared the first draft of this manuscript, and GG and EAA revised it critically and made suggestions for improvements. GG had the idea for the presentation on which this manuscript is based, HS made suggestions for improving the presentation.


There is at present a lack of knowledge of time trends in health related quality of life (HRQL) in common patients with coronary artery disease (CAD) treated in ordinary care. The objective of this study is to assess and compare time trends of health related quality of life (HRQL) and chest pain in patients with coronary artery disease.253 consecutive CAD patients in Stockholm County, Sweden – 197 males/56 females; 60 ± 8 years – were followed during two years. Perceived chest pain symptoms and three global assessments of HRQL were assessed at baseline, after one and after two years. EuroQol-5 dimension (EQ-5D) with a predefined focus on function and symptoms; the broader tapping global estimates of HRQL; EuroQol VAS (EQ-VAS) and Cardiac Health Profile (CHP) were used. Chest pain was ranked according to Canadian Cardiovascular Society (CCS). Change in HRQL was analysed by a repeated measurements ANOVA and chest pain symptoms were analysed by Friedman non-parametric ANOVA.Perceived chest pain decreased during the two years (p < 0.00022); CCS 0: 41–51%; CCS 1: 19–15%; CCS 2: 31–27%; CCS 3: 5–4% and CCS 4: 4–2%. By contrast, HRQL did not change: EQ-5D: 0.76 (CI 0.73–0.79) -0.78 (CI 0.75–0.81), EQ-VAS: 0.68 (CI 0.66–0.71)-0.68 (CI 0.65–0.71) and CHP: 0.66 (CI 0.64–0.69) -0.66 (CI 0.64–0.69).HRQL did not increase despite a reduction in the severity of chest pain during two years. This implies that the major part of HRQL in these consecutive ordinary patients with CAD is unresponsive to change in chest pain symptoms. Disease specific and all-cause mortality in patients with coronary artery disease (CAD) have declined during the last decades . PatientWe identified a cohort of consecutive patients with CAD and assessed chest pain symptoms and perceived quality of life during two years. This was done in connection with a randomised study showing that participatory learning of secondary prevention for general practitioners decreased cholesterol levels in their patients with coronary artery disease .The Department of Medicine at Södertälje Hospital, located in the southernmost part of Stockholm County, Sweden, provides planned and emergency health care for acute and elective cardiac patients in a catchments area of approximately 95 000 habitants. All in- and outpatients with an age less or equal to 70 years who had visited the department during the preceding year, due to symptoms and signs of CAD (ICD-9 code 410–414) were identified (1995). The age limit was chosen due to lack of secondary preventive studies in older patients at time of recruitment. 429 patient records were investigated. 106 patients were excluded, the majority because they did not fulfil the criteria of CAD and a few because of other life threatening disease or because they lived outside the catchments area. 68 of the remaining 323 patients fulfilling the criteria for inclusion refused to participate.Criteria for a confirmed diagnosis of CAD in the patient record were:I: A diagnosis of angina pectoris, either by objective criteria based on coronary angiography, or a pathologic exercise- or stress-test, or a clinical assessment based on typical angina symptoms at exercise with or without ECG-evidence of possible or definite ischemia.II: A diagnosis of myocardial infarction based on either WHO-criteria  or on unHypertension was considered to be present if systolic blood pressure was > 140 mm Hg, if diastolic blood pressure was > 90 mm Hg or if antihypertensive medication was used. Diabetes mellitus was defined as fasting plasma glucose of ≥ 7.0 mmol/l, a self-reported history of diabetes mellitus or treatment for diabetes.The patients were examined and they answered questionnaires including symptoms of angina pectoris and HRQL assessments at inclusion, after one respectively after two years. Baseline characteristics are given in Table The HRQL-questionnaires were self-completed. The Swedish versions of the CHP- and EQ-instruments were used. One research nurse performed all interviews and physical examinations at the department. She also handled all case report forms. All participating patients gave written informed consent. The study complies with the Declaration of Helsinki and was approved by the ethics committee of Karolinska Institutet at Huddinge University Hospital.The patients ranked existence and degree of current angina symptoms according to the Canadian Cardiovascular Society classification (CCS) 0–4  at baselThe definition of HRQL in this study is that 'quality of life' in clinical medicine represents the functional effect of an illness and its consequent therapy upon a patient, as perceived by the patient .The Cardiac health profile questionnaire (CHP) is a disease-specific HRQL questionnaire specially designed for respondents with CAD. CHP assesses the HRQL in a broad perspective. The questionnaire includes questions about physical function/general health, emotional, social and cognitive function. The 16 items in the CHP questionnaire (English version) are shown in Table Multivariate explorative factor analytical methods were used to identify the underlying structure as described in a previous publication . Four inThe generic HRQL questionnaires EuroQol-5 Dimensions (EQ-5D) self-classifier and EuroQol-VAS (EQ-VAS) were used to test the robustness of the analysis of time trends of HRQL as measured by the CHP questionnaire. In the EQ-5D self classifier the respondents classified their own health status in five aspects – mobility, self-care, usual activities, pain/discomfort and anxiety/depression into one of three levels – no problems, moderate problems and severe problems ,15. EachIn the EQ-VAS method the respondent marks present HRQL on a 20 cm vertical measurement scale with labelled anchors 'dead' (0) and 'full health' (100). The individual numbers marked on the scale were divided by 100 in order to obtain EQ-VAS scores between 0 and 1. Both EQ5D methods are easy and rapid generic instruments, well validated and found to be reliable in different cultures and in different diseases including a new validation study in patients after myocardial infarction ,20.The main outcome variable in the present study is change in HRQL and chest pain (CCS) over time. HRQL was analysed by a repeated measurements ANOVA and chest pain symptoms were analysed by Friedman non-parametric ANOVA. The underlying structure of HRQL as assessed by CHP was explored by the factor analytic multivariate technique, principal component (PC) analysis. The 16 item variables were reduced into principal components, i.e. to avoid an influence of multicollinearities and to find orthogonal (independent) factors explaining the variance in the sample ,22. AnotThe characteristics of the study population at baseline are described in Table The prevalence and severity of chest pain symptoms – according to CCS grade – decreased during the two study years as shown in Table An analysis of co-variation between chest pain symptoms and HRQL showed significant correlations between CCS and all three global assessments of HRQL at all three time points, Table A similar analysis of the four independent sub-domains – principal components – showed a correlation between chest pain symptoms and the physical/general health domain at all three time points (p = 0.000000). By contrast, there was no correlation between cognitive function respectively social function domains and chest pain symptoms at any of the time points Table . Figure The prevalence and severity of chest pain symptoms decreased during two years of follow-up as assessed by CCS grade in this unselected cohort of patients with CAD. However, no change was found in global HRQL as assessed by three different HRQL instruments despite the symptom reduction. This indicates that an important part of HRQL in such patients is unresponsive to change in the existence or degree of chest pain. Several explanations to this could be discussed. One possible confounder could be a change in non-cardiac morbidity. However, a separate analysis excluding patients contracting co-morbidity during the study did not change the results. The reason for the unresponsiveness could be the multidimensional construct of HRQL. As shown in table We have in a previous paper found that perceived cognitive function explains the major part of HRQL in this cohort . HoweverA limitation of our study is that that we just assessed perceived function. However, the main aim was to evaluate time trends in illness – both in terms of HRQL and chest pain symptoms – and not primarily to address a comparison between objective function and subjective perception. Furthermore it is known that the correlation is weak between objectively measured cognitive function and perceived cognitive function .Strength of our study is that HRQL was assessed both by three different global instruments and by the independent sub-domains. The response rate was also high. This should increase the robustness of our analysis and conclusions. Another strength is that this large sample of patients were followed over a good [2-year] period. Furthermore, the cohort was unselected and included both patients with and without present physical symptoms. This should increase the ability to understand the comprehensiveness of different assessment instruments and to analyse a putative co variation of change in HRQL and other estimates over time.The comprehensiveness differs between the three HRQL instruments. EQ-VAS is an individually defined global estimate. The 16 items of the CHP questionnaire taps – as compared to other frequently used instruments – a broad perspective of aspects on HRQL. It includes e g questions concerning the ability to select relevant information, and to understand, retain, express and apply knowledge in specific contexts of life. In a previous analysis we showed that questions representing cognitive function explained 43% of the HRQL variation . EQ-5D iThere was a slight but insignificant improvement of HRQL as assessed by EQ-5D . However, the power analysis showed that we could exclude a more clinically relevant improvement in HRQL by 5% or more even in the symptom driven EQ-5D. EQ-5D also showed a better global HRQL by 0.1 (p = 0.0000) than CHP and EQ-VAS. An explanation might be the more narrow scope of EQ-5D tapping mainly physical and emotional aspects of HRQL.It is known that physicians tend to underestimate or fail to recognise functional disabilities that are reported by their patients, especially disabilities related to social activities . The phyA limitation and strength of the present study is of course the unselected sample in concert with the fact that the patients are included from a single geographic area. The study addresses an important issue about responsiveness of HRQL instruments. Our findings of a low sensitivity of the HRQL instruments to change in symptom burden is interesting and challenging both from a clinical and societal perspective. The results should stimulate future research on the complex relations between objectively measured disease and subjectively perceived illness for patients with CAD.The major part of assessed health related quality of life is unresponsive to a decrease in severity of chest pain symptoms in a consecutive cohort of ordinary patients with coronary artery disease. This indicates that a reduction of chest pain symptoms is not sufficient; to improve health related quality of life in patients with coronary artery disease. The dominating dimension of health related quality of life – cognitive function – does not differ between patients with or without chest pain symptoms. The low responsiveness of assessed health related quality of life outcome to a change in symptom burden is interesting and challenging both from a clinical and a public health perspective.The author(s) declare that they have no competing interests.PH and AK shared the conception and analysis of the study. AK was responsible for the statistical analysis, drafting and revising of the article. PH and AK shared the hypothesis and design. PH and AK are both guarantators.