Outcome of paediatric intensive care survivors

The development of paediatric intensive care has contributed to the improved survival of critically ill children. Physical and psychological sequelae and consequences for quality of life (QoL) in survivors might be significant, as has been determined in adult intensive care unit (ICU) survivors. Awareness of sequelae due to the original illness and its treatment may result in changes in treatment and support during and after the acute phase. To determine the current knowledge on physical and psychological sequelae and the quality of life in survivors of paediatric intensive care, we undertook a computerised comprehensive search of online databases for studies reporting sequelae in survivors of paediatric intensive care. Studies reporting sequelae in paediatric survivors of cardiothoracic surgery and trauma were excluded, as were studies reporting only mortality. All other studies reporting aspects of physical and psychological sequelae were analysed. Twenty-seven studies consisting of 3,444 survivors met the selection criteria. Distinct physical and psychological sequelae in patients have been determined and seemed to interfere with quality of life. Psychological sequelae in parents seem to be common. Small numbers, methodological limitations and quantitative and qualitative heterogeneity hamper the interpretation of data. We conclude that paediatric intensive care survivors and their parents have physical and psychological sequelae affecting quality of life. Further well-designed prospective studies evaluating sequelae of the original illness and its treatment are warranted.
Introduction
The development of paediatric intensive care has contributed to improved survival rates in children with critical illnesses [68, 69]. Consequently, new disease patterns have emerged due to long-term complications and effects of the original illness and its treatment. In addition to survival and morbidity, physical and psychological sequelae, as well as the quality of life (QoL) in survivors and in their families are important outcome measures.
Historically, outcome research in paediatrics is either based on an age-specific approach, such as follow-up studies of premature infants [41, 72, 73], or on a more disease-oriented approach, such as follow-up studies in survivors of cardiothoracic surgery or trauma [15, 55, 64, 70]. These studies have shown substantial physical, psychological and neuro-cognitive sequelae, interfering with daily life and normal development. In addition, effects on parents and siblings have been shown [26]. Evaluative research of adult intensive care survivors showed the effect of intensive care treatment per se. Irrespective of the underlying illnesses, sequelae on all domains with effects on QoL were found [2, 19, 58, 75]. In multi-disciplinary paediatric intensive care unit (PICU) populations, reports on outcome are scarce [24, 25].
Based on these observations, we believe that follow-up research of paediatric intensive care survivors and their families is needed to evaluate: (1) physical sequelae and their impact during growth and development; (2) psychological sequelae in patients and their families and their impact on the QoL of patients and family members; and (3) the need for treatment and support after discharge.
The aim of this article is to provide an overview of the available literature concerning the different domains of QoL (i.e. physical, psychological and social functioning) in children surviving paediatric intensive care, including the effect on parents, and to suggest directions for future follow-up research.
Methods
To identify studies eligible for this review, we searched Medline (1966–2006), EMBASE (1974–2006), CINAHL (1982–2006), pre-CINAHL and the Cochrane Library (2006) in March 2006. In the search strategy, all terms mapped to the appropriate MeSH/EMTREE subject headings and “exploded” were used; among them were: paediatric intensive care unit (PICU), septic shock, respiratory insufficiency, meningococcal disease, central venous catheterisation, intubation, physical and psychological sequelae, post-traumatic stress disorder (PTSD), QoL, health status and long-term outcome.
Definitions
Functional health is defined as an individual’s ability to perform normal daily activities, to fulfil usual roles and to maintain health and well-being.
QoL is defined as an individual’s perception of their position in life, in the context of the culture and value systems and in relation to their goals, expectations, standards and concerns [1].
Health-related QoL (HRQoL) is defined as QoL in which a dimension of personal judgement over one’s health and disease is added [21].
Study selection
Studies were selected for review if they met two inclusion criteria: (1) study of a representative population of PICU survivors (defined as a population consisting of medical and/or surgical PICU patients <18 years old) and (2) evaluation of physical sequelae, measurement of QoL or functional health >30 days after PICU discharge. Because of the limited number of studies, the measurement tools did not need to be standardised. Studies with a retrospective and prospective design were included.
Excluded were: (1) studies in homogeneous PICU populations (e.g. survivors of cardiothoracic surgery and trauma) reporting diagnosis-related outcome in particular but not intensive care treatment as such, and (2) studies evaluating mortality only.
Results
Eligible studies and quality of the studies
Twenty-seven studies were found in which one or more aspects of long-term sequelae in PICU survivors and/or their families were described. The patient characteristics, populations, measurement tools and outcomes are described in Tables 1 and 2. The quality criteria are described in Table 3. None of the studies met all of the quality criteria. In studies describing the same outcome aspect, differences in study population, follow-up time and measurement tools make the comparison and synthesis of results difficult.
Table 1Patient characteristics, measurement tools, physical and functional health outcomeReferencePopulationa, nbAgec (yrs)Follow-up timed (yrs)Severity of illnesseLOSf (days)Measurement toolOutcome (n)gInterpretation of outcome23Meningococcal disease0.1–15.38–12GMSPSNANeurological examination1 spastic quadriplegiaMajority of children surviving meningococcal disease neurologically normal.n = 115 (139)Median 5Cognitive tests5 hearing loss4 major impairments35Cardiopulmonary resuscitation (CPR)0–161NANAPCPCh26 normal, 12 mild, 7 moderate, 3 severe disability60% of survivors of CPR neurological normal.n = 44 (48)37 same as prior to CPRLocation, underlying cause and duration of CPR determinants of outcome.43CPR0–171NANAPCPCh54 normal or mild disability, 6 moderate, 5 severe disability80% of survivors of CPR neurological normal.n = 65 (94)POPCiLocation, underlying cause and duration of CPR not determinants of outcome.53Acquired brain injury<3>0.5GCS<9NAGOSjGOS: 23 good recovery, 7 moderate, 5 severe disabilityMajority of children with acquired brain injury dead or disabled.n = 38 (53)BSID-IIkBSID-II: 8 normal, 12 cerebral palsy, 11 cognitive delayNeuro-developmental examination59CPR0–181NAPICUPCPCh5 normal, 1 mild, 4 moderate, 1 severe disability, 2 persistent vegetative state38% of survivors of CPR neurological normal.n = 13Median 5POPCiUnderlying cause and duration of CPR determinants of outcome.6ARDS2–135.6±4.3NANAChest radiography1 SaO2 94% during exerciseARDS survivors at risk for hypoxaemia during exercise.n = 7 (15)Pulmonary function1 reduced diffusion capacity14Meningococcal disease with ARDS0.3–3.70.5–2.1PRISM 12-53%NAPulmonary function1 wheezing for which salbutamolARDS survivors possibly at risk for mild obstructive lung disease.n = 122 obstructive disease22ARDS4.6–15.90.9–4.2NANAPulmonary function3 restrictive or obstructive diseaseARDS survivors at risk for restrictive and obstructive lung disease.n = 9 (12)Electrocardiography EchocardiographyCardiac function normal in all30ARDS5–144.4NANAPulmonary function1 limitations in activityARDS survivors at risk for restrictive and obstructive lung disease.n = 54 restrictive disease48Meningococcal septic shock1.6–15.42.2–4.9NANAPulmonary function18 normal lung functionMeningococcal septic shock survivors at risk for hypoxaemia during exercise.n = 186 SaO2≤95% during exercise74ARDS0.5–160.3–5.5PRISM 18±14%NAChest radiography7 (11) restrictive or obstructive diseaseARDS survivors at risk for restrictive and obstructive lung disease.n = 14 (20)Pulse oximetry4 (7) decreased diffusion capacityPulmonary functionElectrocardiographyEchocardiography62Meningococcal sepsis with renal replacement therapy0.5–152.7–7.1NAPICUGlomerular filtration rate (GFR)2 decreased GFR, proteinuria, hypertensionChildren surviving acute renal failure due to septic shock at risk for long-term renal dysfunction.n = 12 (15)Median 12Serum creatinine1 parenchymal defectProtein excretion in urine1 proteinuriaDMSA scan9Near drowning admitted to PICU0.7–14≥0.5PRISM 71%NAFunctional health:8 severe neurological impairmentMajority of near-drowning survivors lead a normal life.n = 25 (27)GCS≤5Contact primary physician or examination by study facility17 normal or mildly impaired12PICU0–182.5–3NAPICUFunctional health:7% moderate or severe handicapMajority of PICU survivors seem to lead a normal life.n = 775Mean 2.8Written questionnaire or telephone contact with the specialist physician or parents12% mild handicap91% will lead independent life44Bacterial meningitis with respiratory insufficiency0–120.6–6.4PRISMNAFunctional health:12 normalHalf of children surviving severe bacterial meningitis seem to lead a normal life.n = 21 (22)Mean 22%Telephone interview1 independentRange 1–47%2 partially dependent6 dependentaStudied population. bn = studied patients (eligible patients). cAge of patients at admission to PICU (mean±SD or range). dFollow-up time (mean±SD or range). eSeverity of illness (PRISM, Glasgow Coma Score (GCS), Glasgow Meningococcal Septicaemia Prognostic Score (GMSPS) and/or parent rating). fLength of stay (LOS) in PICU and/or hospital (mean±SD or range). gn= examined patients (eligible patients)hPCPC=Paediatric Cerebral Performance Category. iPOPC=Paediatric Overall Performance Category. jGOS=Glasgow Outcome Scale. kBSID-II=Bayley Scales of Infant Development-IITable 2Patient characteristics, measurement tools and psychological and quality of life (QoL) outcomeReferencePopulationa, nbAgec (yrs)Follow-up timed (yrs)Severity of illnesseLOSf (days)Measurement toolOutcome (n)gInterpretation of outcome40PICU children and mothers2.1–15.90.3–1PRISM 0.4–76%PICU 1–30Psychological outcome:Children: Behaviour high 3 (8), SDQ high 3 (21), IES PTSD 3 (29)PICU survivors and their mothers at risk for psychological distress and PTSD.n = 29 (33)Child: Behaviour Check List, SDQh,IESiMothers: GHQ high 11 (26), IES high 13 (27)Mother: GHQj, IESi50PICU5–18Median 0.6Parent rating 10Hospital 4–14Psychological outcome:Children: PTSD 4 (19) PICU, 0 (27) ward IES high 4 (21) PICU, 2 (17) wardPICU survivors and their parents at risk for psychological distress and PTSD.n = 35 (46)Child: PTSDk, SDQh, IESi, depression, anxiety, CSIlParents: PTSD 9(33) PICU, 2 (29) wardGeneral wardParent: GHQj, IESi depressionn = 33 (41)Children and parents51PICU11.3±3.20.5PRISM 25±23%Hospital 13.0Psychological outcome:IES and CMFS dependent on invasive procedures, CMFS and CHLOC on ageStress symptoms in children possibly dependent on number of invasive procedures.n = 60 (69)Children: IESi, CMFSm, CHLOCnGeneral wardn = 60 (69)52PICUMean0.5PRISMNAPsychological outcome:IES higher in high risk, not decreasing over time, IES related with invasive proceduresStress symptoms possibly dependent on invasive procedures. Stress symptoms not decreasing over time.n = 60Low risk 11.5Low risk<34%Children: IESi, CMFSm, CHLOCnHigh risk 11.1High risk≥34%61Meningococcal disease PICU and wardMedian 6.80.3GMSPS 6.9±3.3PICU LOS 0–62Psychological outcome:Child: PTSD 4 (26)PICU survivors and their parents at risk for psychological distress and PTSD.Children and parentsParent rating median 7Hospital LOS 2–87Child: SDQh, IESiMothers: PTSD 22 (58)n = 78 (118)Parent: GHQj, IESiFathers: PTSD 8 (43)4PICU parents25% <1, 25% 1–4, 25% 5–11, 25% >110.2–0.9PRISM 0–26%PICU 1–200Psychological outcome:ASD 87Parents of PICU survivors with ASD more at risk for PTSD.n = 272 (291)Parent rating 1–9Parents: Acute Stress Disease symptomsPTSD 33PTSDk symptoms8PICU1.2±1.3<0.5PRISM 12±7%PICU 10.5±11.5Psychological outcome mothers:Mothers PICU more stress. Stress decreases over time in all groupsMothers of PICU survivors at risk for psychological distress; families at risk for dysfunctioning.n = 31Parent rating 8.3±1.9Parental Stress Scale SCL-90-RoAll families dysfunctioningGeneral wardFAM IIIpn = 32FILEqERn = 32 Mothers20Meningococcal Disease Parents1–180.25–7NANAPsychological outcome parents:High psychological distress in mothers and fathers, not decreasing over timeMothers and fathers of PICU survivors at risk for psychological distress.102 mothers, 90 fathersGHQj27PICU4.61PRISMNAQoL: MAHSCr106 equal to before PICU50% of PICU survivors seem to have the same QoL as before admission; 10% normal QoL.n = 226 (241)n = 223 < 5%58 improvedn = 19 > 16%62 deteriorated26 normal38PICU5.7±3.61PRISMPICU 5.7±5.5QoL: MAHSCr52 improved50% of PICU survivors seem to have good QoL.n = 138 (150)n = 79 < 5%29 deterioratedn = 4 > 16%65 normal after PICU47PICUMedian 2.30.3–2PRISMNAQoL: RAHCs measure of function256 normal QoL60% of PICU survivors seem to have normal QoL.n = 432 (906)Mean 5.5%140 fair QoL9 poor QoL65PICU0–29.32.3–6PRISMPICU 0–57.4QoL: HSUIt GOSuHSUI (727): 608 normal, 29 (very) poor QoL70% of PICU survivors seem to have good QoL.n = 868 (1265)n = 554 < 5%GOS (727): 515 normal, 137 mild disability, 75 moderate/severe disability60% seem to have normal functional health.n = 137 > 16%aStudied population. bn = studied patients (eligible patients). cAge of patients at admission to PICU (mean±SD or range). dFollow-up time (mean±SD or range). eSeverity of illness (PRISM, Glasgow Coma Score, GMSPS and/or parent rating). fLength of stay (LOS) in PICU and/or hospital (mean±SD or range). gn = examined patients (eligible patients).hSDQ=Strength and Difficulties Questionnaire. iIES=Impact of Event Scale. jGHQ=General Health Questionnaire. kPTSD=Post-traumatic stress disorder. lCSI=Child Somatization Inventory. mCMFS=Child Medical Fears Scale. nCHLOC=Child Health Locus Control Scale.oSCL-90-R=Symptom Checklist-90 Revised. pFAM III=Family Assessment Measure III. qFILE=Family Inventory of Life Events and Change. rMAHSC=Multi-attribute health status classification. sRAHC=Royal Alexandra Hospital for Children. tHSUI=Health State Utility index. uGOS=Glasgow Outcome ScoreTable 3Quality assessment of reviewed studiesReferenceSelection bias excludedaSelective loss to follow-up excludedbExposure clearly definedcOutcome clearly defineddControl group includede4yesyesyesyesno6nonoyesyesno8nonoyesyesyes9noyesyesnono12yesyesyesyesno14yesyesyesyesno20yesyesyesyesno22nononoyesno23nonoyesyesyes27yesnoyesyesno30nononoyesno35yesnoyesyesno38nonoyesyesno40noyesyesyesno43nonoyesyesno44yesyesyesyesno47yesnoyesyesno48nonoyesyesno50yesnoyesyesyes51noyesyesyesyes52noyesyesyesyes53yesyesyesyesno59yesnoyesyesno61noyesyesyesno62yesnoyesyesno65nonoyesyesno74yesyesyesyesnoaSelection bias excluded (i.e. exclusion of >10% of the studied population excluded). bSelective loss to follow-up excluded (i.e. description of patients lost to follow-up and comparison with those remaining in the study). cExposure clearly defined (i.e. clear definition of the studied population). dOutcome clearly defined (i.e. clear definition of outcome measures). eComparison with control group (i.e. children admitted to general ward)
Physical and neuro-cognitive sequelae (Table 1)
In 12 studies that included in total 340 patients, aspects of physical and neuro-cognitive sequelae were evaluated.
Neurological evaluation was conducted in five studies including 275 survivors. The majority of the children were neurologically normal. In the remaining children, disabilities such as hearing loss, coordination, cognition and developmental problems turned out to be severe [23, 35, 43, 53, 59].
Pulmonary evaluation was conducted in six studies including 65 patients [6, 14, 22, 30, 48, 74]. Restrictive and obstructive disease and hypoxaemia during exercise was found.
Cardiac evaluation was conducted in two studies including 23 survivors [22, 74]. No abnormalities were found, except for left ventricular hypertrophy in one child.
Renal evaluation was conducted in one study including 12 survivors [62]. In two children, glomerular filtration was impaired, one had hypertension and one had proteinuria.
Psychological sequelae (Table 2)
Various questionnaires were used. Cut-off points for the diagnosis of PTSD differed between studies but all of them showed high scores for PTSD in children and parents.
Psychological evaluation of children was conducted in five studies including 202 children [40, 50–52, 61]. Symptoms of PTSD were found in 11 of 74 evaluated children. In one study, a relation was found between invasive procedures and high scores [52].
Psychological evaluation of parents was conducted in six studies including parents of 547 children [4, 8, 20, 40, 50, 61]. Symptoms of PTSD were found in 72 of 295 evaluated parents. In some studies, a relation was found between high scores and illness severity as perceived by parents [4, 50, 61]. In one study, these high scores decreased over time [8].
Functional health and QoL (Tables 1 and 2)
Evaluation of functional health was conducted in three studies including 821 children [9, 12, 44]. The majority of the children seemed to have normal functional health; the remainder was found to be seriously impaired.
Evaluation of QoL was conducted in four studies including 1,664 children [27, 38, 47, 65]. QoL was evaluated using three different questionnaires. In the majority of children, the QoL was normal or equal to the QoL before PICU admission. In all studies, some of the children had poor QoL.
Discussion
Only 27 studies consisting of 3,444 PICU survivors met our inclusion criteria. The small numbers, heterogeneity of the studied populations and the used measurement tools, the frequent use of non-validated measurement tools and the various aspects of outcomes studied make aggregation of the data and, therefore, strong conclusive statements difficult.
Physical sequelae
The reviewed studies report distinct physical sequelae, including neurological abnormalities in PICU survivors. Standardised neurological examination of PICU survivors was validated in 1994 but very few studies have been carried out since [24, 25]. As neurological problems have a great impact on daily life, standardised evaluation and adequate support and rehabilitation seem to be relevant, similar to in NICU survivors [11, 46, 56].
Follow-up studies evaluating lung function in children are hampered by the small incidence of severe respiratory insufficiency in children [49]. In adult respiratory distress syndrome (ARDS), the recovery of lung function is shown during the first year and physical limitations seem to be partly dependent on lung function [34, 58]. In infants and children, post-natal lung growth may contribute to the improvement of lung function after critical illness. In addition to lung function, the long-term effect of small airway disease should be evaluated, for instance, in children with respiratory syncitial virus infection.
Data on the structured evaluation of cardiac and renal function in paediatric and adult ICU survivors is not available. In young children, septic shock and the need for vasoactive support of the circulation may interact with the developing myocardium and may have persistent effects on cardiac growth and function [10, 67, 77].
Complications of intensive care procedures per se, (e.g. vascular complications due to intra-vascular catheters and side-effects of ototoxic drugs and sedatives) are not evaluated [5, 18, 32, 33, 45, 54, 57, 63]. One can assume the exact incidence of physical sequelae to be higher than has been reported so far.
Psychological sequelae and functional health and QoL
In the reviewed studies, psychological sequelae have been established in 10–14% of survivors and their parents. The comparison of findings is hampered due to different measurement tools and cut-off points for the diagnosis of PTSD and various follow-up intervals. Risk factors accounting for hampered psychological outcome could be diverse (severity of illness, being removed from one’s child, having been witness to the accident, mental health, family functioning, social support, coping strategies and lack of information from the medical team) [17, 26, 29, 31]. Psychological support to improve coping strategies and prevent over-protection might improve psychological outcome in children and parents [3, 28]. Further research is essential to establish the appropriate time and extent of the psychological support needed.
Cognitive sequelae have rarely been studied in the reviewed studies. Adequate neuro-cognitive evaluation is both expensive and time-consuming. Studies in neonatal ICU survivors show substantial cognitive dysfunction with great impact on daily life [7]. Consequently, early intervention, education and rehabilitation are expected to improve daily life [11, 46].
A majority of PICU survivors seem to have unchanged functional health and good QoL. In the reviewed studies, functional health is evaluated by telephone interviews [27, 38, 47, 65]. In most of these studies, the physician rather than the child or its parents evaluates functional health. Ideal (HR)QoL questionnaires should measure all aspects of QoL and preferably be filled in by the children themselves. Proxy investigation of functional health and (HR)QoL (in children <6–8 years of age) is second best [36, 37, 39, 66]. Besides, the pre-morbid state is probably an important factor which is difficult to assess [16].
Suggestions for future follow-up research
The reviewed studies have a number of methodological limitations. Heterogeneity is the most important one. Consensus on all aspects of follow-up research is essential for well-founded conclusions. For example, structured and standardised evaluation of: (1) organ system function with a validated tool such as the Paediatric Logistic Organ Dysfunction (PELOD) score [13, 42, 60, 71]; (2) neuro-cognitive function; (3) complications of PICU treatment; and (4) (HR)QoL are warranted. Multi-centre studies as proposed by the Collaborative Pediatric Critical Care Research Network (CPCCRN) with a uniform approach will provide answers either in general PICU cohorts or in disease-oriented study groups [76].
In conclusion, this review indicates that PICU survivors and their parents may have substantial physical and psychological sequelae interacting with QoL. Because of longer life expectancy, longer follow-up time is warranted, emphasising the consequences for health care in children. We believe that paediatric intensivists and psychologists should be involved as core members of follow-up teams.

quality of life
paediatric intensive care unit
post-traumatic stress disorder
health status
outcome assessment (health care)


Nonparametric IRT analysis of Quality-of-Life Scales and its application to the World Health Organization Quality-of-Life Scale (WHOQOL-Bref)

Background This study investigates the usefulness of the nonparametric monotone homogeneity model for evaluating and constructing Health-Related Quality-of-Life Scales consisting of polytomous items, and compares it to the often-used parametric graded response model.
Introduction
Questionnaires for health-related quality-of-life (HRQoL) measurement are important for several reasons. First, they may be used to compare the mean level of different patient groups with respect to physical, mental and social health. A researcher may want to find out whether these patient groups have different needs with respect to, for example, therapy or medication or whether different adaptations of their environment are in order so as to improve their conditions of living. Second, HRQoL questionnaires are also important for the measurement of mean change––either progress or deterioration––of such groups due to, for example, therapy. The researcher’s interest then lies in the effectiveness of therapy with respect to HRQoL. Third, the total score a patient obtains on an HRQoL questionnaire may be used to diagnose this patient’s general level of physical health and psychological well-being, for example, so as to be able to estimate the budget needed for his/her treatment during a particular period.
To effectively measure HRQoL, we argue that an instrument must meet two requirements. The first requirement is that it is clear what the instrument measures: one overall dimension of HRQoL or several dimensions reflecting different aspects of HRQoL. If the instrument measures one dimension, one can use the total score on all items to obtain an impression of the overall level of HRQoL. If the instrument measures multiple dimensions, it may be recommendable to determine total scores on subsets of items (e.g., domain scores), each reflecting a particular aspect of HRQoL (e.g., HRQoL with respect to physical, psychological, and social limitations) and then assess individuals or compare groups on a profile of scores. These two cases may be characterized as unidimensional and multidimensional measurement.
The second requirement is that the psychometric properties of the items are known and found sufficient. One important psychometric item property is the item’s location on the scale that quantifies the HRQoL aspect of interest. For example, patients are likely to experience fewer problems when engaging in activities like bathing and dressing than in more demanding activities such as shopping and travelling. The items concerning bathing and dressing require a lower level of physical functioning than the other two items. Thus, bathing and dressing are located further to the left (at a lower level of the scale) than shopping and travelling. A good diagnostic HRQoL instrument contains items of which the locations are widely spread along the scale. Such a scale allows for measurement at varying levels of physical functioning and may be used, for example, for assessing mean differences between groups, mean change due to therapy, and individual patients’ levels of physical functioning.
Another important psychometric item property is the item’s discrimination power. This is the degree to which the item distinguishes patients with relatively low psychological well-being levels from patients with relatively high psychological well-being levels. The higher the discrimination power the higher the item’s contribution to reliable measurement ([1], pp. 101–124). A good diagnostic instrument has items with high discrimination power that each contributes effectively to reliable measurement of patients at different locations along the scale. Such an instrument picks up differences between groups, effects of therapy, and individual levels of activity limitation.
Item response theory (IRT) models [1] are becoming more popular as statistical tools for scale construction in the HRQoL context. IRT can be used effectively to investigate the dimensionality of an instrument and the psychometric properties of its constituent items. The goal of this study is to discuss one particular class of IRT models known as nonparametric IRT models [2–5], and to argue that this class in particular provides a general and flexible data analysis framework for studying the dimensionality of a set of polytomously scored items (with dichotomously scored items as special cases) and ascertaining ordinal scales for the measurement of HRQoL aspects which contain items that have varying locations and sufficient discrimination power. Over the past few years, nonparametric IRT models already have been used occasionally for constructing HRQoL scales; see [6–9].
Our point of view is that, given that the researcher has formulated desirable measurement properties, (s)he should construct his/her scale by means of an IRT model that is as general as possible while satisfying the desired measurement properties. Examples of such properties are that the items measure the same dimension, that the measurement level is at least ordinal, and that measurement values are reliable. An HRQoL researcher who has constructed and pre-tested a questionnaire consisting of, say, 40 items is not served well when his/her data are analyzed by means of an IRT model that is unnecessarily restrictive, the result of which is that, say, half of the items are discarded. We will argue that the most general IRT model that serves one’s purposes well, often (but not always) is a nonparametric IRT model.
Many questionnaires are used for assessing differences in HRQoL between groups, change due to therapy, and individual patients’ scale levels, and a general nonparametric IRT model then is the perfect choice for analyzing one’s data. Nonparametric IRT models have several advantages over more-restrictive parametric IRT models [1]: Nonparametric IRT models (1) are based on less-restrictive assumptions, thus they allow more items into the scale while maintaining desirable measurement properties; (2) offer diverse tools for HRQoL analysis that give ample information about the dimensionality of the data and the properties of the items; and (3) provide patient measurement values and item location and discrimination values, which have an interpretation that is close to intuition and therefore easy to interpret for users of HRQoL scales. For computerized adaptive HRQoL testing, more-restrictive parametric IRT models such as the Rasch [10] model (dichotomous items) and the generalized partial credit model [11] and the rating scale model [12] (polytomous items) are more appropriate than nonparametric models.
Parametric IRT models have been used more than nonparametric IRT models, especially in psychological and educational measurement, and also in HRQoL research (e.g., [13–15]). A reason for this may be that nonparametric IRT models were developed later than parametric IRT models. See [16–18] for reviews of nonparametric IRT.
This paper is organized as follows. First, we explain assumptions of IRT and compare parametric IRT and nonparametric IRT. Second, we discuss methods and software from nonparametric IRT that can be used for analyzing the polytomous item scores obtained from HRQoL questionnaires. Third, we use this software to analyze data from the World Health Organization Quality-of-Life Scale (WHOQOL-Bref) [19]. The results are compared to those obtained by means of a parametric IRT model. Finally, we provide recommendations for HRQoL researchers on how to use nonparametric IRT methods for analyzing their data.
Definitions and assumptions
IRT models are suited for the analysis of multi-item questionnaire data that typically result from HRQoL questionnaires. The data are discrete scores representing the responses of N respondents to J items (items are indexed j; j = 1, ..., J). Many HRQoL questionnaires use items that have three or more ordered answer categories represented by three or more ordered scores, also called polytomous item scores. For simplicity, we assume that all items have the same number of ordered answer categories; this number is denoted by M + 1. Let Xj be the random variable representing the discrete score on item j, and let the items be scored Xj = 0,...,M. For example, for an item asking whether one is satisfied with one’s sleep, score 0 may represent ‘very much dissatisfied’ and score M may represent ‘very much satisfied’, and the intermediate scores represent intermediate levels of satisfaction. With dichotomous item scoring, Xj = 0,1, and possible intermediate satisfaction levels are not quantified separately but collapsed into the score categories 0 and 1. Usually, researchers summarize the J item scores for each patient by the total score (sometimes referred to as the sum score), which is formally defined as
. Total score X+ is an estimate of a patient’s true score T; this is the expectation of X+ across independent replications of the measurement procedure ([20], pp. 29–30).
IRT models distinguish observable or manifest variables such as item score Xj and total score X+ from latent variables. These latent variables play the role of summaries of the behavior that is described by the responses to the items. Sometimes, latent variables are interpreted as if they were causal agents driving responses to items and individual differences between patients. We will also use the distinction between latent and manifest variables in our examples. Thus, we assume that patients are characterized by either one latent HRQoL attribute (meaning that measurement is unidimensional) or different HRQoL attributes (meaning that measurement is multidimensional) which together represent the patient’s latent physical, mental or social health. For example, assume that measurement is unidimensional and that the latent variable is mental health or psychological well-being. Psychological well-being then is an unobservable state in each patient, and inferences about it are made on the basis of the manifest responses reported by patients in reaction to the items in an HRQoL questionnaire. Latent variables are denoted by notation θ. If measurement is unidimensional, the IRT model contains one latent variable θ, and if it is multidimensional multiple latent variables are needed. We only consider unidimensional IRT models here; see [21] for a discussion of multidimensional IRT models.
Several families of IRT models for polytomous item scores have been proposed (e.g., [22]). The family of graded response models (GRMs; [23]) is suitable for analyzing ordered item scores collected by means of polytomous response scales [22, 24, 25]. Suppose the item ‘How much do you enjoy life?’ has four ordered answer categories running from ‘Not at all’ (score 0) to ‘Very much’ (score 3); thus Xj = 0,...,3 and M = 3. For this item, GRMs conceptualize the response process by means of four conditional response probabilities, called item-step response functions (ISRFs) and denoted by P(Xj ≥ m|θ)=Pjm(θ) (m = 0,...,M). It may be noted that for m = 0 we have that Pj0(θ) = 1 by definition, irrespective of the latent variable level; thus, this response probability is uninformative about the response process and may be ignored. The ISRF describes the relationship between expressing at least a particular minimum level of enjoying life (i.e., having at least a score of m on the example item) and the latent variable of psychological well-being (θ).
Figure 1a shows an example of the ISRFs of two items having four answer categories each (i.e., M = 3). The solid lines denote the ISRFs for one item and the dashed lines denote the ISRFs for the other item. Two things are noteworthy. First, the ISRFs of different items may intersect but based on the cumulative character of the definition of the ISRFs, the ISRFs of the same item cannot intersect. Second, the ISRFs have been drawn as monotone curves with rather irregular shapes. Such shapes are typical of a nonparametric GRM [24] and found not to prevent a set of items from having favourable measurement properties, as we will see shortly. Figure 2b also shows monotone ISRFs but now having smooth S-shapes, typical of a parametric GRM. This parametric GRM nearly has the same measurement properties as the nonparametric GRM, but because it assumes smooth ISRFs (Fig. 1b) instead of irregular ISRFs (Fig. 1a), it is more restrictive and often leads to the rejection of more items (and thus shorter scales) than its nonparametric counterpart.Fig. 1Examples of item step response functions of (a) a nonparametric item response model and (b) a parametric item response modelFig. 2Examples of (a) flat ISRFs and (b) non-monotone ISRFs
GRMs have the next three assumptions as a point of departure. The first assumption is unidimensionality (UD); that is, each model assumes that one latent variable θ summarizes the variation in the J item scores in the questionnaire. Assumption UD implies that respondents can be ordered meaningfully by means of a single number. The second assumption is local independence (LI); that is, if we condition on θ, the J item scores are statistically independent. An implication of LI is that in a subgroup of patients who have the same θ value, all covariances between item scores are 0. The third assumption is monotonicity (M); that is, the ISRFs are each assumed to be monotone increasing functions in θ (see Figs. 1a and b). Applied to the latent variable of psychological well-being (θ) and the items from the WHOQOL-Bref, assuming UD, LI and M means that we hypothesize that (1) only psychological well-being drives responses to items and has a systematic effect on individual differences in item scores and total scores (UD); (2) given a fixed level of psychological well-being, relationships between concrete aspects of psychological well-being such as represented by the items ‘How much do you enjoy life?’ and ‘Are you able to concentrate?’ are explained completely (i.e., the covariance between these items, conditional on θ, equals 0) (LI); and (3) the higher the level of psychological well-being, the higher the probability that one enjoys life and is able to concentrate.
Parametric and nonparametric graded response models
Parametric Graded Response Model. The ISRFs of the parametric GRM [26] are defined by logistic functions that have the following parameters:δjm: the location parameter of the mth ISRF of item j (i.e., Pjm(θ)) on the scale of θ;αj: the slope parameter or ‘discrimination power’ of item j.
The meaning of these parameters is explained after the ISRF of the parametric GRM is introduced. This ISRF is defined as
Figure 1b shows the logistic ISRFs of two items (M = 3). The two items are denoted by j (solid ISRFs) and k (dashed ISRFs). For each item, the three location parameters are also shown. For item j, by definition we have that δj1 < δj2 < δj3, and for item k by definition δk1 < δk2 < δk3. An ISRF’s location parameter is the value of θ for which the probability of having an item score of at least m equals .5: that is, Pjm(θ) = Pkm(θ) = .5, m = 1,..., M.
Figure 1b also shows that the slopes of the ISRFs of the same item are equal (mathematically, they must be equal or the ISRFs would intersect; this is impossible given the cumulative definition of the ISRFs), but also that the ISRFs of item j are steeper than the ISRFs of item k. Steepness of slopes is evaluated as follows. For ISRF m of item j, consider the point with coordinates (δjm, .5). This is the point in which the slope of a logistic ISRF is steepest, and this steepest slope is taken to be typical for the whole ISRF. Parameter αj expresses this maximum steepness (but is not exactly equal to it). In the example in Fig. 1b, we have that αj > αk.
Nonparametric Graded Response Model. Instead of choosing a parametric function, nonparametric IRT models typically define order restrictions on the ISRFs. The nonparametric GRM, better known as the monotone homogeneity model (MHM) for polytomous items [27, 4], assumes UD, LI, and M: that is, for any pair of θs, say, θa and θb, the MHM assumes that
Thus, the ISRF is monotone non-decreasing in θ; see Fig. 1a for examples of ISRFs that are monotone but not logistic. This assumption says that a higher level of psychological well-being induces a higher probability of obtaining at least an item score of m (i.e., a higher item score). The ISRFs of different items can have any monotone form and be very different. Requiring monotone ISRFs only is less restrictive than requiring monotone logistic ISRFs; thus, the MHM is a more general model for describing the data than the GRM (henceforth, we call the nonparametric GRM by its better known name (in fact, acronym) MHM, and the parametric GRM simply the GRM).
Unlike the GRM, the MHM does not provide numerical estimates of the latent variable θ . Instead, the MHM allows that total score X+ orders patients stochastically on latent variable θ in almost all practical measurement situations [28]. This means that, for two total scores X+ denoted v and w,
[4]. This inequality says that as the total score increases the mean θ also increases (or stays the same). Thus, groups of patients that have higher total scores, on average also have higher latent variable values. This result may not seem spectacular at first sight, but it (1) ascertains an ordinal scale for patient measurement (2) using only observable total scores (without requiring the actual estimation of θ). For the psychological well-being example, if the MHM fits the data, ordering patients by means of the total score by implication orders them on the latent variable θ.
Also, the MHM does not provide numerical estimates of the item parameters δ and α. Instead, a distinction can be made between drawing information about item functioning from estimates of the complete ISRFs and item parameters typical of the MHM. Estimates of the complete ISRFs provide much information about the exact relationship between the item scores and the latent variable [16, 29]. ISRFs that are relatively flat or fail to be monotone can be studied in much detail so as to reveal why they dysfunction.
Figure 2a shows three relatively flat ISRFs of a hypothetical item (M = 3). This item does not distinguish low θ and high θ patients well. It may be noted that the ISRFs do not all need to be flat simultaneously, but given that they  cannot intersect if one ISRF is flat others are likely to be relatively flat as well and the item as a whole contributes little to the reliable ordering of patients on θ.
Figure 2b shows three non-monotone ISRFs of another hypothetical item. Each shows relative good distinction between low θ values and between above-average θ values but bad distinction just below the middle of the scale. Again, non-intersection of the ISRFs of the same item implies that often several ISRFs simultaneously show such disturbing non-monotonicities. For the example item one may conclude that when the questionnaire contains few items, which are effective in the high θ area, this item may be retained to cover this area even though this would be at the expense of measurement quality just below the middle of the scale.
For each item, the MHM framework provides M location parameters and a scalability coefficient, which provides information about item discrimination. The location parameters are the proportions of the population of interest, which have at least a score m on item j, and which are denoted by πjm, m = 1, ..., M. For the same item, due to non-intersection of ISRFs we have that πj1 ≥  ... ≥ πjM, whereas in the GRM item location parameters are ordered oppositely, δj1 ≤ ... ≤ δjM. The item scalability coefficient Hj (e.g., [2], pp. 148–153; [18], chap. 4) summarizes the discrimination power of an item across its M ISRFs. The numerical Hj value is determined by the interplay of the slope and the location of the ISRFs of all J items and the distribution of the latent variable θ [30], and it expresses how well item j separates patients given the ISRFs of item j relative to the other items’ ISRFs and the distribution of θ. Mathematically, holding constant this distribution and the location of all items’ ISRFs, coefficient Hj is higher when the slopes of ISRFs of item j are steeper [18].
Nonparametric versus parametric graded response models
Relationships Among Models. The MHM shares assumptions UD, LI and M with the GRM, but the MHM is less restrictive with respect to the shape of the ISRFs. Thus, the MHM is more general than the GRM or, equivalently, the GRM is a special case of the MHM. This hierarchy implies that, if the GRM fits the data, by implication the more general MHM also fits but if the MHM fits the data, this does not imply that the GRM also fits. Fit of the GRM then needs to be investigated separately. Because of the hierarchical relationship, for any data set the MHM fits as least as many items as the GRM (Table 1).Table 1Comparison of monotone homogeneity model (MHM) and graded response model (GRM)Nonparametric IRT (MHM)Parametric IRT (GRM)Restrictiveness of modelsLow; many items admitted to the scaleHigh; fewer items admitted to the scaleInterpretation of parametersIntuitively appealingMore-complicatedParameters(typical range)    Person levelT, X+θ(−3 ≤ θ ≤ 3)    ISRF locationπjmδjm(−3 ≤ δjm ≤ 3)    ISRF discriminationHjαj(0.5 ≤ αj ≤ 2.5)Data analysisExploratory, data as point of departureConfirmatory, model as null hypothesisApplicationsComparing groupsMeasuring changeDiagnosing patientsComparing groupsMeasuring changeDiagnosing patientsConstructing item banksAdaptive testing
Patient and Item Parameters. The MHM and the GRM provide the following patient and item parameters (also, see Table 1):For patient measurement, the MHM uses total score X+ to order patients on latent variable θ. Because total score X+ has an easy interpretation and, moreover, in many IRT models X+ and θ tend to correlate extremely high suggesting a strong linear relationship [31], total score X+ may be preferred in practice. Total score X+ is the sum of the rating scale scores on the J items, whereas estimates of θ are expressed on a logit scale, which does not have a straightforward interpretation for users of HRQoL scales. In general, the ordinal relationship of θ with total score X+ (which can be approximated well by a linear relation) enables users to switch between scales, and use the one that suits their goals best.Because item location δjm is expressed on the same scale as latent variable θ, it also has an interpretation in logits. For many users, proportion πjm, the proportion of patients who have at least an item score of m, has an easier interpretation.Item discrimination αj gives the maximum slope of the logistic ISRF irrespective of the locations of the other ISRFs of item j and the other items in the questionnaire, and irrespective of the distribution of θ. Thus, information on ISRF slopes is absolute in the sense that a particular αj value does not provide information on the item’s suitability for measurement in a particular group (characterized by a particular distribution of θ) by means of a set of J items (characterized by particular location and slope parameters). On the other hand, item scalability coefficient Hj depends explicitly on the interplay of the distribution of θ, the spread of locations of the ISRFs, and the slopes of the ISRFs. In particular, keeping two of these factors fixed, Hj tends to increase in the third. This dependence on the distribution of θ and the item properties informs the researcher precisely how well item j separates patients with low and high θ values in the particular group of patients under consideration using the particular set of items. The difference between absolute slope information (αj) and relative slope information (Hj) is illustrated as follows.
Two data sets of size N = 5,000 and five items (J = 5) (M = 3 for each item) were generated using the item parameters in Table 2. The first data set came from a hypothetical clinical population with θ ∼ N(−2,1) (i.e., low psychological well-being level), and the second data set came from a hypothetical healthy population with θ ∼ N(1,1) (i.e., high psychological well-being level). It may be noted that for each of the five items, αj = 1.4 by definition, irrespective of ISRF location parameters and the θ distribution. The Hj values in the clinical group were computed and found to range from .36 to .39. In the healthy group, the Hj values were found to be smaller for items 1, 4, and 5 (H1 = .26, H4 = H5 = .25). In the clinical group, the ISRFs of these three items were located more closely to the middle of the θ distribution (Fig. 3 shows this for Item 4) such that higher Hj values resulted, but in the healthy group the ISRFs were located further in the lower tail of the θ distribution (Fig. 3) resulting in lower Hj values (.3 is considered minimally acceptable; [2], chap. 5). Thus, the items 1, 4, and 5 are well suited for measurement in the clinical group but not in the healthy population, despite their overall discrimination power expressed by αj = 1.4, for J = 1, ..., 5. For item 3, H3 was a little higher in the healthy group because the second and third ISRFs discriminate particularly well at higher ranges of θ. For item 2, the location parameters of the ISRFs were widely spread across the θ distribution, resulting in good discrimination both at lower and higher levels of θ in both distributions. We conclude that item scalability coefficient Hj has the advantage that it takes the item (and not the individual ISRF) as a unit and depends simultaneously on the θ distribution, the slopes of the ISRFs, and the spread of the locations of the ISRFs. Thus, Hj informs the researcher whether item j discriminates well in the group under consideration using the particular set of items, whereas item discrimination αj provides information about the discrimination power irrespective of the patient group under consideration and the item properties of item j and the other items in the scale.Table 2Example of item parameters of the graded response model (GRM), and Item H Values in two different populations with normally distributed latent traitsItem jItem parameters GRMHjα jδj1δj2δj3θ∼N(–2,1)θ∼N(1,1)11.4–2.4–2.2–1.00.360.2621.4–4.0–2.01.00.370.3931.4–1.02.02.50.390.4241.4–3.0–2.0–1.00.360.2551.4–2.5–2.0–1.50.360.25Note: Hj values are based on simulated data for sample size N = 5,000Fig. 3Three ISRFs of the same item (Item 4) relative to two different distributions of the latent variable
Confirmatory and exploratory data analysis. In general, before IRT models are accepted as reasonable descriptions of the data their goodness-of-fit to these data must be investigated and assessed. In general, goodness-of-fit research is different for parametric and nonparametric IRT (with the GRM and the MHM as special cases, respectively). In a parametric IRT analysis the model often serves as null-hypothesis and it is tested whether this null-hypothesis must be rejected or may be supported by the data. Nonparametric IRT analysis in general takes the data as point of departure and (1) instead of positing a unidimensional or multidimensional latent variable structure analyzes the data to find its true dimensionality, and (2) instead of positing a logistic or other functional shape estimates the ISRFs from the data so as to diagnose the items’ functioning [16, 18]. This research strategy renders nonparametric IRT a more flexible data-analysis tool than parametric IRT. One could also characterize this distinction as confirmatory (parametric IRT) versus exploratory (nonparametric IRT) (Table 1).
Application of parametric and nonparametric IRT. If a nonparametric IRT model such as the MHM fits the data, the result is a scale on which patients can be ordered by means of the total score X+. This total score has a strong linear correlation with latent variable θ. Such a scale suffices in many applications. Examples are the comparison of groups, the measurement of change due to therapy, and the establishment of the patient’s psychological well-being level as low, medium, or high (Table 1). The practical advantage of nonparametric IRT models over parametric IRT models is that the scales they produce contain more items thus reducing the risk of wasting items that have non-logistic but monotone ISRFs that discriminate well in (part of) the group under consideration (e.g., Fig. 2b). Such items contribute well to reliable measurement. In addition, rejection of such items may also harm the coverage of the latent attribute.
If a fitting parametric IRT model is obtained for a set or a subset of the items, one has a parsimonious description of the item characteristics, and one can use the estimated item parameters to scale the items, and the estimated θs as interval level measures to locate patients on this scale. If a large set of items, also known as an item bank [32], is available, and if a parametric IRT model fits the item bank, parametric IRT models have the advantage that the patient’s θ can be assessed using different sets of items from the item bank. This may be useful for the measurement of change when change is so large that the set of items that was used initially no longer captures the higher or lower θ levels needed for the second measurement, thus necessitating the use of other items. Another application of parametric IRT is computerized adaptive testing (CAT), which selects items that match the patient’s θ level well from a huge item bank so as to optimize accuracy of θ measurement. In principle, CAT requires different item sets for different θ values (Table 1).
Nonparametric IRT analysis in practice
Software for nonparametric IRT analysis. Several programs are available for data analysis using nonparametric IRT but not each can handle polytomous item scores. We briefly discuss the programs MSP ([33]; also see [4]) and TestGraf98 [34]. Both programs are used regularly and, together, they provide the researcher with a clear and informative picture of (1) the dimensionality of the data, (2) the (lack of) monotonicity of the ISRFs, and (3) estimates of item locations and item discrimination.
Program MSP uses the MHM as the main analysis model but another nonparametric model not discussed here is also included in the program. Basically the program consists of two parts. The first part of MSP is an algorithm for exploring the dimensionality of the data ([2], chap. 5; [4], chap. 5). The algorithm uses item scalability coefficient Hj to select items. Items that are related to the same latent variable θ are selected one by one on the basis of their Hj value. Suppose that one latent variable drives the responses to one subset of the items, another latent variable drives the responses to another subset of items, and so on. Then, the algorithm selects mutually exclusive clusters of items each of which is driven by a different latent variable.
The second part of MSP provides several statistical tools for exploring the shape of the ISRFs [29]. This is most useful after the dimensionality of the data has been ascertained. For example, due to their strong positive tendency the non-monotone ISRFs in Fig. 2b may have an Hj value that is high enough for the items to be selected in a unidimensional cluster, but the non-monotonicity also may distort parts of the ordinal scale defined by the items in the cluster. MSP estimates ISRFs by means of a number of discrete points that are connected to form a jagged ‘curve’. Figure 4a shows four discrete ISRFs of the same item (i.e., M = 4) each estimated by means of eleven points. The researcher can manipulate the number of estimated points. If more points are estimated from the same data more details of the ISRF become visible (i.e., bias is reduced) but because for each point fewer data are available, accuracy decreases. In statistics, this is known as the bias-accuracy trade-off, and it is advisable to try several options to reach a good decision. MSP tests observable deviations from monotonicity for significance. Figure 4a also shows a ‘mean’ ISRF (dashed curve), which we may call the item score function (ISF) and which is not standard output of MSP. This function must be monotone nondecreasing.Fig. 4(a) Discrete estimates of four ISRFs (solid curves; from MSP) of Item 16 (‘Satisfied doing daily activities?’, from Physical Health and Well Being domain), and the corresponding discrete item-score function (dashed curve), (b) also estimated as a continuous curve (from Testgraf98)
Program TestGraf98 [34] can be used for studying the shape of the ISF. Unlike MSP, TestGraf98 produces continuous estimates of response functions ([35]; here, only the ISFs); and like MSP, TestGraf98 shows graphical displays of these estimates that can be manipulated with respect to bias and accuracy, and also here it is advisable to try several options. The quality of the decision can be improved by using the confidence envelopes for the estimated ISFs for statistical testing. Figure 4b shows an example of an estimated ISF (solid curve) and its confidence envelopes (dashed curves), which were estimated by means of TestGraf98.
Research strategies for nonparametric IRT analysis. MSP provides a method for investigating the dimensionality of the data, and MSP and TestGraf98 both can be used to investigate assumption M. The investigation of dimensionality and monotonicity serves to identify the items that together constitute an ordinal patient scale for the same latent variable.
For investigating dimensionality, MSP offers the researcher the possibility to set a positive lower bound c on Hj. Under the MHM, the lowest admissible value is c = .0; MSP’s default is c = .3 ([2], chap. 5; [4, 33]). This default value ascertains a lower bound on the overall discrimination power of the items (but researchers are free to choose a higher value) and, as a result, item clusters consist only of sufficiently discriminating items that measure the same latent variable. Thus, MSP aims to produce unidimensional scales that allow accurate patient measurement.
TestGraf98 estimates the ISFs (e.g., Fig. 4b) by means of the nonparametric regression method known as kernel smoothing (e.g., [36], chap. 2; [35]). The availability of confidence envelopes for the continuous ISF estimates provides detailed information of (lack of) monotonicity for each item. TestGraf98 provides these estimates irrespective of the dimensionality of the data. Thus, a good research strategy is to first investigate item-set dimensionality by means of MSP and then use MSP and TestGraf98 to study the ISRFs and the ISFs in dimensionally distinct clusters. See [37] for another method for assessing the shape of these curves.
A real-data example: The World Health Organization Quality-of-Life Scale
The WHOQOL-Bref was developed for assessing individuals’ perception and feelings of their daily life. The questionnaire starts with two items, which ask for global estimates of one’s quality of life, and then continues with 24 items covering four domains: (a) physical health and well-being (seven items); (b) psychological health and well-being (six items); (c) social relations (three items); and (d) environment (eight items). The two general items were left out of the analysis. In agreement with their numbering in the WHOQOL-Bref, the other 24 items were numbered from 3 to 26. Examples of items are:Do you have enough energy for daily life? (physical domain)How much do you enjoy life? (psychological domain)How satisfied are you with your personal relationships? (social domain)How safe do you feel in your daily life? (environmental domain)
Each item uses a five-point rating scale (i.e., Xj = 0, ..., 4 ); the higher the item score, the better one’s quality of life on the specific domain covered by the item.
The data were collected by undergraduate psychology students of Tilburg University as part of a course Research Practical in the academic year 2005–2006. Students were instructed to strive for a sample of participants equally distributed across both sexes and the following age categories: 30–39, 40–49, 50–59, and more than 60 years. The final sample consisted of N = 589 respondents from the Dutch population. Of these respondents, 55% were women, mean age was 55.2 years (SD = 14.6), 32% had completed community college or university, 36% had completed vocational school, 20% had high school at most, and 12% had only elementary school or less.
N = 55 cases had missing item scores. Missing values were estimated using two-way imputation. Comparable to an analysis-of-variance layout, this method uses both a person effect and an item effect for estimating a missing score (for details, see [38]). MSP and TestGraf98 were used to analyze these data and construct one or more scales, thus illustrating the possibilities of the MHM. For the sake of comparison, we ran a principal component analysis and a GRM scale analysis on the data.
Results
Sample statistics of item and scale scores
Table 3 shows that the mean item scores ranged from 2.58 (Item 20: ‘Satisfied with sex life?’) to 3.46 (Item 25: ‘Moving around well?’). The mean X+ scores were 21.04 (physical domain), 17.01 (psychological domain), 8.52 (social domain), and 24.27 (environmental domain). Correlations between the domain scores ranged from .37 (between physical and social domains) to .51 (between physical and environmental domains). Table 3Results from MSP item selection procedure (Item clusters, item Hj values, and total H), and Item Hj values and total H for each content domainMSP item selection procedureHj per content domainc = .3c = .4jMean1212345Physical health and well-being3Distraction due to paina3.04.59.59.4010Experiencing energya2.98.43.53.4615Satisfied with sleep2.66.22––––––.2825Moving around well3.46.36.43.4116Satisfied doing daily activities2.84.41.56.524Need medical treatment for daily functioninga3.17.59.59.4317Satisfied work capacity2.89.40.57.52Scale value21.04.43Psychological health and well-being5Enjoying life2.66.34.42.377Being able to concentrate2.80.32–––––.2918Satisfied with yourself2.95.41.48.4511Acceptance physical appearance3.23.33–––––.3526Experiencing negative feelingsa2.72.30–––––.346Life meaningful2.66.30–––––.37Scale value17.01.36Social relations19Satisfied relationship with other people3.06.34.50.5020Satisfied with sex life2.58.30.42.4221Satisfied support from others2.88.28–.40.40Scale value8.52.44Environment8Feeling safe in daily life3.08.30–––––.3322Satisfied living conditions3.13.43–––––.4312Enough financial resources3.08.33.53.4223Satisfied getting adequate health care2.87.29–.52.3613Availability information needed in daily life3.06.34.53.4014Opportunities leisure2.90.34.49.399Healthy environment2.88.29––––––.3124Satisfied with transport in daily life3.28.34.52.40Scale value24.27.38aReversely scored items
Dimensionality analysis
Principal components analysis. Dimensionality was explored by means of a principal components analysis using polychoric correlations. The ratio of the first to the second eigenvalue of the polychoric correlation matrix was 8.428/1.986 = 4.24. A ratio of 4:1 is taken as evidence of considerable strength of the first dimension (e.g., [39]). The first factor explained 32.4% of the variance. A confirmatory factor analysis of the four a priori domain scales of the WHOQOL-bref improved fit over the one-factor model (P ≤ 0.001). However, the factors correlated from .50 (physical and social domain) to .79 (psychological and social domain). The explorative factor analysis in conjunction with the confirmative factor analysis justifies the assumption of a general HRQoL dimension underlying each scale.
Monotone homogeneity model analysis. Next, MSP was used treating all 24 items as a fixed scale. The MHM does not allow negatively correlating items in one scale. Item 4 (‘Need medical treatment for daily functioning?’) and Item 20 (‘Satisfied with sex life?’) correlated negatively but not significantly (P > .05); thus all 24 items were used for analysis. The item Hj values (not tabulated) ranged from .21 (Item 15: ‘Satisfied with sleep?’) to .40 (Item 22: ‘Satisfied with living conditions?’ and Item 10: ‘Enough energy for everyday life?’). The total-scale H coefficient was equal to .30. The results suggest that the items tend to cover one latent HRQoL aspect, which, however, induces only weak general association between the items. Thus, in addition to this common aspect it seems reasonable to also look for more-specific HRQoL aspects that are covered by subsets of items.
Dimensionality was investigated by means of the MSP search algorithm using several c values, starting with .3 (default) and then increasing c with steps of .05 in each next analysis round. We only report results for c = .3 and c = .4 (other values did not reveal interesting results). For c = .3, one scale consisting of 18 items (H = .35) and one scale consisting of 2 items (H = .59) were found (Table 3). The four remaining items were not selected because their Hj values were under .3 (i.e., .22 and .28, .29, and .29). The 18-item scale had a rather heterogeneous content. The 2-item scale asked about distraction due to pain (Item 3) and the need for medical treatment for daily functioning? (Item 4). Thus, their high scalability may be explained by the use of palliative medicines.
For c = .4, five scales were found consisting of 6, 2, 3, 2, and 3 items, respectively (Table 3). The first scale consisted of items from the physical domain and the psychological domain. The other scales consisted of items from one domain. Scale 2 again covered Item 3 and Item 4. Scales 3 and 4 covered environmental-domain aspects. Scale 5 contained all social-domain items.
Thus, for default c = .3, 18 of the 24 items were selected in one scale. The pattern of item selection for higher c values such as c = .4 showed that the item set progressively crumbled into many smaller scales while other items remained unselected. Sijtsma and Molenaar ([4], pp. 80–86; see also [7] [40]) argued that this typical pattern of results gives evidence that the 18-item set constitutes a unidimensional scale. The total-scale H equaled .35, giving evidence of weak scalability ([2], p. 185). Most of the item Hj values were between .3 and .4, also suggesting a weak relationship with the latent variable ([2], p. 185).
Finally, it was investigated whether the four a priori identified item domains could be considered as separate scales. Table 3 (last column) shows that the total-scale H values ranged from .36 (environmental domain) to .44 (social domain). Thus, based on Mokken’s classification of scales [2] the four a priori item domains constituted weak to medium scales. Two items had H values just smaller than c = .3 (i.e., Item 15 (H15 = .28): ‘Satisfied with sleep?’ and Item 18 (H18 = .29): ‘Satisfied with yourself?’). The content domains may be considered as unidimensional clusters of items measuring distinct aspects of HRQoL, each of which are related to a more general underlying HRQoL construct. Because of their conceptual clarity, the remaining analyses were done on the a priori defined item domains.
Monotonicity assessment in each item domain
For each item domain, MSP was used to assess the ISRFs’ shapes. First, ISRFs were estimated accurately (i.e., many cases were used to estimate separate points of the ISRFs) but at the expense of possible bias (i.e., only few points were estimated). Second, ISRFs were estimated with little bias (i.e., many points were estimated) but at the expense of accuracy (i.e., few cases were used to estimate each point).
For the physical domain, the first analysis (high accuracy, more risk of bias) revealed four items of which one or more ISRFs showed minor violations of monotonicity, but none of these violations were significant (5% level, one-tailed test, because only sample decreases are tested as violations; increases support monotonicity). The second analysis (more inaccuracy, less bias) revealed that for all seven items one or more ISRFs showed one or more local decreases, but none them were significant. Figure 5a shows the local, nonsignificant decreases in the ISRFs for Item 3 (‘Distraction due to pain?’).Fig. 5Four ISRFs of Item 3 (‘Distraction due to pain?’, from Physical Health and Well Being domain) showing nonsignificant violations of assumption M and rejected by the GRM: (a) Results from MSP (including the ISF); (b) results from Testgraf98 (ISF and confidence envelopes); (c) and results from Multilog7.0 (GRM)
For the psychological domain, for both analyses (i.e., high accuracy versus little bias) two items were found which had ISRFs showing significant local decreases. For example, Fig. 6a shows for Item 7 (‘Being able to concentrate?’) that the estimate of ISRF P72(θ) = P(X7 ≥ 2|θ) (the second curve from the top) has several local decreases, the largest of which was significant (5% significance level, P = 0.019). The estimate of ISRF P73(θ) = P(X7 ≥ 3|θ) shows two small decreases; they were not significant. For the social domain and the environmental domain, no significant violations of the monotonicity assumption were found. It can be concluded that assumption M holds for each of the four scales.Fig. 6Four ISRFs of Item 7 (‘Being able to concentrate?’) showing significant violations of assumption M and rejected by the GRM: (a) Results from MSP (including the ISF); (b) results from Testgraf98 (ISF and confidence envelopes); and (c) results from Multilog7.0 (GRM)
Next, TestGraf98 was used to investigate assumption M for the ISFs. Several sample sizes were used for estimating curve fragments of the ISFs and balancing the bias-accuracy trade-off. Figure 5b shows the estimated ISF of Item 3 (‘Distraction due to pain?’). The confidence envelopes show that the local decrease of the estimated ISF can be ignored safely. For the estimated ISF of Item 7 (‘Being able to concentrate?’), Fig. 6b suggests a violation of assumption M for high latent variable levels (i.e., θ > 2). With MSP the user specifies the minimum number of observations used for estimating each point of an ISRF, but Testgraf98 controls the bias-accuracy trade-off by means of a bandwidth parameter, also to be specified by the user but without being able to control the number of observations for estimating separate curve fragments. The effect may be that, in particular at the lower end and higher end of the scale, the ISF is estimated very inaccurately. Combining the results from MSP (no significant decreases of the ISRFs at the higher end of the θ scale) and Testgraf98 (a smooth monotone increasing ISF in the middle of the θ scale), we may conclude that assumption M holds for item scores. Thus, the expected item score monotonically increases in the latent variable.
Comparison of the MHM with the GRM
The GRM was fitted using Multilog7.0 [41]. Estimation problems occurred for item scoring 0–4 because some score categories were (almost) empty. This was resolved by combining scores 0 and 1 into score 0, and re-scoring the items 0–3. Table 4 provides the estimated slope parameters (αj) and the three location parameters (δjm, j = 1, 2, 3), and also the estimated item Hj coefficients (also in Table 3, last column) and the nonparametric location parameters (πjm, j = 1, 2, 3). The location parameters indicate that the items are relatively popular (highly endorsed).Table 4Results of monotone homogeneity model (MHM) scale analysis and estimated item parameters from the graded response model (GRM)MHMGRM bjHjπj1πj2πj3πj4αjδj1δj2δj3Physical health and well-being3Distraction due to painb.40.99.93.72.401.14−2.71−0.990.4910Experiencing energya.46.98.95.79.451.97−2.24−0.630.5615Satisfied with sleep.28.99.95.70.340.85−2.09−0.791.7825Moving around well.41.99.96.91.611.23−3.08−2.24−0.4116Satisfied doing daily activities.52.98.93.72.213.96−1.59−0.580.874Need medical treatment for daily functioninga.43.98.92.75.231.23−2.82−1.290.2417Satisfied work capacity.52.99.98.77.203.58−1.56−0.690.81Scale value.43Psychological health and well-being5Enjoying life.371.00.98.61.071.93−2.79−0.381.977Being able to concentrate.29.99.98.77.200.82−4.270.681.6918Satisfied with yourself.451.00.93.64.151.99−2.85−0.971.1211Acceptance physical appearance.35.99.98.80.451.08−3.93−1.560.2226Experiencing negative feelingsa.341.00.97.61.081.11−2.84−0.601.906Life meaningful.371.00.96.62.231.90−2.71−0.351.94Scale value.36Social relations19Satisfied relationship with other people.50.99.96.82.292.52−2.21−1.100.9720Satisfied with sex life.40.97.88.58.151.32−3.19−0.901.3821Satisfied support from others.42.99.97.72.201.38−1.88−0.341.62Scale value.44Environment8Feeling safe in daily life.331.00.98.78.321.12−4.12−1.370.8422Satisfied living conditions.43.98.90.68.341.96−2.71−1.270.6312Enough financial resources.42.99.95.70.441.84−2.22−0.730.2123Satisfied getting adequate health care.36.99.97.68.231.32−2.84−0.931.3313Availability information needed in daily life.40.99.95.72.211.64−3.21−0.980.6014Opportunities leisure.39.99.98.75.341.61−1.94−0.690.609Healthy environment.31.99.98.83.321.02−3.95−0.891.4324Satisfied with transport in daily life.40.99.97.86.451.66−2.77−1.550.17Scale value.38a Reversely scored items; b For the GRM analysis, items were recoded by collapsing item scores 0 and 1 into item score 1
Several methods are available for assessing the fit of the GRM, but many are problematic and no generally accepted standard goodness-of-fit method for the GRM is presently available ([42] pp. 85–89). We investigated goodness-of-fit of the GRM by means of posterior predictive assessment [43], which can provide graphical and numerical evidence about model fit. Model fit was investigated at the level of items. The most interesting results were found for items from the physical and psychological domains. Results are only discussed for these domains. For the physical domain, Item 3 (‘Distraction due to pain?’) and Item 7 (‘Being able to concentrate’) showed significant misfit (P < .01). In Fig. 5c, the curve made up by dots connected by straight line pieces represents the estimated nonparametric ISF of Item 3, the solid curve represents the expected ISF of Item 3 under the GRM, and the dotted curves represent the 95% confidence envelopes. The GRM rejects this item. Thus, modeling the jagged pattern of the estimated ISF by means of logistic ISRFs having the same slopes would do injustice to the data. However, it is noteworthy that Item 3 has good measurement properties (Table 4: e.g., H3 = .40) under the more general MHM, and from this model’s perspective it might be retained in the scale.
Item 7 (‘Being able to concentrate?’) was a popular item (Table 4; πj1 = .99, πj2 = .98; also, see Fig. 6a, upper two ISRFs). As a result, the GRM could not be estimated accurately; item parameters were estimated very inaccurately (standard errors >.25). Figure 6c shows evidence of misfit at θ > 2, for which the observed ISF fell outside the 95% confidence interval. This means that the GRM gives biased results for θ > 2. The nonparametric estimates of the ISRFs were monotone. This provides evidence that the MHM adequately fitted Item 7. However, H7 = .29, which is rather low. A reason to keep this item in the scale is that it may help measuring differences at the lower and middle ranges of the θ scale, which are the most relevant ranges for measuring HRQoL.
Summary of the scale properties
The WHOQOL-Bref is most often used in scientific research (e.g., epidemiological studies and clinical trails) and by health professionals (e.g., to assess treatment efficacy) [19]. The nonparametric MHM analyses revealed that the scales have adequate properties for comparing groups on the underlying HRQoL aspects. Each of the four domains of the WHOQOL-Bref constitutes a unidimensional scale, each scale measuring a different aspect of HRQoL in addition to a weak common HRQoL attribute. This justifies reporting both separate domain scores and possibly an overall HRQoL score. The scalability results showed that the domain scales are weak to moderate, with scalability coefficients H ranging from .36 to .44. The test-score reliabilities of the four domain scores were .82, .76, .66, and .81, respectively. The rank correlations between sum score X+ and the estimated θ from the GRM varied from .91 (‘physical health’) to .96 (‘environment’) (Pearson correlations ranged from .94 (‘physical health’) to .99 (‘environment’)). Thus, X+ and estimated θ carry nearly the same rank order (and numerical) information. This interesting result further justifies the use of the nonparametric MHM for scale analysis, and the use of X+ for (at least) ordinal measurement of persons.
Because the item-score distributions were severely skewed to the left, the lower response categories 0 and 1 were ineffective for HRQoL measurement in the general population. The locations of the ISRFs for the higher response categories 2, 3, and 4 were well spread along the θ scale. The ISRFs’ discrimination power as reflected by the Hj values often was in the weak to medium range. Thus, the higher response categories are modestly informative across a wide range on the θ scale.
The relatively short WHOQOL-Bref may also be considered for use as a tool for assessing HRQoL at the individual level in clinical and medical settings. For example, the WHOQOL-Bref may be used to evaluate whether a patient’s HRQoL has improved after taking medication. An interesting feature of a fitting IRT model is that psychometric properties can be evaluated conditionally on the latent variable. For example, the measurement error of X+ can be evaluated at different values of the latent variable. TestGraf98 provides graphical information about the standard error of measurement based on the MHM model. For example, for the physical-health domain Testgraf98 estimated a standard error of measurement ranging from 2.8 for X+ ≤ 20 to 1.8 for X+ ≥ 28. Thus, to be significant at the 5% significance level differences between two observed X+ scores have to be larger than  for X+ ≤ 20, and larger than  for X+ ≥ 28 (e.g., see [44], p. 209). This is a substantial standard error of measurement relative to the length of scale. This relatively large measurement error appears to be consistent with the observed Hj values, which indicate weak to moderate scalability. For the other three content domains, the standard error of measurement was also substantial. Thus, caution has to be exercised when drawing conclusions about differences and changes in individual levels of HRQoL based on observed X+ scores from the WHOQOL-Bref and any other HRQOL measure—see [45].
Discussion
This study explained how the nonparametric monotone homogeneity model contributes to the construction of scales for the measurement of HRQoL. The MHM is more general than parametric IRT models [24], such as the much-used parametric graded response model [26] but also the partial credit model [13–15] and the generalized partial credit model [11]. Hemker et al. [46] showed that all known parametric IRT models for polytomous items are special cases of the nonparametric MHM. This means that any item set satisfying the requirements of a parametric IRT model for polytomous items also satisfies the requirements of the nonparametric MHM. Given the greater generality and flexibility of the nonparametric MHM, which results in longer scales, and because X+ and estimated θ carry the same rank order information (based on the approximate stochastic ordering property of θ given X+), the nonparametric MHM is highly suited for person measurement.
In an HRQoL context, often little is known about the psychometric properties of new questionnaires. A typical nonparametric MHM analysis explores the dimensionality of the data by capitalizing on model assumptions such as monotonicity (MSP), and studies the shapes of the ISRFs and the ISFs in order to learn more about the (mal-)functioning of individual items (MSP and TestGraf98). This results in scales on which groups can be compared and changes monitored without making unduly restrictive assumptions about the data.
The properties of any IRT model only hold for the application at hand when the model fits the data. In case of misfit, the structure of the model does not match the structure of the data. One cause of misfit is that the data are multidimensional while the model assumes unidimensionality. Another cause of misfit is that the real ISRFs may not be monotone or that they are monotone but fail to have the logistic shape assumed by many parametric models. Other causes of misfit, such as a multiple-group structure as in differential item functioning (e.g., [47]) or person misfit [48] were not considered here.
When the MHM fits the data, the researcher may decide to also investigate goodness of fit of the GRM or other parametric IRT models for polytomous items. The choice of a parametric model may be based on the flexibility of the model. For example, the partial credit model only has item location parameters but assumes the slopes of the response functions to be the same within and between items, whereas the generalized partial credit model also allows for varying slope parameters between items, just as the GRM. If one pursues a parametric IRT analysis, misfit may be a good reason to resort to a nonparametric IRT model and still have an ordinal patient scale. If CAT is pursued, one of the parametric models is a better option provided the model fits the data well. In an HRQoL context, CAT indeed could prove to be successful because patients have a definitive interest in providing truthful answers (in the educational context, in which CAT originated, CAT requires that items be kept secret. This requires item banks often containing hundreds of calibrated items). As a result, in HRQoL measurement CAT presently meets with a growing interest (e.g., [49–54]).