Rasch Analysis of the Ocular Surface Disease Index (OSDI)

The Ocular Surface Disease Index (OSDI) is perhaps the most frequently used survey instrument for the assessment of ocular surface disease severity in dry eye research. It has been used in a variety of studies, both in the assessment of the severity of dry eye associated with certain conditions1–3 and in the investigation of the efficacy of dry eye treatments.4–7 The instrument, introduced in 1997 by the Outcomes Research Group (Allergan Inc., Irvine, CA),8 consists of 12 items that assess symptoms, functional limitations, and environmental factors related to dry eye. Each item has the same five-category Likert-type response option, and each of the three subscales has its own question type. Initial investigations of the reliability of the OSDI were conducted using Classical Test Theory methods, that is, using the Cronbach's α statistic to assess the internal consistency of the items.9 There are several excellent references on Classical Test Theory and its use in survey research.10,11 Schiffman et al.9 found that Cronbach's α for the OSDI was 0.92, and a factor analysis revealed three subscales (symptoms, environmental triggers, and vision-related function). They reported acceptable test–retest repeatability, but found that OSDI scores did not correlate particularly well with clinical tests for dry eye. Specifically, correlations for all subjects between OSDI score and tear break-up time, Schirmer's test, lissamine green, and fluorescein staining in the worse eye ranged from −0.21 to +0.19 and none of the correlations was statistically significant. The use of Classical Test Theory to score survey instruments and evaluate their reliability has been criticized for several reasons. One of these is the treatment of Likert-type survey data as continuous, rather than ordinal. Another is the assignment of equal weight to each survey item in the calculation of an overall score, when one could imagine situations in which items may require different levels of the underlying trait for endorsement. For these reasons, the common method of generating an overall score from an instrument—summing and averaging the ordinal-level responses—is open to criticism. Also, the approach of using Cronbach's α to assess reliability does not provide information regarding the behavior of individual survey items, only the instrument as a whole. A family of models known collectively as Item Response Theory (IRT) provides an alternative approach to the scoring and evaluation of survey instruments. The models have roots in education and aptitude testing, and seek to quantify the amount of some latent trait using survey responses. Rasch analysis, also sometimes referred to as a one-parameter logistic model, is often considered as the simplest of the IRT models.12,13 Rasch analysis was initially developed for educational testing, but is now frequently used in healthcare research to evaluate and score survey instruments.14–18 It provides interval-level data from survey responses, including estimates of the amount of the latent trait displayed by each subject (“person measure”) and the amount of the trait necessary to respond in a certain way to each item (“item measure”).12 Moreover, Rasch analysis also provides item fit statistics that indicate whether the individual item is contributing to the measurement of the latent trait. There are excellent descriptions of Rasch analysis and its use with survey instruments published elsewhere.12,19–21 Other IRT models include parameters in addition to the item difficulty parameter used in Rasch analysis. For instance, some models include an item discrimination parameter that allows for differences in the slope of the curve describing response probabilities for individual items.22 Rasch models are restrictive in the sense that data that do not conform to the model are not considered to be consistent with measurement, and the curves describing response probabilities are not free to vary between items. Other IRT models seek to describe the data as best as possible using extra parameters such as item discrimination. Massof23 published a study in which he compared a Rasch model to a two-parameter logistic IRT model (the Muraki model) using data from visual functioning questionnaires. He demonstrated that the item discrimination parameter of the Muraki model was inversely proportional to the item fit statistics of the Rasch model. There has been some recent work in the area of evaluation of dry eye survey instruments with Rasch analysis. Gothwal et al.14 examined the measurement properties of the McMonnies questionnaire using Rasch analysis. They found that person separation was inadequate for discriminating between more than two strata of dry eye severity and, therefore, the McMonnies questionnaire did not function as a valid measure to discriminate across disease severity. Johnson and Murphy24 developed the Ocular Comfort Index (OCI) to measure ocular surface disease symptoms using Rasch analysis. The instrument they developed has 12 items and a seven-category response structure. Person separation was good and all 12 of the final items had adequate Rasch fit statistics. Simpson et al.25 evaluated the Dry Eye Questionnaire, the McMonnies questionnaire, and the OSDI. One purpose of this study was to evaluate the Rasch item fit statistics of the instruments and use them to determine whether the surveys were unidimensional. For the OSDI, the authors found that all items had fit statistics within the acceptable range. Other aspects of the analysis, such as the functioning of the category structure and person separation statistics, were not reported. Pesudovs and Noble26 evaluated a single-item faces scale for measuring pain associated with severe ocular surface disease. They applied Rasch analysis to refine the category structure of the instrument. The study also used the scale to demonstrate the potential of Rasch analysis to increase sensitivity to changes after treatment for ocular surface disease, finding an increased effect size with Rasch analysis compared with conventional raw scores.26 In light of this, we hypothesize that the application of Rasch analysis to responses to the OSDI from patients with dry eye disease might be beneficial in further understanding its psychometric properties. Thus, the purpose of this study was to investigate the OSDI using Rasch analysis in a sample of females 50 years of age and older who were participating in a study of dry eye in postmenopause and who had been previously diagnosed with dry eye or reported significant ocular dryness and irritation.