A Proportional Odds / Hazards Approach to Analyzing Likert Scale Data

Linear scoring coupled with parametric regression or ANOVA often is used to analyze responses measured on a Likert scale. Alternatively, we have had success in numerous cases by modeling the same data under relaxed assumptions of the proportional odds model. This is implemented conveniently by procedure LOGISTIC, where a test of the proportional odds parallelism assumption is displayed automatically. When this test fails, a satisfactory model sometimes results by switching from the default cumulative logit link to use of the complementary loglog link, implemented by option LINK=CLOGLOG. Model fit by means of the log-log link also can be assessed using the CLOGLOG option, after minimal data manipulation. Either link induces the equivalent of a proportional hazards model. Interpretation of proportional odds and hazards models in the context of a Likert scale response is illustrated using data obtained from a survey of 849 teachers and administrators involved in implementing a new teaching program in their schools. Of 8 items relating to the perceived degree of implementation of the program, the proportional hazards model represents responses for seven items very well. This paper is directed at survey specialists who employ Likert scale instruments and who are familiar with logistic regression. INTRODUCTION Recently, our assistance was requested to aid in the analysis of a survey in which each of eight Likert-scale questions was considered a response variable. Because there were only four levels of the scale, we did not want to treat it as linear, nor did we want to lose the ordinal nature of the data by treating it as strictly nominal. This led us to use the proportional odds model available in SAS procedure LOGISTIC. However, when we fitted separate models to the eight questions, the proportional odds assumption, i.e., parallelism assumption, was met in only one case. As a consequence, we decided to try the complementary log-log (CLL) link, which, as Agresti (1984) pointed out, is equivalent to fitting a proportional hazards model. This proved successful for seven of the eight questions. Since these appear to be new innovations in the analysis of Likert-scale data, we present an example where the parallelism assumption failed using the logit link but did not fail with the CLL link, and we will also present an example where the parallelism assumption did not fail with either link. TEACHING PROGRAM DATA Data were collected from surveys given to teachers and administrators in schools that had implemented a new teaching program. The survey asked eight questions about components of the program. The teachers rated the degree of desirability on a 1 to 4 Likert scale (1=undesirable and 4=highly desirable), and they also rated the degree of present implementation on the same scale. The number of years the teacher had been involved in the program was also asked in the survey. This paper will focus on N=804 teacher’s responses excluding responses from administrators. The question of interest was: Does the perception of degree of present implementation of each of the eight components vary significantly between years of implementation (1,2,3+) or among degree of desirability (1 to 4)? Note: Because of the low counts, levels 1 and 2 from the Likert-scale were combined as 2. METHOD Initially, each component was analyzed with a traditional proportional odds model with logit link. However, since it gave an adequate fit in only one case, we were forced to use a more complex proportional hazards model based on a complementary log-log link. In this model, the response was the answer to the degree of implementation, and the explanatory variables were years of implementation (categorical), degree of desirability (linear), and their interaction. In the case where the parallelism assumption failed for both links, logistic regression was used with the response defined to be 0 if the answer was 1 or 2, i.e., “failure” in some sense, and 1 if the answer was 3 or 4, corresponding to “success”. For all types of models, the initial model allowed tests of significance of the main effects and the interaction. If the interaction was non-significant at α=0.05, it was removed from the model. COMPARISON OF PROPORTIONAL ODDS AND HAZARDS MODELS When modeling ordinal response data, either the proportional odds model or the proportional hazards model could be used. The proportional odds model is used most frequently, probably because it is a natural extension of the logistic regression model from binary response to ordinal response with more than two categories. However, we have found that when the parallelism assumption fails for the proportional odds model, it is sometimes met by using the proportional hazards model. The two models are shown below, and Table 1 shows a brief comparison of the models. PROPORTIONAL ODDS MODEL WITH LOGIT LINK For three levels of response with answers 1 and 2 combined, the proportional odds model requires two equations, namely ) ' e 1 1 ] | 2 I P[Q ] | 2 I i P[Q 1 ] | 2 I i P[Q ln ]) | 2 I Logit(P[Q