AS 03 Collinearity in Mixed Models

Strong correlations among the independent variables in a model can lead to large standard errors for the estimates. There are well documented methods for evaluating the impact of collinearity in linear regression models but these methods are geared to models with continuous variables only. As an alternative, the R package “perturb” can be used to evaluate the impact of collinearity by adding random noise (perturbations) to selected variables. This method can deal with transformations, interaction effects, categorical variables. The perturb package has now been enhanced to work with the “nlme” and “lme4” packages for mixed models in R. This makes it suitable for collinearity in mixed models with a random slope variable or sparse categories in a levels variable. INTRODUCTION Collinearity occurs when two or more independent variables in a model are strongly correlated. This need not be a problem given sufficient data but it can result in large standard errors for the estimates. This makes sense intuitively, an attempt is made to estimate the separate effects of two variables that are nearly the same. This cannot be done with a high degree of certainty unless there is a good deal of information (i.e. a large dataset). Large standard errors can mean unstable results, where a small modification to the data would lead to substantially different inferences. There are two main methods for evaluating collinearity in linear regression models. Most software packages can calculate VIF (Variance InFlation) statistics that indicate how much larger the standard error of an estimate would be due to collinearity. Conditioning indices are a second option and have the advantage that they can detect collinearity among a set of variables. Dorman et al. [2] give a more complete overview of methods for detecting and for dealing with collinearity. These methods are geared to multiple linear regression with continuous independent variables. They are less suitable if a model includes transformed variables or interaction effects. These derived variables are treated in the model as regular independent variables. This does not do justice to the underlying problem, that a small change to the untransformed variable, or to the variables comprising the interaction effect, could affect the model results. This is the rationale for the perturb package in R. This program evaluates collinearity by adding small random values (perturbations) to specified variables in a model, re-estimating the model, and repeating this process e.g. 100 times. The model estimates can then be evaluated to determine how much these random perturbations have affected the results. The perturbances are applied to the untransformed variables or to the variables included in an interaction effect, making it possible to evaluate how small changes are propagated to derived model terms. This principle can be extended to categorical variables where a probability of misclassification is specified. The perturb package can also be used in generalized linear models such as logistic regression. The latest version of perturb can also be used in mixed models estimated by the nlme or lme4 packages. This can take into account that a variable for a random slope is correlated with other variables in the mixed model. Another application that comes to mind is evaluating the impact of changes to a levels variable such as the site where data was collected. The site variable often contains small categories, applying random reclassifications can indicate how robust the results actually are. ADDING PERTURBANCES The perturb package evaluates collinearity by adding small, normally distributed random values to selected variables in a model. The SD of the normal distribution can be specified to control the magnitude of these perturbations. After adding the perturbances to the original variable, any derived variables are re-calculated. For example, the perturbance would be added to the linear term of a variable, then the squared term would be calculated from the modified variable. The perturbances thus indirectly affect model terms such as transformations or interaction effects that are derived from those variables. The model is re-estimated for a specified number of iterations (default 100), adding perturbations and deriving model terms at each iteration. Model coefficients are collected at each iteration, then summarised using descriptive statistics in order to evaluate the stability of the coefficients under the changes applied to the data. PhUSE EU Connect 2018 2 This approach has the advantage that it provides concrete information on how strongly model results could be affected by changes of the magnitude selected. The basic principles apply to any regression-like model, including generalised linear models and mixed models. On the other hand, it can be difficult to choose an appropriate magnitude for the perturbations. For some variables, a certain amount of measurement error can be reasonable to expect. In other cases, increasing magnitudes for the perturbations could be used to show that model outcomes are stable unless strong changes to the data were to occur. PERTUBANCES FOR CATEGORICAL VARIABLES The perturb package extends this principle to categorical variables. Although it may not be self-evident, sparse categories or categories nearing 100% can be can be seen as a collinearity problem. Note that a categorical variable can be modelled by creating a set of dummy variables, one for each category except the reference category. Each dummy variable has the value of 1 for the category it belongs to and 0 otherwise. A dummy variable for a sparse category will consist almost entirely of zeros, which means a high negative correlation with the intercept. A dummy variable for a category near 100% will consist almost entirely of ones and will have a high positive correlation with the intercept. Perturb assess the stability of coefficients for categorical variables by randomly reclassifying a certain percentage of cases at each iteration. This is the equivalent to adding small random numbers to continuous variables. However, the reclassification procedure in perturb is not as straightforward as this sounds. The reason is that as such, reclassifying will cause small categories to increase in relative size while large categories will shrink. Table 1 provides an example of reclassification probabilities with a 90% chance for each case to remain in the same category and a 5% chance tor the case to be moved to one of the two other categories. Table 1: Reclassification probabilities Reclassified Original a b c a 0.90 0.05 0.05 b 0.05 0.90 0.05 c 0.05 0.05 0.90 Table 2 shows the expected frequency distribution of a reclassified variable with frequencies 10, 70 and 20 for categories a, b, and c respectively. On the whole, reclassifying this variable would the largest category to shrink and the other two categories to increase. Table 2: Example of a reclassified variable Reclassified Original a b c total a 9 0.5 0.5 10 b 3.5 63 3.5 70 c 1 1 18 20 total 13.5 64.5 22 100 There is a method to derive reclassification probabilities that will not affect the relative size of categories. Table 3 contains reclassification probabilities that will produce the same expected frequencies for the original and reclassified categories. Table 4 shows the expected frequencies using these reclassification probabilities. The totals of the original and reclassified variable are now identical. Table 3: Reclassification probabilities for the categories in Table 2 Reclassified Original a b c a 0.854 0.096 0.050 b 0.014 0.966 0.020 c 0.025 0.070 0.905 PhUSE EU Connect 2018 3 Table 4: Expected frequencies using the reclassification probabilities from Table 3 Reclassified Original a b c total a 8.54 0.96 0.50 10 b 0.96 67.64 1.40 70 c 0.50 1.40 18.10 20 total 10 70 20 100 The reclassification probabilities are derived using loglinear models for square tables. These models are based on patterns of association, e.g. Table 3 uses a model that allows a certain odds of staying in the same category versus moving to a different category (the diagonal of the table). For movers, the odds of moving are the same for all categories. But for a given frequency distribution, the expected frequencies of the reclassified variable are the same as for the original variable. The perturb program supports many loglinear models for square tables. Some of these models are suitable for ordered categories, for example to make small changes in category number more likely than large changes. This reclassification method ensures that the expected frequency distribution of the reclassified variable is as close as possible to the original variable. This corresponds with the method used for continuous variables, where the random perturbations have a mean of 0. It would also be undesirable to have large categories shrink and small categories increase. That would increase the likelihood of stable results whereas the interest is whether results might become less stable. PERTURBANCES IN MIXED MODELS A mixed model expands on the linear regression model by allowing random intercepts, random slopes, a covariance structure among repeated measurements. A random intercept, possibly together with a random slope, can take the intraclass correlation into account within certain levels in which the population can be classified. For example, measurements within research sites could be relatively similar due to certain procedures or instruments. The resulting intraclass correlation can result in inefficient estimators (the standard errors are too large). The correlation among repeated measures can also lead to inefficient estimators. Mixed models are more flexible but also more complex than linear regression. Still, the basic principles of a perturbation analysis can be extended to mixed models in a straightforward fashion. Random perturbations can be added to a (continuous) variable tha