Fixed effects model

In statistics, a fixed effects model is a statistical model in which the model parameters are fixed or non-random quantities. This is in contrast to random effects models and mixed models in which all or some of the model parameters are random variables. In many applications including econometrics and biostatistics a fixed effects model refers to a regression model in which the group means are fixed (non-random) as opposed to a random effects model in which the group means are a random sample from a population. Generally, data can be grouped according to several observed factors. The group means could be modeled as fixed or random effects for each grouping. In a fixed effects model each group mean is a group-specific fixed quantity.

In panel data where longitudinal observations exist for the same subject, fixed effects represent the subject-specific means. In panel data analysis the term fixed effects estimator (also known as the within estimator) is used to refer to an estimator for the coefficients in the regression model including those fixed effects (one time-invariant intercept for each subject).

Qualitative description

Such models assist in controlling for omitted variable bias due to unobserved heterogeneity when this heterogeneity is constant over time. This heterogeneity can be removed from the data through differencing, for example by subtracting the group-level average over time, or by taking a first difference which will remove any time invariant components of the model.

There are two common assumptions made about the individual specific effect: the random effects assumption and the fixed effects assumption. The random effects assumption is that the individual-specific effects are uncorrelated with the independent variables. The fixed effect assumption is that the individual-specific effects are correlated with the independent variables. If the random effects assumption holds, the random effects estimator is more efficient than the fixed effects estimator. However, if this assumption does not hold, the random effects estimator is not consistent. The Durbin–Wu–Hausman test is often used to discriminate between the fixed and the random effects models.

Formal model and assumptions

Consider the linear unobserved effects model for  observations and  time periods:
 for  and 

Where:
  is the dependent variable observed for individual  at time .
  is the time-variant  (the number of independent variables) regressor vector.
  is the  matrix of parameters.
  is the unobserved time-invariant individual effect. For example, the innate ability for individuals or historical and institutional factors for countries.
  is the error term.

Unlike ,  cannot be directly observed.

Unlike the random effects model where the unobserved  is independent of  for all , the fixed effects (FE) model allows  to be correlated with the regressor matrix . Strict exogeneity with respect to the idiosyncratic error term  is still required.

Statistical estimation

Fixed effects estimator 
Since  is not observable, it cannot be directly controlled for. The FE model eliminates  by de-meaning the variables using the within transformation:

where , , and .

Since  is constant,  and hence the effect is eliminated. The FE estimator  is then obtained by an OLS regression of  on .

At least three alternatives to the within transformation exist with variations.

One is to add a dummy variable for each individual  (omitting the first individual because of multicollinearity). This is numerically, but not computationally, equivalent to the fixed effect model and only works if the sum of the number of series and the number of global parameters is smaller than the number of observations. The dummy variable approach is particularly demanding with respect to computer memory usage and it is not recommended for problems larger than the available RAM, and the applied program compilation, can accommodate.

Second alternative is to use consecutive reiterations approach to local and global estimations. This approach is very suitable for low memory systems on which it is much more computationally efficient than the dummy variable approach.

The third approach is a nested estimation whereby the local estimation for individual series is programmed in as a part of the model definition. This approach is the most computationally and memory efficient, but it requires proficient programming skills and access to the model programming code; although, it can be programmed including in SAS.

Finally, each of the above alternatives can be improved if the series-specific estimation is linear (within a nonlinear model), in which case the direct linear solution for individual series can be programmed in as part of the nonlinear model definition.

First difference estimator 

An alternative to the within transformation is the first difference transformation, which produces a different estimator. For :

The FD estimator  is then obtained by an OLS regression of  on .

When , the first difference and fixed effects estimators are numerically equivalent. For , they are not. If the error terms  are homoskedastic with no serial correlation, the fixed effects estimator is more efficient than the first difference estimator. If  follows a random walk, however, the first difference estimator is more efficient.

Equality of fixed effects and first difference estimators when T=2 
For the special two period case (), the fixed effects (FE) estimator and the first difference (FD) estimator are numerically equivalent. This is because the FE estimator effectively "doubles the data set" used in the FD estimator. To see this, establish that the fixed effects estimator is:

Since each  can be re-written as , we'll re-write the line as:

Chamberlain method

Gary Chamberlain's method, a generalization of the within estimator, replaces  with its linear projection onto the explanatory variables. Writing the linear projection as:

this results in the following equation:

which can be estimated by minimum distance estimation.

Hausman–Taylor method

Need to have more than one time-variant regressor () and time-invariant
regressor () and at least one  and one  that are uncorrelated with
.

Partition the  and  variables such that  where  and  are uncorrelated with . Need .

Estimating  via OLS on  using  and  as instruments yields a consistent estimate.

Generalization with input uncertainty 

When there is input uncertainty for the  data, , then the  value, rather than the sum of squared residuals, should be minimized. This can be directly achieved from substitution rules:
,

then the values and standard deviations for  and  can be determined via classical ordinary least squares analysis and variance-covariance matrix.

Use to test for consistency 

Random effects estimators may be inconsistent sometimes in the long time series limit, if the random effects are misspecified (i.e. the model chosen for the random effects is incorrect). However, the fixed effects model may still be consistent in some situations. For example, if the time series being modeled is not stationary, random effects models assuming stationarity may not be consistent in the long-series limit. One example of this is if the time series has an upward trend. Then, as the series becomes longer, the model revises estimates for the mean of earlier periods upwards, giving increasingly biased predictions of coefficients. However, a model with fixed time effects does not pool information across time, and as a result earlier estimates will not be affected.

In situations like these where the fixed effects model is known to be consistent, the Durbin-Wu-Hausman test can be used to test whether the random effects model chosen is consistent. If  is true, both  and  are consistent, but only  is efficient. If  is true the consistency of  cannot be guaranteed.

See also
 Random effects model
 Mixed model
 Dynamic unobserved effects model

 Fixed-effect Poisson model

Notes

References

External links
Fixed and random effects models
  Examples of all ANOVA and ANCOVA models with up to three treatment factors, including randomized block, split plot, repeated measures, and Latin squares, and their analysis in R

Analysis of variance
Regression models