# R Programming/Linear Models

R Programming
* R Basics
** Introduction
** Sample Session
** Manage your workspace
** Settings and Installation
** Documentation
** Control Structures
** Working with functions
** Debugging
** Using C or Fortran
** Utilities
** Estimation utilities
** Packages
* Data Management
** Data types
** Working with data frames
** Importing and exporting data
** Text Processing
** Times and Dates
* Graphics
** Grammar of graphics
* Publication quality output
* Descriptive Statistics
* Mathematics
** Optimization
** Probability Distributions
** Random Number Generation
* Statistical Core Methods
** Maximum Likelihood
** Method of Moments
** Bayesian Methods
** Bootstrap
** Multiple Imputation
** Nonparametric Methods
* Regression Models
** Linear Models
** Quantile Regression
** Binomial Models
** Multinomial Models
** Tobit And Selection Models
** Count Data Models
** Duration Analysis
* Time Series
* Factor Analysis
* Classification
** Ordination
** Clustering
* Network Analysis
* High Performance Computing
** Profiling R code
** Parallel computing with R
* Appendix
** Sources
** Index

## Standard linear model

In this section we present estimation functions for the standard linear model estimated by ordinary least squares (OLS). Heteroskedasticity and endogeneity are treated below. The main estimation function is lm.

### Fake data simulations

We first generate a fake dataset such that there is no hetereoskedasticity, no endogeneity and no correlation between the error terms. Therefore the ordinary least square estimator is unbiased and efficient. We choose a model with two variables and take all the coefficients equal to one.
$y_{i}=1+x_{1,i}+x_{2,i}+u_{i}$

### Least squares estimation

* The standard function to estimate a simple linear model is lm.
* lsfit performs the least square procedure but the output is not formatted in fashionable way.
* ols (Design) is another alternative.
We estimate the model using lm. We store the results in fit and print the result using summary which is the standard function.
There are some alternative to display the results.
* display in the arm package is one of them.
* coefplot (arm) graphs the estimated coefficients with confidence intervals. This is a good way to present the results.
* mtable in the memisc package can display the results of a set of regressions in the same table.
fit is a list of objects. You can see the list of these objects by typing names(fit). We can also apply functions to fit.
We can get the estimated coefficients using fit$coeff or coef(fit).
se.coef (arm) returns the standard error of the estimated coefficients.
The vector of fitted values can be returned via fit$fitted, fitted(fit) or the predict function. The predict function also returns standard error and confidence intervals for predictions.
The vector of residuals:
The number of degrees of freedom :

#### Confidence intervals

We can get the confidence intervals using confint or conf.intervals in the alr3 package.

#### Tests

coeftest (lmtest) performs the Student t test and z test on coefficients.
linear.hypothesis (car) performs a finite sample F test on a linear hypothesis or an asymptotic Wald test using $\mathrm {X} ^{2}$ statistics.
See also waldtest (lmtest) for nested models.

#### Analysis of variance

We can also make an analysis of variance using anova.

#### Model Search and information criteria

The stats4 package includes AIC and BIC function:
The step functions performs a model search using the Akaike Information Criteria.

#### Zelig

* The method is also supported in Zelig

### Bayesian estimation

* MCMCregress (MCMCpack)
* BLR (BLR)

## Heteroskedasticity

* See the lmtest and sandwich packages.
* gls (nlme) computes the generalized least squares estimator.
* See "Cluster-robust standard errors using R" (pdf) by Mahmood Arai. He suggests two functions for cluster robust standard errors. clx allow for one-way clustering and mclx for two-way clustering. They can be loaded with the following command source("http://people.su.se/~ma/clmclx.R").

## Robustness

Cook's distance
Influence plot:
Leverage plots:
Bonferroni's outlier test:
See also outlier.t.test in the alr3 package.
* inf.index in the alr3 package computes all the robustness statistics (Cook's distance, studentized residuals, outlier test, etc)
* rlm performs a robust estimation
* See UCLA example
* See also the robustbase package

## Instrumental Variables

* ivreg in the AER package
* tsls in the sem package.
* It is also possible to use the gmm command in the gmm package. See Methods of moments for an example.

### Fake data simulations

We first simulate a fake data set with x correlated to u, z and u independent and x correlated with z. Thus x is an endogenous explanatory variable of y and z is a valid instrument for x.

### Two stage least squares

Then we estimate the model with OLS (lm) and IV using z as an instrument for x.
We plot the results :

## Panel Data

plm (plm) implements the standard random effect, fixed effect, first differences methods. It is similar to Stata's xtreg command.
Note that plm output are not compatible with xtable and mtable for publication quality output.
* lme4 and gee implements random effect and multilevel models.
* See also BayesPanel

### Random effects model

To implement a random effects model we generate a fake data set with 1000 observations over 5 time periods.
We estimate the random effect model with the plm function and the model = "random" option.

### Fixed effects model

For a fixed effects model we generate a fake dataset and we correlate the fixed effects f with covariates :
We first transform our data in a plm data frame using plm.data. We estimate the fixed model using plm with model = "within" as an option. Then, we compare the estimate with the random effect model and perform an Hausman test. At the end, we plot the density of the fixed effects.

### Dynamic panel data

* pgmm (plm) implements the Arellano Bond estimation procedure. It is similar to xtabond2 in Stata.

## Simultaneous equations model

For a [:w:Simultaneous_equations_model|simultaneous equations model] the following packages are needed :
* sem package
* systemfit package

## References

* ↑ Christian Kleiber and Achim Zeileis (2008). Applied Econometrics with R. New York: Springer-Verlag. ISBN 978-0-387-77316-2. URL http://CRAN.R-project.org/package=AER
* ↑ Yves Croissant, Giovanni Millo (2008). Panel Data Econometrics in R: The plm Package. Journal of Statistical Software 27(2). URL http://www.jstatsoft.org/v27/i02/.
* ↑ M Arellano, S Bond "Some tests of specification for panel data: Monte Carlo evidence and an application to employment equations" - The Review of Economic Studies, 1991
* ↑ David Roodman, XTABOND2: Stata module to extend xtabond dynamic panel data estimator, http://ideas.repec.org/c/boc/bocode/s435901.html

## External links

* Visualization of regression coefficients