# R Programming/Binomial Models

R Programming
* R Basics
** Introduction
** Sample Session
** Manage your workspace
** Settings and Installation
** Documentation
** Control Structures
** Working with functions
** Debugging
** Using C or Fortran
** Utilities
** Estimation utilities
** Packages
* Data Management
** Data types
** Working with data frames
** Importing and exporting data
** Text Processing
** Times and Dates
* Graphics
** Grammar of graphics
* Publication quality output
* Descriptive Statistics
* Mathematics
** Optimization
** Probability Distributions
** Random Number Generation
* Statistical Core Methods
** Maximum Likelihood
** Method of Moments
** Bayesian Methods
** Bootstrap
** Multiple Imputation
** Nonparametric Methods
* Regression Models
** Linear Models
** Quantile Regression
** Binomial Models
** Multinomial Models
** Tobit And Selection Models
** Count Data Models
** Duration Analysis
* Time Series
* Factor Analysis
* Classification
** Ordination
** Clustering
* Network Analysis
* High Performance Computing
** Profiling R code
** Parallel computing with R
* Appendix
** Sources
** Index
In this section, we look at the binomial model. We have one outcome which is binary and a set of explanatory variables.
This kind of model can be analyzed using a linear probability model. However a drawback of this model for the parameter of the Bernoulli distribution is that, unless restrictions are placed on $\beta$, the estimated coefficients can imply probabilities outside the unit interval $[0,1]$ . For this reason, models such as the logit model or the probit model are more commonly used. If you want to estimate a linear probability model, have a look at the linear models page.

## Logit model

The model takes the form : $y_{i}\sim Bernoulli(\pi _{i})$ with the inverse link function : $\pi _{i}={\frac {exp(x_{i}'\beta )}{(1+exp(x_{i}'\beta ))}}$. It can be estimated using maximum likelihood or using bayesian methods.

### Maximum likelihood estimation

* The standard way to estimate a logit model is glm function with family binomial and link logit.
* lrm (Design) is another implementation of the logistic regression model.
* There is an implementation in the Zelig package.
In this example, we simulate a model with one continuous predictor and estimate this model using the glm function.

#### Zelig

The Zelig' package makes it easy to compute all the quantities of interest.
We develop a new example. First we simulate a new dataset with two continuous explanatory variables and we estimate the model using zelig with the model = "logit" option.
* We the look at the predicted values of y at the mean of x1 and x2
* Then we look at the predicted values when x1 = 0 and x2 = 0
* We also look at what happens when x1 changes from the 3rd to the 1st quartile.
* ROC Curve in the verification package.
* Zelig has a rocplot function.
* See UCLA Statistical Computing example

### Bayesian estimation

* bayesglm in the arm package
* MCMClogit in the MCMCpack for a bayesian estimation of the logit model.

## Probit model

The probit model is a binary model in which we assume that the link function is the cumulative density function of a normal distribution.
We simulate fake data. First, we draw two random variables x1 and x2 in any distributions (this does not matter). Then we create the vector xbeta as a linear combination of x1 and x2. We apply the link function to that vector and we draw the binary variable y as Bernouilli random variable.

### Maximum likelihood

We can use the glm function with family=binomial(link=probit) option or the probit function in the sampleSelection package which is a wrapper of the former one.

### Bayesian estimation

* MCMCprobit (MCMCpack)

### See Also

* There is an example of a probit model with R on the UCLA statistical computing website.

## Semi-Parametric models

* Klein and Spady estimator is implemented in the np package (see npindex with method = "kleinspady" option).

## References

* ↑ Kosuke Imai, Gary King, and Oliva Lau. 2008. "logit: Logistic Regression for Dichotomous Dependent Variables" in Kosuke Imai, Gary King, and Olivia Lau, "Zelig: Everyone's Statistical Software," http://gking.harvard.edu/zelig
* ↑ UCLA statistical computing probit example http://www.ats.ucla.edu/stat/R/dae/probit.htm
* ↑ Klein, R. W. and R. H. Spady (1993), “An efficient semiparametric estimator for binary response models,” Econometrica, 61, 387-421.
* ↑ Tristen Hayfield and Jeffrey S. Racine (2008). Nonparametric Econometrics: The np Package. Journal of Statistical Software 27(5). URL http://www.jstatsoft.org/v27/i05/.