# R Programming/Optimization

R Programming
* R Basics
** Introduction
** Sample Session
** Manage your workspace
** Settings and Installation
** Documentation
** Control Structures
** Working with functions
** Debugging
** Using C or Fortran
** Utilities
** Estimation utilities
** Packages
* Data Management
** Data types
** Working with data frames
** Importing and exporting data
** Text Processing
** Times and Dates
* Graphics
** Grammar of graphics
* Publication quality output
* Descriptive Statistics
* Mathematics
** Optimization
** Probability Distributions
** Random Number Generation
* Statistical Core Methods
** Maximum Likelihood
** Method of Moments
** Bayesian Methods
** Bootstrap
** Multiple Imputation
** Nonparametric Methods
* Regression Models
** Linear Models
** Quantile Regression
** Binomial Models
** Multinomial Models
** Tobit And Selection Models
** Count Data Models
** Duration Analysis
* Time Series
* Factor Analysis
* Classification
** Ordination
** Clustering
* Network Analysis
* High Performance Computing
** Profiling R code
** Parallel computing with R
* Appendix
** Sources
** Index
* optimize is devoted to one dimensional optimization problem.
* optim, nlm, ucminf (ucminf) can be used for multidimensional optimization problems.
* nlminb for constrained optimization.
* quadprog, minqa, rgenoud, trust packages
* Some work is done to improve optimization in R. See Updating and improving optim, Use R 2009 slides, the R-forge optimizer page and the corresponding packages including optimx.

## Numerical Methods
### One dimensional problem

The one dimensional problem :

### Newton-Raphson

* nlm provides a Newton algorithm.
* maxLik package for maximization of a likelihood function. This package includes the Newton Raphson method.
* newtonraphson in the spuRs package.

### BFGS

* The BFGS method

### Conjugate gradient method

* optim with method="cg".

### Trust Region Method

* "trust" package for trust region method

### The Nelder-Mead simplex method

* The Nelder Mead Method
* The boot package includes another simplex method

## Simulation methods
### Simulated Annealing

* The Simulated Annealing is an algorithm which is useful to maximise non-smooth functions. It is pre implemented in optim.

### Genetic Algorithm

* rgenoud package for genetic algorithm
* gaoptim package for genetic algorithm
* ga general purpose package for optimization using genetic algorithms. It provides a flexible set of tools for implementing genetic algorithms search in both the continuous and discrete case, whether constrained or not.

## References
### Citations

* ↑ Updating and improving optim, Use R 2009 slides http://www.agrocampus-ouest.fr/math/useR-2009/slides/Nash+Varadhan.pdf
* ↑ R-forge optimizer http://optimizer.r-forge.r-project.org/
* ↑ Jasjeet Sekhon homepage : http://sekhon.berkeley.edu/rgenoud/
* ↑ gaoptim on CRAN: http://cran.r-project.org/web/packages/gaoptim/index.html
* ↑ ga on CRAN: http://cran.r-project.org/web/packages/GA/index.html/

### Sources

* Venables and Ripley, Chapter 16.
* Cameron and Trivedi, Microeconometrics, Chapter 10.
* Braun and Murdoch, first course in statistical programming with R (a very good reference on optimization using R), Chapter 7.