JuliaOpt/Optim.jl — pull request #50 by timholy (2014-03-11 03:34:37+00:00)
Title: WIP: constrained optimization

This is not done yet, but I thought I should put it out there so people can comment sooner rather than later, if it looks like this is going in the wrong direction.

When finished, this should implement all the standard inequality constraints; equality constraints may come, but likely will be a little later (I have a pressing need for the inequalities, but not the equalities). The only algorithm I've implemented is a central-path-following interior-point solver---but I'd venture to say that if you could only pick one, interior-point is the way to go.

Using this, a version of `nnls` (just two lines, `objective = linlsq(A, b); result = interior(objective, initial_x, bounds, method=:newton)`) is 100x faster on a 50x50 problem compared to the existing implementation of `nnls` in `Optim`.

Left to do:

- [ ] Add support for constraints to other linesearch algorithms
- [ ] Implement value, gradient, and hessian for barrier functions of linear and nonlinear constraints
- [ ] Add more tests

In addition to adding support for constrained optimization, this makes several relevant improvements to `hz_linesearch`, in particular ones that are smarter about some of the odd behaviors of floating-point numbers, and others that will make constrained optimization perform significantly better. Thanks to @JeffBezanson for his quick work on https://github.com/JuliaLang/julia/issues/6097, making it feasible to debug some of the darker corners of `hz_linesearch`.

One final point: note that this changes how the `problems` directory is handled. I changed it because it wasn't obvious to me that we wanted this to be a regular part of loading `Optim`. Perhaps we should go one step further, and only load it when running tests (and not even have `testpaths` in `Optim.jl`).