pluskid/Mocha.jl — pull request #12 by pluskid (2014-12-03 04:42:23+00:00)
Title: Solver Refactoring

Refactoring the solver code to separate the general solver loop and the updating rules for specific solver types. As a by-product, a Nesterov solver is added. It is a re-formulated version from the paper

> *Advances in Optimizing Recurrent Networks*. [arXiv:1212.0901](http://arxiv.org/abs/1212.0901)

Other refactoring:

* Loading saved models is now moved from `Snapshot` coffee break into the solver. Now we could load both a JLD snapshot and a HDF5 general model file (the latter will corresponds to fine-tuning on a trained model).
* The forward-backward in the solver loop is changed to backward-forward with an extra initializing forward call. Now the objective value in iteration n is actually for iteration n, instead of iteration n-1.
* The coffee breaks are cleaned up by removing the coffee break time type. Now all the coffee breaks are called after the n-th iteration is computed. An initial coffee break is also called (after no iteration computed yet).