lisa-lab/pylearn2 — pull request #913 by skearnes (2014-05-17 23:43:45+00:00)
Title: One-step hyperparameter grid search

The are some good examples of using YAML templates with % formatting to perform hyperparameter grid searches, but wouldn't it be nice if you could perform a grid search in one step using train.py? This PR adds the capability to use train.py to grid search with a single YAML file containing a YAML template and a hyperparameter grid (see the tests for examples).

Additional functionality includes:
* Save only `n_best` models, ranked by a monitor channel value.
* Special handling of `TrainCV` templates by `GridSearch` so you can select the best model(s) from each cross-validation fold.
* `GridSearchCV` class, which averages scores across all cross-validation folds to get the best hyperparameters and optionally retrain the best model(s) on the full dataset using those hyperparameters.

__Note__: Some of the `GridSearchCV` functionality in this pull request relies on code that is waiting to be merged into pylearn2, specifically some modifications to the cross-validation module (#889).