lisa-lab/pylearn — Issue #67: train_example unit test
Author: goodfeli
Created: 2012-03-03 16:35:04+00:00
State: open

Make a unit test based on scripts/train_example

It should do basically the same thing as "train.py cifar_grbm_smd.yaml" but with these changes:

-don't use the real dataset. the preprocessing is expensive, and cifar10 is a reasonably big download. we don't want to force everyone who uses the library to download cifar10. randomly generated data might be ok, or maybe we could host a .npy file with a few hundred thousand examples somewhere that the unit test can download if the file isn't found
-set save_freq to 0, so the test doesn't modify the filesystem

The test should work by checking that the initial and final values in the monitor are pretty close to the values obtained by some original version of the code that we believe to be working. It should also check the length of the monitor to make sure convergence is happening at the same rate.

One question about how to implement this test: do we want it to load the yaml file and then modify the object to use different data? This would require adding an interface for changing the dataset of a Train object. An alternative is to manually update the test to remain similar to the yaml file, but it seems like it would be easy for the test and the yaml to get out of sync this way.