lisa-lab/pylearn2 — pull request #171 by SinaHonari (2013-03-06 02:22:23+00:00)
Title: removing some bugs in sgd.py

Here I have made 3 major changes:

1) I have added a variable name "dataset_name" to the MonitorBasedLRAdjuster class such that the user can specify with dataset to monitor. The current version is old and gives an error like the following. Instead it should look for 'train_objective' or 'test_objective':

Traceback (most recent call last):
File "/u/honaris/pythonlibs/Pylearn2/pylearn2/scripts/train.py", line 141, in
train_obj.main_loop()
File "/u/honaris/pythonlibs/pylearn2/train.py", line 116, in main_loop
self.run_callbacks_and_monitoring()
File "/u/honaris/pythonlibs/pylearn2/train.py", line 140, in run_callbacks_and_monitoring
extension.on_monitor(self.model, self.dataset, self.algorithm)
File "/u/honaris/pythonlibs/pylearn2/training_algorithms/sgd.py", line 394, in on_monitor
v = monitor.channels['objective'].val_record
KeyError: 'objective'

2) In the same class given the description of the MonitorBasedLRAdjuster I have changed this part:
elif v[-1] < self.low_trigger * v[-2]
The previous code did not make sense.

3) Finally, in the LinearDecayOverEpoch and LinearDecay classes I have made this change:
new_lr = self._init_lr - self._step * (self._count - self.start)
The current version throws an error when the current epoch equals (saturate-start) saying that the new_lr is negative. Calling this class with these variable gives you an error at epoch 20:

!obj:pylearn2.training_algorithms.sgd.LinearDecayOverEpoch {
start: 10,
saturate: 30,
decay_factor: 0.01
}