Repository: lisa-lab/pylearn2

Issue:
termination_criteria.EpochCounter with new_epochs=False does not check before running
I am using `termination_criteria.EpochCounter` with `new_epochs=False` along with `monitor.push_monitor` with `transfer_experience=True`. The model loaded has already seen more than or equal to `max_epochs` but the main training loop still performs one epoch before terminating. 

This is a bug; no training should occur since we are at or exceeding `max_epochs`. 



Discussion:
I also see that. EpochCounter with 0 epoch will also train for one epoch.

From memory, the problem is that the loop check if we should stop after
training and not before. So it must train at least one time.

On Sun, Aug 17, 2014 at 11:10 PM, Alex Rothberg notifications@github.com
wrote:

> I am using termination_criteria.EpochCounter with new_epochs=False along
> with monitor.push_monitor with transfer_experience=True. The model loaded
> has already seen more than or equal to max_epochs but the main training
> loop still performs one epoch before terminating.
> 
> This is a bug; no training should occur since we are at or exceeding
> max_epochs.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/lisa-lab/pylearn2/issues/1078.

One option would be to have a `setup` method, like the extensions, that is called initially. This method could either return a boolean or raise to indicating that no training is needed.

Or we could move this part

``` python
continue_learning = (
                    self.algorithm.continue_learning(self.model) and
                    extension_continue
                )
assert continue_learning in [True, False, 0, 1]
if not continue_learning:
    break
```

to the beggining of the loop after exceeded_time_budget

Just need to preset extension_continue to True for first iteration.

@nouiz I just noticed the exceeded_time_budget addition to Train. Is the implementation of interruptible training done?

The exceeded_time_budget isn't related to the interruptible training. The
stop/resume isn't done.

The exceeded_time_budget have the same problem. If you set a budget of 0s,
it still do at least on iteration.

On Mon, Aug 18, 2014 at 12:36 PM, Xavier Bouthillier <
notifications@github.com> wrote:

> Or we could move this part
> 
> continue_learning = (
>                     self.algorithm.continue_learning(self.model) and
>                     extension_continue
>                 )assert continue_learning in [True, False, 0, 1]if not continue_learning:
>     break
> 
> to the beggining of the loop after exceeded_time_budget
> 
> Just need to preset extension_continue to True for first iteration.
> 
> @nouiz https://github.com/nouiz I just noticed the exceeded_time_budget
> addition to Train. Is the implementation of interruptible training done?
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/lisa-lab/pylearn2/issues/1078#issuecomment-52518487.

The break point is before the training execution and the test is dt>=time_budget so it should not train for 1 epoch as it is.

@madisonmay What do you think the best way to deal with this is? The issue is that the termination condition is currently checked after running one full epoch.

Hi @cancan101,

Didn't realize I had been CC'ed on this issue until just now... will take a look tonight.

Was unfortunately pretty busy today and didn't get a chance to give this issue a thorough look, but may have some time tomorrow to submit a patch.

Finally got some dedicated time to take a look at this issue and put up a pull request with a fix.  
Code review and general comments welcome.

The previous fix introduced a change in the training procedure. See #1148. The PR has been reverted and the bug stated here is present again. 


Resolving patch:
diff --git a/pylearn2/termination_criteria/__init__.py b/pylearn2/termination_criteria/__init__.py
--- a/pylearn2/termination_criteria/__init__.py
+++ b/pylearn2/termination_criteria/__init__.py
@@ -12,6 +12,7 @@
 import functools
 import numpy as np
 
+
 class TerminationCriterion(object):
     """
     A callable used to determine if a TrainingAlgorithm should quit
@@ -33,8 +34,8 @@ def continue_learning(self, model):
             True or False as described above
         """
 
-        raise NotImplementedError(str(type(self)) + " does not implement " +
-                                  "continue_learning.")
+        raise NotImplementedError(
+            str(type(self)) + " does not implement " + "continue_learning.")
 
 
 class MonitorBased(TerminationCriterion):
@@ -55,6 +56,7 @@ class MonitorBased(TerminationCriterion):
         has only one channel, this channel will be used; otherwise, an
         error will be raised.
     """
+
     def __init__(self, prop_decrease=.01, N=5, channel_name=None):
         self._channel_name = channel_name
         self.prop_decrease = prop_decrease
@@ -205,8 +207,7 @@ class EpochCounter(TerminationCriterion):
     Parameters
     ----------
     max_epochs : int
-        Number of epochs (i.e. calls to this object's `__call__`
-        method) after which this termination criterion should
+        Number of epochs after which this termination criterion should
         return `False`.
     new_epochs : bool, optional
         If True, epoch counter starts from 0. Otherwise it
@@ -219,18 +220,17 @@ def __init__(self, max_epochs, new_epochs=True):
 
     def initialize(self, model):
         if self._new_epochs:
-            self._epochs_done = 0
+            self._epochs_offset = model.monitor.get_epochs_seen()
         else:
-            # epochs_seen = 1 on first continue_learning() call
-            self._epochs_done = model.monitor.get_epochs_seen() - 1
+            self._epochs_offset = 0
 
     @functools.wraps(TerminationCriterion.continue_learning)
     def continue_learning(self, model):
-        if not hasattr(self, "_epochs_done"):
+        if not hasattr(self, "_epochs_offset"):
             self.initialize(model)
 
-        self._epochs_done += 1
-        return self._epochs_done < self._max_epochs
+        current_epoch = model.monitor.get_epochs_seen() - self._epochs_offset
+        return current_epoch < self._max_epochs
 
 
 class And(TerminationCriterion):
diff --git a/pylearn2/train.py b/pylearn2/train.py
--- a/pylearn2/train.py
+++ b/pylearn2/train.py
@@ -140,26 +140,31 @@ def main_loop(self, time_budget=None):
         t0 = datetime.now()
         self.setup()
         if self.algorithm is None:
-            self.run_callbacks_and_monitoring()
-            while True:
+            extension_continue = self.run_callbacks_and_monitoring()
+            # First check if the model is already beyond the stop criteria of
+            # training, if so, just return directly.
+            continue_learning = (self.model.continue_learning() and
+                                 extension_continue)
+            assert continue_learning in [True, False, 0, 1]
+            while continue_learning:
                 if self.exceeded_time_budget(t0, time_budget):
                     break
 
                 rval = self.model.train_all(dataset=self.dataset)
                 if rval is not None:
-                    raise ValueError("Model.train_all should not return " +
-                                     "anything. Use Model.continue_learning " +
-                                     "to control whether learning continues.")
+                    raise ValueError(
+                        "Model.train_all should not return anything. Use "
+                        "Model.continue_learning to control whether "
+                        "learning continues.")
                 self.model.monitor.report_epoch()
                 extension_continue = self.run_callbacks_and_monitoring()
                 freq = self.save_freq
-                if freq > 0 and self.model.monitor.get_epochs_seen() % freq == 0:
+                if freq > 0 and \
+                    self.model.monitor.get_epochs_seen() % freq == 0:
                     self.save()
                 continue_learning = (self.model.continue_learning() and
                                      extension_continue)
                 assert continue_learning in [True, False, 0, 1]
-                if not continue_learning:
-                    break
         else:
             if not hasattr(self.model, 'monitor'):
                 # TODO: is this really necessary? I just put this error here
@@ -193,36 +198,44 @@ def main_loop(self, time_budget=None):
                     val=self.total_seconds,
                     data_specs=(NullSpace(), ''),
                     dataset=self.model.monitor._datasets[0])
-            self.run_callbacks_and_monitoring()
+            extension_continue = self.run_callbacks_and_monitoring()
 
-            while True:
+            # First check if the model is already beyond the stop criteria of
+            # training, if so, just return directly.
+            continue_learning = (
+                self.algorithm.continue_learning(self.model) and
+                extension_continue
+            )
+            assert continue_learning in [True, False, 0, 1]
+            while continue_learning:
                 if self.exceeded_time_budget(t0, time_budget):
                     break
 
                 with log_timing(log, None, level=logging.DEBUG,
                                 callbacks=[self.total_seconds.set_value]):
                     with log_timing(
-                            log, None, final_msg='Time this epoch:',
-                            callbacks=[self.training_seconds.set_value]):
+                        log, None, final_msg='Time this epoch:',
+                        callbacks=[self.training_seconds.set_value]
+                    ):
                         rval = self.algorithm.train(dataset=self.dataset)
                     if rval is not None:
-                        raise ValueError("TrainingAlgorithm.train should not "
-                                         "return anything. Use "
-                                         "TrainingAlgorithm.continue_learning "
-                                         "to control whether learning "
-                                         "continues.")
+                        raise ValueError(
+                            "TrainingAlgorithm.train should not return "
+                            "anything. Use "
+                            "TrainingAlgorithm.continue_learning to "
+                            "control whether learning continues."
+                        )
                     self.model.monitor.report_epoch()
                     extension_continue = self.run_callbacks_and_monitoring()
                     if self.save_freq > 0 and \
-                       self.model.monitor.get_epochs_seen() % self.save_freq == 0:
+                        self.model.monitor.get_epochs_seen() % \
+                        self.save_freq == 0:
                         self.save()
                 continue_learning = (
                     self.algorithm.continue_learning(self.model) and
                     extension_continue
                 )
                 assert continue_learning in [True, False, 0, 1]
-                if not continue_learning:
-                    break
 
         self.model.monitor.training_succeeded = True
 


Test patch:
diff --git a/pylearn2/termination_criteria/tests/test_init.py b/pylearn2/termination_criteria/tests/test_init.py
--- a/pylearn2/termination_criteria/tests/test_init.py
+++ b/pylearn2/termination_criteria/tests/test_init.py
@@ -2,7 +2,6 @@
 Tests for pylearn2.termination_criteria.__init__ functions and classes.
 """
 
-
 from pylearn2.termination_criteria import EpochCounter
 
 from pylearn2.datasets.dense_design_matrix import DenseDesignMatrix
@@ -19,9 +18,7 @@ def test_epoch_counter():
     Test epoch counter with max_epochs={True,False}
     """
 
-    N = 5
-
-    def produce_train_obj(new_epochs, model=None):
+    def produce_train_obj(new_epochs, max_epochs, model=None):
         if model is None:
             model = MLP(layers=[Softmax(layer_name='y',
                                         n_classes=2,
@@ -34,35 +31,62 @@ def produce_train_obj(new_epochs, model=None):
         dataset = DenseDesignMatrix(X=np.random.normal(size=(6, 3)),
                                     y=np.random.normal(size=(6, 2)))
 
-        epoch_counter = EpochCounter(max_epochs=N,
+        epoch_counter = EpochCounter(max_epochs=max_epochs,
                                      new_epochs=new_epochs)
 
-        algorithm = SGD(batch_size=2, learning_rate=0.1,
+        algorithm = SGD(batch_size=2,
+                        learning_rate=0.1,
                         termination_criterion=epoch_counter)
 
-        return Train(dataset=dataset,
-                     model=model,
-                     algorithm=algorithm)
+        return Train(dataset=dataset, model=model, algorithm=algorithm)
 
     def test_epochs(epochs_seen, n):
         assert epochs_seen == n, \
             "%d epochs seen and should be %d" % (epochs_seen, n)
 
-    # Tests for N new epochs
-    train_obj = produce_train_obj(new_epochs=True)
+    # Tests for 5 new epochs
+    train_obj = produce_train_obj(new_epochs=True, max_epochs=5)
     train_obj.main_loop()
-    test_epochs(train_obj.model.monitor.get_epochs_seen(), N)
-    train_obj = produce_train_obj(new_epochs=True, model=train_obj.model)
+    test_epochs(train_obj.model.monitor.get_epochs_seen(), 5)
+    train_obj = produce_train_obj(new_epochs=True,
+                                  max_epochs=5,
+                                  model=train_obj.model)
     train_obj.main_loop()
-    test_epochs(train_obj.model.monitor.get_epochs_seen(), 2 * N)
+    test_epochs(train_obj.model.monitor.get_epochs_seen(), 2 * 5)
 
-    # Tests for N max epochs
-    train_obj = produce_train_obj(new_epochs=False)
+    # Tests for 5 max epochs
+    train_obj = produce_train_obj(new_epochs=False, max_epochs=5)
+    train_obj.main_loop()
+    test_epochs(train_obj.model.monitor.get_epochs_seen(), 5)
+    train_obj = produce_train_obj(new_epochs=False,
+                                  max_epochs=5,
+                                  model=train_obj.model)
     train_obj.main_loop()
-    test_epochs(train_obj.model.monitor.get_epochs_seen(), N)
+    test_epochs(train_obj.model.monitor.get_epochs_seen(), 5)
 
-    # Try training while already reached max_epochs, should stop after 1 epoch
-    # on first continue_learning() call
-    train_obj = produce_train_obj(new_epochs=False, model=train_obj.model)
+    # Tests for 0 new epochs
+    train_obj = produce_train_obj(new_epochs=True, max_epochs=0)
+    before_train = train_obj.model.get_weights()
+    train_obj.main_loop()
+    after_train = train_obj.model.get_weights()
+    test_epochs(train_obj.model.monitor.get_epochs_seen(), 0)
+    assert np.all(before_train == after_train)
+
+    train_obj = produce_train_obj(new_epochs=True,
+                                  max_epochs=0,
+                                  model=train_obj.model)
+    before_train = train_obj.model.get_weights()
+    train_obj.main_loop()
+    after_train = train_obj.model.get_weights()
+    test_epochs(train_obj.model.monitor.get_epochs_seen(), 0)
+    assert np.all(before_train == after_train)
+
+    # Tests for 0 max epochs
+    train_obj = produce_train_obj(new_epochs=False,
+                                  max_epochs=0,
+                                  model=train_obj.model)
+    before_train = train_obj.model.get_weights()
     train_obj.main_loop()
-    test_epochs(train_obj.model.monitor.get_epochs_seen(), N+1)
+    after_train = train_obj.model.get_weights()
+    test_epochs(train_obj.model.monitor.get_epochs_seen(), 0)
+    assert np.all(before_train == after_train)
diff --git a/pylearn2/train_extensions/tests/test_roc_auc.py b/pylearn2/train_extensions/tests/test_roc_auc.py
--- a/pylearn2/train_extensions/tests/test_roc_auc.py
+++ b/pylearn2/train_extensions/tests/test_roc_auc.py
@@ -62,7 +62,7 @@ def test_roc_auc_one_vs_one():
                     max_epochs: 1,
                 },
                 !obj:pylearn2.termination_criteria.MonitorBased {
-                    channel_name: train_y_roc_auc,
+                    channel_name: train_roc_auc,
                     prop_decrease: 0.,
                     N: 1,
                 },
@@ -112,7 +112,7 @@ def test_roc_auc_one_vs_one():
                     max_epochs: 1,
                 },
                 !obj:pylearn2.termination_criteria.MonitorBased {
-                    channel_name: train_y_roc_auc,
+                    channel_name: train_roc_auc-0vX,
                     prop_decrease: 0.,
                     N: 1,
                 },
@@ -173,7 +173,7 @@ def test_roc_auc_one_vs_one():
                     max_epochs: 1,
                 },
                 !obj:pylearn2.termination_criteria.MonitorBased {
-                    channel_name: train_y_roc_auc,
+                    channel_name: train_roc_auc-0v1,
                     prop_decrease: 0.,
                     N: 1,
                 },