internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pytorch/ignite

Issue:
Add annotations to attach event handlers
See #37 for details


Discussion:

Resolving patch:
diff --git a/ignite/trainer/trainer.py b/ignite/trainer/trainer.py
--- a/ignite/trainer/trainer.py
+++ b/ignite/trainer/trainer.py
@@ -103,6 +103,29 @@ def add_event_handler(self, event_name, handler, *args, **kwargs):
         self._event_handlers[event_name].append((handler, args, kwargs))
         self._logger.debug("added handler for event % ", event_name)
 
+    def on(self, event_name, *args, **kwargs):
+        """
+        Decorator shortcut for add_event_handler
+
+        Parameters
+        ----------
+        event_name: enum
+            event from ignite.trainer.TrainingEvents to attach the
+            handler to
+        args:
+            optional args to be passed to `handler`
+        kwargs:
+            optional keyword args to be passed to `handler`
+
+        Returns
+        -------
+        None
+        """
+        def decorator(f):
+            self.add_event_handler(event_name, f, *args, **kwargs)
+            return f
+        return decorator
+
     def _fire_event(self, event_name):
         if event_name in self._event_handlers.keys():
             self._logger.debug("firing handlers for event %s ", event_name)


Test patch:
diff --git a/tests/ignite/trainer/test_trainer.py b/tests/ignite/trainer/test_trainer.py
--- a/tests/ignite/trainer/test_trainer.py
+++ b/tests/ignite/trainer/test_trainer.py
@@ -410,3 +410,32 @@ def test_create_supervised():
     assert loss == approx(17.0)
     assert model.weight.data[0, 0] == approx(1.3)
     assert model.bias.data[0] == approx(0.8)
+
+
+def test_on_decorator():
+    max_epochs = 5
+    num_batches = 3
+    training_data = _create_mock_data_loader(max_epochs, num_batches)
+
+    trainer = Trainer(MagicMock(return_value=1), MagicMock())
+
+    class Counter(object):
+        def __init__(self, count=0):
+            self.count = count
+
+    started_counter = Counter()
+
+    @trainer.on(TrainingEvents.TRAINING_ITERATION_STARTED, started_counter)
+    def handle_training_iteration_started(trainer, started_counter):
+        started_counter.count += 1
+
+    completed_counter = Counter()
+
+    @trainer.on(TrainingEvents.TRAINING_ITERATION_COMPLETED, completed_counter)
+    def handle_training_iteration_completed(trainer, completed_counter):
+        completed_counter.count += 1
+
+    trainer.run(training_data, max_epochs=max_epochs)
+
+    assert started_counter.count == 15
+    assert completed_counter.count == 15
Repository: pytorch/ignite

Issue:
Complete engines.rst sphinx package reference



Discussion:

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -40,12 +40,14 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    'sphinx.ext.autodoc',
+    'sphinx.ext.autosummary',
     'sphinx.ext.doctest',
     'sphinx.ext.intersphinx',
     'sphinx.ext.todo',
+    'sphinx.ext.coverage',
     'sphinx.ext.mathjax',
-    'sphinx.ext.viewcode',
+    'sphinx.ext.napoleon',
+    'sphinx.ext.viewcode'
 ]
 
 # Add any paths that contain templates here, relative to this directory.
diff --git a/ignite/engines/engine.py b/ignite/engines/engine.py
--- a/ignite/engines/engine.py
+++ b/ignite/engines/engine.py
@@ -6,6 +6,7 @@
 
 
 class Events(Enum):
+    """Events that are fired by the :class:`ignite.engines.Engine` during execution"""
     EPOCH_STARTED = "epoch_started"
     EPOCH_COMPLETED = "epoch_completed"
     STARTED = "started"
@@ -16,6 +17,7 @@ class Events(Enum):
 
 
 class State(object):
+    """An object that is used to pass internal and user-defined state between event handlers"""
     def __init__(self, **kwargs):
         self.iteration = 0
         self.output = None


Test patch:
Repository: pytorch/ignite

Issue:
Update the "Concepts" part in the documentation
Recently we've added a possibility to handle custom events, so we need to keep "Concepts" documentation updated.

There is also a bad docstring rendering of parameters on methods
- [add_event_handler](https://pytorch.org/ignite/engine.html#ignite.engine.Engine.add_event_handler)
- [fire_event](https://pytorch.org/ignite/engine.html#ignite.engine.Engine.fire_event)
- [on](https://pytorch.org/ignite/engine.html#ignite.engine.Engine.fire_event)
- [register_events](https://pytorch.org/ignite/engine.html#ignite.engine.Engine.register_events)


Discussion:
@vfdev-5 : also there is a minor bug in the ModelCheckpoint docstring, the save_as_state_dict parameter is not in the list.
Please can you give me your policy how I should proceed and then I will correct it (I do not have much experience in contributing to other's work). Thanks in advance!
@rpatrik96 welcome on board of open-source :) Thanks for pointing out other undocumented parts.

In brief how to fix a docstrings:
- fork the repository and clone the fork to your machine
- fix the docstring python files, e.g. ModelCheckpoint docstring
- to update "Concepts" of the documentation you need to edit files from [here](https://github.com/pytorch/ignite/tree/master/docs/source).
- how to build the doc? Follow the same path as [travis CI](https://github.com/pytorch/ignite/blob/master/.travis.yml#L85)
  - to install dependencies execute the [previous lines](https://github.com/pytorch/ignite/blob/master/.travis.yml#L79)
- finally run in a separate terminal: `cd docs/build/ && python3 -m http.server 8080` to display built docs locally. Like this you can check if fixes are working.
- if everything is fine -> commit and push to your fork -> create PR to original repository
    
And for a general context take a look also at [CONTRIBUTING](https://github.com/pytorch/ignite/blob/master/CONTRIBUTING.md)

Do not hesitate to ask details on the points if you have other question.

HTH

Resolving patch:
diff --git a/ignite/handlers/checkpoint.py b/ignite/handlers/checkpoint.py
--- a/ignite/handlers/checkpoint.py
+++ b/ignite/handlers/checkpoint.py
@@ -42,6 +42,8 @@ class ModelCheckpoint(object):
             in the directory 'dirname'
         create_dir (bool, optional):
             If True, will create directory 'dirname' if it doesnt exist.
+        save_as_state_dict (bool, optional):
+            If True, will save only the `state_dict` of the objects specified, otherwise the whole object will be saved.
 
     Notes:
           This handler expects two arguments: an `Engine` object and a `dict`


Test patch:
Repository: pytorch/ignite

Issue:
tqm_logger: metric_names is currently not optional 
Hi,

https://github.com/pytorch/ignite/blob/master/ignite/contrib/handlers/tqdm_logger.py#L75
This line should be modified to make `metric_names` optional. Here is a suggestion:
```
if metric_names is not None and not isinstance(metric_names, list):
    raise TypeError("metric_names should be a list, got {} instead".format(type(metric_names)))
```

Thanks


Discussion:
thanks for pointing it out! I'll fix that ASAP

Resolving patch:
diff --git a/ignite/contrib/handlers/tqdm_logger.py b/ignite/contrib/handlers/tqdm_logger.py
--- a/ignite/contrib/handlers/tqdm_logger.py
+++ b/ignite/contrib/handlers/tqdm_logger.py
@@ -47,6 +47,7 @@ def _update(self, engine, metric_names=None):
 
         if metric_names is not None:
             if not all(metric in engine.state.metrics for metric in metric_names):
+                self._close(engine)
                 raise KeyError("metrics not found in engine.state.metrics")
 
             metrics = {name: '{:.2e}'.format(engine.state.metrics[name]) for name in metric_names}
@@ -72,7 +73,7 @@ def attach(self, engine, metric_names=None):
             engine (Engine): engine object
             metric_names (list): (Optional) list of the metrics names to log as the bar progresses
         """
-        if not isinstance(metric_names, list):
+        if metric_names is not None and not isinstance(metric_names, list):
             raise TypeError("metric_names should be a list, got {} instead".format(type(metric_names)))
 
         engine.add_event_handler(Events.EPOCH_COMPLETED, self._close)


Test patch:
diff --git a/tests/ignite/contrib/handlers/test_pbar.py b/tests/ignite/contrib/handlers/test_pbar.py
--- a/tests/ignite/contrib/handlers/test_pbar.py
+++ b/tests/ignite/contrib/handlers/test_pbar.py
@@ -66,3 +66,23 @@ def step(engine, batch):
 
     with pytest.raises(KeyError):
         trainer.run(data=data, max_epochs=1)
+
+
+def test_pbar_no_metric_names(capsys):
+
+    n_epochs = 2
+    loader = [1, 2]
+    engine = Engine(update_fn)
+
+    pbar = ProgressBar()
+    pbar.attach(engine)
+
+    engine.run(loader, max_epochs=n_epochs)
+
+    captured = capsys.readouterr()
+    err = captured.err.split('\r')
+    err = list(map(lambda x: x.strip(), err))
+    err = list(filter(None, err))
+    actual = err[-1]
+    expected = u'Epoch 2: [1/2]  50%|█████      [00:00<00:00]'
+    assert actual == expected
Repository: pytorch/ignite

Issue:
[feature request]Changing how the loss metric get the shape information
Recently I was working with multi-output models and I was using a pattern that I believe to
be common. The model returns a tuple, for instance `y_pred = (ypred_1, ypred_2)` with the correspondent y of the form `y = (y_1, y_2)` with the `loss_fn` accepting as arguments `ypred` and  `y` (`loss_fn(ypred, y)`).

However I have run into problems  when using the Loss metric due to the batch size used on the update function of this metric calling the attribute shape of the, presumed, torch.Tensor directly.

I have as suggestion to change the Loss metric class to accept an extra function, for instance `batch_size = lambda x: x.shape[0]` to recover the current behavior and to permit to access the shape attribute from a tuple/list.



Discussion:
@labor00 thanks for the feedback! It's true that `Loss` is not working for a generic loss function. Previously, we decided to ask users [to override `Loss` function](https://github.com/pytorch/ignite/pull/112) in such cases. However maybe it's time to rethink the purpose of `Loss` and add such feature... 

cc @jasonkriss 

@labor00 just want to make sure I'm understanding... You are trying to use the `Loss` metric with a custom defined `loss_fn`, right?

The original motivation/purpose for the `Loss` metric was to work with built-in PyTorch loss functions. Thus, the requirement that y and y_pred be tensors. That being said, I don't really have a problem with the solution/change you suggest. @vfdev-5 I think you're right. A PR would be welcome here, in my eyes.
Yes I am using  a custom loss. 

Resolving patch:
diff --git a/ignite/metrics/loss.py b/ignite/metrics/loss.py
--- a/ignite/metrics/loss.py
+++ b/ignite/metrics/loss.py
@@ -20,12 +20,16 @@ class Loss(Metric):
             The output is is expected to be a tuple (prediction, target) or
             (prediction, target, kwargs) where kwargs is a dictionary of extra
             keywords arguments.
+        batch_size (callable): a callable taking a target tensor that returns the
+            first dimension size (usually the batch size).
 
     """
 
-    def __init__(self, loss_fn, output_transform=lambda x: x):
+    def __init__(self, loss_fn, output_transform=lambda x: x,
+                 batch_size=lambda x: x.shape[0]):
         super(Loss, self).__init__(output_transform)
         self._loss_fn = loss_fn
+        self._batch_size = batch_size
 
     def reset(self):
         self._sum = 0
@@ -42,8 +46,9 @@ def update(self, output):
         if len(average_loss.shape) != 0:
             raise ValueError('loss_fn did not return the average loss')
 
-        self._sum += average_loss.item() * y.shape[0]
-        self._num_examples += y.shape[0]
+        N = self._batch_size(y)
+        self._sum += average_loss.item() * N
+        self._num_examples += N
 
     def compute(self):
         if self._num_examples == 0:


Test patch:
Repository: pytorch/ignite

Issue:
ParamScheduler docs missing
No docs on `ParamScheduler` and related classes on the [site](https://pytorch.org/ignite/contrib/handlers.html).


Discussion:

Resolving patch:
diff --git a/ignite/contrib/engines/__init__.py b/ignite/contrib/engines/__init__.py
--- a/ignite/contrib/engines/__init__.py
+++ b/ignite/contrib/engines/__init__.py
@@ -2,6 +2,3 @@
 
 from ignite.contrib.engines.tbptt import create_supervised_tbptt_trainer
 from ignite.contrib.engines.tbptt import Tbptt_Events
-
-
-__all__ = ["create_supervised_tbptt_trainer", "Tbptt_Events"]
diff --git a/ignite/contrib/handlers/__init__.py b/ignite/contrib/handlers/__init__.py
--- a/ignite/contrib/handlers/__init__.py
+++ b/ignite/contrib/handlers/__init__.py
@@ -3,5 +3,3 @@
     LinearCyclicalScheduler, CosineAnnealingScheduler
 
 from ignite.contrib.handlers.tqdm_logger import ProgressBar
-
-__all__ = ['ProgressBar']


Test patch:
Repository: pytorch/ignite

Issue:
Master branch nightly build depending on torch_nightly
We need a build of ignite to allow people try nightly version of PyTorch together with ignite. Since the PyTorch is nightly built, it might be better to call this package `pytorch-ignite-nightly` which is also a nightly build of master branch, but with the dependencies on `torch` changed to `torch_nightly`.

cc: @vfdev-5 

NB: I'm not sure the dependency should be `torch_nightly` or `torch-nightly`, because the install command has `torch_nightly` but the output of pip says `torch-nightly`:
```
gaoxiang@pc ~ $ sudo pip install torch_nightly -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html
Looking in links: https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html
Collecting torch_nightly
  Downloading https://download.pytorch.org/whl/nightly/cpu/torch_nightly-1.0.0.dev20181029-cp37-cp37m-linux_x86_64.whl (85.9MB)
    100% |████████████████████████████████| 85.9MB 698kB/s 
Installing collected packages: torch-nightly
Successfully installed torch-nightly-1.0.0.dev20181029
```
@soumith Do you think we should change these two names to make them consistent?


Discussion:

Resolving patch:
diff --git a/nightly/setup_nightly.py b/nightly/setup_nightly.py
new file mode 100644
--- /dev/null
+++ b/nightly/setup_nightly.py
@@ -0,0 +1,47 @@
+import os
+import io
+import re
+from setuptools import setup, find_packages
+import datetime
+
+
+def read(*names, **kwargs):
+    with io.open(os.path.join(os.path.dirname(__file__), *names),
+                 encoding=kwargs.get("encoding", "utf8")) as fp:
+        return fp.read()
+
+
+def find_version(*file_paths):
+    version_file = read(*file_paths)
+    version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", version_file, re.M)
+    if version_match:
+        return version_match.group(1)
+    raise RuntimeError("Unable to find version string.")
+
+
+readme = read('README.rst')
+
+VERSION = find_version('ignite', '__init__.py')
+
+requirements = ['enum34;python_version<"3.4"', 'torch_nightly']
+
+setup(
+    # Metadata
+    name='pytorch-ignite-nightly',
+    version=datetime.datetime.now().strftime("%Y%m%d"),
+    author='PyTorch Core Team',
+    author_email='soumith@pytorch.org',
+    url='https://github.com/pytorch/ignite',
+    description='A lightweight library to help with training neural networks in PyTorch.',
+    long_description=readme,
+    license='BSD',
+
+    # Package info
+    packages=find_packages(exclude=('tests', 'tests.*',)),
+
+    zip_safe=True,
+    install_requires=requirements,
+    dependency_links=[
+        "https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html"
+    ]
+)


Test patch:
Repository: pytorch/ignite

Issue:
Fix new flake8 error
Impacts `master`
https://travis-ci.org/pytorch/ignite/jobs/486833029#L466
```
$ flake8
./ignite/metrics/accuracy.py:42:17: E117 over-indented
./tests/ignite/metrics/test_metrics_lambda.py:11:9: E117 over-indented
```

and `nightly`
https://travis-ci.org/pytorch/ignite/jobs/486949548#L452



Discussion:
~~Fixed via latest commit in #420~~

~~Lines were actually over indented.~~

Resolving patch:
diff --git a/ignite/metrics/accuracy.py b/ignite/metrics/accuracy.py
--- a/ignite/metrics/accuracy.py
+++ b/ignite/metrics/accuracy.py
@@ -39,7 +39,7 @@ def _check_shape(self, output):
             raise ValueError("y and y_pred must have compatible shapes.")
 
         if self._is_multilabel and not (y.shape == y_pred.shape and y.ndimension() > 1 and y.shape[1] != 1):
-                raise ValueError("y and y_pred must have same shape of (batch_size, num_categories, ...).")
+            raise ValueError("y and y_pred must have same shape of (batch_size, num_categories, ...).")
 
         return y_pred, y
 


Test patch:
diff --git a/tests/ignite/metrics/test_metrics_lambda.py b/tests/ignite/metrics/test_metrics_lambda.py
--- a/tests/ignite/metrics/test_metrics_lambda.py
+++ b/tests/ignite/metrics/test_metrics_lambda.py
@@ -8,18 +8,18 @@
 
 class ListGatherMetric(Metric):
 
-        def __init__(self, index):
-            super(ListGatherMetric, self).__init__()
-            self.index = index
+    def __init__(self, index):
+        super(ListGatherMetric, self).__init__()
+        self.index = index
 
-        def reset(self):
-            self.list_ = None
+    def reset(self):
+        self.list_ = None
 
-        def update(self, output):
-            self.list_ = output
+    def update(self, output):
+        self.list_ = output
 
-        def compute(self):
-            return self.list_[self.index]
+    def compute(self):
+        return self.list_[self.index]
 
 
 def test_metrics_lambda():
Repository: pytorch/ignite

Issue:
Improve to_onehot to any shape
Idea is to improve `to_onehot` to accept any input shape: (N, ) and (N, H, W, D1, D2, ...).
Currently supported shape is `(N, )`. Thoughts ?


Discussion:

Resolving patch:
diff --git a/ignite/utils.py b/ignite/utils.py
--- a/ignite/utils.py
+++ b/ignite/utils.py
@@ -42,6 +42,8 @@ def apply_to_type(input_, input_type, func):
 
 
 def to_onehot(indices, num_classes):
-    """Convert a tensor of indices to a tensor of one-hot indicators."""
-    onehot = torch.zeros(indices.size(0), num_classes, device=indices.device)
+    """Convert a tensor of indices of any shape `(N, ...)` to a
+    tensor of one-hot indicators of shape `(N, num_classes, ...)`.
+    """
+    onehot = torch.zeros(indices.shape[0], num_classes, *indices.shape[1:], device=indices.device)
     return onehot.scatter_(1, indices.unsqueeze(1), 1)


Test patch:
diff --git a/tests/ignite/test_utils.py b/tests/ignite/test_utils.py
--- a/tests/ignite/test_utils.py
+++ b/tests/ignite/test_utils.py
@@ -39,3 +39,18 @@ def test_to_onehot():
     actual = to_onehot(indices, 4)
     expected = torch.eye(4)
     assert actual.equal(expected)
+
+    y = torch.randint(0, 21, size=(1000,))
+    y_ohe = to_onehot(y, num_classes=21)
+    y2 = torch.argmax(y_ohe, dim=1)
+    assert y.equal(y2)
+
+    y = torch.randint(0, 21, size=(4, 250, 255))
+    y_ohe = to_onehot(y, num_classes=21)
+    y2 = torch.argmax(y_ohe, dim=1)
+    assert y.equal(y2)
+
+    y = torch.randint(0, 21, size=(4, 150, 155, 4, 6))
+    y_ohe = to_onehot(y, num_classes=21)
+    y2 = torch.argmax(y_ohe, dim=1)
+    assert y.equal(y2)
Repository: pytorch/ignite

Issue:
Version selector on gh-pages like in pytorch docs
I think that many of us are installing the ignite using the master branch.

It would be nice if the ignite documentation also contains the master docs like https://pytorch.org/docs/versions.html. 


What do you think?

Best,
Igor.


Discussion:
@igormq today the documentation reflects the master version and not the stable 0.1.0. 
I agree that it would be nice to add such selector between versions.
I think the easiest way is to host the documentation on readthedocs.org
@miguelvr yes, it is pretty easy to deal with that in `readthedocs.org`, but today our docs are at `pytorch.org/ignite` and not all projects have this opportunity :)
yeah, but `pytorch.org` has the version selector, so I guess we can still do it.

Since you brought that up: 

It is a shame that this project does not appear on https://pytorch.org/ecosystem, or is linked in the main page in some way.
> It is a shame that this project does not appear on https://pytorch.org/ecosystem, or is linked in the main page in some way.

@miguelvr yes, we talked about that with @soumith, he told that he will work on it, to highlight ignite and skorch, and we are still waiting :)
Maybe we can use this plugin : https://github.com/Robpol86/sphinxcontrib-versioning
Probably installed from https://github.com/Robpol86/sphinxcontrib-versioning/pull/46
* Up to date fork that works with Sphinx==1.8.5 - https://github.com/sphinxcontrib-versioning-ng/sphinxcontrib-versioning
* working prototype here - https://anmolsjoshi.github.io/ignite_sphinx_try/

@anmolsjoshi, your working prototype looks great and seems like doing what we need ...

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -12,9 +12,9 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
-# import os
-# import sys
-# sys.path.insert(0, os.path.abspath('.'))
+import os
+import sys
+sys.path.insert(0, os.path.abspath('../..'))
 import ignite
 import sphinx_rtd_theme
 
@@ -25,7 +25,15 @@
 author = 'Torch Contributors'
 
 # The short X.Y version
-version = 'master (' + ignite.__version__ + ' )'
+try:
+    version = os.environ['code_version']
+    if 'master' in version:
+        version = 'master (' + ignite.__version__ + ')'
+    else:
+        version = version.replace('v', '')
+except KeyError:
+    version = ignite.__version__
+
 # The full version, including alpha/beta/rc tags
 release = 'master'
 


Test patch:
Repository: pytorch/ignite

Issue:
Reset should_terminate between runs
Following the [comment](https://github.com/pytorch/ignite/issues/227#issuecomment-410770504), currently, Engine's `should_terminate`/`should_terminate_single_epoch` are not reset between runs. IMO, starting a run we need to set them arguments to `False`



Discussion:

Resolving patch:
diff --git a/ignite/engine/engine.py b/ignite/engine/engine.py
--- a/ignite/engine/engine.py
+++ b/ignite/engine/engine.py
@@ -130,7 +130,7 @@ class TBPTT_Events(Enum):
                 TIME_ITERATION_COMPLETED = "time_iteration_completed"
 
             TBPTT_event_to_attr = {TBPTT_Events.TIME_ITERATION_STARTED: 'time_iteration',
-                                   TBPTT_Events.TIME_ITERATION_STARTED: 'time_iteration'}
+                                   TBPTT_Events.TIME_ITERATION_COMPLETED: 'time_iteration'}
 
             engine = Engine(process_function)
             engine.register_events(*TBPTT_Events, event_to_attr=TBPTT_event_to_attr)
@@ -363,6 +363,7 @@ def run(self, data, max_epochs=1):
         """
 
         self.state = State(dataloader=data, max_epochs=max_epochs, metrics={})
+        self.should_terminate = self.should_terminate_single_epoch = False
 
         try:
             self._logger.info("Engine run starting with max_epochs={}.".format(max_epochs))


Test patch:
diff --git a/tests/ignite/engine/test_engine.py b/tests/ignite/engine/test_engine.py
--- a/tests/ignite/engine/test_engine.py
+++ b/tests/ignite/engine/test_engine.py
@@ -713,3 +713,22 @@ def test_create_supervised_with_metrics():
 
     state = evaluator.run(data)
     assert state.metrics['mse'] == 12.5
+
+
+def test_reset_should_terminate():
+
+    def update_fn(engine, batch):
+        pass
+
+    engine = Engine(update_fn)
+
+    @engine.on(Events.ITERATION_COMPLETED)
+    def terminate_on_iteration_10(engine):
+        if engine.state.iteration == 10:
+            engine.terminate()
+
+    engine.run([0] * 20)
+    assert engine.state.iteration == 10
+
+    engine.run([0] * 20)
+    assert engine.state.iteration == 10