Repository: pytorch/ignite

Issue:
Remove handler feature in Engine
I recently encountered the cases where I would like to remove a handler and optionally replace it with another one. The following code is used:
```python
def remove_handler(engine, handler, event_name):
    assert event_name in engine._event_handlers    
    engine._event_handlers[event_name] = [(h, args, kwargs) 
                                          for h, args, kwargs in engine._event_handlers[event_name] 
                                            if h != handler]
```

@alykhantejani @jasonkriss what do you guys think about inserting such method into `Engine` ?
cc @anmolsjoshi 


Discussion:
@vfdev-5 Do you mind giving an example use case for this?
Sure, maybe these examples I have may look a bit particular:
1) engine that logs to the tensorboard on different datasets at different groups:
```python
    tb_logger = TensorboardLogger(log_dir=log_dir)

    tb_handler = tb_output_handler(tag="training", output_transform=lambda x: x)
    tb_logger.attach(stats_collector,
                     log_handler=tb_handler,
                     event_name=Events.ITERATION_COMPLETED)

    stats_collector.run(config.train_loader, max_epochs=1)

    remove_handler(stats_collector, tb_handler, Events.ITERATION_COMPLETED)
    tb_logger.attach(stats_collector,
                     log_handler=tb_output_handler(tag="validation", output_transform=lambda x: x),
                     event_name=Events.ITERATION_COMPLETED)

    stats_collector.run(config.val_loader, max_epochs=1)
```
2) A handler that when attached to the engine adds some internal handlers and when finished cleans out after itself.


Thanks @vfdev-5. I'm still trying to understand the motivation...

1. Would it maybe more clear here to use two stats_collectors rather than having to remember to remove the handler?

2. Is there a benefit to cleaning up after it finishes?
> Would it maybe more clear here to use two stats_collectors rather than having to remember to remove the handler?

@jasonkriss it means that we need to copy previous code to reinit another `stats_collectors` ... 

> Is there a benefit to cleaning up after it finishes?

More on the context: I wrote a basic time profiler that collects times spent in handlers, batch preparation and process function. To do this the profiler adds manually its handlers in the special order: the first and the last handlers on the events. Profiler adds its handlers **once** the `run` is called to avoid the situation when user adds other handlers after `profiler.attach(trainer)` is called.
So, once `trainer.run(train_loader)` is done, profiler computed what it needs and should remove its added handlers. Otherwise, on the second call of  `trainer.run(train_loader)` there will be twice more attached handlers and times collection will be difficult or even faulty.

**EDIT**: even for symmetry with the methods we already have: add, has, event handler. Remove is missing :)

Got it! Thanks for the added context. Helps a lot. Let's go ahead and add it then.

Resolving patch:
diff --git a/ignite/engine/engine.py b/ignite/engine/engine.py
--- a/ignite/engine/engine.py
+++ b/ignite/engine/engine.py
@@ -163,6 +163,23 @@ def has_event_handler(self, handler, event_name=None):
                     return True
         return False
 
+    def remove_event_handler(self, handler, event_name):
+        """Remove event handler `handler` from registered handlers of the engine
+
+        Args:
+            handler (callable): the callable event handler that should be removed
+            event_name: The event the handler attached to.
+
+        """
+        if event_name not in self._event_handlers:
+            raise ValueError("Input event name '{}' does not exist".format(event_name))
+
+        new_event_handlers = [(h, args, kwargs) for h, args, kwargs in self._event_handlers[event_name]
+                              if h != handler]
+        if len(new_event_handlers) == len(self._event_handlers[event_name]):
+            raise ValueError("Input handler '{}' is not found among registered event handlers".format(handler))
+        self._event_handlers[event_name] = new_event_handlers
+
     def _check_signature(self, fn, fn_description, *args, **kwargs):
         exception_msg = None
 


Test patch:
diff --git a/tests/ignite/engine/test_engine.py b/tests/ignite/engine/test_engine.py
--- a/tests/ignite/engine/test_engine.py
+++ b/tests/ignite/engine/test_engine.py
@@ -150,6 +150,38 @@ def test_has_event_handler():
     assert not engine.has_event_handler(m, Events.EPOCH_STARTED)
 
 
+def test_remove_event_handler():
+    engine = DummyEngine()
+
+    with pytest.raises(ValueError, match=r'Input event name'):
+        engine.remove_event_handler(lambda x: x, "an event")
+
+    def on_started(engine):
+        return 0
+
+    engine.add_event_handler(Events.STARTED, on_started)
+
+    with pytest.raises(ValueError, match=r'Input handler'):
+        engine.remove_event_handler(lambda x: x, Events.STARTED)
+
+    h1 = MagicMock()
+    h2 = MagicMock()
+    handlers = [h1, h2]
+    m = MagicMock()
+    for handler in handlers:
+        engine.add_event_handler(Events.EPOCH_STARTED, handler)
+    engine.add_event_handler(Events.EPOCH_COMPLETED, m)
+
+    assert len(engine._event_handlers[Events.EPOCH_STARTED]) == 2
+    engine.remove_event_handler(h1, Events.EPOCH_STARTED)
+    assert len(engine._event_handlers[Events.EPOCH_STARTED]) == 1
+    assert engine._event_handlers[Events.EPOCH_STARTED][0][0] == h2
+
+    assert len(engine._event_handlers[Events.EPOCH_COMPLETED]) == 1
+    engine.remove_event_handler(m, Events.EPOCH_COMPLETED)
+    assert len(engine._event_handlers[Events.EPOCH_COMPLETED]) == 0
+
+
 def test_args_and_kwargs_are_passed_to_event():
     engine = DummyEngine()
     kwargs = {'a': 'a', 'b': 'b'}