Repository: ContinualAI/avalanche

Issue:
Should Replay and GDumb have their own Strategy?
Do you think it would be useful for the final user to have also a simple _GDdumb_  and _Replay_ strategies as well? 

Even if we add a bit of redundancy I guess some user would prefer to delegate all the inner logic to a single strategy object rather than using a plugin.

It would also make more clear the strategies we offer in avalanche:

`from avalanche.training.strategies import Naive, Replay, GDumb`

What do you think?


Discussion:
I think this is an important question. I think there should be rules/heuristics to let user tell when to use strategy from when to use plugins. Currently, I am using plugins for reusable code and use strategy otherwise. 
For example, If I am running a CL strategy I could be interested in the combination between that strategy and Replay. Hence, a plugin for Replay. GDumb adopts a specific kind of replay, substituting the entire training set at each step with the replay memory. This could be adopted in other strategies as well.
Still, maybe this line of reasoning is too general and it would (almost) never require to write a strategy.
I think the best approach is to create strategies by implementing inner (and well isolated) mechanisms as plugins.
This way we satisfy both usability, easy of use and modularity.
So, we could start by creating new strategy for both replay and GDumb which internally use existing plugins. If we agree on that I can start right away
yes please, go ahead! :)

Resolving patch:
diff --git a/avalanche/training/strategies/__init__.py b/avalanche/training/strategies/__init__.py
--- a/avalanche/training/strategies/__init__.py
+++ b/avalanche/training/strategies/__init__.py
@@ -1,2 +1,4 @@
 from .cl_naive import *
 from .cl_cumulative import *
+from .cl_replay import *
+from .cl_gdumb import *
\ No newline at end of file
diff --git a/avalanche/training/strategies/cl_gdumb.py b/avalanche/training/strategies/cl_gdumb.py
new file mode 100644
--- /dev/null
+++ b/avalanche/training/strategies/cl_gdumb.py
@@ -0,0 +1,112 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+################################################################################
+# Copyright (c) 2020 ContinualAI Research                                      #
+# Copyrights licensed under the CC BY 4.0 License.                             #
+# See the accompanying LICENSE file for terms.                                 #
+#                                                                              #
+# Date: 1-06-2020                                                              #
+# Author(s): Andrea Cossu                                                      #
+# E-mail: contact@continualai.org                                              #
+# Website: clair.continualai.org                                               #
+################################################################################
+
+from __future__ import absolute_import
+from __future__ import division
+# Python 2-3 compatible
+from __future__ import print_function
+
+from typing import Optional, Sequence
+
+import torch
+from torch.nn import Module
+from torch.optim import Optimizer
+
+from avalanche.training.strategies import Naive
+from avalanche.training.plugins import GDumbPlugin
+
+from avalanche.training.skeletons import TrainingFlow
+
+from avalanche.evaluation import EvalProtocol
+from avalanche.training.skeletons import StrategySkeleton
+
+
+class GDumb(Naive):
+    """
+    The GDumb strategy is built on top of the GDumb plugin and Naive strategy.
+    It retrain at each step the model with patterns taken solely from
+    replay memory. It is up to the user logic to decide whether to 
+    reinitialize model at each step or proceed with finetuning.
+    Original paper retrain from scratch at each step.
+    """
+
+    def __init__(self, model: Module, classifier_field: str,
+                 optimizer: Optimizer, criterion: Module,
+                 mem_size: int, reinit_model_before_step: bool = True,
+                 reinit_function: object = None,
+                 train_mb_size: int = 1, train_epochs: int = 1,
+                 test_mb_size: int = None, device=None,
+                 evaluation_protocol: Optional[EvalProtocol] = None,
+                 plugins: Optional[Sequence[StrategySkeleton]] = None):
+        """
+        Creates an instance of the GDumb strategy.
+
+        :param model: The model.
+        :param classifier_field: The name of the classifier field. Used when
+            managing heads in Multi-Task scenarios.
+        :param optimizer: The optimizer to use.
+        :param criterion: The loss criterion to use.
+        :param mem_size: Replay memory size
+        :param reinit_model_before_step: if True, model will be reinitialized 
+            before each step according to `reinit_function` or uniformly if
+            `reinit_function` is not provided. Default True.
+        :param reinit_function: a function which takes as input the model 
+            and return the model with weights initialized.
+            Used only if `reinit_model_before_step` is True. Default None.
+        :param train_mb_size: The train minibatch size. Defaults to 1.
+        :param train_epochs: The number of training epochs. Defaults to 1.
+        :param test_mb_size: The test minibatch size. Defaults to 1.
+        :param device: The device to use. Defaults to None (cpu).
+        :param evaluation_protocol: The evaluation protocol. Defaults to None.
+        :param plugins: Plugins to be added 
+            (GDumb plugin will be added by default). Defaults to None.
+        """
+
+        gdumb_plugin = GDumbPlugin(mem_size=mem_size)
+        if plugins is None:
+            plugins = [gdumb_plugin]
+        else:
+            plugins.append(gdumb_plugin)
+
+        super(GDumb, self).__init__(
+            model, classifier_field, optimizer, criterion, train_mb_size,
+            train_epochs, test_mb_size, device, evaluation_protocol, plugins)
+
+        self.reinit_model_before_step = reinit_model_before_step
+        self.reinit_function = reinit_function
+
+
+    @TrainingFlow
+    def before_training(self):
+        """
+        Reinitialize model before each step, if needed.
+        """
+
+        if self.reinit_model_before_step:
+            if self.reinit_function is not None:
+                self.reinit_function(self.model)
+            else:
+                self._init_weights()
+
+
+    def _init_weights(self):
+        """
+        Default init function
+        """
+        with torch.no_grad():
+            for p in self.model.parameters():
+                torch.nn.init.uniform_(p, -0.1, 0.1)
+
+
+__all__ = ['GDumb']
diff --git a/avalanche/training/strategies/cl_replay.py b/avalanche/training/strategies/cl_replay.py
new file mode 100644
--- /dev/null
+++ b/avalanche/training/strategies/cl_replay.py
@@ -0,0 +1,109 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+################################################################################
+# Copyright (c) 2020 ContinualAI Research                                      #
+# Copyrights licensed under the CC BY 4.0 License.                             #
+# See the accompanying LICENSE file for terms.                                 #
+#                                                                              #
+# Date: 1-06-2020                                                              #
+# Author(s): Andrea Cossu                                                      #
+# E-mail: contact@continualai.org                                              #
+# Website: clair.continualai.org                                               #
+################################################################################
+
+from __future__ import absolute_import
+from __future__ import division
+# Python 2-3 compatible
+from __future__ import print_function
+
+from typing import Optional, Sequence
+
+import torch
+from torch.nn import Module
+from torch.optim import Optimizer
+
+from avalanche.training.strategies import Naive
+from avalanche.training.plugins import ReplayPlugin
+
+from avalanche.evaluation import EvalProtocol
+from avalanche.training.skeletons import StrategySkeleton
+from avalanche.training.skeletons import TrainingFlow
+
+
+class Replay(Naive):
+    """
+    The replay strategy is built on top of the Replay plugin and Naive strategy.
+    It mix patterns from the current step with patterns from the previous steps
+    in order to mitigate forgetting.
+    """
+
+    def __init__(self, model: Module, classifier_field: str,
+                 optimizer: Optimizer, criterion: Module,
+                 mem_size: int, reinit_model_before_step: bool = True,
+                 reinit_function: object = None,
+                 train_mb_size: int = 1, train_epochs: int = 1,
+                 test_mb_size: int = None, device=None,
+                 evaluation_protocol: Optional[EvalProtocol] = None,
+                 plugins: Optional[Sequence[StrategySkeleton]] = None):
+        """
+        Creates an instance of the Replay strategy.
+
+        :param model: The model.
+        :param classifier_field: The name of the classifier field. Used when
+            managing heads in Multi-Task scenarios.
+        :param optimizer: The optimizer to use.
+        :param criterion: The loss criterion to use.
+        :param mem_size: Replay memory size
+        :param reinit_model_before_step: if True, model will be reinitialized 
+            before each step according to `reinit_function` or uniformly if
+            `reinit_function` is not provided. Default True.
+        :param reinit_function: a function which takes as input the model 
+            and return the model with weights initialized.
+            Used only if `reinit_model_before_step` is True. Default None.        
+        :param train_mb_size: The train minibatch size. Defaults to 1.
+        :param train_epochs: The number of training epochs. Defaults to 1.
+        :param test_mb_size: The test minibatch size. Defaults to 1.
+        :param device: The device to use. Defaults to None (cpu).
+        :param evaluation_protocol: The evaluation protocol. Defaults to None.
+        :param plugins: Plugins to be added 
+            (Replay plugin will be added by default). Defaults to None.
+        """
+
+        replay_plugin = ReplayPlugin(mem_size=mem_size)
+        if plugins is None:
+            plugins = [replay_plugin]
+        else:
+            plugins.append(replay_plugin)
+
+        super(Replay, self).__init__(
+            model, classifier_field, optimizer, criterion, train_mb_size,
+            train_epochs, test_mb_size, device, evaluation_protocol, plugins)
+
+        self.reinit_model_before_step = reinit_model_before_step
+        self.reinit_function = reinit_function
+
+
+    @TrainingFlow
+    def before_training(self):
+        """
+        Reinitialize model before each step, if needed.
+        """
+
+        if self.reinit_model_before_step:
+            if self.reinit_function is not None:
+                self.reinit_function(self.model)
+            else:
+                self._init_weights()
+
+
+    def _init_weights(self):
+        """
+        Default init function
+        """
+        with torch.no_grad():
+            for p in self.model.parameters():
+                torch.nn.init.uniform_(p, -0.1, 0.1)
+
+
+__all__ = ['Replay']


Test patch:
diff --git a/tests/test_strategies.py b/tests/test_strategies.py
--- a/tests/test_strategies.py
+++ b/tests/test_strategies.py
@@ -14,6 +14,8 @@
 
 import unittest
 
+import torch
+
 from torchvision.datasets import MNIST
 from torchvision.transforms import ToTensor, Compose
 from torch.optim import SGD
@@ -24,8 +26,8 @@
 from avalanche.evaluation.metrics import ACC
 from avalanche.benchmarks.scenarios import \
     create_nc_single_dataset_sit_scenario, DatasetPart, NCBatchInfo
-from avalanche.training.strategies import Naive, Cumulative
-from avalanche.training.plugins import ReplayPlugin, GDumbPlugin
+from avalanche.training.strategies import Naive, Cumulative, Replay, GDumb
+#from avalanche.training.plugins import ReplayPlugin, GDumbPlugin
 
 device = 'cpu'
 
@@ -64,11 +66,18 @@ def test_replay(self):
                 ACC(num_class=nc_scenario.n_classes)
             ])
 
-        strategy = Naive(model, 'classifier', optimizer, criterion,
+        def reinit(m):
+            with torch.no_grad():
+                for p in m.parameters():
+                    torch.nn.init.uniform_(p, -1, -1)
+
+        strategy = Replay(model, 'classifier', optimizer, criterion,
+                mem_size=200, reinit_model_before_step=True,
+                reinit_function=reinit, # None to use default init function
                 evaluation_protocol=eval_protocol,
                 train_mb_size=100, 
-                train_epochs=4, test_mb_size=100, device=device,
-                plugins=[ReplayPlugin(mem_size=10)])
+                train_epochs=4, test_mb_size=100, device=device, plugins=None
+                )
 
         self.run_strategy(nc_scenario, strategy)
 
@@ -107,28 +116,21 @@ def test_gdumb(self):
                 ACC(num_class=nc_scenario.n_classes)
             ])
 
-        strategy = Naive(model, 'classifier', optimizer, criterion,
+        def reinit(m):
+            with torch.no_grad():
+                for p in m.parameters():
+                    torch.nn.init.uniform_(p, -1, 1)
+
+        strategy = GDumb(model, 'classifier', optimizer, criterion,
+                mem_size=2000, reinit_model_before_step=True,
+                reinit_function=reinit, # None to use default init function
                 train_mb_size=64,
                 evaluation_protocol=eval_protocol,
                 train_epochs=10, test_mb_size=100, device=device,
-                plugins=[GDumbPlugin(2000)])
-
-        print('Starting experiment...')
-        results = []
-        batch_info: NCBatchInfo
-        for batch_info in nc_scenario:
-            print("Start of step ", batch_info.current_step)
-
-            # retrain from scratch each step
-            strategy.model = SimpleMLP()
-            strategy.optimizer = SGD(strategy.model.parameters(), lr=1e-2)
-            strategy.train(batch_info, num_workers=4)
-            print('Training completed')
-
-            print('Computing accuracy on the whole test set')
-            results.append(strategy.test(batch_info, DatasetPart.COMPLETE,
-                                            num_workers=4))
+                plugins=None)
 
+        self.run_strategy(nc_scenario, strategy)
+    
 
     def load_dataset(self):