Repository: IBM/aihwkit (language: python)

Issue:
Include analog device presets
## Description and motivation

While using the family of `RPUConfig` devices and adjust their parameters manually offers a great deal of flexibility, for some use cases being able to start with a set of pre-defined interesting configurations would be useful. This would help the users being familiar with the different options, serve as a basis for exploring the toolkit, and allow using "well-known" configurations off the shelf for a number of purposes.

## Proposed solution

Add a number of curated analog device presets to the package, representing devices in existing literature or other specific configurations that seem relevant or interesting.

## Alternatives and other information

<!-- A clear and concise description of any alternative solutions or features you've considered. -->

Device and configuration presets
## Related issues

#133 

## Description

Addition of device and config presets. 

## Details

Device presets (called `XXXPresetDevice`) are parameter settings of devices, implemented by inheriting the underlying device data class. They can be used to configure a tile instead of the base classes like `ConstantStepDevice`.

Config presets (called `XXXPreset`) are parameter settings of the full `rpu_config` and could be used instead of defining ones own. They might specify a particular tile model choice and analog optimizer.  For instance, 

```python 
model = AnalogLinear(4, 2, rpu_config=TikiTakaReRamSBPreset())
```

would use Tiki-taka optimizer (using two ReRAM crossbars with the `ReRamSBPresetDevice` device model) and otherwise default parameters. As before, parameter choices can be adapted by giving keyword arguments or overwriting the field values. 



Discussion / hints:
Closed via #144


Fix patch:
diff --git a/.pylintrc b/.pylintrc
index 926bc95b..33293dbf 100644
--- a/.pylintrc
+++ b/.pylintrc
@@ -346,7 +346,7 @@ contextmanager-decorators=contextlib.contextmanager
 # List of members which are set dynamically and missed by pylint inference
 # system, and so shouldn't trigger E1101 when accessed. Python regular
 # expressions are accepted.
-generated-members=get_layer,get_tile,get_rpu_config,bias
+generated-members=get_layer,get_tile,get_rpu_config,bias,preset_cls
 
 # Tells whether missing members accessed in mixin class should be ignored. A
 # mixin class is detected if its name ends with "mixin" (case insensitive).
diff --git a/src/aihwkit/simulator/presets/__init__.py b/src/aihwkit/simulator/presets/__init__.py
index 8eb468e4..6107a65d 100644
--- a/src/aihwkit/simulator/presets/__init__.py
+++ b/src/aihwkit/simulator/presets/__init__.py
@@ -15,6 +15,7 @@
 from .configs import (
     # Single device configs.
     ReRamESPreset, ReRamSBPreset, CapacitorPreset, EcRamPreset, IdealizedPreset,
+    GokmenVlasovPreset,
     # 2-device configs.
     ReRamES2Preset, ReRamSB2Preset, Capacitor2Preset, EcRam2Preset, Idealized2Preset,
     # 4-device configs.
diff --git a/src/aihwkit/simulator/rpu_base_src/rpu_base_devices.cpp b/src/aihwkit/simulator/rpu_base_src/rpu_base_devices.cpp
index f8910718..7aa3e446 100644
--- a/src/aihwkit/simulator/rpu_base_src/rpu_base_devices.cpp
+++ b/src/aihwkit/simulator/rpu_base_src/rpu_base_devices.cpp
@@ -345,6 +345,7 @@ void declare_rpu_devices(py::module &m) {
       .def_readwrite("gamma_down_dtod", &LinearStepParam::ls_decrease_down_dtod)
       .def_readwrite("allow_increasing", &LinearStepParam::ls_allow_increasing_slope)
       .def_readwrite("mean_bound_reference", &LinearStepParam::ls_mean_bound_reference)
+      .def_readwrite("write_noise_std", &LinearStepParam::write_noise_std)
       .def_readwrite("mult_noise", &LinearStepParam::ls_mult_noise)
       .def("__str__", [](LinearStepParam &self) {
         std::stringstream ss;
@@ -355,6 +356,7 @@ void declare_rpu_devices(py::module &m) {
   py::class_<SoftBoundsParam, PySoftBoundsParam, PulsedParam>(
       m, "SoftBoundsResistiveDeviceParameter")
       .def_readwrite("mult_noise", &SoftBoundsParam::ls_mult_noise)
+      .def_readwrite("write_noise_std", &SoftBoundsParam::write_noise_std)
       .def(py::init<>())
       .def("__str__", [](SoftBoundsParam &self) {
         std::stringstream ss;
@@ -370,6 +372,7 @@ void declare_rpu_devices(py::module &m) {
       .def_readwrite("gamma_down", &ExpStepParam::es_gamma_down)
       .def_readwrite("a", &ExpStepParam::es_a)
       .def_readwrite("b", &ExpStepParam::es_b)
+      .def_readwrite("write_noise_std", &ExpStepParam::write_noise_std)
       .def("__str__", [](ExpStepParam &self) {
         std::stringstream ss;
         self.printToStream(ss);


Test patch:
diff --git a/tests/helpers/decorators.py b/tests/helpers/decorators.py
index 2892621c..305158b7 100644
--- a/tests/helpers/decorators.py
+++ b/tests/helpers/decorators.py
@@ -48,8 +48,8 @@ def class_name(cls, _, params_dict):
 def parametrize_over_layers(layers: List, tiles: List, biases: List) -> Callable:
     """Parametrize a TestCase over different kind of layers.
 
-    The ``TestCase`` will be repeated for each combination of
-    `layer`, `tile` and `bias`.
+    The ``TestCase`` will be repeated for each combination of `layer`, `tile`
+    and `bias`.
 
     Args:
         layers: list of layer descriptions.
@@ -80,3 +80,25 @@ def class_name(cls, _, params_dict):
         [object_to_dict(layer, tile, bias) for
          layer, tile, bias in product(layers, tiles, biases)],
         class_name_func=class_name)
+
+
+def parametrize_over_presets(presets: List) -> Callable:
+    """Parametrize a TestCase over different kind of presets.
+
+    Note that this decorator expects a list of Presets, as opposed to a list of
+    helper objects.
+
+    Args:
+        presets: list of presets.
+
+    Returns:
+        The decorated TestCase.
+    """
+
+    def class_name(cls, _, params_dict):
+        """Return a user-friendly name for a parametrized test."""
+        return '{}_{}'.format(cls.__name__, params_dict['preset_cls'])
+
+    return parameterized_class(
+        [{'preset_cls': preset} for preset in presets],
+        class_name_func=class_name)
diff --git a/tests/test_presets.py b/tests/test_presets.py
new file mode 100644
index 00000000..37ceb9e4
--- /dev/null
+++ b/tests/test_presets.py
@@ -0,0 +1,70 @@
+# -*- coding: utf-8 -*-
+
+# (C) Copyright 2020, 2021 IBM. All Rights Reserved.
+#
+# This code is licensed under the Apache License, Version 2.0. You may
+# obtain a copy of this license in the LICENSE.txt file in the root directory
+# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
+#
+# Any modifications or derivative works of this code must retain this
+# copyright notice, and modified files need to carry a notice indicating
+# that they have been altered from the originals.
+
+"""Tests for analog presets."""
+
+from torch import Tensor
+
+from aihwkit.simulator.tiles.analog import AnalogTile
+from aihwkit.simulator.presets import (
+    ReRamESPreset, ReRamSBPreset, CapacitorPreset, EcRamPreset, IdealizedPreset,
+    GokmenVlasovPreset,
+    ReRamES2Preset, ReRamSB2Preset, Capacitor2Preset, EcRam2Preset, Idealized2Preset,
+    ReRamES4Preset, ReRamSB4Preset, Capacitor4Preset, EcRam4Preset, Idealized4Preset,
+    TikiTakaReRamESPreset, TikiTakaReRamSBPreset, TikiTakaCapacitorPreset,
+    TikiTakaEcRamPreset, TikiTakaIdealizedPreset
+)
+from .helpers.decorators import parametrize_over_presets
+from .helpers.testcases import AihwkitTestCase
+
+
+@parametrize_over_presets(
+    [
+        ReRamESPreset, ReRamSBPreset, CapacitorPreset, EcRamPreset, IdealizedPreset,
+        GokmenVlasovPreset,
+        ReRamES2Preset, ReRamSB2Preset, Capacitor2Preset, EcRam2Preset, Idealized2Preset,
+        ReRamES4Preset, ReRamSB4Preset, Capacitor4Preset, EcRam4Preset, Idealized4Preset,
+        TikiTakaReRamESPreset, TikiTakaReRamSBPreset, TikiTakaCapacitorPreset,
+        TikiTakaEcRamPreset, TikiTakaIdealizedPreset
+    ]
+)
+class PresetTest(AihwkitTestCase):
+    """Test for analog presets."""
+
+    def test_tile_preset(self):
+        """Test instantiating a tile that uses a preset."""
+        out_size = 2
+        in_size = 3
+
+        rpu_config = self.preset_cls()
+        analog_tile = AnalogTile(out_size, in_size, rpu_config, bias=False)
+
+        learning_rate = 0.123
+
+        # Use [out_size, in_size] weights.
+        weights = Tensor([[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]])
+
+        # Set some properties in the simulators.Tile.
+        analog_tile.set_learning_rate(0.123)
+        analog_tile.set_weights(weights)
+
+        # Assert over learning rate.
+        self.assertAlmostEqual(analog_tile.get_learning_rate(), learning_rate)
+        self.assertAlmostEqual(analog_tile.get_learning_rate(),
+                               analog_tile.tile.get_learning_rate())
+
+        # Assert over weights and biases.
+        tile_weights, tile_biases = analog_tile.get_weights()
+        self.assertEqual(tuple(tile_weights.shape), (out_size, in_size))
+        self.assertEqual(tile_biases, None)
+        # TODO: disabled as the comparison needs to take into account noise
+        # self.assertTensorAlmostEqual(tile_weights, weights)