Repository: intel/dffml

Issue:
configuration: Change "arg" to "plugin"
`dffml/base.py` and some other places interpret `dict`s with `"arg"` and `"config"` keys in a special way.

`"arg"` is the name of the plugin to load, and `"config"` is the config for that plugin.

Therefore, `"arg"` should really be `"plugin"`

https://github.com/intel/dffml/blob/01d921ce610d809ef3a941d828c8d8c3b4b853ba/dffml/util/data.py#L21-L49

Discussion:
lgtm-com[bot]: This pull request **introduces 1 alert** and **fixes 1** when merging 975d447f63915ea6fc80ff53bf237408c951f2cc into a058e290c78d45341996cc56e0befca3040cc01e - [view on LGTM.com](https://lgtm.com/projects/g/intel/dffml/rev/pr-ef81290fcf82f6ae6ea33be3feb26021d2fcf139)

**new alerts:**

* 1 for \_\_eq\_\_ not overridden when adding attributes

**fixed alerts:**

* 1 for \_\_eq\_\_ not overridden when adding attributes
codecov-io: # [Codecov](https://codecov.io/gh/intel/dffml/pull/521?src=pr&el=h1) Report
> Merging [#521](https://codecov.io/gh/intel/dffml/pull/521?src=pr&el=desc) into [master](https://codecov.io/gh/intel/dffml/commit/a058e290c78d45341996cc56e0befca3040cc01e&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/intel/dffml/pull/521/graphs/tree.svg?width=650&height=150&src=pr&token=B6TSSyBYRi)](https://codecov.io/gh/intel/dffml/pull/521?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master     #521   +/-   ##
=======================================
  Coverage   87.86%   87.86%           
=======================================
  Files         100      100           
  Lines        6361     6361           
  Branches      944      944           
=======================================
  Hits         5589     5589           
  Misses        601      601           
  Partials      171      171           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/intel/dffml/pull/521?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/intel/dffml/pull/521?src=pr&el=footer). Last update [a058e29...975d447](https://codecov.io/gh/intel/dffml/pull/521?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

pdxjohnny: @sakshamarora1 can you merge master and put this under the unreleased section?

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b47e17809..48eb67680 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+### Changed
+- Renamed `"arg"` to `"plugin"`.
 
 ## [0.3.6] - 2020-04-04
 ### Added
diff --git a/dffml/base.py b/dffml/base.py
index 6aa57842b..dcc310920 100644
--- a/dffml/base.py
+++ b/dffml/base.py
@@ -180,7 +180,7 @@ def convert_value(arg, value):
 
 def is_config_dict(value):
     return bool(
-        "arg" in value
+        "plugin" in value
         and "config" in value
         and isinstance(value["config"], dict)
     )
@@ -192,7 +192,7 @@ def _fromdict(cls, **kwargs):
             value = kwargs[field.name]
             config = {}
             if is_config_dict(value):
-                value, config = value["arg"], value["config"]
+                value, config = value["plugin"], value["config"]
             value = convert_value(mkarg(field), value)
             if inspect.isclass(value) and issubclass(value, BaseConfigurable):
                 # TODO This probably isn't 100% correct. Figure out what we need
@@ -200,11 +200,11 @@ def _fromdict(cls, **kwargs):
                 value = value.withconfig(
                     {
                         value.ENTRY_POINT_NAME[-1]: {
-                            "arg": None,
+                            "plugin": None,
                             "config": {
                                 key: value
                                 if is_config_dict(value)
-                                else {"arg": value, "config": {}}
+                                else {"plugin": value, "config": {}}
                                 for key, value in config.items()
                             },
                         }
diff --git a/dffml/df/memory.py b/dffml/df/memory.py
index 6d016a61e..5b9b13d7e 100644
--- a/dffml/df/memory.py
+++ b/dffml/df/memory.py
@@ -599,7 +599,10 @@ class MemoryOperationNetwork(BaseOperationNetwork, BaseMemoryDataFlowObject):
     @classmethod
     def args(cls, args, *above) -> Dict[str, Arg]:
         cls.config_set(
-            args, above, "ops", Arg(type=Operation.load, nargs="+", default=[])
+            args,
+            above,
+            "ops",
+            Arg(type=Operation.load, nargs="+", default=[]),
         )
         return args
 
diff --git a/dffml/util/data.py b/dffml/util/data.py
index 0dbea118f..5a6b24f59 100644
--- a/dffml/util/data.py
+++ b/dffml/util/data.py
@@ -28,16 +28,16 @@ def traverse_config_set(target, *args):
     """
     >>> traverse_config_set({
     ...     "level": {
-    ...         "arg": None,
+    ...         "plugin": None,
     ...         "config": {
     ...             "one": {
-    ...                 "arg": 1,
+    ...                 "plugin": 1,
     ...                 "config": {},
     ...             },
     ...         },
     ...     },
     ... }, "level", "one", 42)
-    {'level': {'arg': None, 'config': {'one': {'arg': 42, 'config': {}}}}}
+    {'level': {'plugin': None, 'config': {'one': {'plugin': 42, 'config': {}}}}}
     """
     # Seperate the path down from the value to set
     path, value = args[:-1], args[-1]
@@ -45,10 +45,10 @@ def traverse_config_set(target, *args):
     last = target
     for level in path:
         if not level in current:
-            current[level] = {"arg": None, "config": {}}
+            current[level] = {"plugin": None, "config": {}}
         last = current[level]
         current = last["config"]
-    last["arg"] = value
+    last["plugin"] = value
     return target
 
 
@@ -56,10 +56,10 @@ def traverse_config_get(target, *args):
     """
     >>> traverse_config_get({
     ...     "level": {
-    ...         "arg": None,
+    ...         "plugin": None,
     ...         "config": {
     ...             "one": {
-    ...                 "arg": 1,
+    ...                 "plugin": 1,
     ...                 "config": {},
     ...             },
     ...         },
@@ -72,7 +72,7 @@ def traverse_config_get(target, *args):
     for level in args:
         last = current[level]
         current = last["config"]
-    return last["arg"]
+    return last["plugin"]
 
 
 def traverse_get(target, *args):
diff --git a/docs/contributing/codebase.rst b/docs/contributing/codebase.rst
index b40ab9d8c..929b7ce2c 100644
--- a/docs/contributing/codebase.rst
+++ b/docs/contributing/codebase.rst
@@ -112,7 +112,7 @@ For example:
 .. code-block:: yaml
 
     model:
-      arg: tfdnnc
+      plugin: tfdnnc
       config:
         epochs: 400
         steps: 4000
@@ -134,8 +134,8 @@ For example:
           length: 10
           name: work
 
-Currently, ``arg`` needs to be renamed to ``plugin``, it signifies the plugin to
-load. ``config`` is the ``...Config`` class as a dict for that plugin.
+Here, ``plugin`` is the ``...Arg`` class which signifies the plugin to load.
+``config`` is the ``...Config`` class as a dict for that plugin.
 
 The command line equivalent for the model is...
 
@@ -152,8 +152,6 @@ The command line equivalent for the model is...
           commits:int:10 \
           work:int:10 \
 
-The reason it's called ``arg`` right now is because the parsing of the command
-line came first and the argument is stored there when it's not nested.
 
 setup.py
 --------
diff --git a/scripts/docs.py b/scripts/docs.py
index afada0865..5d9c0f535 100644
--- a/scripts/docs.py
+++ b/scripts/docs.py
@@ -72,20 +72,24 @@ def sanitize_default(default):
 def build_args(config):
     args = []
     for key, value in config.items():
-        arg = value["arg"]
-        if arg is None:
+        plugin = value["plugin"]
+        if plugin is None:
             continue
         build = ""
         build += "- %s: %s\n" % (
             key,
-            data_type_string(arg.get("type", str), arg.get("nargs", None)),
+            data_type_string(
+                plugin.get("type", str), plugin.get("nargs", None)
+            ),
         )
-        if "default" in arg or "help" in arg:
+        if "default" in plugin or "help" in plugin:
             build += "\n"
-        if "default" in arg:
-            build += "  - default: %s\n" % (sanitize_default(arg["default"]),)
-        if "help" in arg:
-            build += "  - %s\n" % (arg["help"],)
+        if "default" in plugin:
+            build += "  - default: %s\n" % (
+                sanitize_default(plugin["default"]),
+            )
+        if "help" in plugin:
+            build += "  - %s\n" % (plugin["help"],)
         args.append(build.rstrip())
     if args:
         return "**Args**\n\n" + "\n\n".join(args)
diff --git a/service/http/docs/api.rst b/service/http/docs/api.rst
index 5bf4d9989..877f64662 100644
--- a/service/http/docs/api.rst
+++ b/service/http/docs/api.rst
@@ -111,17 +111,17 @@ sources.
     {
       "csv": {
         "source": {
-          "arg": null,
+          "plugin": null,
           "config": {
             "csv": {
-              "arg": null,
+              "plugin": null,
               "config": {
                 "filename": {
-                  "arg": {},
+                  "plugin": {},
                   "config": {}
                 },
                 "readwrite": {
-                  "arg": {
+                  "plugin": {
                     "type": "bool",
                     "action": "store_true",
                     "default": false
@@ -129,7 +129,7 @@ sources.
                   "config": {}
                 },
                 "allowempty": {
-                  "arg": {
+                  "plugin": {
                     "type": "bool",
                     "action": "store_true",
                     "default": false
@@ -137,14 +137,14 @@ sources.
                   "config": {}
                 },
                 "label": {
-                  "arg": {
+                  "plugin": {
                     "type": "str",
                     "default": "unlabeled"
                   },
                   "config": {}
                 },
                 "key": {
-                  "arg": {
+                  "plugin": {
                     "type": "str",
                     "default": null
                   },
@@ -207,16 +207,16 @@ URL this ``POST`` request is sent to is.
 
     {
       "source": {
-        "arg": null,
+        "plugin": null,
         "config": {
           "filename": {
-            "arg": [
+            "plugin": [
               "dataset.csv"
             ],
             "config": {}
           },
           "readwrite": {
-            "arg": [
+            "plugin": [
               true
             ],
             "config": {}
@@ -237,16 +237,16 @@ The following is an example request body to configure a model. The URL this
 
   {
     "model": {
-      "arg": null,
+      "plugin": null,
       "config": {
         "directory": {
-          "arg": [
+          "plugin": [
             "/home/user/modeldirs/mymodel"
           ],
           "config": {}
         },
         "features": {
-          "arg": [
+          "plugin": [
             {
               "name": "Years",
               "dtype": "int",
diff --git a/service/http/docs/python.rst b/service/http/docs/python.rst
index d28e20cf6..b0cee4605 100644
--- a/service/http/docs/python.rst
+++ b/service/http/docs/python.rst
@@ -24,16 +24,16 @@ This is an example of how you can use the web API from Python.
   PARAMS = {
     {
       "model": {
-        "arg": null,
+        "plugin": null,
         "config": {
           "directory": {
-            "arg": [
+            "plugin": [
               "/home/user/modeldirs/mymodel"
             ],
             "config": {}
           },
           "features": {
-            "arg": [
+            "plugin": [
               {
                 "name": "Years",
                 "dtype": "int",
diff --git a/service/http/examples/javascript/config.json b/service/http/examples/javascript/config.json
index a7ddc5933..961b410c4 100644
--- a/service/http/examples/javascript/config.json
+++ b/service/http/examples/javascript/config.json
@@ -1,9 +1,9 @@
 {
   "source": {
-    "arg": null,
+    "plugin": null,
     "config": {
       "filename": {
-        "arg": [
+        "plugin": [
           "mydataset.json"
         ],
         "config": {}
diff --git a/service/http/examples/javascript/index.js b/service/http/examples/javascript/index.js
index e9eb77764..cace69be2 100644
--- a/service/http/examples/javascript/index.js
+++ b/service/http/examples/javascript/index.js
@@ -20,10 +20,10 @@ var runit = async function() {
 
   await training_source.configure("csv", "my_training_dataset", {
     "source": {
-      "arg": null,
+      "plugin": null,
       "config": {
         "filename": {
-          "arg": [
+          "plugin": [
             "my_training_dataset.csv"
           ],
           "config": {}
@@ -50,10 +50,10 @@ var runit = async function() {
 
   await test_source.configure("csv", "my_test_dataset", {
     "source": {
-      "arg": null,
+      "plugin": null,
       "config": {
         "filename": {
-          "arg": [
+          "plugin": [
             "my_test_dataset.csv"
           ],
           "config": {}
@@ -82,10 +82,10 @@ var runit = async function() {
 
   await model.configure("scikitlr", "mymodel", {
     "model": {
-      "arg": null,
+      "plugin": null,
       "config": {
         "predict": {
-          "arg": [{
+          "plugin": [{
             "name": "Salary",
             "dtype": "int",
             "length": 1
@@ -93,7 +93,7 @@ var runit = async function() {
           "config": {}
         },
         "features": {
-          "arg": [{
+          "plugin": [{
             "name": "Years",
             "dtype": "int",
             "length": 1


Test patch:
diff --git a/tests/df/test_memory.py b/tests/df/test_memory.py
index 60b4516a3..4c3c312e4 100644
--- a/tests/df/test_memory.py
+++ b/tests/df/test_memory.py
@@ -45,22 +45,22 @@ class TestMemoryRedundancyChecker(AsyncTestCase):
             MemoryRedundancyChecker.args({}),
             {
                 "rchecker": {
-                    "arg": None,
+                    "plugin": None,
                     "config": {
                         "memory": {
-                            "arg": None,
+                            "plugin": None,
                             "config": {
                                 "kvstore": {
-                                    "arg": Arg(
+                                    "plugin": Arg(
                                         type=BaseKeyValueStore.load,
                                         default=MemoryKeyValueStore,
                                     ),
                                     "config": {
                                         "withargs": {
-                                            "arg": None,
+                                            "plugin": None,
                                             "config": {
                                                 "filename": {
-                                                    "arg": Arg(type=str),
+                                                    "plugin": Arg(type=str),
                                                     "config": {},
                                                 }
                                             },
diff --git a/tests/source/test_file.py b/tests/source/test_file.py
index 3c0c8da9c..41e9fce69 100644
--- a/tests/source/test_file.py
+++ b/tests/source/test_file.py
@@ -44,17 +44,17 @@ class TestFileSource(AsyncTestCase):
             FileSource.args({}),
             {
                 "source": {
-                    "arg": None,
+                    "plugin": None,
                     "config": {
                         "file": {
-                            "arg": None,
+                            "plugin": None,
                             "config": {
                                 "filename": {
-                                    "arg": Arg(type=str),
+                                    "plugin": Arg(type=str),
                                     "config": {},
                                 },
                                 "readwrite": {
-                                    "arg": Arg(
+                                    "plugin": Arg(
                                         type=bool,
                                         action="store_true",
                                         default=False,
@@ -62,7 +62,7 @@ class TestFileSource(AsyncTestCase):
                                     "config": {},
                                 },
                                 "allowempty": {
-                                    "arg": Arg(
+                                    "plugin": Arg(
                                         type=bool,
                                         action="store_true",
                                         default=False,
@@ -70,7 +70,9 @@ class TestFileSource(AsyncTestCase):
                                     "config": {},
                                 },
                                 "tag": {
-                                    "arg": Arg(type=str, default="untagged"),
+                                    "plugin": Arg(
+                                        type=str, default="untagged"
+                                    ),
                                     "config": {},
                                 },
                             },
diff --git a/tests/test_base.py b/tests/test_base.py
index aac553ff9..02143ebea 100644
--- a/tests/test_base.py
+++ b/tests/test_base.py
@@ -45,18 +45,21 @@ class TestAutoArgsConfig(unittest.TestCase):
             FakeTesting.args({}),
             {
                 "test": {
-                    "arg": None,
+                    "plugin": None,
                     "config": {
                         "fake": {
-                            "arg": None,
+                            "plugin": None,
                             "config": {
-                                "num": {"arg": Arg(type=float), "config": {}},
+                                "num": {
+                                    "plugin": Arg(type=float),
+                                    "config": {},
+                                },
                                 "files": {
-                                    "arg": Arg(type=str, nargs="+"),
+                                    "plugin": Arg(type=str, nargs="+"),
                                     "config": {},
                                 },
                                 "features": {
-                                    "arg": Arg(
+                                    "plugin": Arg(
                                         type=Feature.load,
                                         nargs="+",
                                         action=list_action(Features),
@@ -64,13 +67,13 @@ class TestAutoArgsConfig(unittest.TestCase):
                                     "config": {},
                                 },
                                 "name": {
-                                    "arg": Arg(
+                                    "plugin": Arg(
                                         type=str, help="Name of FakeTesting"
                                     ),
                                     "config": {},
                                 },
                                 "readonly": {
-                                    "arg": Arg(
+                                    "plugin": Arg(
                                         type=bool,
                                         action="store_true",
                                         default=False,
@@ -78,11 +81,13 @@ class TestAutoArgsConfig(unittest.TestCase):
                                     "config": {},
                                 },
                                 "label": {
-                                    "arg": Arg(type=str, default="unlabeled"),
+                                    "plugin": Arg(
+                                        type=str, default="unlabeled"
+                                    ),
                                     "config": {},
                                 },
                                 "source": {
-                                    "arg": Arg(
+                                    "plugin": Arg(
                                         type=BaseSource.load,
                                         default=JSONSource,
                                     ),
diff --git a/tests/util/test_cli.py b/tests/util/test_cli.py
index 7f29886bf..d53aa83cb 100644
--- a/tests/util/test_cli.py
+++ b/tests/util/test_cli.py
@@ -204,19 +204,19 @@ class TestArg(unittest.TestCase):
             parsed,
             {
                 "rchecker": {
-                    "arg": None,
+                    "plugin": None,
                     "config": {
                         "memory": {
-                            "arg": None,
+                            "plugin": None,
                             "config": {
                                 "kvstore": {
-                                    "arg": ["withargs"],
+                                    "plugin": ["withargs"],
                                     "config": {
                                         "withargs": {
-                                            "arg": None,
+                                            "plugin": None,
                                             "config": {
                                                 "filename": {
-                                                    "arg": ["somefile"],
+                                                    "plugin": ["somefile"],
                                                     "config": {},
                                                 }
                                             },