Repository: ipython/traitlets
Pull request #322 (2016-09-22 15:53:01)

Use argparse to configure container-traits from command-line
- The idea was to  scan all conf-classes to collect `List/Set/Tuple/Dict` traits and use them to setup *argparse* for reading cmd-line args either with action='append' or nargs='*'`.
- Additionally, do the same for any matching aliases (for flags this does not make sense).
- But eventually had to feed all config-traits through *argparse* when classes are provided.
The reason is that any "regular"(not handled by *argparse*) trait with a name that is a prefix of some container-trait will get miss-classified as that container-trait (disabling this [`allow_abbrev`](https://docs.python.org/3/library/argparse.html#allow-abbrev) behavior is not supported in py2).


## Design
1. The `Application` on `parse_command_line()` uses its `classes` field to collects all relevant `HasTraits` classes and subclasses to push them into the `KVArgParseConfigLoader` constructor;
2. The `KVArgParseConfigLoader.load_config()` invokes `_add_arguments()` for all *list/Set/tuple* traits found on the collected classes , and appends them as *argparse* arguments according to the `multiplicity` metadata found on these traits:

    1. `multiplicity == 'append'`: use `add_argument(..., action='append')` (default if metadata missing);
    2.  `multiplicity == [ '*' | '+' | N ]`: use `add_argument(..., nargs=multiplicity)`.

3. During the above step, all container traits are collected in the new loder's field `argparse_traits` to be used in the next step to create the appropriate config-tree instance.
4. After *argparse* has parsed cmd-line arguments, the `KVArgParseConfigLoader._convert_to_config()` py-evaluates any lists-of-values (or list-of-tuples for `Dict` traits) one by one before creating and assigning the appropriate instance (i.e. `dict` for `Dict` traits) into the returned config-instance .

## Example
Assuming this 2 configurables + 1 app:

```python
class Base(Configurable): 
    a = List([]).tag(config=True)

class Sub(Base): 
    atuple = Tuple([]).tag(config=True, multiplicity='*')
    aset = Set(set()).tag(config=True, multiplicity='append')

class Cmd(Application):
    classes = [Sub]  ## Note on #289: frequently list given like that.
    aliases = {
        'a':  'Base.a', 
        'aa': 'Sub.a', 
        't':  'Sub.atuple', 
        's':  'Sub.aset'
    }
```

Then, in command-line it should be possible to enter all these commands:

```
cmd  --Base.a=1 -a=foo      ## Base.a = [1, 'foo']
cmd  --aa=1  --Sub.a=foo    ## Sub.a  = [1, 'foo']
cmd  -t 1 ab cd 4  --Sub.aset a  -s b  -s a  ## Sub.b = (1, 'ab', 'cd', 4), Sub.set = {'a', 'b'}
cmd  --Base.a=1 --Base.a=2  --Sub.a=foo      ## Base.a = [1, 2], Sub.a = ['foo']
```
* Notice that in the last case, `Sub` class trumps over any `Base` values.

For a `Dict` trait, the cmd-line syntax is this:

      cmd   --Base.adict kk=vv -D=key1=v1  -D key2=True -Dkey3="'True'"

resulting in this:

    Base.adict = {'kk': 'v', 'key1': 'v1', 'key2': True, 'key3: 'True'}


## Future
- Merge parts of `KVArgParseConfigLoader` into `ArgParseConfigLoader` class - actually I would try to compress the *loader* hierarchy.

ankostis: @minrk At [`traitlets/traitlests.py#1378`](https://github.com/ipython/traitlets/blob/master/traitlets/traitlets.py#L1378) it tests whether a variable can be called; why is it using `type(var) is types.FunctionType` instead of `iscallable(var)`?

The same question applies for [all such tests](http://stackoverflow.com/questions/624926/how-to-detect-whether-a-python-variable-is-a-function).
minrk: @ankostis I suspect checking for `callable` is the right thing to do in those cases. I've updated the checks in #323. That's a real old line, probably from the time when Python 3 had removed `callable`, which it's since put back.
ankostis: @minrk  PLEASE REVIEW the code + specs in the OP, to eventual become material for the manual.

ankostis: PR works fine - maybe some TCs could also test more fringe cases, e.g. what happens when a List trait without defaults and without `allow_null` does not receive cmd-line values, etc.
But I had some mild problems extending it to Dict-traits.

In general, for a next release it might make more sense to drop the "mixed" parsing of args when conf-classes are available and delegate everything to *arparse*.  
Specifically to rely on `type` functions of `ArgumentParser.add_argument()` method instead of `_exec_config_str()`, and introduce a new method that assembles all the collected values (list, dictionary) for some config-trait.
The later was the problem I mentoned for extending this PR also for Dict-traits.

By the way, why in [loader.py#L517](https://github.com/ipython/traitlets/blob/master/traitlets/config/loader.py#L517) it uses eval to set the Config tree?
ankostis: ankostis asked:
> By the way, why in [loader.py#L517](https://github.com/ipython/traitlets/blob/master/traitlets/config/loader.py#L517) it uses eval to set the Config tree?

Answering myself, I found 329db369 where @ellisonbg in 2009(!) did this change:

```diff
-            if v is not NoDefault:
-                setattr(self.config, k, v)
+            if v is not NoConfigDefault:
+                exec_str = 'self.config.' + k + '= v'
+                exec exec_str in locals(), globals()
```

unfortunately without explaining the reason - the only relative part of the commit message must be this:

> ...  The config system now supports config 
  subsections, like config.Global.  These sections, which must start
  with an uppercase char, are auto-created upon a getattr.

When using `setattr()` many TCs fail with this message:

```
Traceback (most recent call last):
  File "D:\Work\traitlets.git\traitlets\config\tests\test_loader.py", line 336, in test_seq_traits
    config = cl.load_config(argv, aliases=aliases)
  File "D:\Work\traitlets.git\traitlets\config\loader.py", line 761, in load_config
    self._convert_to_config()
  File "D:\Work\traitlets.git\traitlets\config\loader.py", line 903, in _convert_to_config
    self._exec_config_str(k, v, trait=trait)
  File "D:\Work\traitlets.git\traitlets\config\loader.py", line 529, in _exec_config_str
    setattr(self.config, lhs, value)
  File "D:\Work\traitlets.git\traitlets\config\loader.py", line 287, in __setattr__
    self.__setitem__(key, value)
  File "D:\Work\traitlets.git\traitlets\config\loader.py", line 272, in __setitem__
    'char must be Config instances: %r, %r' % (key, value))
ValueError: values whose keys begin with an uppercase char must be Config instances: 'CSub.e', [1, 'a', 'bcd']
```

Any explaination why?
ankostis: With the latest commit, also `Dict` traits can be set by cmd-line *argparse*  options.
Design OP document updated.
minrk: A simple setattr doesn't work because it's nested. It can be A.B.c. You could parse out the name and make a nested series of setattrs. I'm not sure that would help anything. I don't think it's relevant to this PR. 
ankostis: Thanks. So `eval` is not used for the `globals()`, and I can actually remove it; will re-write last commit.

You see, after my changes, in py2.7 I was getting [the following error](https://travis-ci.org/ankostis/traitlets/jobs/162436926) in `exec` line, and I had to add `locals()` and probably `globals()`.  My question was whether the `eval` was needed because internally the `config` object did something with the globals

```
../../../virtualenv/python2.7.9/lib/python2.7/site-packages/_pytest/python.py:463: in _importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=True)
../../../virtualenv/python2.7.9/lib/python2.7/site-packages/py/_path/local.py:641: in pyimport
    __import__(modname)
traitlets/config/__init__.py:6: in <module>
    from .application import *
traitlets/config/application.py:19: in <module>
    from traitlets.config.configurable import Configurable, SingletonConfigurable
traitlets/config/configurable.py:12: in <module>
    from .loader import Config, LazyConfigValue, _is_section_key
E   SyntaxError: unqualified exec is not allowed in function '_exec_config_str' because it contains a nested function with free variables (loader.py, line 528)
```
ankostis: @minrk there is a subtlety with the new code where half args are fed into *arparse* and the rest parsed as "unknown-args".  Since `ArgumentParser`  is configured with `allow_abbrev=True`, any regular trait with name that is a prefix of some container-trait will be miss-classified by argparse as that container-trait.

Can be dealt:
- by disabling `allow_abbrev`, BUT the flag was introduced in >py3.5 and behavior cannot change in older pythons, or
- by feeding all traits into *argparse*, and keep abbreviations for all traits!

The later may be beneficial to the users of programs (i.e. Jupyter) that have long options, so abbreviations would be a real bonus to them.

In any case, I retrofitted app-TCs to detect it the problem.
ankostis: Well, tried the "future" (7678ffc78, all traits parsed with *arparse*, the 2nd option above), and it worked like charm, except from 1 TC.  I had to disable the `test-app.test_ipython_cli_priority()` TC because of the following error: 

```
 File "D:\Work\traitlets.git\traitlets\config\tests\test_application.py", line 208, in test_ipython_cli_priority
    app.parse_command_line(['--v=cli'])
  File "<decorator-gen-4>", line 2, in parse_command_line
  File "D:\Work\traitlets.git\traitlets\config\application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  ... 
  File "D:\Apps\WinPython-64bit-3.5.2.1\python-3.5.2.amd64\lib\argparse.py", line 1497, in _check_conflict
    conflict_handler(action, confl_optionals)
  File "D:\Apps\WinPython-64bit-3.5.2.1\python-3.5.2.amd64\lib\argparse.py", line 1506, in _handle_conflict_error
    raise ArgumentError(action, message % conflict_string)
argparse.ArgumentError: argument --TestApp.config_file_loaded: conflicting option string: --TestApp.config_file_loaded
```

The situation is a kind of magic: the code failing, till that point is exactly the same with the previous TC.  Actually there is this comment on the TC:

>  this test is almost entirely redundant with above,
> but we can keep it around in case of subtle issues creeping into
> the exact sequence IPython follows.

@minrk  any ideas why it is failing?
minrk: I'm not sure where argparse is getting the conflicting string, that's a bit baffling to me. But it is pretty important that this test passes without modification to the test case.

I'll try to think about what's going on.
minrk: #325 fixes the issue with the failing test. It was an unrelated bug that this PR triggered a test for. Yay!
ankostis: @minrk wrote:
> I'm not sure where argparse is getting the conflicting string, that's a bit baffling to me. 

Look at the conflicting definitions of [`Bar.b`](https://github.com/ankostis/traitlets/blob/c1453515eb360b7aea575261136ef32d407ade70/traitlets/config/tests/test_application.py#L50) and [`Bar.bdict`](https://github.com/ankostis/traitlets/blob/c1453515eb360b7aea575261136ef32d407ade70/traitlets/config/tests/test_application.py#L54) on `traitlets/config/tests/test_application.py`.  If user specifies `Bar.b=1`, *arparse* assumes abbreviation of `Bar.bdict=1` which of course is not what we want.


> But it is pretty important that this test passes without modification to the test case.

PR rebased on latest master to get #325, and still `test-app.test_ipython_cli_priority()` TC fails.
See https://travis-ci.org/ankostis/traitlets/jobs/162732578
minrk: I hadn't merged #325 yet, so rebasing wouldn't have picked it up. It's merged now, though.

Patch:
diff --git a/traitlets/config/application.py b/traitlets/config/application.py
index a3f790a..91a6f12 100644
--- a/traitlets/config/application.py
+++ b/traitlets/config/application.py
@@ -413,7 +413,7 @@ class Application(SingletonConfigurable):
         self.print_options()
 
         if classes:
-            help_classes = self._classes_in_config_sample()
+            help_classes = self._classes_with_config_traits()
             if help_classes:
                 print("Class parameters")
                 print("----------------")
@@ -565,8 +565,10 @@ class Application(SingletonConfigurable):
 
         # flatten flags&aliases, so cl-args get appropriate priority:
         flags,aliases = self.flatten_flags()
+        classes = tuple(self._classes_with_config_traits())
         loader = KVArgParseConfigLoader(argv=argv, aliases=aliases,
-                                        flags=flags, log=self.log)
+                                        flags=flags, log=self.log,
+                                        classes=classes)
         self.cli_config = deepcopy(loader.load_config())
         self.update_config(self.cli_config)
         # store unparsed args in extra_args
@@ -636,7 +638,7 @@ class Application(SingletonConfigurable):
         self.update_config(new_config)
 
 
-    def _classes_in_config_sample(self):
+    def _classes_with_config_traits(self):
         """
         Yields only classes with own traits, and their subclasses.
 
@@ -672,7 +674,7 @@ class Application(SingletonConfigurable):
         """generate default config file from Configurables"""
         lines = ["# Configuration file for %s." % self.name]
         lines.append('')
-        for cls in self._classes_in_config_sample():
+        for cls in self._classes_with_config_traits():
             lines.append(cls.class_config_section())
         return '\n'.join(lines)
 
diff --git a/traitlets/config/configurable.py b/traitlets/config/configurable.py
index bbc1060..86171c6 100644
--- a/traitlets/config/configurable.py
+++ b/traitlets/config/configurable.py
@@ -10,7 +10,15 @@ from copy import deepcopy
 import warnings
 
 from .loader import Config, LazyConfigValue, _is_section_key
-from traitlets.traitlets import HasTraits, Instance, observe, observe_compat, default
+from traitlets.traitlets import (
+    HasTraits,
+    Instance,
+    Container,
+    Dict,
+    observe,
+    observe_compat,
+    default,
+)
 from ipython_genutils.text import indent, dedent, wrap_paragraphs
 
 
@@ -227,7 +235,20 @@ class Configurable(HasTraits):
         """
         assert inst is None or isinstance(inst, cls)
         lines = []
-        header = "--%s.%s=<%s>" % (cls.__name__, trait.name, trait.__class__.__name__)
+        header = "--%s.%s" % (cls.__name__, trait.name)
+        if isinstance(trait, (Container, Dict)):
+            multiplicity = trait.metadata.get('multiplicity', 'append')
+            if isinstance(trait, Dict):
+                sample_value = '<key-1>=<value-1>'
+            else:
+                sample_value = '<%s-item-1>' % trait.__class__.__name__.lower()
+            if multiplicity == 'append':
+                header = "%s=%s..." % (header, sample_value)
+            else:
+                header = "%s %s..." % (header, sample_value)
+        else:
+            header = '%s=<%s>' % (header, trait.__class__.__name__)
+        #header = "--%s.%s=<%s>" % (cls.__name__, trait.name, trait.__class__.__name__)
         lines.append(header)
         if inst is not None:
             lines.append(indent('Current: %r' % getattr(inst, trait.name), 4))
diff --git a/traitlets/config/loader.py b/traitlets/config/loader.py
index 3a75350..f424c64 100644
--- a/traitlets/config/loader.py
+++ b/traitlets/config/loader.py
@@ -6,7 +6,7 @@
 
 import argparse
 import copy
-import logging
+import functools as fnt
 import os
 import re
 import sys
@@ -17,7 +17,7 @@ from ipython_genutils.path import filefind
 from ipython_genutils import py3compat
 from ipython_genutils.encoding import DEFAULT_ENCODING
 from six import text_type
-from traitlets.traitlets import HasTraits, List, Any
+from traitlets.traitlets import (HasTraits, Container, List, Dict, Any)
 
 #-----------------------------------------------------------------------------
 # Exceptions
@@ -62,36 +62,36 @@ class ArgumentParser(argparse.ArgumentParser):
 
 class LazyConfigValue(HasTraits):
     """Proxy object for exposing methods on configurable containers
-    
+
     Exposes:
-    
+
     - append, extend, insert on lists
     - update on dicts
     - update, add on sets
     """
-    
+
     _value = None
-    
+
     # list methods
     _extend = List()
     _prepend = List()
-    
+
     def append(self, obj):
         self._extend.append(obj)
-    
+
     def extend(self, other):
         self._extend.extend(other)
-    
+
     def prepend(self, other):
         """like list.extend, but for the front"""
         self._prepend[:0] = other
-    
+
     _inserts = List()
     def insert(self, index, other):
         if not isinstance(index, int):
             raise TypeError("An integer is required")
         self._inserts.append((index, other))
-    
+
     # dict methods
     # update is used for both dict and set
     _update = Any()
@@ -102,14 +102,14 @@ class LazyConfigValue(HasTraits):
             else:
                 self._update = set()
         self._update.update(other)
-    
+
     # set methods
     def add(self, obj):
         self.update({obj})
-    
+
     def get_value(self, initial):
         """construct the value from the initial one
-        
+
         after applying any insert / extend / update changes
         """
         if self._value is not None:
@@ -120,7 +120,7 @@ class LazyConfigValue(HasTraits):
                 value.insert(idx, obj)
             value[:0] = self._prepend
             value.extend(self._extend)
-        
+
         elif isinstance(value, dict):
             if self._update:
                 value.update(self._update)
@@ -129,10 +129,10 @@ class LazyConfigValue(HasTraits):
                 value.update(self._update)
         self._value = value
         return value
-    
+
     def to_dict(self):
         """return JSONable dict form of my data
-        
+
         Currently update as dict or set, extend, prepend as lists, and inserts as list of tuples.
         """
         d = {}
@@ -161,10 +161,10 @@ class Config(dict):
     def __init__(self, *args, **kwds):
         dict.__init__(self, *args, **kwds)
         self._ensure_subconfig()
-    
+
     def _ensure_subconfig(self):
         """ensure that sub-dicts that should be Config objects are
-        
+
         casts dicts that are under section keys to Config objects,
         which is necessary for constructing Config objects from dict literals.
         """
@@ -174,11 +174,11 @@ class Config(dict):
                     and isinstance(obj, dict) \
                     and not isinstance(obj, Config):
                 setattr(self, key, Config(obj))
-    
+
     def _merge(self, other):
         """deprecated alias, use Config.merge()"""
         self.merge(other)
-    
+
     def merge(self, other):
         """merge another config object into this one"""
         to_update = {}
@@ -194,13 +194,13 @@ class Config(dict):
                     to_update[k] = v
 
         self.update(to_update)
-    
+
     def collisions(self, other):
         """Check for collisions between two config objects.
-        
+
         Returns a dict of the form {"Class": {"trait": "collision message"}}`,
         indicating which values have been ignored.
-        
+
         An empty dict indicates no collisions.
         """
         collisions = {}
@@ -214,7 +214,7 @@ class Config(dict):
                     collisions.setdefault(section, {})
                     collisions[section][key] = "%r ignored, using %r" % (mine[key], theirs[key])
         return collisions
-    
+
     def __contains__(self, key):
         # allow nested contains of the form `"Section.key" in config`
         if '.' in key:
@@ -222,15 +222,15 @@ class Config(dict):
             if first not in self:
                 return False
             return remainder in self[first]
-        
+
         return super(Config, self).__contains__(key)
-    
+
     # .has_key is deprecated for dictionaries.
     has_key = __contains__
-    
+
     def _has_section(self, key):
         return _is_section_key(key) and key in self
-    
+
     def copy(self):
         return type(self)(dict.copy(self))
 
@@ -248,7 +248,7 @@ class Config(dict):
                 value = copy.copy(value)
             new_config[key] = value
         return new_config
-    
+
     def __getitem__(self, key):
         try:
             return dict.__getitem__(self, key)
@@ -456,12 +456,12 @@ class PyFileConfigLoader(FileConfigLoader):
             raise ConfigFileNotFound(str(e))
         self._read_file_as_dict()
         return self.config
-    
+
     def load_subconfig(self, fname, path=None):
         """Injected into config file namespace as load_subconfig"""
         if path is None:
             path = self.path
-        
+
         loader = self.__class__(fname, path)
         try:
             sub_config = loader.load_config()
@@ -471,13 +471,13 @@ class PyFileConfigLoader(FileConfigLoader):
             pass
         else:
             self.config.merge(sub_config)
-    
+
     def _read_file_as_dict(self):
         """Load the config file into self.config, with recursive loading."""
         def get_config():
             """Unnecessary now, but a deprecation warning is more trouble than it's worth."""
             return self.config
-        
+
         namespace = dict(
             c=self.config,
             load_subconfig=self.load_subconfig,
@@ -496,14 +496,8 @@ class CommandLineConfigLoader(ConfigLoader):
     here.
     """
 
-    def _exec_config_str(self, lhs, rhs):
-        """execute self.config.<lhs> = <rhs>
-        
-        * expands ~ with expanduser
-        * tries to assign with literal_eval, otherwise assigns with just the string,
-          allowing `--C.a=foobar` and `--C.a="foobar"` to be equivalent.  *Not*
-          equivalent are `--C.a=4` and `--C.a='4'`.
-        """
+    def _parse_config_value(self, rhs):
+        """Python-evaluates any cmd-line argument values."""
         rhs = os.path.expanduser(rhs)
         try:
             # Try to see if regular Python syntax will work. This
@@ -513,8 +507,25 @@ class CommandLineConfigLoader(ConfigLoader):
         except (NameError, SyntaxError, ValueError):
             # This case happens if the rhs is a string.
             value = rhs
+        return value
+
+    def _exec_config_str(self, lhs, rhs, trait=None):
+        """execute self.config.<lhs> = <rhs>
+
+        * expands ~ with expanduser
+        * tries to assign with literal_eval, otherwise assigns with just the string,
+          allowing `--C.a=foobar` and `--C.a="foobar"` to be equivalent.  *Not*
+          equivalent are `--C.a=4` and `--C.a='4'`.
+        """
+        if isinstance(trait, Dict):
+            value = {r[0]: self._parse_config_value(r[1])
+                                for r in rhs}
+        elif isinstance(rhs, (list, tuple)):
+            value = [self._parse_config_value(r) for r in rhs]
+        else:
+            value = self._parse_config_value(rhs)
 
-        exec(u'self.config.%s = value' % lhs)
+        exec(u'self.config.%s = value' % lhs, None, locals())
 
     def _load_flag(self, cfg):
         """update self.config from a flag, which can be a dict or Config"""
@@ -688,12 +699,17 @@ class KeyValueConfigLoader(CommandLineConfigLoader):
 class ArgParseConfigLoader(CommandLineConfigLoader):
     """A loader that uses the argparse module to load from the command line."""
 
-    def __init__(self, argv=None, aliases=None, flags=None, log=None,  *parser_args, **parser_kw):
+    def __init__(self, argv=None, aliases=None, flags=None, log=None, classes=None,
+                 *parser_args, **parser_kw):
         """Create a config loader for use with argparse.
 
         Parameters
         ----------
 
+        classes : optional, list
+          The classes to scan for *container* config-traits and decide
+          for their "multiplicity" when adding them as *argparse* arguments.
+
         argv : optional, list
           If given, used to read command-line arguments from, otherwise
           sys.argv[1:] is used.
@@ -718,6 +734,7 @@ class ArgParseConfigLoader(CommandLineConfigLoader):
         self.argv = argv
         self.aliases = aliases or {}
         self.flags = flags or {}
+        self.classes = classes or ()
 
         self.parser_args = parser_args
         self.version = parser_kw.pop("version", None)
@@ -725,7 +742,7 @@ class ArgParseConfigLoader(CommandLineConfigLoader):
         kwargs.update(parser_kw)
         self.parser_kw = kwargs
 
-    def load_config(self, argv=None, aliases=None, flags=None):
+    def load_config(self, argv=None, aliases=None, flags=None, classes=None):
         """Parse command line arguments and return as a Config object.
 
         Parameters
@@ -738,11 +755,7 @@ class ArgParseConfigLoader(CommandLineConfigLoader):
         self.clear()
         if argv is None:
             argv = self.argv
-        if aliases is None:
-            aliases = self.aliases
-        if flags is None:
-            flags = self.flags
-        self._create_parser(aliases, flags)
+        self._create_parser(aliases, flags, classes)
         self._parse_args(argv)
         self._convert_to_config()
         return self.config
@@ -753,11 +766,16 @@ class ArgParseConfigLoader(CommandLineConfigLoader):
         else:
             return []
 
-    def _create_parser(self, aliases=None, flags=None):
+    def _create_parser(self, aliases=None, flags=None, classes=None):
         self.parser = ArgumentParser(*self.parser_args, **self.parser_kw)
-        self._add_arguments(aliases, flags)
+        self._add_arguments(aliases, flags, classes)
 
-    def _add_arguments(self, aliases=None, flags=None):
+    def _parse_config_traits(self):
+        for cls in self.classes:
+            for trait, traitname in cls.class_traits(config=True).items():
+                yield ()
+
+    def _add_arguments(self, aliases=None, flags=None, classes=None):
         raise NotImplementedError("subclasses must implement _add_arguments")
 
     def _parse_args(self, args):
@@ -772,45 +790,93 @@ class ArgParseConfigLoader(CommandLineConfigLoader):
         for k, v in vars(self.parsed_data).items():
             exec("self.config.%s = v"%k, locals(), globals())
 
+## For Dict traits, describes the cmd-line option as `key=value`
+_kv_opt_pattern = re.compile(r'^([^=]+)=(.*)$')
+
+def _kv_opt(traitname, opt_value):
+    """
+    Used as `type` when adding args into :meth:`ArgumentParser.add_argument()`
+    corresponding to config Dict-traits.
+    """
+    m = _kv_opt_pattern.match(opt_value)
+    if not m:
+        raise ArgumentError("Expecting <key>=<value> for Dict-trait '%s', got %r!"
+                            % (traitname, opt_value))
+    return m.groups()
+
+
 class KVArgParseConfigLoader(ArgParseConfigLoader):
     """A config loader that loads aliases and flags with argparse,
     but will use KVLoader for the rest.  This allows better parsing
     of common args, such as `ipython -c 'print 5'`, but still gets
     arbitrary config with `ipython --InteractiveShell.use_readline=False`"""
 
-    def _add_arguments(self, aliases=None, flags=None):
+    def _add_arguments(self, aliases=None, flags=None, classes=None):
         self.alias_flags = {}
         # print aliases, flags
         if aliases is None:
             aliases = self.aliases
         if flags is None:
             flags = self.flags
+        if classes is None:
+            classes = self.classes
         paa = self.parser.add_argument
-        for keys,value in aliases.items():
+
+        ## An index of all container traits collected::
+        #
+        #     { <traitname>: (<trait>, <argparse-kwds>) }
+        #
+        #  Used to add the correct type into the `config` tree.
+        #  Used also for aliases, not to re-collect them.
+        self.argparse_traits = argparse_traits = {}
+        for cls in classes:
+            for traitname, trait in cls.class_traits(config=True).items():
+                argname = '%s.%s' % (cls.__name__, traitname)
+                argparse_kwds = {'type': text_type}
+                if isinstance(trait, (Container, Dict)):
+                    multiplicity = trait.metadata.get('multiplicity', 'append')
+                    if multiplicity == 'append':
+                        argparse_kwds['action'] = multiplicity
+                    else:
+                        argparse_kwds['nargs'] = multiplicity
+                    if isinstance(trait, Dict):
+                        argparse_kwds['type'] = fnt.partial(_kv_opt, traitname)
+                argparse_traits[argname] = (trait, argparse_kwds)
+                paa('--'+argname, **argparse_kwds)
+
+        for keys, traitname in aliases.items():
             if not isinstance(keys, tuple):
                 keys = (keys, )
             for key in keys:
-                if key in flags:
-                    # flags
-                    nargs = '?'
-                else:
-                    nargs = None
-                if len(key) is 1:
-                    paa('-'+key, '--'+key, type=text_type, dest=value, nargs=nargs)
+                argparse_kwds = {'type': text_type, 'dest': traitname}
+                ## Is alias for a sequence-trait?
+                #
+                if traitname in argparse_traits:
+                    argparse_kwds.update(argparse_traits[traitname][1])
+                    if 'action' in argparse_kwds:
+                        ## A flag+alias should have `nargs='?'` multiplicity,
+                        #  but base config-property had 'append' multiplicity!
+                        #
+                        if key in flags:
+                            raise ArgumentError(
+                                "The alias `%s` for the 'append' sequence "
+                                "config-trait `%s` cannot be also a flag!'"
+                                % (key, traitname))
                 else:
-                    paa('--'+key, type=text_type, dest=value, nargs=nargs)
-        for keys, (value, help) in flags.items():
+                    if key in flags:
+                        argparse_kwds['nargs'] = '?'
+                keys = ('-'+key, '--'+key) if len(key) is 1 else ('--'+key, )
+                paa(*keys, **argparse_kwds)
+
+        for keys, (value, _) in flags.items():
             if not isinstance(keys, tuple):
                 keys = (keys, )
             for key in keys:
                 if key in self.aliases:
-                    #
                     self.alias_flags[self.aliases[key]] = value
                     continue
-                if len(key) is 1:
-                    paa('-'+key, '--'+key, action='append_const', dest='_flags', const=value)
-                else:
-                    paa('--'+key, action='append_const', dest='_flags', const=value)
+                keys = ('-'+key, '--'+key) if len(key) is 1 else ('--'+key, )
+                paa(*keys, action='append_const', dest='_flags', const=value)
 
     def _convert_to_config(self):
         """self.parsed_data->self.config, parse unrecognized extra args via KVLoader."""
@@ -826,8 +892,11 @@ class KVArgParseConfigLoader(ArgParseConfigLoader):
                 # it was a flag that shares the name of an alias
                 subcs.append(self.alias_flags[k])
             else:
+                trait = self.argparse_traits.get(k)
+                if trait:
+                    trait = trait[0]
                 # eval the KV assignment
-                self._exec_config_str(k, v)
+                self._exec_config_str(k, v, trait=trait)
 
         for subc in subcs:
             self._load_flag(subc)


Test patch:
diff --git a/traitlets/config/tests/test_application.py b/traitlets/config/tests/test_application.py
index dcc3c2f..f9efe5c 100644
--- a/traitlets/config/tests/test_application.py
+++ b/traitlets/config/tests/test_application.py
@@ -13,7 +13,7 @@ import logging
 import os
 import sys
 from io import StringIO
-from unittest import TestCase
+from unittest import TestCase, skip
 
 try:
     from unittest import mock
@@ -34,7 +34,7 @@ from traitlets.config.application import (
 
 from ipython_genutils.tempdir import TemporaryDirectory
 from traitlets.traitlets import (
-    Bool, Unicode, Integer, List, Dict
+    Bool, Unicode, Integer, List, Tuple, Set, Dict
 )
 
 class Foo(Configurable):
@@ -42,12 +42,16 @@ class Foo(Configurable):
     i = Integer(0, help="The integer i.").tag(config=True)
     j = Integer(1, help="The integer j.").tag(config=True)
     name = Unicode(u'Brian', help="First name.").tag(config=True)
-
+    la = List([]).tag(config=True)
+    fdict = Dict().tag(config=True, multiplicity='+')
 
 class Bar(Configurable):
 
     b = Integer(0, help="The integer b.").tag(config=True)
     enabled = Bool(True, help="Enable bar.").tag(config=True)
+    tb = Tuple(()).tag(config=True, multiplicity='*')
+    aset = Set().tag(config=True, multiplicity='+')
+    bdict = Dict().tag(config=True)
 
 
 class MyApp(Application):
@@ -64,6 +68,9 @@ class MyApp(Application):
                     ('fooi', 'i') : 'Foo.i',
                     ('j', 'fooj') : 'Foo.j',
                     'name' : 'Foo.name',
+                    'la': 'Foo.la',
+                    'tb': 'Bar.tb',
+                    'D': 'Bar.bdict',
                     'enabled' : 'Bar.enabled',
                     'log-level' : 'Application.log_level',
                 })
@@ -115,6 +122,35 @@ class TestApplication(TestCase):
         self.assertEqual(config.Bar.enabled, False)
         self.assertEqual(config.MyApp.log_level,50)
 
+    def test_config_seq_args(self):
+        app = MyApp()
+        app.parse_command_line("--la 1 --tb AB 2 --Foo.la=ab --Bar.aset S1 S2 S1".split())
+        config = app.config
+        self.assertEqual(config.Foo.la, [1, 'ab'])
+        self.assertEqual(config.Bar.tb, ['AB', 2])
+        self.assertEqual(config.Bar.aset, 'S1 S2 S1'.split())
+        app.init_foo()
+        self.assertEqual(app.foo.la, [1, 'ab'])
+        app.init_bar()
+        self.assertEqual(app.bar.aset, {'S1', 'S2'})
+        self.assertEqual(app.bar.tb, ('AB', 2))
+
+    def test_config_dict_args(self):
+        app = MyApp()
+        app.parse_command_line(
+            "--Foo.fdict a=1 b=b c=3 "
+            "--Bar.bdict k=1 -D=a=b -D 22=33 "
+            .split())
+        fdict = {'a': 1, 'b': 'b', 'c': 3}
+        bdict = {'k': 1, 'a': 'b', '22': 33}
+        config = app.config
+        self.assertDictEqual(config.Foo.fdict, fdict)
+        self.assertDictEqual(config.Bar.bdict, bdict)
+        app.init_foo()
+        self.assertEqual(app.foo.fdict, fdict)
+        app.init_bar()
+        self.assertEqual(app.bar.bdict, bdict)
+
     def test_config_propagation(self):
         app = MyApp()
         app.parse_command_line(["--i=10","--Foo.j=10","--enabled=False","--log-level=50"])
@@ -436,4 +472,4 @@ def test_help_output():
 
 if __name__ == '__main__':
     # for test_help_output:
-    MyApp.launch_instance()
\ No newline at end of file
+    MyApp.launch_instance()
diff --git a/traitlets/config/tests/test_configurable.py b/traitlets/config/tests/test_configurable.py
index fb2e2d5..f94eab4 100644
--- a/traitlets/config/tests/test_configurable.py
+++ b/traitlets/config/tests/test_configurable.py
@@ -5,6 +5,7 @@
 # Distributed under the terms of the Modified BSD License.
 
 import logging
+import sys
 from unittest import TestCase
 
 from pytest import mark
@@ -57,11 +58,44 @@ if PY3:
 class Foo(Configurable):
     a = Integer(0, help="The integer a.").tag(config=True)
     b = Unicode('nope').tag(config=True)
+    flist = List([]).tag(config=True)
+    fdict = Dict().tag(config=True)
 
 
 class Bar(Foo):
     b = Unicode('gotit', help="The string b.").tag(config=False)
     c = Float(help="The string c.").tag(config=True)
+    bset = Set([]).tag(config=True, multiplicity='+')
+    bdict = Dict().tag(config=True, multiplicity='+')
+
+foo_help=u"""Foo(Configurable) options
+-------------------------
+--Foo.a=<Int>
+    Default: 0
+    The integer a.
+--Foo.b=<Unicode>
+    Default: 'nope'
+--Foo.fdict=<key-1>=<value-1>...
+    Default: {}
+--Foo.flist=<list-item-1>...
+    Default: []"""
+
+bar_help=u"""Bar(Foo) options
+----------------
+--Bar.a=<Int>
+    Default: 0
+    The integer a.
+--Bar.bdict <key-1>=<value-1>...
+    Default: {}
+--Bar.bset <set-item-1>...
+    Default: set()
+--Bar.c=<Float>
+    Default: 0.0
+    The string c.
+--Bar.fdict=<key-1>=<value-1>...
+    Default: {}
+--Bar.flist=<list-item-1>...
+    Default: []"""
 
 
 class TestConfigurable(TestCase):
@@ -140,8 +174,12 @@ class TestConfigurable(TestCase):
         self.assertEqual(c.b, 'and')
         self.assertEqual(c.c, 20.0)
 
+    @mark.skipif(sys.version_info < (3, ),
+                 reason="Set, Int in py2 get printed as 'set([])', 'Integer'!'")
     def test_help(self):
         self.assertEqual(MyConfigurable.class_get_help(), mc_help)
+        self.assertEqual(Foo.class_get_help(), foo_help)
+        self.assertEqual(Bar.class_get_help(), bar_help)
 
     def test_help_inst(self):
         inst = MyConfigurable(a=5, b=4)
@@ -164,7 +202,6 @@ class TestSingletonConfigurable(TestCase):
         self.assertEqual(Bar.initialized(), False)
         self.assertEqual(Bam.initialized(), False)
         bam = Bam.instance()
-        bam == Bar.instance()
         self.assertEqual(Bar.initialized(), True)
         self.assertEqual(Bam.initialized(), True)
         self.assertEqual(bam, Bam._instance)
@@ -435,7 +472,7 @@ class TestLogger(TestCase):
         logger = logging.getLogger('test_warn_match')
         cfg = Config({'A': {'bat': 5}})
         with self.assertLogs(logger, logging.WARNING) as captured:
-            a = TestLogger.A(config=cfg, log=logger)
+            TestLogger.A(config=cfg, log=logger)
 
         output = '\n'.join(captured.output)
         self.assertIn('Did you mean one of: `bar, baz`?', output)
@@ -443,7 +480,7 @@ class TestLogger(TestCase):
 
         cfg = Config({'A': {'fool': 5}})
         with self.assertLogs(logger, logging.WARNING) as captured:
-            a = TestLogger.A(config=cfg, log=logger)
+            TestLogger.A(config=cfg, log=logger)
 
         output = '\n'.join(captured.output)
         self.assertIn('Config option `fool` not recognized by `A`.', output)
@@ -451,7 +488,7 @@ class TestLogger(TestCase):
 
         cfg = Config({'A': {'totally_wrong': 5}})
         with self.assertLogs(logger, logging.WARNING) as captured:
-            a = TestLogger.A(config=cfg, log=logger)
+            TestLogger.A(config=cfg, log=logger)
 
         output = '\n'.join(captured.output)
         self.assertIn('Config option `totally_wrong` not recognized by `A`.', output)
diff --git a/traitlets/config/tests/test_loader.py b/traitlets/config/tests/test_loader.py
index 50c8659..0fca256 100644
--- a/traitlets/config/tests/test_loader.py
+++ b/traitlets/config/tests/test_loader.py
@@ -5,7 +5,6 @@
 # Distributed under the terms of the Modified BSD License.
 
 import copy
-import logging
 import os
 import pickle
 import sys
@@ -24,6 +23,8 @@ from traitlets.config.loader import (
     KVArgParseConfigLoader,
     ConfigError,
 )
+from traitlets import (HasTraits, Union, List, Tuple, Dict, Int, Unicode)
+from traitlets.config import Configurable
 
 
 pyfile = """
@@ -121,14 +122,14 @@ class TestFileCL(TestCase):
         f = os.fdopen(fd, 'w')
         f.write('{}')
         f.close()
-        
+
         with JSONFileConfigLoader(fname, log=log) as config:
             config.A.b = 1
-        
+
         with self.assertRaises(TypeError):
             with JSONFileConfigLoader(fname, log=log) as config:
                 config.A.cant_json = lambda x: x
-        
+
         loader = JSONFileConfigLoader(fname, log=log)
         cfg = loader.load_config()
         assert cfg.A.b == 1
@@ -172,17 +173,24 @@ class TestFileCL(TestCase):
         with self.assertRaises(ValueError):
             cl.load_config()
 
+def _parse_int_or_str(v):
+    try:
+        return int(v)
+    except:
+        return str(v)
 
 class MyLoader1(ArgParseConfigLoader):
-    def _add_arguments(self, aliases=None, flags=None):
+    def _add_arguments(self, aliases=None, flags=None, classes=None):
         p = self.parser
         p.add_argument('-f', '--foo', dest='Global.foo', type=str)
         p.add_argument('-b', dest='MyClass.bar', type=int)
         p.add_argument('-n', dest='n', action='store_true')
         p.add_argument('Global.bam', type=str)
+        p.add_argument('--list1', action='append', type=_parse_int_or_str)
+        p.add_argument('--list2', nargs='+', type=int)
 
 class MyLoader2(ArgParseConfigLoader):
-    def _add_arguments(self, aliases=None, flags=None):
+    def _add_arguments(self, aliases=None, flags=None, classes=None):
         subparsers = self.parser.add_subparsers(dest='subparser_name')
         subparser1 = subparsers.add_parser('1')
         subparser1.add_argument('-x',dest='Global.x')
@@ -220,6 +228,14 @@ class TestArgParseCL(TestCase):
         self.assertEqual(config.n, True)
         self.assertEqual(config.Global.bam, 'wow')
 
+    def test_list_args(self):
+        cl = MyLoader1()
+        config = cl.load_config('--list1 1 wow --list2 1 2 3 --list1 B'.split())
+        self.assertEqual(list(config.Global.keys()), ['bam'])
+        self.assertEqual(config.Global.bam, 'wow')
+        self.assertEqual(config.list1, [1, 'B'])
+        self.assertEqual(config.list2, [1, 2, 3])
+
 
 class TestKeyValueCL(TestCase):
     klass = KeyValueConfigLoader
@@ -242,7 +258,7 @@ class TestKeyValueCL(TestCase):
         # non-literal expressions are not evaluated
         self.assertEqual(config.Foo.Bam.value, 'list(range(10))')
         self.assertEqual(config.D.C.value, 'hi there')
-    
+
     def test_expanduser(self):
         cl = self.klass(log=log)
         argv = ['--a=~/1/2/3', '--b=~', '--c=~/', '--d="~/"']
@@ -251,7 +267,7 @@ class TestKeyValueCL(TestCase):
         self.assertEqual(config.b, os.path.expanduser('~'))
         self.assertEqual(config.c, os.path.expanduser('~/'))
         self.assertEqual(config.d, '~/')
-    
+
     def test_extra_args(self):
         cl = self.klass(log=log)
         config = cl.load_config(['--a=5', 'b', '--c=10', 'd'])
@@ -260,24 +276,24 @@ class TestKeyValueCL(TestCase):
         self.assertEqual(config.c, 10)
         config = cl.load_config(['--', '--a=5', '--c=10'])
         self.assertEqual(cl.extra_args, ['--a=5', '--c=10'])
-    
+
     def test_unicode_args(self):
         cl = self.klass(log=log)
         argv = [u'--a=épsîlön']
         config = cl.load_config(argv)
         self.assertEqual(config.a, u'épsîlön')
-    
+
     def test_unicode_bytes_args(self):
         uarg = u'--a=é'
         try:
             barg = uarg.encode(sys.stdin.encoding)
         except (TypeError, UnicodeEncodeError):
             raise skip("sys.stdin.encoding can't handle 'é'")
-        
+
         cl = self.klass(log=log)
         config = cl.load_config([barg])
         self.assertEqual(config.a, u'é')
-    
+
     def test_unicode_alias(self):
         cl = self.klass(log=log)
         argv = [u'--a=épsîlön']
@@ -285,6 +301,17 @@ class TestKeyValueCL(TestCase):
         self.assertEqual(config.A.a, u'épsîlön')
 
 
+class CBase(HasTraits):
+    a = List().tag(config=True)
+    b = List().tag(config=True, multiplicity='*')
+    c = List().tag(config=True, multiplicity='append')
+    adict = Dict().tag(config=True)
+
+class CSub(CBase):
+    d = Tuple().tag(config=True)
+    e = Tuple().tag(config=True, multiplicity='+')
+    bdict = Dict().tag(config=True, multiplicity='*')
+
 class TestArgParseKVCL(TestKeyValueCL):
     klass = KVArgParseConfigLoader
 
@@ -294,13 +321,36 @@ class TestArgParseKVCL(TestKeyValueCL):
         config = cl.load_config(argv, aliases=dict(a='A.a', b='A.b'))
         self.assertEqual(config.A.a, os.path.expanduser('~/1/2/3'))
         self.assertEqual(config.A.b, '~/1/2/3')
-    
+
     def test_eval(self):
         cl = self.klass(log=log)
         argv = ['-c', 'a=5']
         config = cl.load_config(argv, aliases=dict(c='A.c'))
         self.assertEqual(config.A.c, u"a=5")
-    
+
+    def test_seq_traits(self):
+        cl = self.klass(log=log, classes=(CBase, CSub))
+        aliases = {'a3': 'CBase.c', 'a5': 'CSub.e'}
+        argv = ("--CBase.a A --CBase.a 2 --CBase.b 1 2 3 --a3 AA --CBase.c BB "
+                "--CSub.d 1 --CSub.d BBB --CSub.e 1 a bcd").split()
+        config = cl.load_config(argv, aliases=aliases)
+        self.assertEqual(config.CBase.a, ['A', 2])
+        self.assertEqual(config.CBase.b, [1, 2, 3])
+        self.assertEqual(config.CBase.c, ['AA', 'BB'])
+
+        self.assertEqual(config.CSub.d, [1, 'BBB'])
+        self.assertEqual(config.CSub.e, [1, 'a', 'bcd'])
+
+    def test_dict_traits(self):
+        cl = self.klass(log=log, classes=(CBase, CSub))
+        aliases = {'D': 'CBase.adict', 'E': 'CSub.bdict'}
+        argv = ["--CBase.adict", "k1=v1", "-D=k2=2", "-D", "k3=v 3", "-E", "k=v", "22=222"]
+        config = cl.load_config(argv, aliases=aliases)
+        self.assertDictEqual(config.CBase.adict,
+                {'k1': 'v1', 'k2': 2, 'k3': 'v 3'})
+        self.assertEqual(config.CSub.bdict,
+                {'k': 'v', '22': 222})
+
 
 class TestConfig(TestCase):
 
@@ -368,12 +418,12 @@ class TestConfig(TestCase):
     def test_builtin(self):
         c1 = Config()
         c1.format = "json"
-    
+
     def test_fromdict(self):
         c1 = Config({'Foo' : {'bar' : 1}})
         self.assertEqual(c1.Foo.__class__, Config)
         self.assertEqual(c1.Foo.bar, 1)
-    
+
     def test_fromdictmerge(self):
         c1 = Config()
         c2 = Config({'Foo' : {'bar' : 1}})
@@ -389,7 +439,7 @@ class TestConfig(TestCase):
         self.assertEqual(c1.Foo.bar, 1)
         self.assertEqual(c1.Foo.baz, 2)
         self.assertNotIn('baz', c2.Foo)
-    
+
     def test_contains(self):
         c1 = Config({'Foo' : {'baz' : 2}})
         c2 = Config({'Foo' : {'bar' : 1}})
@@ -397,14 +447,14 @@ class TestConfig(TestCase):
         self.assertIn('Foo.baz', c1)
         self.assertIn('Foo.bar', c2)
         self.assertNotIn('Foo.bar', c1)
-    
+
     def test_pickle_config(self):
         cfg = Config()
         cfg.Foo.bar = 1
         pcfg = pickle.dumps(cfg)
         cfg2 = pickle.loads(pcfg)
         self.assertEqual(cfg2, cfg)
-    
+
     def test_getattr_section(self):
         cfg = Config()
         self.assertNotIn('Foo', cfg)
@@ -440,7 +490,7 @@ class TestConfig(TestCase):
         foo = cfg['foo']
         assert isinstance(foo, LazyConfigValue)
         self.assertIn('foo', cfg)
-    
+
     def test_merge_no_copies(self):
         c = Config()
         c2 = Config()