Add new cfg module

As described here:

  http://wiki.openstack.org/CommonConfigModule

The module implements an API for defining configuration options and
reading values for those options that a user may have set in a config
file or on the command line.

The module is successfully in use in both Nova and Glance. Some work
remains in Nova to switch from using it under a gflags compatible
shim layer, but Glance is using it fully.

There doesn't appear to be any blockers to other projects moving over
to it fairly easily. Swift would perhaps be the next project to tackle.

Just to go through potential future compatibility concerns:

  - Nova (the scroundrel) hackily uses the private ConfigOpts::_oparser
    in order to disable interspersed args. This was just for nova-manage
    and can probably be resolved some other way. In any case, Nova
    shouldn't switch to openstack-common's cfg API until it removes this
    hack.

  - the CommonConfigOpts subclass set of logging related options is
    perhaps assuming too much about what configuration options should
    be common across all the projects. However, it seems a fairly sane
    set and the worst that can happen is that projects avoid using it.

  - the parameters to the Opt constructor fairly closely mirror
    optparse, but they're fairly generic and shouldn't prevent us from
    switching to e.g. argparse

  - stuff like %prog expansion in the ConfigOpt's usage ctor param is a
    similar concern, but it's a very minor concern.

  - find_config_files() search path is perhaps too much policy for
    openstack-common; however, it is probably as generic as it could
    be and projects which need a different policy can just not use
    the function

On the whole, I think we're in good shape wrt future compatibility.

Change-Id: I279a9db7806d80aff3b9b085b4a9e4fb193662f9
Fix some cfg test case naming conflicts

As pointed out by Vish, there are duplicates of:

  OverridesTestCase::test_default_override
  OverridesTestCase::test_override
  SadPathTestCase::test_conf_file_not_found

Also, rename the ConfigFileOptsTestCase so it is obvious they
don't clash with the tests by the same name in CliOptsTestCase.

Change-Id: I1d650d05d32501623cfed8f0b6399858d101ae02
Add cfg test case for exceptions' __str__ methods

Increases coverage from 93% to 97%

Change-Id: I6a41b31e29238831fe2a888d5d64dc0bffd770c0
Get cfg test cases to 100%

 * Test cfg.find_config_files() sys.argv usage
 * Test boolean values in cfg config files
 * Finish off incomplete cfg bad value test case
 * Test register_opts() and register_cli_opts()
 * Test the quiet ignoring of option/group re-registration
 * Test cfg print_usage()
 * Test explicit option group titles

Change-Id: Icbe4b7c48d4785551f06873821d1be758adf942c
Add support to cfg for disabling interspersed args

Implements blueprint cfg-disable-interspersed-args

Nova currently relies on cfg being implemented with optparse because it
uses optparse's disable_interspersed_args()

The use case for this is if you do:

  $> nova-manage --verbose create --project foo --user bar

you want invoking ConfigOpts() to return:

  ['create', '--project', 'foo', '--user', 'bar']

as the "extra" args rather than aborting when it doesn't recognize the
--project arg.

This is a reasonable use case for cfg to support and it should just have
{disable,enable}_interspersed_args() methods.

If we ever switch from optparse to argparse, we'll do something like this:

  parser.add_argument('--verbose')
  ...
  parser.add_argument(
    'extra_args',
    nargs=argparse.REMAINDER if disable_interspersed_args else '*')
  ...
  ns = parser.parse_args(...)
  extra_args = ns.extra_args

i.e. we will need an 'extra_args' multi-value positional argument in any
case and we'll just pass nargs=REMAINDER if we want trailing options to
be included in the extra args.

Change-Id: I3ecb7dc18230327cf5aaaa7d832224e64aafa40c
Add cfg test case for recursive substitution

i.e. test that if blaa='blaa', foo='$blaa' and bar='$foo' that
the value of bar after substitutions is 'blaa'

Change-Id: I01d370832a871603b7cb47bfb3546f6aaad8c34d
Add the Mapping interface to cfg.ConfigOpts

Implements blueprint cfg-mapping interface

With cfg, option values are accessed via attributes on ConfigOpts
objects e.g.

  conf = ConfigOpts()
  conf.register_opt(StrOpt('foo'))
  conf()
  print conf.foo

One use case that isn't easily supported with option values represented
this way is iterating over all the registered options. Standard
interfaces for listing attributes on an object aren't suitable because
they will list more than just the options.

For this use case alone, it's worth having ConfigOpts implement the
mapping interface. That way we can do e.g.

  for opt, value in conf.items():
      print "Option %s = %s" % (opt, value)

It's interesting to compare argparse's approach to this problem - option
values are attributes on a Namespace object which has no attributes or
methods to pollute the namespace of option names. This is a nice
approach, but would mean that we would be passing around both a
ConfigOpts object and a Namespace object. That's a bit too much overhead,
and the mapping interface provides a usable workaround where there is a
conflict.

Change-Id: Ic113919a20291048f962999229c76884ebdd5ad8
PEP8 type comparison cleanup

Fixes bug #910295

The None, True, and False values are singletons.

All variable *comparisons* to singletons should use 'is' or 'is not'.
All variable *evaluations* to boolean should use 'if' or 'if not'.

"== None", "== True", "== False", and "!= None" comparisons in sqlalchemy's
where(), or_(), filter(), and_(), and select() functions should not be changed.

Incorrect comparisons or evaluations in comments were not changed.

Change-Id: I087f0883bf115b5fe714ccfda86a794b9b2a87f7
Makes common/cfg.py raise AttributeError

  * fixes bug 915039
  * includes test

Change-Id: I67b886be3b5af3763f52fffe54085975d61d61eb
Implements blueprint separate-nova-volumeapi

[...]
 ** Removes flag osapi_extension and replaces with osapi_compute_extension and osapi_volume_extension
[...]

Change-Id: I4c2e57c3cafd4e1a9e2ff3ce201c8cf28326afcd
Disable ConfigParser interpolation (lp#930270)

This breaks e.g.

  volume_name_template=volume-%08x
  instance_name_template=instance-%08x

and is not part of the API contract anyway. We use $opt based value
interpolation.

Change-Id: I7ba566ae7c9a77322b52c67c5e1ffbffb760f0fc
Backslash continuations (misc.)

Fixes bug #925166

This patch for packages which have few backslash continuations.

Follow up patches will be for packages network, scheduler, virt,
db/sqlalchemy, tests, and api/openstack.

Change-Id: I4200010b47b33fa8b9115b5d379b543200f6668d
PEP8 cleanup (openstack-common)

Fixes bug #930625

Remove backslash continuations in openstack-common.
Fix type checking taboos.

Change-Id: I49ddb9ff5fa5af760dcfccb52cb4793b71e02f19
cfg: unneeded multiple inheritance

Fixed bug #927650

In python=<2.6, collections.Mapping inherits from
collections.Sized, collections.Iterable, and collections.Container
which are also subclasses of object.

Change-Id: I6238c683324127abd9fb637748a10b6bdb2961e0
cfg: fix a small comment typo

Change-Id: I2646d7e674ef3d1759558e820f051cc5e7f3b4ae
Add ConfigOpts.print_help()

Keystone uses this optparse method.

Change-Id: Ic840b2fb2234a12cd94ca671a8d90cd2affe3a5e
fix restructuredtext formatting in docstrings

blueprint sphinx-doc-cleanup
bug 94516

- Correct parameter declarations, list formatting, cross-references, etc.
- We don't need "let" in generate_autodoc_index.sh since we aren't doing math.
- Change conf.py to not prefix class and function names with full namespace
  in generated output to save width on the screen.

Change-Id: I9adc8681951913fd291d03e7142146e9d46841df
Fix bug 954488

Change-Id: I99b764310c575e70aff4a6790e8ba8f55e43deeb
Avoid leaking secrets into config logging.

Implements bp cfg-password-options

Allow options to be declared secret so that their value is
obfuscated before logging.

Change-Id: Ie2168d218b029d9c12fa5b48342cd5b17b2cc77a
Finish implementing MultiStrOpt

Fixes bug 955308

Previously only multiple string options from the CLI were supported.
This change adds support for config files too and merges the results
from both CLI and config files.

Change-Id: I642408c03ed295fac050105fd4380940e876f228