Allow policy.json resource vs constant check

Adds the ability to check any resource's field against a constant
(literal, or string) in the policy.json file.

For instance, I can ensure that only users with field `enabled` set to
False can be deleted with the following rule:
"identity:delete_user": "False:%(target.user.enabled)s",

Or that only the `Member` role can be granted:
"identity:create_grant": "'Member':%(target.role.name)s",

Change-Id: I99688fd88c229b7566ddf6e4933412b6fd0cbdb2
Implements: blueprint policy-constant-check
DocImpact
Add to_dict() method to apiclient Resource

Resource class in different openstack clients (ceilometerclient,
glanceclient, heatclient, ironiccilent, tuskarclient) has to_dict()
method so it would be nice to have this method in common code to
avoid code duplication.

blueprint common-client-library-2

Change-Id: I1db6c12a1f798de7f7fafd0c34fb0ef523610153
Removed set_loaded() method from Resource class

set_loaded() method should not be available in class public API cause its
usage can break internal class logic. Base Resource class(method __getattr__)
can behave incorrectly if access to attribute _loaded is provided to end users.

method get() of Resource class also should be private, because it's
related to internal class logic and doesn't have any meaning from
outside of the class

blueprint common-client-library-2

Change-Id: I491f9b54c7074546cf738c6150666bfcc3966a78
Automatic retry db.api query if db connection lost

Current patch allows us to retry db.api method call if
database connection was lost temporally.

It can be done in a such way:
 - We should use annotate all methods which are safe to retry with
   @safe_for_db_retry, if it's totally safe to retry these functions
 - And we must allow db-retry feature in config (`use_db_reconnect` option
   in a `database` group). It's will allow db-retry for all db.api
   methods.

If function suppose to be retried and it's got `db-gone-away` error,
it's will calls in a loop until it succeeds, or until maximum retries
count will be reached.
Maximum retries count and timeout between retries attempts are configurable.

We use SQLA engine.dialect.is_disconnect() function to determine
broken db connection so it's should work for all supported SQLA
backends (MySQL, PostgreSQL, DB2 and SQLite supported as well).

Added wrapper for session and query methods. This wrapper catch
exceptions caused by broken db connection and re-raise it as
DBConnectionError exception.

Note, that it's safely to use db-retry feature only if all db.api
queries use single transaction.

At the moment this feature is experimental, so disabled by default.

Previous attepmt - https://review.openstack.org/#/c/10797/
'Proof-of-concept' patch in Nova https://review.openstack.org/#/c/35610/

blueprint db-reconnect

Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>

Change-Id: Ifd848ea70a62fb9f7ccbc8bfd22a10ab9d3135c9
Format sql in db.sqlalchemy.session docstring

Some SQL in the module wasn't marked as such so it wasn't
nicely highlighted.

Change-Id: If902796886cb614262b305b221fc9890d3fd1c87
Don't share periodic_task instance data in a class attr

The time of the last task run for a given periodic task was being
stored in a class attribute, which meant all instances of a class
shared the same last run time regardless of when they were
actually run.  This may not have been a problem in actual use of
the code because I believe in practice periodic task classes
are created only once, but it broke the unit tests because
subsequent runs of a task might behave differently depending on
whether a previous test had already run a task.

To me, this behavior is unintuitive and wrong so it should be
changed.

This commit also fixes the test cases to properly verify whether
the number of task runs expected actually happened.  The improper
call validation was the only reason these tests passed before.

Change-Id: I78fc249fd33bc41f3fa8f301414bc60463cadb07
Closes-Bug: #1275151
Cleanup config fixture docstrings

The docstring was defined for the whole class rather than for
the method that it was describing.

Change-Id: I8123f10eb682661c453347f03fbdb1032ad616da
Enhance config fixture to support registering options

Add register_opt() and register_opts() to the config fixture. This
allows tests to define configuration options that will not persist
in the global CONF object over multiple tests.

Minor changes to the config fixture tests to use the same fixture
that the test case uses as .config_fixture instead of an
independantly (non setUp) instantiation of the fixture.

Change-Id: Ic125b705644ae091cd6696e7d8619a3238a8c866
Add tool for helping with library graduation

This new script prepares a copy of the oslo-incubator repository
to house a stand-alone oslo library by revising the history,
retaining commits for files that are part of the new library and
discarding commits for files that are not. It then uses the
new oslo-cookiecutter library to inject any missing structural
files so the packaging and tests will work.

Change-Id: I1f513a773589f034975ae6ee54c5a418e4bfce25
Make sure rpc_notifier opts exist

The notifier tests try to override config opts registered by
rpc_notifier, but rpc_notifier may not be imported before those
tests run.  Use import_opt to ensure the options are registered
before the test tries to override them.

Change-Id: I5f01eed286c71ea2706225d2ca0380df153bd975
Closes-Bug: #1275154
Log unit test improvements

The log unit tests were often failing when run in parallel, which
was caused by incorrect dependencies between tests.  Some tests
were not cleaning up after themselves properly, and the changes
they made were influencing other tests.  This commit makes all of
the tests independent of each other and ensures that proper
cleanup is being done after the tests that alter global log state.

Change-Id: I9f8bb4096963077737e556d2808791123cec441b
Closes-Bug: #1275890
Related-Bug: #1272035
Clear time override in test_periodic

The periodic tasks tests were not clearing their time override
properly, so it was persisting into the timeutils test cases when
they were run first.

Change-Id: Idaa2bddba4943f5f8776a883228c56d8503097e7
Closes-Bug: #1275152
Related-Bug: 1272035
Merge "Cleanup config fixture docstrings"
Add support for translating log levels separately

Use separate message decorating functions for translating log
messages than regular error messages. Each log level uses
its own translation domain, to allow translators to prioritize
translations.

bp log-messages-translation-domain

Change-Id: I05d5c4d019404a07571b1fb101e028e9d9a962a6
Merge "Format sql in db.sqlalchemy.session docstring"
Fix usage of dict.keys in apiclient.exceptions

Method dict.keys returns a dict_keys object in Python 3, which is
not indexable, so we get exception TypeError ('dict_keys' object does not
support indexing). This is fix of usage dict object in module exceptions.

Related to bp make-python3-compatible

Change-Id: Ie827f408cd23d4baa900f2ae1b074d13a5cd6f0e
Merge "Allow policy.json resource vs constant check"
Merge "Enhance config fixture to support registering options"
Merge "Add to_dict() method to apiclient Resource"
Merge "Removed set_loaded() method from Resource class"