Avoid failure of test_basic_report on 32 bit OS

report.generators.threading module uses sys._current_frames() to
get thread id, but it will get a negative number on 32 bit operating
system, which will cause test_guru_meditation_report.test_basic_report
fail since it expects a positive number.

This patch add an optional character '-' to pattern parameter of
re.match to avoid such problem.

Change-Id: If844894dc1f1032c78d6ed78d221aac27c42712e
Closes-Bug: #1264906
Merge same db calls for some quota methods

self.db.quota_get_all_by_project is called twice in method limit_check()
and reserve(),
self.db.quota_get_all_by_project_and_user is called twice in method
get_settable_quotas()

Change-Id: Ibe41f49afe16c220ca663a7ad13b61acc08c7775
Closes-bug: 1277068
Deleted duplicated method in cliutils.

Method import_class in cliutils duplicates method in importutils module,
and method import_versioned_module is more related to import tools,
so it is moved in appropriate module.

bp common-client-library-2

Change-Id: I7b8892b2885f8b05a8106300371481209e99749d
Enhance tests for LOG.deprecated

The tests for LOG.deprecated were incorrect because they were
clearing the log buffers on every call.

When this is corrected, it shows that LOG.deprecated is not working
as expected since it logs multiple times when the test indicates
that it should only log once.

Change-Id: Ic0e4925c372e5948c2140daa07ab3362fb25a306
Partial-Bug: #1266812
Fix deprecated messages sent multiple times

Calling LOG.deprecated('...') multiple times with the same log
message was causing the message to be logged multiple times. The
message should only have been logged once.

Change-Id: Ieab0bf2656761759966cf16f1de05bcb41cd1be5
Closes-Bug: #1266812
Fix parsing of UC errors in sqlite 3.7.16+/3.8.2+

The new version of libsqlite changed the format of error messages.
Code of oslo.db helpers must be updated appropriately. This was meant
to be fixed by I86e9673ac409b3b3e881bea7cb8b3516f320388d, but that
change doesn't cope well with the new format of column names.

Closes-Bug: #1259915

Change-Id: Iec39c8804ea47307a98d1b79899ff74af9404bf6
Allow policy.json resource vs constant check

Adds the ability to check any resource's field against a constant
(literal, or string) in the policy.json file.

For instance, I can ensure that only users with field `enabled` set to
False can be deleted with the following rule:
"identity:delete_user": "False:%(target.user.enabled)s",

Or that only the `Member` role can be granted:
"identity:create_grant": "'Member':%(target.role.name)s",

Change-Id: I99688fd88c229b7566ddf6e4933412b6fd0cbdb2
Implements: blueprint policy-constant-check
DocImpact
Add to_dict() method to apiclient Resource

Resource class in different openstack clients (ceilometerclient,
glanceclient, heatclient, ironiccilent, tuskarclient) has to_dict()
method so it would be nice to have this method in common code to
avoid code duplication.

blueprint common-client-library-2

Change-Id: I1db6c12a1f798de7f7fafd0c34fb0ef523610153
Removed set_loaded() method from Resource class

set_loaded() method should not be available in class public API cause its
usage can break internal class logic. Base Resource class(method __getattr__)
can behave incorrectly if access to attribute _loaded is provided to end users.

method get() of Resource class also should be private, because it's
related to internal class logic and doesn't have any meaning from
outside of the class

blueprint common-client-library-2

Change-Id: I491f9b54c7074546cf738c6150666bfcc3966a78
Use Context Manager for reading JSON configs

Avoids leaking a fd when there's an exception.

Change-Id: Iacd10edc58037192a2e710c47a6caf0da40c6d71
Automatic retry db.api query if db connection lost

Current patch allows us to retry db.api method call if
database connection was lost temporally.

It can be done in a such way:
 - We should use annotate all methods which are safe to retry with
   @safe_for_db_retry, if it's totally safe to retry these functions
 - And we must allow db-retry feature in config (`use_db_reconnect` option
   in a `database` group). It's will allow db-retry for all db.api
   methods.

If function suppose to be retried and it's got `db-gone-away` error,
it's will calls in a loop until it succeeds, or until maximum retries
count will be reached.
Maximum retries count and timeout between retries attempts are configurable.

We use SQLA engine.dialect.is_disconnect() function to determine
broken db connection so it's should work for all supported SQLA
backends (MySQL, PostgreSQL, DB2 and SQLite supported as well).

Added wrapper for session and query methods. This wrapper catch
exceptions caused by broken db connection and re-raise it as
DBConnectionError exception.

Note, that it's safely to use db-retry feature only if all db.api
queries use single transaction.

At the moment this feature is experimental, so disabled by default.

Previous attepmt - https://review.openstack.org/#/c/10797/
'Proof-of-concept' patch in Nova https://review.openstack.org/#/c/35610/

blueprint db-reconnect

Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>

Change-Id: Ifd848ea70a62fb9f7ccbc8bfd22a10ab9d3135c9
Handle exception messages with six.text_type

Handle exception messages with six.text_type tools instead of using
string. This is necessary becase a message can contain
non-ascii characters and it can cause the unicode error:
"UnicodeError: Message objects do not support str() because they may
contain non-ascii characters. Please use unicode() or translate()
instead."

Change-Id: I3312b75731c1ce49e2f1069c83e1a868805409d3
Closes-Bug: #1273301
Format sql in db.sqlalchemy.session docstring

Some SQL in the module wasn't marked as such so it wasn't
nicely highlighted.

Change-Id: If902796886cb614262b305b221fc9890d3fd1c87
Don't share periodic_task instance data in a class attr

The time of the last task run for a given periodic task was being
stored in a class attribute, which meant all instances of a class
shared the same last run time regardless of when they were
actually run.  This may not have been a problem in actual use of
the code because I believe in practice periodic task classes
are created only once, but it broke the unit tests because
subsequent runs of a task might behave differently depending on
whether a previous test had already run a task.

To me, this behavior is unintuitive and wrong so it should be
changed.

This commit also fixes the test cases to properly verify whether
the number of task runs expected actually happened.  The improper
call validation was the only reason these tests passed before.

Change-Id: I78fc249fd33bc41f3fa8f301414bc60463cadb07
Closes-Bug: #1275151
Cleanup config fixture docstrings

The docstring was defined for the whole class rather than for
the method that it was describing.

Change-Id: I8123f10eb682661c453347f03fbdb1032ad616da
Enhance config fixture to support registering options

Add register_opt() and register_opts() to the config fixture. This
allows tests to define configuration options that will not persist
in the global CONF object over multiple tests.

Minor changes to the config fixture tests to use the same fixture
that the test case uses as .config_fixture instead of an
independantly (non setUp) instantiation of the fixture.

Change-Id: Ic125b705644ae091cd6696e7d8619a3238a8c866
Don't raise MySQL 2013 'Lost connection' errors

Closes-bug: #1276510

Although this code error is part of the `ping_listener` tests, it was
missing from the list of known - and ignored - connection errors.

The code error 2013 refers to a connection lost during a query, see:
https://dev.mysql.com/doc/refman/5.0/en/error-messages-client.html#error_cr_server_lost

Without this error code, the session won't try to reconnect to mysql,
which makes `max_retries` useless.

Change-Id: If2667cf83ac656666ffe00aa90e3454516b080db
Merge "Handle exception messages with six.text_type"
Merge "Use Context Manager for reading JSON configs"
Add tool for helping with library graduation

This new script prepares a copy of the oslo-incubator repository
to house a stand-alone oslo library by revising the history,
retaining commits for files that are part of the new library and
discarding commits for files that are not. It then uses the
new oslo-cookiecutter library to inject any missing structural
files so the packaging and tests will work.

Change-Id: I1f513a773589f034975ae6ee54c5a418e4bfce25