Merge from oslo-incubator

Merging processutils.py and dependencies from oslo-incubator.
Primarily, I want change d6b55fb but the rest come along for
the ride.

0d7296f Add kwargs to jsonutils.load(s) functions
3310d8d update new requests logger to default WARN
33afb20 Fix broken formatting of processutils.execute log statement
3d90045 Backport code for i18n to check lazy at runtime
433fa0b Make logging_context_format_string optional in log.set_defaults
4d9328c Python 3: enable tests/unit/test_log.py
5fd77eb Ability to customize default_log_levels for each project
6751b30 Remove extra whitespace
726d00a Adjust oslo logging to provide adapter is enabled for
ac92c06 Add default log level for websocket
cb5a804 Move `mask_password` to strutils
d6b55fb Remove `processutils` dependency on `log`
e184dd3 Fix exception message in openstack.common.processutils.execute
ef37e03 Added missing jsonutils.dump() function

Change-Id: I567f514faed5a8467602257d35a98625801daf22
Closes-Bug: #1349890
guestagent/mysql: Remove unused function

This commit remove the dead function
`replace_mycnf_with_template`.

Closes-bug: 1351361
Change-Id: Ie9a17a972e069b27740c96188f176271599142f4
allow both ipv4 and ipv6 hostnames by default

Refactored the code to use netaddr rather than re-implementing IP
address validation. This gives us IPv6 support for free.

DocImpact
The hostname_require_ipv4 configuration option is being
removed in favor of hostname_require_valid_ip. The latter is more
inclusive and validates both IPv4 and IPv6 addresses.

Change-Id: I36c76169f2ab3bb7f54f67db522309a49ae61f7a
Closes-Bug: #1343192
Load trove API extensions using stevedore

The current trove impl loads trove API extensions by searching on a
single file path which is exposed as a conf property. This results in a
less than optimal approach from a consumer extension POV. This change
replaces the single extension path approach with dyanmic loading  using
stevedore. Consumers can now bind into the API extensions using the
'trove.api.extensions' entry point; a standard means to define extension
points. Moreover this change refactors to the
trove.openstack.common.extensions logic into trove.common.extensions.

In addition this change includes base unit tests to ensure the
existing trove proper extension points are loaded and that some basic
checks are in place W/R/T validating trove extension points.

Change-Id: Id3e712b536aef3bc9c9d1603367cdc0d4816b970
Implements: blueprint dynamic-extension-loading
Unit Tests for Mysql replication functionality

Implemented unit tests for the API methods in mysql manager.

Partially implements: blueprint replication-v1

Change-Id: Ifa64630457ddd771d98985823d92b4bab3412890
Adjusted audit logging for trove.instance module

The trove.instance module is logging some verbose debug data at the
info level, and some info messages at the debug level. Some log
entries are also lacking context because of missing identifiers such
as InstanceID and TenantID.

Changed logging level of various messages to conform to standard,
added references to various ID's as needed.

Change-Id: I3bb569529befa2b2497d2d5a094e3306e569422c
Partial-Bug: #1324206
Mysql guest agent functionality for replication

Implement new strategy for replication
Implement replication strategy for mysql binlog replication.
Supporting methods in mysql service
Implement API methods in mysql manager
Define configuration settings for replication

Co-authored by: Nikhil Manchanda <SlickNik@gmail.com>
Co-authored by: Greg Lucas <glucas@tesora.com>

Partially implements: blueprint replication-v1
Change-Id: I70f0b5c37fe3c2d42426029bb627c141965eb524
show stdout/err from failed command execution

When executing a command the mysql service status command it is useful
to see the actual output of the failing command in debug mode, not just
the exit status.

Add a log_output_on_error parameter to execute_with_timeout() so this
can be reused.

Change-Id: Iff931d0489754303e099e8567bca3237fd476f8f
Closes-Bug: #1340124
Add new checklinks tox environment

This is a new non-voting job that will be used for changes of the
manual. It checks that URLs in the manual are reachable and valid.

Change-Id: I0f34d2fa2a0334276623dfa708f4196a16e20dc1
Move usage_timeout out of guest options

In an effort to further separate the guest and task manager,
the usage_timeout has been moved into the common cfg. The
datastores have no need for this. the default is now set as
the largest timeout, cassandra.

DocImpact: This change modifies a configuration parameter.
partially implements: blueprint refactoring-datastore-options-in-cfg
Co-Author: Michael Basnight <mbasnight@gmail.com>

Change-Id: I3b4b2a9bffb40df4531a78c29d1e155e59436948
Added the bind_host configuration option when launching the API

Closes-bug: 1356411
Change-Id: Iad69754d60fc88d503ac7f87621ae7b5d561f9c2
Use netifaces to lookup IP address on guest agent

The current way of looking up the IP address on the guest
doesn't correctly work across all platforms. Use netifaces
instead which works correctly across all platforms.

Change-Id: I3d5febb1dd801fdf7801613cd843e7c00f308016
Merge "guestagent/mysql: Remove unused function"
Merge "Add new checklinks tox environment"
Merge "Merge from oslo-incubator"
Set the python hash seed that tox uses to 0

The current trove unit tests don't work correctly with the random
hash seed that the new tox sets by default.

This change forces the new tox to use 0 as the hash seed which
should unblock the gate until we can fix the trove unit tests
to run correctly with a random hash seed.

This change will probably need to be backported to the stable
branches to ensure that the tox tests keep working there as well.

Change-Id: Ib5c6d4d166c847ce94ec70740c5753d338687865
handle repeating mysqld options containing equals

Let load_mysqld_options handle repeating options such as:
"--plugin-load=blackhole=ha_blackhole.so
--plugin-load=federated=ha_federated.so" without crashing by storing all
the mysql_options we get in a dictionary with lists as values.

Change-Id: I634532e13bab80d476e4a1e8558b23da33656f60
Closes-Bug: #1348652
Merge "Mysql guest agent functionality for replication"
Snapshot component for replication

Implements the snapshot capability for replication by
taking a backup of the master site, then using it
to populate the slave, then turning on replication.

Co-Authored-By: Greg Lucas <glucas@tesora.com>
Co-Authored-By: Nikhil Manchanda <SlickNik@gmail.com>

Partially implements: blueprint replication-v1
Change-Id: Ibd3214b2ffafb250f0cc794c6ee0ff6b0d2ed7da
Look up trove instance by ID instead of name

update_statuses_on_time_out looks up the instance by name instead of ID.
This potentially means that we update the wrong instance to ERROR, since
instance name is not unique. This patchset fixes this issue by looking
up the trove instance by ID instead of name.

Change-Id: If9c7a518f50098390567d8e4df0f948b7d3be2c6
Closes-bug: 1356647