Support configdrive from HTTP(S)

This commit adds support for passing an HTTP(S) URL as the
`configdrive` parameter for the `prepare_image` command.

This will load the entire image in memory. Although the configdrive
is limited to 64MB, this is not ideal and should be made better
in the future (likely when imaging gets refactored).

Change-Id: I6f14b9059920d2cef108c30d027c5486fa0b7e9a
Implements: blueprint expose-configdrive
Improve/add docstrings for CommandResult classes

Added missing docstrings and improved existing docstrings for every
method in the CommandResult classes.

Change-Id: I71ca54a6ee2e3df2705b60bfae7c5a29a2b4657d
Partial-bug: 1367915
Merge "Improve/add docstrings for CommandResult classes"
Use _ instead of - for config options

Config options containing "-" don't work when reading
configuration from a file; change these to "_". Leave
the old options for backwards compatibility.

Closes-Bug: #1386486

Change-Id: Ie10a548191effa2a505309d9d4ee32003f9676a2
Merge "Use _ instead of - for config options"
Updated from global requirements

Change-Id: I9a48d7388515daa61e4fd7748480cfe1fb11aaaa
Fix badly named function _get_kernel_params()

Renamed _get_kernel_params() to _get_agent_params() and
appropriate module-level attribute KPARAMS to APARAMS.

Updated agent tests to match new function name.

Change-Id: I4cdf1defee7487785967a092c81c906d40bf3214
Closes-Bug: #1367917
Workflow documentation is now in infra-manual

Replace URLs for workflow documentation to appropriate parts of the
OpenStack Project Infrastructure Manual.

Change-Id: I7f7196b740006b8a2d7123ad6e781e35718da07d
Merge "Workflow documentation is now in infra-manual"
Use LLDP to get switch port mapping

Provides a function to listen for LLDP packets on the network.
Listens on one or all of the network interfaces, and then parses the found
packets.

Change-Id: I1545a41f46cd0916aab9c43ce036865454fa66e0
Co-Authored-By: Josh Gachnang <josh@pcsforeducation.com>
Do not log configdrive contents ever

Currently this logline contains the full, base64'd configdrive. This
changes it to log the file it came from instead.

Change-Id: I40fa4b8d889ce08fa20f7b0841b5abf122499676
Closes-bug: 1401244
Adding support for decommissioning

The pluggable hardware manager should be able be used for decommissioning,
such as erasing disks, updating firmwares, changing BIOS settings, and
anything else specific hardware needs. This works with the agent driver
to store state in Ironic and request out of band reboots (which may be
required to make BIOS/firmware updates work). Also allows the agent to do
some basic verification of the node's properties, to ensure they haven't
changed, such as a stick of memory dying.

Change-Id: Ia42a8e75d2b8887b2351de97c2137fe21a7f4041
(cherry picked from commit aa27598f2b56614d76214805b0d2a8dde99e1aae)
Add versioning to Agent decommission

As the agent will likely be rebooting while decommissioning (to apply BIOS/
firmware upgrades), it is possible the version of the agent will change
between boots. Including a version string in the Hardware Manager allows
the Hardware Manager builder to bump versions for any breaking changes.
I recommend that the entire decommission process be restarted if there
is a version mismatch, but that will be up to individual drivers to
implement.

This change also adds a new return value for Agent REST commands called
'DECOM_VERSION_MISMATCH' that will signal no command was run, and
Ironic should update its stored value. A 'hardware_manager_version'
will be returned with AsyncCommandResult, and should be set in the
Ironic driver_info field and sent with any decommission command.

Change-Id: I2ae0e93573e0348f023aa124a07bb16921b5c53e
(cherry picked from commit 05059e14d665ace405c4766cc2f089dfe8d2eb12)
Add command metrics to IPA API

This will allow the agent to report timing and counter data for commands
that are run on the agent (starting with Standby commands). It will allow a
properly configured agent to send metrics to a remote or locally packaged
statsd. It is also extensible, so other metrics systems can easily be used.

Change-Id: Ic08d4ff78b6fb614b474b956a32eac352a14262a
(cherry picked from commit 72d3941c254a5a090afeb779aae5a9bd32c99827)