ipmitool driver raises DriverLoadError

When the 'ipmitool' utility is not present in the system path,
the IPMITool driver will now raise a DriverLoadError during
initialization.

Change-Id: Iba8c3ad7053c600487ce4eb829ef57c708d8f154
Eliminate races in Conductor _check_deploy_timeouts

Re-check filters after we grab a lock in case things changed.

Tests are also re-factored to not use the DB. Tests are added for all
of these missing test cases.

Also: 'maintenance' was dropped from the filters in this review:

I646540e334dec05682640c05cedb315e0ee355bc

I don't see any explanation of why it was removed in that patch.

It seems reasonable that someone could set maintenance on a node that
is stuck in DEPLOYWAIT. And we would not want to touch it while it is
in maintenance. So, this patch re-adds it to the filters and makes
sure to re-check it after having a lock.

Also: When this was being reviewed, it was caught that
NoFreeConductorWorker should actually just completely exit the periodic
task. That change is included here too while re-factoring this.

Closes-Bug: 1332860

Change-Id: I9e73ca4a1d90bd92bdf83821dae72a30f7b885a9
Add some real-world testing on DiskPartitioner

This patch adds some integration tests for DiskPartitioner by using
it on a temporary file. As a side effect, this patch adds
disk_partitioner.list_partitions() helper, which is used by the test,
but can by used for anything else later.

The new tests are currently skipped on gate-ironic-python26, as
there is no `parted` utility there.
Also these tests do not cover destroy_disk_metadata, as it can't
be executed on a file.

Change-Id: I53bf2d207b7d15a0673be6a6786c47500b9eb499
Closes-Bug: #1306153
PXE to pass hints to ImageCache on how much space to reclaim

After previous patch PXE triggers cache clean up, if there's not enough
disk space for deployment. However, standard clean up may reclaim too
little space, due to configuration. This patch implements hints to
ImageCache, so that it tries to reclaim exactly required amount
of disk space.

Note that this amount is calculated as total size of images multiplied
by two as an attempt to account for images converting to raw format.

Change-Id: I8f95b7e6334a37e9e6e98278a5b61a9ace7221b5
Closes-Bug: #1316168
Mock pyghmi lib in unit tests if not present

Mock the pyghmi library from within ironic/tests/drivers so that
test_ipminative.py can import ironic/drivers/ipminative and run the unit
tests on it, even when the external library is not present. Also
corrects pep8 issue in ipminative by grouping the imports.

Change-Id: I870724bf48346857b1170463fcc7d22c08aca3bd
Add genconfig tox job for sample config file generation

If someone tries to run generate_sample.sh outside a venv
or without installing project dependencies, you could encounter
error messages related to missing or outdated dependencies.

This change wraps generate_sample.sh execution in tox. Wrapping it
in a tox job makes sure all dependencies found in requirements.txt and
test-requirements.txt are properly installed prior to its execution.

This change does not propose the removal of the sample config file
already in place in the repository and it does not propose adding
it to the list of jobs automatically ran by tox.

This change is merely the introduction of a more convenient way
to execute generate_sample.sh in a proper venv.

You can now (re)generate the sample config file with this tox job:

  tox -egenconfig

Change-Id: I1058a306a462ba8349a4433c78070fabd6d03e39
Merge "ipmitool driver raises DriverLoadError"
Merge "Add some real-world testing on DiskPartitioner"
Merge "Eliminate races in Conductor _check_deploy_timeouts"
Merge "Mock pyghmi lib in unit tests if not present"
Implement security groups and firewall filtering methods

Implement security groups and firewall filtering related methods in the
Nova Ironic Driver, the methods implemented are:

* refresh_security_group_rules() invoked when security group rules
  are updated

* refresh_security_group_members() invoked when instances are
  added/removed to a security group.

* refresh_provider_fw_rules() invoked when a firewall rule is added or
  removed from the list of rules.

* refresh_instance_security_rules() invoked when an instance gets added
  to or removed from the security group the instance is a member of or if
  the group gains or looses a rule.

* ensure_filtering_rules_for_instance() used to ensure that an instance's
  filtering rules are enabled.

* unfilter_instance() to stop filtering the instance.

Change-Id: Ib464dc6df4b03c952d564fdd4185ffbab59a8bc8
Do not delete pxe_deploy_{kernel, ramdisk} on tear down

As part of the instance_info work it was agreed that the deploy kernel
and ramdisk IDs should be passed to Node when enrolling it and not be
present on the Flavor anymore so that we can enable Ironic to support
different deploy provisioning methods using the same Flavor. The problem
is that right now Ironic is always removing the pxe_deploy_kernel and
pxe_deploy_ramdisk parameters from the driver_info as part of the tear
down process, but it shouldn't because those parameters should be constant
across multiple deployments to the node.

This patch fixing it by adding a backward compatible layer that would
work for both: The old way, by having the deploy kernel/ramdisk ID in
the flavor. And the new way, passing the deploy kernel/ramdisk as part
of the enrollment process of the node.

If the deploy kernel and ramdisk IDs are present on the flavor, the
Ironic Nova driver will stash it on driver_info when deploying the
node and delete it from driver_info when unprovisioning the node, if
it's not on the Flavor, the Ironic Nova driver does neither of those
(and either it's already on the node's driver_info and things work,
or the node fails on validate)

Implements: blueprint add-node-instance-info
Change-Id: I67e2deffaa55a754a7649f8863f7ac3c033dbf6f
Imported Translations from Transifex

Change-Id: I6b551aa202303c9d0c98d2b22900c4cc236e29ae
Merge "Add genconfig tox job for sample config file generation"
Merge "Imported Translations from Transifex"
Merge "Implement security groups and firewall filtering methods"
Merge "Do not delete pxe_deploy_{kernel, ramdisk} on tear down"
Remove from_chassis/from_nodes from the API doc

The flags from_chassis/from_nodes are internal flags and are used
to indicate whether the requests coming to the controller are
coming from the top-level resource and so prevent some actions like
creating/deleting/updating a resource to happen, this flags were being
passed to the class constructor and were appearing in the auto-generated
API documentation as a misleading parameter that could be passed as part
of the resource URI.

This patch is removing those flags from the class constructors.

Closes-Bug: #1329389
Change-Id: I6b6ba2ab303fb485350f52569539ac405ec5ade3
Fixing hardcoded glance protocol

glance protocol was hardcoded as "http".
But someone may be use https for glance like us.

We are already have configuration for glance protocol.
So we should use it.

Change-Id: Iebde9efa5c4bb531b5b8a80061d98ccc360b622c
Closes-Bug: 1334489
Update "Exercising the Services Locally" doc

The sample config used to have several drivers enabled, but this was changed
to default to the 'pxe_ipmitool' production driver. The developer documentation
for "Exercising the Services Locally", has examples using the 'fake_ipmitool'
driver. It won't work since only the pxe_ipmitool driver is available. This adds
instructions for updating the config file to enable fake_ipmitool instead of
pxe_ipmitool.

Users sometimes encounter the 'NodeLocked' exception while trying out the
sample "ironic" commands, due to the periodic sync_power_state task locking
the node. Documentation was added with instructions on how to turn off this
background sync_power_state task via a config setting.

Change-Id: Id53db08df59d13ca7a52505d6b3c0931551f4bf4