XenAPI: Refactor Fake to create pools, SRs and VIFs automatically

Required for changing the default SR to be the pool default - see
I116f65ece94f839e63b085480ba4f8974119d2bb
This also fixes some minor bugs in fake.py shown up by the tests

Change-Id: I19fc0747648b97a0912d28008981a84f70107d7b
port baremetal_nodes API into v3 part1

This changeset only copies the v2 files into the appropriate v3
directories unchanged. The copy as-is will not be loaded by either
the v2 or v3 extension loaders. The second changeset will then
make the changes required for it to work as a v3 extension.

This is being tried in order to make reviewing of extension
porting easier as gerrit will display only what is actually
changed for v3 rather than entirely new files.

Change-Id: I180bbf7306e7d3bffa4d6093fedfd52e4baf5215
port BaremetalNodes API into v3 part2

This patch contains the changes required to adapt the BaremetalNodes
extension and the corresponding unittest to the v3 framework

Change-Id: Iceef736a3821a4628afc34be29a04766421d3e47
Add expected_errors for extension extended_volumes v3

Partially implements bp v3-api-extension-versioning

Change-Id: I724cf10f23eead727ad3a91ab0b313524f68ca52
Add rpcapi tests for shelving calls

Adds tests to verify that compute RPC calls have the correct version and
pass the correct arguments.

Change-Id: Idb931139d3a7e6f47cbb474807c619f0ae3cd4fb
xenapi: Pass string arguments to popen

The current BitTorrent code was passing integer arguments to `make_subprocess`
which would in turn pass those to `popen`. This would cause a failure because
the semantics of the underlying `execve` function is to only accept a list of
strings.

The solution is to cast the arguments to a string before passing them into
`make_subrpocess`.

Fixes bug 1201594

Change-Id: I8372795ca032420641c449cb07bb47f9252ff4ef
Use valid IP addresses values in tests

Currently we use strings like 'fake_address', 'localhost', etc
as IP addresses values in DP API tests. This works great when
running tests on SQLite or MySQL backends, where IP addresses
are stored as plain strings. But in PostgreSQL there is a special
column type (INET), which checks inserted values to be valid IP
addresses, so tests fail.

This patch ensures we use valid IP addresses values instead of
arbitrary strings in DB API tests.

Blueprint: db-api-tests-on-all-backends

Change-Id: I2026bab612fcac09b245f6745810b6200f1f2443
Change EC2 client tokens to use system_metadata

An EC2 client token is created as a result of RunInstances. The initial
implementation called create_tags which uses per-instance metadata.
However, we can't update per-instance metadata while an instance is still
building because put the metadata updates in the messaging queue.

We avoid this problem by using system_metadata instead of per-instance
metadata. As an added bonus, users can't tamper with the token via the tags
API.

Fixes bug #1202404

Change-Id: I8ff28d75154a30fe3b521926453b9e9cbdd9fead
Fix iptables rules when metadata_host=127.0.0.1

When metadata_host is set to 127.0.0.1 metdata_forward will use the
REDIRECT target instead of the DNAT target in iptables rules. The
REDIRECT target results in an implicit DNAT to the primary address
of the incoming interface. As a result requests coming in from VMs
will have a destination address of the gateway brought up by
nova-network.

metadate_accept will now allow traffic destinatined for any local
address on the metadata_port when metadata_host is set to
127.0.0.1.

Change-Id: I877befe6f7c1020d2160194b6f6cab8297cb086c
Fixes: bug #1202356
Sync eventlet_backdoor from oslo-incubator

Includes this commit:

 c7c55b2 Improve usability when backdoor_port is nonzero

Change-Id: I06c0af7179bc2d0078e2cb757bef8551d8ce8b77
Raise exception if both port and fixed-ip are in requested networks.

When both fixed-ip and port (uuid) are specified as part of the
request during server creation, the user specified fixed-ip must
be ignored because it has already been allocated for the port. For
compute osapi v3, we raise a BadRequest Exception to ensure the user
understands why he/she is not getting what he/she asks for.

This patch also address some minor nits in other error messages.

Fixed bug #1190777

Change-Id: Iaf606637a8acf9d20fc6cae0bed5c9a3ea0bf611
Fix flavor v3 follow API v3 rules

* Change the parameter naming style for index and detail
* Add decorator expected_errors

Partially implements bp v3-api-extension-versioning

DocImpact

Change-Id: Iae6fd27e665388f4e6b24a3685ea7ed39e78a89d
Per-project-user-quotas for more granularity

Implements blueprint per-user-quotas.

Fixes bug 968175

Based on the original quotas structure.

NOTE:
quota_instances, quota_cores, quota_ram, quota_key_pairs and
quota_security_groups are supported per user.

Add CRUD methods for project user quotas API. DocImpact
 - Shows quotas for a user.
   GET v2/{tenant_id}/os-quota-sets/{tenant_id}?user_id={user_id}
 - Updates quotas for a user.
   POST v2/{tenant_id}/os-quota-sets/{tenant_id}?user_id={user_id}

Add commands for project user quotas management.
 - Show user quotas:
   nova-manage project quota --project <Project name> --user <User name>
 - Update/Create user quotas:
   nova-manage project quota --project <Project name> --user <User name>
   --key <key> --value <value>

Change-Id: I24af1f6bc439d5d740303c6fe176a9bffe754579
Fix extension rescue follow API v3 rules

* Add decorator expected_errors
* Change 'adminPass' to 'admin_pass'

Partially implements bp v3-api-extension-versioning

DocImpact

Change-Id: Ibfc78801bf91768c62f2b055e4f663396d66f66d
xenapi: Fix console rotate script

First time run of the console rotate script failed, due to the fact that
the logs did not exist at that time.

Fixes bug 1204440

Change-Id: I49b5569c1ba3349d911caace42685fa0f19a1f66
Sync models for AgentBuild, Aggregates, AggregateHost tables.

DB models definition should be up-to-date with DB schema obtained after
applying of migrations. Currently, there are no tests enforcing this and
it's really hard to analyze what indexes should be added, and what indexes
we have at the moment.

So the first step is it to fix all models, and add missing descriptions
about Indexes and UniqueConstraints in __table_args__ and use explicit
'nullable' parameter in columns description.

Add empty __table_args__ parameter to Table

blueprint db-sync-models-with-migrations

Change-Id: I76e064cf11d8b0df473c6085271b48a81fe31702
Fix policy failure on image_metadata calls

Image metadata create/update/delete calls were not catching failures
returned by Glance and translating them into nice returns for API
callers.  Now ImageNotAuthorized failures are caught and passed back to
the caller as 403s.

bug 1204646

Change-Id: Ib9e451531d8102b6089e2cfc9be16d1bd635d59d
Fix the is_volume_backed_instance check

Changing the check for volume backed instance to handle instances with
no root_device_name

Fixes LP bug #1197237

Change-Id: If0e2dff98a55ed1addf8a010849389f85db427f6
Update oslo.config.generator

pick 01e5c8e Change file bit of sample config file generator
pick a6ddb0a config.generator: remove the total number of options

Change-Id: I18ae09b54a4e0c7f972479b464f681adca47493e
Merge "Sync models for AgentBuild, Aggregates, AggregateHost tables."