Skip autoscaling test until more reliable

AutoScalingTest has issues which will require rethinking how
this test is implemented, including:
- the optimal ConsumeStopSeconds will depend on the overhead
  of boot time, which varies greatly between bare metal and
  virt-on-virt
- every combination of distro/release/flavor will result in a
  different baseline memory usage, making it difficult to
  determine the ScaleDownThreshold to set

This change skips this test and tracks against the bug:
Partial-Bug: #1257575

Change-Id: I0695a7efacdbeab39c895a97cb45496fcaf97e35
Update tempest hacking regarding unit tests

This commit just updates some of the hacking documentation around unit
tests. It also adds a new hacking rule to ensure that setUpClass isn't
used for the unit tests.

Partially implements bp unit-tests

Change-Id: Ie8b1c9f1312a467265d53bc28ee905fa1b5fbb53
Moves negative tests from api/compute/servers/test_virtual_interfaces

This change set moves the negative tests in test_virtual_interfaces to
a new file test_virtual_interfaces_negative

Also changes line # 43, rand_name(#$%*&^%) to uuid4() to get the invalid
server ID in negative file and removes usage of modules directly.

Note: HACKING.rst
The second requirement is that all negative tests must be added to a negative
test file. If such a file doesn't exist for the particular resource being
tested a new test file should be added

Change-Id: Ia6b4c927112318b5b6b4583511537d1cbe020f56
Add testcases for volume

Adds tests for those GET API functions that accept params

List_volumes and list_volumes_with_detail for nova volume extension
List_volumes and list_volumes_with_detail for Cinder API
List_snapshots and list_snapshots_with_detail for Cinder API
Fixed typo in volume snapshot json client to make it consistent
with the corresponding xml client method "list_snapshots_with_detail"

Partial-Bug: #1086590

Change-Id: Ie703fa8729d838203c12293a7ce1ce942ec83a81
add admin server tests

- use regular user to resize vm in test_resize_server_using_overlimit_*
- rebuild server when it's in SHUTOFF and ERROR state
- add parameter 'raise_on_error' in wait_for_server_status

Change-Id: Ibd04a1fdaab74fe59bcf16cc99d4ecf4821f4ae2
Test for the update extra route

Add tests for neutron API '/routers/router_id'.

Change-Id: I989267a71cf00e6a7982ab99bce42d0aba74e3d2
Fix display_name of volume for test_volumes_list

In test_volumes_list, three volumes are created for some tests.
These display_name seems wrong like "volume-%s1393726625json".
The interface name should be inserted after "volume-", but it is
appended to the tail.
This patch fixes this volume name.

Change-Id: Iee55d2405e888569455a2325fe6b2843ddf9de9d
Separate negative tests in flavors/test_flavors

Separate negative tests to their own file. Restructured
invalid_id and nonexistent_id tests.

bp negative-test-files

Change-Id: I4b00353bcfef03e59cd3e628fb12774e0bbdcbd5
Add the external gateway interface to vpn router

Change-Id: Ib24080f0895a322576c837566976ade9fc4d8069
Closes-Bug: #1258379
Fix the upper values of test_network_quotas

In test_network_quotas scenario test, hard-coded values are used as
upper value.
These should be the actual quota values.

Change-Id: Ief1b12e13d0a191f2db78540d249c72c0dcc7d19
Closes-Bug: #1252999
Separate negative tests for list_floating_ips

Add negative tests for test_list_floating_ips to a specific file.
Since no floating_ips are needed to setup for negative test,
generation of non existent id can be simplified.

bp negative-test-files

Change-Id: I1d3ac0956ea3f69bbd5bde897f0a18fc0e182930
Add tests for keystone OS-TRUST v3 API

Add tests providing coverage of the keystone v3 OS-TRUSTS extension,
as documented at:
https://github.com/openstack/identity-api/blob/master/\
openstack-identity-api/v3/src/markdown/identity-api-v3-os-trust-ext.md

This covers all the documented interfaces to create/delete/list/get
trusts and roles delegated by trusts, except those impacted by bug
1246383 and 1245590, which will be submitted separately.

A subsequent patch will add coverage for consuming trusts via the tokens API.

Note only the JSON API is currently tested due to bug #1246941

Partially-Implements: blueprint keystone-trust-api
Change-Id: I310951f070107f3af33b52fcd2c8e54d82654088
Remove generic_setup_package() function

The generic_setup_package() function was previously run at init which
was causing a dependency on a config when a file was imported. This
commit takes that functionality and puts it in a classmethod and calls
that in setUpClass. There was nothing in generic_setup_package() that
needed to be run during so init, so this moves it to the proper place.

Change-Id: I32fa0fb1e230a98a3a9ca732df2d013ca4f13dce
Fix Neutron VPNaaS Test

VPNService requires Router is connected to external gateway.
In this commit, we connected the test router to external network.

Change-Id: Ib87583219efd8efd959c9c7edb813c3a70e7d20c
Closes-Bug:1259282
Make Heat's non_empty_stack usable without a server

We don't really need to test with a server to check Heat's
API responses. So just use a trivial resource type (a LaunchConfig)
to create some objects that we can query against. Now we can use
this test to gate.

Change-Id: I71bc065d8a1d2064ca90406543a376ce4d477275
Heat: check response fields

Make sure that we get the expected fields in each response.

Change-Id: Ie3d56304ffc7c80c186e100fc8d83fb78dff3044
Change unstable test which gets console output

We don't necessarily get "REBOOT" status of a server because the
state is ephemeral. As the result, the test fails sometimes.
This patch changes the server status from "REBOOT" to "SHUTOFF" for
stable test condition.

Closes-Bug: #1014683
Change-Id: I4b06b148b5d9bceda0cef2d2bb5f3b72325928ac
Add testcases for security groups

Adds tests for those GET API functions that accept params

List_security_groups Nova API

Skip security groups list by 'all_tenants' search filter test
if neutron is enabled because neutron do not support 'all_tenants'
search filter.

Change-Id: I5867acd03c3d184911281c673ce804455f269f57
Partial-Bug: #1086590
Fix the scope to share a server between tests

In test_server_actions.py, one server is created in setUpClass()
and the server would be shared between all tests.
If the server status is not 'ACTIVE' before each test, rebuild_
server() deletes it and rebuilds a server again.

The server_id scope which is passed to rebuild_server() is class
variable not instance variable. In addition, the server_id scope
which is stored with the return value of rebuild_server() is
instance variable like the following:

  [[instance]]                         [[class]]
  self.server_id = self.rebuild_server(self.server_id)

If we need to rebuild a server, the server_id, which is passed to
rebuild_server(), cannot be changed due to its scope. As the result,
many server creations happens in each test.
Then it causes "Quota exceeded" exception.

This patch fixes the scope of server_id for sharing the same server
between each tests.

Change-Id: I72b4aa73a6c77cce3a8e777d4f78913fb0986f66
Closes-Bug: #1257641
Don't have tox install pre-release software

Override the tox pip install command to avoid using the "--pre" option
which is the default in tox.  "--pre" means "Include pre-release and
development versions."  By default, pip will only install stable
versions of software, and that is the behavior we want.

Change-Id: I3aa2a69c5bae54932137c78cc711236a0ca24602