Add test providers directory

The test-providers.d directory concentrate test provider
resources, such as configuration files, download directory
and the final test filesystem location itself.

What is a test provider?

It is a directory with .py test files and .cfg config
files, plus test resources, that will be the sources
of tests inside virt-test. The test suite tracks which
providers are available by looking into .ini files that
contain information about them, such as where they are
located in, their names and what types of virt tests
they contain. A README file in the test-providers.d
directory explains how the .ini files are structured.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.data_dir: Remove unused module

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.data_dir: Add test provider dir APIs

We'll frequently need to locate providers, so this
provides an appropriate API to do that.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.asset: Add test provider related APIs

Create an API to read from all test provider resources,
and return a list of dicts with information, that will
be used to download them. Also, add APIs to download
said test providers.

Use the autotest API autotest.client.test_config, so we
can make the functions more concise. Take the time to
make get_asset_info() shorter using this API.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
run: Add --update providers option

Instead of having test providers to be updated every
time the suite executes, update them only if asked to.

Introduce the needed hooks in the bootstrap function to
have the test providers installed on the 1st virt-test
execution.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.data_dir: Add get_backend_dir API

It is very frequent that we want to find out where a
given test backend dir is. So introduce that API, that
can be used when moving all the top level test dirs to
/backends.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.data_dir: Introduce get_backend_cfg_path API

Since it is also a fairly common operation inside the
test suite to find a file inside the config directory
of a given test backend.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Move all virt backend directories to 'backends'

In order to organize the backends used in virt-test,
currently:

 * qemu
 * libvirt
 * v2v
 * openvswitch
 * libguestfs
 * lvsb
 * generic (in fact, code that can be used among
            multiple backends)

Move them to a sub directory 'backends'. Make the
necessary adjustments in the code to load tests
properly.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.asset: Add get_known_backends() function

That is far better than keeping an inventory of them
manully.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.standalone_test: Look for tests in test providers

Rather than in backend dirs.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.bootstrap: Filter all top level cfg dirs on config filter

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.asset: Change get_all_test_provider_names name and behavior

With just a little more effort, we can get test provider
names by backend, rather than simply a dumb list. So
get_all_test_provider_names changes to get_test_provider_names
and we can optionally filter by backend.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
virttest.bootstrap: Construct subtests.cfg from test provider info

With this, virt-test can finally run the tests from
test providers specifically.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Remove generic backend dir

As now backends will just hold config files, generic
is no longer necessary.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Mass removal of files that belong to test providers

Rather than the core virt-test. Those files should
now come from test providers. The only files remaining
in backend dirs should be:

cfg/tests-shared.cfg
cfg/tests.cfg

With some few exceptions, handled case by case
(example - default_tests in libvirt).

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>