Docker: Add docker test harness v0.0.1

Control file searches down subtest module(s) and executes any
test.test subclasses via the steps engine.  This is because
there may be situations or tests which require rebooting
the test client.  The steps engine stores harness state
across reboots.  It then allows resumption of testing
with --continue option to the autotest client.

When running via autotest/autotest client, all subtests
are executed by default.  A custom list of subtests may
be provided via ``-a, --args`` in the form of subtest
module names as a CSV string.

Within submodule scope, any modules within the 'lib'
directory are made available in the 'docker' namespace.
This was done due to possible namespace clash when
autotest processes the 'docker' test name or if
``--args docker`` is used (for some reason).

Configuration processing is handled by the
``docker.config.Config`` class.  It uses the builtin
ConfigParser to load sections from all files under the
``config.d`` subdirectory.  Each section will be
supplimented (or may override) any options defined
in the ``defaults.ini`` file [DEFAULTS] section.

Instances of the ``Config`` class are regular python
dictionaries copied from an internal cache.
The dictionary keys are the section names, and values
are regular python dictionaries containing the
options key/value items for that section.  Modifications
is allowed, however they will remain local to the copy.

To support private configurations, version checking
is supported via the ``version`` module.  The helper
function ``check_version`` errors any test if the
'config_version' key from the test config section
is less-than the version encoded in the library module.
It is up to the user to make sure their config is modern.

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Add simple docker version subtest

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Add simple docker help subtest

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Add build test

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Add import subtest

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Add documentation

*  A README.rst file including warning & link to main documentation
   This is only needed because github displays it by default.
*  A index.rst file with everything included in one place to make
   searching easier.
*  Sphinx conf.py and Makefile to support building local
   copies of documentation using.
*  Added check before each test that documentation
   and API api versions match.  Subtest modules
   already verify configuration and API versions
   match.

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Version 0.1.1 updates

*  Support custom and/or private test modules and/or configurations.
   Otherwise modifying the default configs. will make version control
   grumpy.  Anything (includeing replacement defaults) defined inside
   the new config_custom directory tree will override content from the
   config_defaults (formerly config.d) tree.

*  Updated .gitignore file to ignore all files in config_custom
   subdirectory.

*  Subtests now support optionally setting the # of test iterations from
   their config section with the 'iterations' key.  If not present,
   it defaults to the class-attribute value.

*  Subtests and SubSubtests automatically log their configuration
   settings into the results/.../keyval file (including defaults).
   They will log a warning if no configuration section
   is found.  In this case, their config attribute is
   a copy of the defaults.

*  Added additional comments in various places, further clarifing
   decisions.  Updated documentation to version 0.1.1 reflecting
   all changes.

*  Added dockercmd module with classes to make running docker commands
   within subtests easier. Added unittests for dockercmd module.  Updated
   all bundled subtests to use dockercmd where possible

*  Added simple run_pylint.sh shell-script to run pylint across all python
   modules and tests.  Does not yet properly handle autotest deps. Will
   likely be replaced by inspektor.

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Version 0.2.1 updates

*  It was a mistake to design ``dockercmd.DockerCmd()`` as a subclass
   representing it's result.  It made extending data and behavior
   nearly impossible since state was effectively fixed at init time.
   Reworked dockercmd interfaces to support ``timeout`` and ``ignore_error``
   parameters properly w/ help from Jiří Župka <jzupka@redhat.com>.
   Simplified the interface into a base-class with sub-classes to specialize
   it to particular purposes.  This also had the consequence of
   greatly simplifying unittesting.

*  Most tests were having to run each output checker function
   on both ``stdout`` and ``stderr`` from a ``CmdResult`` instance.  It makes
   more sense to bundle this data with the output checking as
   a configurable/extensible set of behaviors.  Jiří Župka
   <jzupka@redhat.com> pointed out that the exception-raising
   nature of the checks should be optional behavior.  Moved
   the former ``output.usage_check()`` and ``crash_check()`` functions
   under a new class, ``OutputGood()``.  It takes a ``CmdResult`` as
   input, then either raises an exception or represents a ``True``/``False``
   value upon scanning ``stdin``/``stdout``.  The set of scanner/checker
   methods is configurable and easily expandable w/o changing
   the interface.

*  Added very basic unittests for ``output`` module.

*  Copy-pasted the special ``mock()`` function around in unittests.
   Bad-practice since it should go into a central module.  However,
   I already had a problem with it accidentally being visible inside
   a test, where it became a ugly, nasty, difficult to track problem.
   It needs to be de-duplicated at some point, but only if/when it can
   be done safely.  For now, it's duplicated inside all unittests and
   the documentation's ``config.py``

*  Moved version-check inside ``Subtest`` class under ``__init__``
   instead of ``initialize()``.  Formerly, it was causing failures
   during ``cleanup()`` which shouldn't even be running if there's
   a version mis-match.

*  Brought some exceptions from autotest under the ``xceptions``
   module.  This is mostly to help prevent subtest code from
   needing to import 'autotest.client.shared.error'.  It also
   makes debugging easier since we can tell where exceptions
   are coming from.

*  Re-worked the build subtest so that it is entirely self-contained
   (not requiring any external content).  This requires a static
   version of ``busybox`` is available on the system.  The benefit
   is the build-test produces much more predictable output.  It's
   also a potential technique/content for a base-level image we
   could use at some point for other testing.

*  Updated other tests to make use of new DockerCmd and output interfaces.

*  Fixed run_pylint.sh problem with importing autotest packages

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Add helpers for frequently used docker image/repository operations

The dockertest.images module defines several independent interfaces using
an abstract-base-class pattern.   They are extended through a few subclasses
to meet basic needs. There is an assumption that the full 64-character long
IDs are safer to use than the short, 12-character ones.  It is also assumed
that callers are in the best position to decide what if any 'tag' content
is used and how to mangle repo/image names.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Add docker cli 'image' test

Signed-off-by: Jiří Župka <jzupka@redhat.com>
Docker: Add run tests

*  Added [DEFAULTS] options to specify a image for testing purposes

*  Added a function to config module to help setting '' dictionary
   values to None.  Updated unittests.

*  Added a full_name_from_defaults static method to images.DockerImage
   to help with putting together a image full_name baesd on (above)
   new image configuration options.

*  Added three tests for the run command + configuration

*  Bumped revision number to 0.2.3 (API additions only, no changes)

*  Bumped all default config versions to 0.2.3

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Version 0.3.1 updates

*  Make SubSubtest class from 'dockerimport' test into
   a standard interface.  It will be used again later
   by the 'pull' test.

*  Allow the tester to specify on the autotest command-line,
   which subsubtests to run, and in what order.  For example,
   to start only the subsubtest "good" under the "pull" test:

   ./autotest-local --verbose run docker --args="docker_cli/pull:good"

*  Many coding style and whitespace fixes.

*  Frequently, subsubtests need access to their own and parent
   test configuration options.  Sometimes there are subtest
   options which should also apply to subsubtests.  Since the
   subsubtest config. is coppied into a python-dict anyway,
   it's trivial to inject the parent subtests config options
   as well.  This helps avoid possibly duplicate lines in
   subsubtest configurations.

*  Several image module class methods have no defined
   return type.  Make this part of the interface standard
   to avoid any doubt/confusion.

Signed-off-by: Jiří Župka <jzupka@redhat.com>

*  Removed individual 'config_version' overrides from all subtest
   configurations.  Master, [DEFAULTS] now specifies for all
   bundled test config versions.  In practice, testing environments
   will copy the shipped defaults under config_custom/ which overrides
   but is ignored by git.  Therefor, the intent of protecting config.
   customizations from changes in dockertest API still applies.  It
   simply requires less work when adding new shipped-test configs.

*  Ignore standard files produced/consumed by cscope e.g.

   find `pwd`/ -name "*.py" -o -name "*.ini" > cscope.files && \
       cscope -bv -i cscope.files -f cscope.out'

*  In an effort to reduce the work of API/test/config/docs
   version tracking, updated dockertest.Subtest base class
   to pull it's 'version' from it's configuration.  Unless
   overridden, this will now come from default.ini's [DEFAULTS]
   section.  Reducing development work while still flagging
   possible breakage of custom tests/config due to API
   changes.

*  To help separate 'input' from 'output' in subtests, added
   a 'stuff' attribute to the subtest base class.  This gets
   initialized to an empty python-dict.  After that it is
   totally ignored by the API so subtests may do with it
   as they please.

*  Similar to above, added a 'subStuff' attribute to the
   subtest.SubSubtest base class.

*  Updated all bundled subtests to use 'stuff' or 'subStuff'
   accordingly.

*  Added some more info to default subtest and subsubtest
   non-critical logging messages.

*  Updated/added some new comments to code in several places.

*  To avoid any possible confusion about relationships,
   renamed the SubSubtest.test attribute to
   SubSubtest.parentSubtest  Updated all bundled subtests
   accordingly.

*  Added a note to the subtest.SubSubtest class that it's
   interface is *like* that of Subtest, however it is-not
   itself a Subtest subclass.  This provides room for future
   expansion and/or divergence from the Subtest interface
   if ever needed.  This class may be renamed in the
   future (it's a pain to type).

*  Added/Updated docstrings on new subtest.SubSubtestCaller
   class.  Changed dockerimport subtest to use this new
   base class.

*  Updated main documentation for all the above changes
   and new test content.

Signed-off-by: Chris Evich <cevich@redhat.com>
docker: Adds subtests for pull command.

Subtests Pull contains some more subsubtests.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
Docker: Upstream push

*  Ran ``git filter-branch --subdirectory-filter docker -- --all`` on
   docker subdirectory.
*  Prepared documentation for inclusion into upstream autotest project
   on github.

Signed-off-by: Chris Evich <cevich@redhat.com>
dockertest.images: Fix repo_split_p regexp

Docker looks for either '.' or ':' in order to distinguish between
repo_addr or user name, thus the name could contain other characters
and not just \w.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
LICENSE: Fix the project name in LICENSE

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Merge pull request #4 from ldoktor/framework2

LICENSE: Fix the project name in LICENSE
Merge pull request #2 from ldoktor/framework

dockertest.images: Fix repo_split_p regexp
Docker: Revision 0.3.2 Updates (no interface changes)

*  Cleaned up some minor pylint problems

*  Added an 's' on the end of images_unittest.py

*  Added simple unittests script to execute them
   all in parallel.

*  Updated documentation to include subtest.SubSubtestCaller
   class.  It was mistakenly not added previously.

*  Added minimal networking module with ContainerPort class.
   Follows very closely to the same pattern as images module.

*  Added minimal containers module also mirroring closely
   with the images pattern.  However there are quite a few
   more container attributes, only some are needed for
   creation, the rest are simply stuck onto the class.

*  Added missing __init__ method to DockerImagesCLI and
   DockerContainersCLI to fill in default params.

*  Added standard autotest test exceptions to xceptions module.
   This makes it easier for tests (they don't need to import
   big-long autotest module).  It also allows us to see the
   different between a test-raised exception and harness-raised.

*  Extended DockerImagesCLI and DockerContainersCLI
   interface by turning docker_cmd() into a public method.
   This allows the possibility for tests to perform checks or
   modifications when the command being used is the subject
   of a test.  Changed the timeout parameter to be non-defaulting.
   Callers may override this behavior if needed.

*  Extended CLI interfaces for images and containers with
   new name 'clic'.  This is for when the image or ps command
   is the test subject and therefor output checkers need to
   run against it's output.

*  Added annother output module 'checker' to watch for an
   'error: ' string in output.  Updated all negative subtests
   to properly invert the checker results.

*  Updated build subtest to log a message when postprocess
   failed to remove the built image.

*  Added output_check method and override to pull tests.
   Updated negative subsubtest 'wrong_registry_addr' to properly
   fail on SUCCESSFUL checks (error-check now catches here)

*  Updated run subtest to use utils.signal_pid() which has some
   extra testing-related checks involved.

*  Added additional verbosity to check in version subtest.

Signed-off-by: Chris Evich <cevich@redhat.com>
Docker: Add test for 'ps -a' subcommand

Signed-off-by: Chris Evich <cevich@redhat.com>