database migration file: add the required newline on the generated file

At first, I thought it would be better to leave the auto generated file
as is, but since this will become quite verbose on future runs of
check_patch.py, it's better to add it.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
database: increase size of TKO's test attributes value

Turns out that the definition of the test attribute value size set
in the tko models file (frontend/tko/models.py) was a lot smaller
than the size on the SQL schema file:

schema.sql snippet:

   CREATE TABLE `tko_test_attributes` (
      `value` varchar(1024) DEFAULT NULL,

frontent/tko/models.py snippet:

   class TestAttribute(dbmodels.Model, model_logic.ModelExtensions):
       value = dbmodels.CharField(blank=True, max_length=300)

This has been the case since 2009, according to commit f756779f.

So, increase the size so that attributes with larger values do not
get truncated.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
profilers.systemtap : fix import error when using systemtap profiler

Now under profiler/systemtap miss the file __init__.py, so when using
systemtap profiler will report import error like:
    "no module named systemtap"
This patch fix this error, and add some systemtap scripts for qemu test

Signed-off-by: Yunping Zheng <yunzheng@redhat.com>
documentation: added client docs structure

Signed-off-by: Cleber Rosa <crosa@redhat.com>
client/shared/distro.py: introduce distro detection API

This new API offers a way to precisely detect not only the vendor
OS, but also specifics such as the version and release of the
currently running distro.

Probes for new distributions are quite simple to create. That and
the basic usage of the API is properly documented.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
client/shared/software_manager.py: use distro detection API

Signed-off-by: Cleber Rosa <crosa@redhat.com>
client/shared/distro.py: add Probe based on Python's Standard Library

Ross Brattain pointed us to the platform.dist() function on Python's
Standard Library. We realized it's valuable to use it in addition
to our custom Probes.

So this introduces a Probe that relies on platform.dist(), that will
have its result used if no other (more specific) Probe exists.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
AsyncJob: implement support for stdin as string

AsyncJob did not support filling the stdin pipe with a string input.
Resolve this by creating a new thread function to write stdin_string
in 512 byte chunks to the subprocess stdin PIPE.

Extract the drainer thread to be a protected staticmethod so we can
unittest it.

Refactor the thread functions to eliminate branches and lookups during
the while loop.

Instead of checking for None, check for the actual methods we use.
Only check once if Lock is None, otherwise in the case of None replace
acquire() and lock() with a noop like bool.

v2:
Remove lock from stdin_thread since no other thread will access stdin
Add try/finally to _fd_drainer to release the lock after an exception

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Import mock.py 1.0.1 from http://www.voidspace.org.uk/python/mock/

mock is a library for testing in Python. It allows you to replace parts
of your system under test with mock objects and make assertions about
how they have been used.

mock is now part of the Python standard library, available as
`unittest.mock
<http://docs.python.org/py3k/library/unittest.mock.html#module-unittest.mock>`_
in Python 3.3 onwards.

Licensed under BSD-3-Clause.

Justification:

Since this particular mock library has been accepted in upstream Python 3.3 it
reflects community acceptance and suggests we should move towards using
this mock as a standard mock library for unittests.

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
client/shared/mock.py: disable pylint false positives

So that further pylint runs will not be verbose about them.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Add unittests for AsyncJob. These tests use the stdlib unittest.mock module backported from Python 3.3.

v2:
Unit tested on Windows, had to patch out resource, signal, and fcntl

Add patch for subprocess.Popen since Windows doesn't support the
preexec_fn parameter that is used in the BgJob __init__

Replace Thread MagicMock with a class since the MagicMock wasn't working
on Windows for some reason.

v3:
move mock to autotest.client.shared.mock, remove patch for resource,
signal, and fcnlt

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
utils/external_packages.py: increase maximum package size

Signed-off-by: Cleber Rosa <crosa@redhat.com>
gwt: update to version 2.5.1

This task is described on issue #606.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
frontent/client: simple fix to make it work with gwt 2.5.1

Signed-off-by: Cleber Rosa <crosa@redhat.com>
parse_ssh_path: fix regex to reject urls with no netloc

parse_ssh_path("ssh:///foo") does not raise an exception.  It should
raise an exception because there is no netloc in the URL.

Replace * with + in regex, to raise an exception if there is not one or more
characters in netloc.

parse_ssh_path could be replaced with urlparse.urlsplit in Python 2.6+

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
base_packages_unittest: add tests for parse_ssh_path

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Add various backports of Python 2.6 builtins: next(), any(), all(), bin()

This module contains four backports of Python 2.6 methods:
next() from Ross Brattain
any() from http://stackoverflow.com/questions/3785433/python-backports-for-some-methods
all() from http://stackoverflow.com/questions/3785433/python-backports-for-some-methods
bin() from http://code.activestate.com/recipes/576847/

These backports should be conditionally imported upon exception so they
do not shadow the builtin function on newer Pythons.

Example:

try:
    any([True])
except NameError:
    from autotest.client.shared.backports import any

Justification:

Using backports for builtin functions allows for forward-compatibility and
better performance on newer python versions.

bin() is used heavily when decoding hardware register bits.
any() and all() are useful for generators and list comprehensions
next() is useful because of the default argument, and it is consistent with
Python 3 usage.

v2: strip leading left zeros to match builtin bin()

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
backports_unittest: unit tests for backported any(), all(), bin(), next()

v2: test backports.bin against builtin bin for a larger range of numbers

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
backports: import OrderedDict backport by Raymond Hettinger

http://code.activestate.com/recipes/576693-ordered-dictionary-for-py24/

Documentation:
http://docs.python.org/2/library/collections.html#collections.OrderedDict

Changes from v1:
 * follow some pylint suggestions, disable other checks

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
backports: import namedtuple backport by Raymond Hettinger

http://code.activestate.com/recipes/500261-named-tuples/

Documentation:
http://docs.python.org/2/library/collections.html#collections.namedtuple

Changes from v1:
 * follow some pylint suggestions, disabled some checks

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>