tko/autotest-db-delete-job: Do a better job handling MySQL errors

When trying to delete a job from the db, sometimes all
conditions for it to be deleted may not be fulfilled.
This usually happens when the job that the user tried
to delete was still on the scheduler queue, see issue #656.

So handle exceptions that might happen during the operation,
and try to make an educated guess about what happened
and how the user can work around the problem.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
contrib/install-autotest-server.sh: add steps for installing git

So that the autotest source code tree can be fetched and the other
packages installed via `autotest-install-packages-deps`.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
contrib/install-autotest-server.sh: use autotest-install-packages-deps

Revert back to the previous way of installing server package deps.
With the separate step that installs git this now works when the
user has the install script only, and not a checked out tree.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
contrib/install-autotest-server.sh: use `ip` instead of `ifconfig`

When attempting to detect the IP address that the Autotest server
will be listening on.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
frontend/pkgdeps.py: remove non-existing ubuntu package

Signed-off-by: Cleber Rosa <crosa@redhat.com>
client.shared.test: Add call trace back to log.

When debug qmp_basic_rehl6.py, got following output:
11:54:18 INFO | Running function: qmp_basic_rhel6.run_qmp_basic_rhel6()
11:54:18 ERROR| Test failed: TestFail: 'qemu' key is not of type '<type 'unicode'>', it's '<type 'dict'>'
11:54:18 ERROR| /usr/code/autotest/client/tests/virt/virttest/video_maker.py:195: DeprecationWarning: gst.Bin.add_many() is deprecated, use gst.Bin.add()
11:54:18 ERROR|   pipeline.add_many(source, decoder, encoder, container, output)

11:55:08 ERROR| child process failed
11:55:10 INFO | 		FAIL	virt.qemu.Host_RHEL.6.2.repeat1.qcow2.virtio_blk.smp2.virtio_net.RHEL.6.4.x86_64.qmp_basic_rhel6	virt.qemu.Host_RHEL.6.2.repeat1.qcow2.virtio_blk.smp2.virtio_net.RHEL.6.4.x86_64.qmp_basic_rhel6	timestamp=1370663710	localtime=Jun 08 11:55:10	'qemu' key is not of type '<type 'unicode'>', it's '<type 'dict'>'
11:55:10 INFO | 	END FAIL	virt.qemu.Host_RHEL.6.2.repeat1.qcow2.virtio_blk.smp2.virtio_net.RHEL.6.4.x86_64.qmp_basic_rhel6	virt.qemu.Host_RHEL.6.2.repeat1.qcow2.virtio_blk.smp2.virtio_net.RHEL.6.4.x86_64.qmp_basic_rhel6	timestamp=1370663710	localtime=Jun 08 11:55:10
11:55:10 INFO | END GOOD	----	----	timestamp=1370663710	localtime=Jun 08 11:55:1

It is difficult for us to debug the fail.

Signed-off-by: Feng Yang <fyang@redhat.com>
site_tests: remove client/site_tests/__init__.py to prevent conflicts

For site specific tests I use an external self-contained git repo in
client/site_tests/.git

This __init__.py conflicts with this external repo.  Remove it.  This
should have the side effect of preventing git from creating the
directory client/site_tests since git does not track directories, only
files.  This is fine because site_tests should not be version controlled
by autotest-core.

Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
utils/external_packages.py: Fix Python module versions comparison.

The is_needed function was comparing Python module versions with string
comparison.

This resulted in an installation bug on Fedora 18 triggered by the
python-paramiko package. Fedora 18 is currently shipping with paramiko
1.10.1 and it compared incorrectly to the 1.7.5 version required by
Autotest.

The fix uses distutils.version.LooseVersion for Python module versions
comparison. StrictVersion cannot be used because the Autotest required
python-matplotlib version of 0.98.5.3 is an invalid version number with
StrictVersion.

Signed-off-by: Vinson Lee <vlee@twitter.com>
cmdparser: add new command bootstrap

Because beaker uses the 'push' method for running tests, there has always
been a catch-22 on how to start autotest after provisioning the box.

Currently, beaker uses a daemon called 'beah' to do all the initial
bootstrapping.  In order to move away from beah we needed another
alternate approach.

Bill P. suggested to use an env variable.  So that is what I did.
When autotest runs from the /etc/init.d on bootup it will look for
'AUTOTEST_HARNESS'.  If that is set, call 'autotest bootstrap'.  The
env will tell autotest to use the harness beaker, download an initial
control and pass that to the 'run' command.  autotest will proceed as normal.

This is kind of an ugly hack but I could not think of another solution to deal
with bootstrapping after provisioning.

v2:  look for harness args in the environment, empty control files silently
     exit autotest (to handle reservesys case)

Signed-off-by: Don Zickus <dzickus@redhat.com>
harness: support for beaker's xml

Beaker passes its recipes through xml files.  This patch just converts the xml
recipe into a python dictionary.  Only used by beaker's get_recipe command.
Pretty straightforward.

Beaker folks should review this.  Not very interesting to autotest folks other
than style and syntax.

Signed-off-by: Don Zickus <dzickus@redhat.com>
harness: support beaker proxy protocol

This patch contains the pieces to communicate with Beaker's new stable API.

It is a thin layer that takes commands from the harness and translates them into
the appropriate http GET, POST or PUT command.

One thing I did to make debugging easier is to allow the ability to write to
files locally (offline mode).  The harness doesn't care where the data is sent,
so writing locally allow me to quickly test and address problems without complicating
it with broken http requests.

I also implemented chunk uploading to minimize memory usage.

I spent most of my time testing the log uploading and task result reporting
aspect of the API.  Everything seems to work great.  I expect the 'status'
and 'watchdog' parts to just work.

Mostly of interest to the beaker folks.  Autotest folks can review for style,
syntax and entertainment.

v2: python lint fixes

Signed-off-by: Don Zickus <dzickus@redhat.com>
harness: new beaker harness

This is the bulk of the harness file for enabling beaker.

The idea is to bootstrap it by downloading the remote xml file, parse it,
convert it into a control file and pass that to autotest.

The rest of the harness has various hacks to deal with passing info
from autotest to beaker.

Two of the biggest pieces are converting the xml recipe into a control
file.  This will still need tweaking.

The other piece is trying to figure out when recipe/tasks/sub-tasks
start and stop.  Again probably needs tweaking.

Overall, this patch has been tested with some stub recipes and responds
well.  Work still needs to be done on figuring out which files to
upload and real workflow testing.

This patch will need to be reviewed by autotest and beaker folks.

v2: rewrote file uploading to handle reboot case (save state), python
    lint fixes, handle reservesys case, param fixes
v3: Some renaming, small fixes, and squashing the beaker hook for quick
    commands into it.

Signed-off-by: Don Zickus <dzickus@redhat.com>
client.kernel: Adding handling of ppc processors

Add some code to handle building kernel on ppc machines
(32 bit versions). Also, reorganize handling on s390
and s390x.

Signed-off-by: Alexander Graf <agraf@suse.de>
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
client.kernel_config: Introduce feature_enabled API

Check in the given kernel config file whether a given
feature is enabled. Refactor the function modules_needed
to use it.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
client.kernel_config: Add unittests

Add simple unittests for functions inside the kernel_config
module.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
client.kernel: Add support to generate U-Boot linux images

If the kernel config has CONFIG_DEFAULT_UIMAGE enabled,
we want to build a uImage (U-Boot compatible linux image),
so set the build target accordingly.

Signed-off-by: Alexander Graf <agraf@suse.de>
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Django update: install version 1.5.1 with build_externals.py

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Django update: set the DEBUG setting when running tests

Enabling the DEBUG setting is the most straightforward way to prevent Django
from enforcing conditions that should be enforced during production, but not
necessarily under a test environment (such as when running unittests. One
example is the HOSTS_ALLOWED check that was introduced in Django 1.5.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Django update: make doctest work on different Django versions

The original doctest recording was too specific to the output generated
by older versions of Django. Instead, catch the exception that should be
caught, and only if a different exception is generated raise it to
force a unittest error.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Django update: ship with less secure but working configuration

Signed-off-by: Cleber Rosa <crosa@redhat.com>