qemu-testlib: Add python helper and simplify shell

The current code has a race since it greps for *any* qemu process
running, even if it isn't the one we started. This leads to some sanity
tests potentially failing on machines where multiple sets of sanity tests
are running.

To resovle this and some other ugly code issues, add a python script
to accurately walk the process tree and find the qemu process. We can
then replace all the shell functions attempting this which happen to
work in many cases but not all.

Also clean up some of the error handling so its more legible.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
scripts/qemutestlib: Add better process debugging and fix process group issue

In single testing with a shutdown scenario the processes are cleaned up correctly
but the manual cleanup fall back used for a minimal image do not work properly.
This patch fixes the kill commands to revert to non-process groups, fixing
the hung process issues that were occuring.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemuimage-testlib: Fix IP address handling

Remove some pointless code and also fix the return handling
for the function since it returns null, not 0 as the comments
would suggest.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemuimage-testlib: Add delay to work around races in qemu startup

Qemu changes pid when starting up. On a loaded machine, this can result
in the incorrect pid being returned. Since qemu will take a few seconds to
boot anyway, we might as well delay a short while and allow things to settle
which should fix various race issues being seen on the autobuilder.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemuimage-testlib: Add extra debugging and sanity check

Check for a zero IP address since its clearly incorrect if that value
is found. Also add debugging for cases where we can't find the qemu
process. A process listing is handy to help understand what the problem
might be.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqemu: Improve error handling/exit codes

runqemu-internal is sourced so should be returning with an error code in
case of errors. runqemu needs to deal with this.

This patch fixes up the various error paths so we're consistent and get
a sane exit status for runqemu which helps a lot in its use in the qemu
runtime testing on the autobuilder.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
scripts/runqemu-internal: Fix lock races

There are two problems here. Firstly the grep command is unanchored so
pid 345 will match against 12345 and so on.

The second issue is that there are several context switched between attempting
the lock and then writing the pid to it.

Between the two issues, there were issues appearing on the autobuilder due
to these conflicts. This patch replaces the mechanism with flock on fd 8
which should be a safer mechanism to use.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
scripts/qemu-testlib: Add more debugging information

This extra information should allow better forensics if the sanity tests
fail as they're currently doing occasionaly on the autobuilder for unknown
reasons.

The patch also tightens up certain checks to remove pointless noise and
error output from the logs.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemuimagetest: fix erroneous ps errors when qemu couldn't be started

The helper script was printing an error to stdout when it couldn't find
any qemu child processes; output this error to stderr instead and
redirect stderr to /dev/null when running from qemuimage-testlib so that
QEMUPID is actually blank if there are no qemu instances found.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemuimagetest: collect and print runqemu output

If runqemu (or qemu itself) fails we need to know why, so tee out to a
log file and print it when we can't find the qemu process or determine
its IP address.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemuimage-testlib: Capture stderror in the logs as well as stdout

This allows error messages to be captured in the logs which is helpful.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>