autotest/virt-test — pull request #1449 by jzupka (2014-02-17 14:10:18+00:00)
Title: RFC:  Remote commander (remote python shell)

**Allows start commands and collect results really easy way in remote system.**

How to test create new test and read in lower part:
```python
import time
import logging
from autotest.client.shared import error

@error.context_aware
def run(test, params, env):
"""
Qemu reboot test:
1) Log into a guest
3) Send a reboot command or a system_reset monitor command (optional)
4) Wait until the guest is up again
5) Log into the guest to verify it's up again

:param test: QEMU test object
:param params: Dictionary with the test parameters
:param env: Dictionary with test environment.
"""

timeout = float(params.get("login_timeout", 240))
vm = env.get_all_vms()[0]
vm.wait_for_login(timeout=timeout)

cmd = vm.remote_commander()
commands = []
for _ in xrange(2):
commands.append(cmd.async.shell("ip a"))

for c in commands:
logging.info(c.wait().stdout)
```

Another testing:

The remote commander consists of 3 parts messenger,
remote_master and remote_runner (python shell).

The messenger is base class for remote_master and remote_runner
which is used for sending picked data over IO. IOs are implemented
like wrappers to special type of IO. For example python streams,
file descriptors, paramiko class. ...

data => cPickle.dump(data) => IO_wrapper_OUT.write()
IO_wrapper_IN.read(len, timeout) => cPickle.load(data) => data

messanger could be used for general communication over pipe
or similar IO interface.

The remote_master is responsible for passing commands to
remote_runner and receiving results of commands.

example:
remote_master.CommanderMaster(inw, outw, False)

where:
inw is wrapper to input stream from remote_runner -> remote_master
outw is wrapper to output stream from remote_master -> remote_runner
False disables debug mode during which debug log are printed.

The remote_runner is responsible for starting commands
in system where the remote_runner running.

Command can be started in 4 modes:
1) blocking:
Remote master wait until command on remote_runner is finished and
after that it returns results.
cmd = remote_master.CommanderMaster(inw, outw, False)
a = cmd.shell("ip a") # returns cmd
shell("ip a") -> remote_runner
remote_master <- basecmd object
wait -> fork new process
wait -> wait till end of process
a <- remote_master <- results
a.stdout # stdout of command
a.stderr # stderr of command
a.results # what ever what command returns. (needs to be picleable).

2) async:
It is possible start lots of async command on remote_runner simultaneous.
cmd = remote_master.CommanderMaster(inw, outw, False)
a = cmd.async.shell("ip a") # returns cmd
async.shell("ip a") -> remote_runner
a <- remote_master <- basecmd object
do whatever -> fork new process
do whatever -> wait till end of process
a.wait_response() # wait for any response from remote command
# stdout, stderr, results
a.wait() # wait until command return result.
a <- remote_master <- results
a.stdout # stdout of command
a.stderr # stderr of command
a.results # what ever what command returns. (needs to be pickleable).

3) nohup:
It is possible start lots of nohup command on remote_runner simultaneous.
It is also possible loose connection with remote machine (kill remote_runner)
and after that reconnect to running nohup process.
There could be problem with really long outputs from nohup process the buffer
in communication interface could overflow. Keep it in mind!!!
cmd = remote_master.CommanderMaster(inw, outw, False)
a = cmd.nohup.shell("ip a") # returns cmd
nohup.shell("ip a") -> remote_runner
a <- remote_master <- basecmd object
do whatever -> daemonize new process
-> create communication interface to nohup process.
do whatever -> wait till end of process

stdin, stdout, stderr = ssh.exec_command("/usr/bin/env python /tmp/remote_runner.py agent", 0)

#!!!!!!! reconnect to remote_runner interface.
cmd = remote_master.CommanderMaster(ParamikoIOWrapperIn(stdout),
ParamikoIOWrapperOut(stdin),
False)
#!!!!!!! set new commander for old commands.
a.set_commander(cmd) -> reconnect to nohup communication interface.
-> continue in communication.
a.wait_response() # wait for any response from remote command
# stdout, stderr, results
a.wait() # wait until command return result.
a <- remote_master <- results

a.stdout # stdout of command
a.stderr # stderr of command
a.results # what ever what command returns. (needs to be pickleable).

4) manage:
It starts command in remote_runner process without forking.
It could be dangerous but it is really useful for control of remote_runner.
For example it is possible start function cmd.manage.exit() for exiting of remote_runner.
Remote master wait until command on remote_runner is finished and
after that it returns results.
cmd = remote_master.CommanderMaster(inw, outw, False)
a = cmd.manage.add_function # return cmd
manage.add_function(remote_master.get_source(func_name)) -> remote_runner
remote_master <- basecmd object
wait -> wait till end of process
a <- remote_master <- results
b = cmd.async.func_name(args..)
a.stdout # stdout of command
a.stderr # stderr of command
a.results # what ever what command returns. (needs to be pickleable).

There are also prepared helper function in remote_runner.
shell # start command in shell (bash ... )
interactive # start interactive python shell
send_msg # sends msg to specific command
add_function # for register class of function to global namespace of runner
# if add_function is started in manage mode then all commands
# could use this new function and it is possible to call the function
# like normal command from remote_master.
copy_file # send file to remote system very naive implementation.
import_src # import python module to python. (copied file or python libs)
exit # kill the remote_runner.