Package: executor
Theexecutorpackage is a simple wrapper for Python’ssubprocessmodule
that makes it very easy to handle subprocesses on UNIX systems with proper
escaping of arguments and error checking:An object oriented interface is used to execute commands using sane but
customizable (and well documented) defaults.Remote commands (executed overSSH) are supported using the same object
oriented interface, as are commands insidechroots(executed usingschroot).There’s also support for executing a group of commands concurrently in
what’s called a “command pool”. The concurrency level can be customized and
of course both local and remote commands are supported.The package is currently tested on Python 2.7, 3.5, 3.6, 3.7, 3.8 and PyPy. For
usage instructions please refer to following sections and thedocumentation.InstallationUsageCommand linePython APIContactLicenseInstallationTheexecutorpackage is available onPyPIwhich means installation should be
as simple as:$pipinstallexecutorThere’s actually a multitude of ways to install Python packages (e.g. theper
user site-packages directory,virtual environmentsor just installing
system wide) and I have no intention of getting into that discussion here, so
if this intimidates you then read up on your options before returning to these
instructions ;-).UsageThere are two ways to use theexecutorpackage: As the command line programexecutorand as a Python API. The command line interface is described below
and there are also some examples of simple use cases of the Python API.Command linePython APICommand lineUsage:executor [OPTIONS] COMMAND …Easy subprocess management on the command line based on the Python package with
the same name. The “executor” program runs external commands with support for
timeouts, dynamic startup delay (fudge factor) and exclusive locking.You can think of “executor” as a combination of the “flock” and “timelimit”
programs with some additional niceties (namely the dynamic startup delay and
integrated system logging on UNIX platforms).Supported options:OptionDescription-t,--timeout=LIMITSet the time after which the given command will be aborted. By defaultLIMITis counted in seconds. You can also use one of the suffixes “s”
(seconds), “m” (minutes), “h” (hours) or “d” (days).-f,--fudge-factor=LIMITThis option controls the dynamic startup delay (fudge factor) which is
useful when you want a periodic task to run once per given interval but the
exact time is not important. Refer to the--timeoutoption for acceptable
values ofLIMIT, this number specifies the maximum amount of time to sleep
before running the command (the minimum is zero, otherwise you could just
include the command “sleep N && …” in your command line :-).-e,--exclusiveUse an interprocess lock file to guarantee that executor will never run
the external command concurrently. Refer to the--lock-timeoutoption
to customize blocking / non-blocking behavior. To customize the name
of the lock file you can use the--lock-fileoption.-T,--lock-timeout=LIMITBy default executor tries to claim the lock and if it fails it will exit
with a nonzero exit code. This option can be used to enable blocking
behavior. Refer to the--timeoutoption for acceptable values ofLIMIT.-l,--lock-file=NAMECustomize the name of the lock file. By default this is the base name of
the external command, so if you’re running something generic like “bash”
or “python” you might want to change this :-).-v,--verboseIncrease logging verbosity (can be repeated).-q,--quietDecrease logging verbosity (can be repeated).-h,--helpShow this message and exit.Python APIBelow are some examples of how versatile theexecute()function is. Refer to
the API documentation onRead the Docsfor (a lot of) other use cases.Checking status codesProviding inputGetting outputRunning commands as rootEnabling loggingRunning remote commandsRunning remote commands concurrentlyChecking status codesBy default the status code of the external command is returned as a boolean:>>> from executor import execute
>>> execute('true')
TrueIf an external command exits with a nonzero status code an exception is raised,
this makes it easy to do the right thing (never forget to check the status code
of an external command without having to write a lot of repetitive code):>>> execute('false')
Traceback (most recent call last):
  File "executor/__init__.py", line 124, in execute
    cmd.start()
  File "executor/__init__.py", line 516, in start
    self.wait()
  File "executor/__init__.py", line 541, in wait
    self.check_errors()
  File "executor/__init__.py", line 568, in check_errors
    raise ExternalCommandFailed(self)
executor.ExternalCommandFailed: External command failed with exit code 1! (command: bash -c false)TheExternalCommandFailedexception exposescommandandreturncodeattributes. If you know a command is likely to exit with a nonzero status code
and you wantexecute()to simply return a boolean you can do this instead:>>> execute('false', check=False)
FalseProviding inputHere’s how you can provide input to an external command:>>> execute('tr a-z A-Z', input='Hello world from Python!\n')
HELLO WORLD FROM PYTHON!
TrueGetting outputGetting the output of external commands is really easy as well:>>> execute('hostname', capture=True)
'peter-macbook'Running commands as rootIt’s also very easy to execute commands with super user privileges:>>> execute('echo test > /etc/hostname', sudo=True)
[sudo] password for peter: **********
True
>>> execute('hostname', capture=True)
'test'Enabling loggingIf you’re wondering how prefixing the above command withsudowould
end up being helpful, here’s how it works:>>> import logging
>>> logging.basicConfig()
>>> logging.getLogger().setLevel(logging.DEBUG)
>>> execute('echo peter-macbook > /etc/hostname', sudo=True)
DEBUG:executor:Executing external command: sudo bash -c 'echo peter-macbook > /etc/hostname'Running remote commandsTo run a command on a remote system usingSSHyou can use theRemoteCommandclass, it works as follows:>>> from executor.ssh.client import RemoteCommand
>>> cmd = RemoteCommand('localhost', 'echo $SSH_CONNECTION', capture=True)
>>> cmd.start()
>>> cmd.output
'127.0.0.1 57255 127.0.0.1 22'Running remote commands concurrentlyTheforeach()function wraps theRemoteCommandandCommandPoolclasses to
make it very easy to run a remote command concurrently on a group of hosts:>>> from executor.ssh.client import foreach
>>> from pprint import pprint
>>> hosts = ['127.0.0.1', '127.0.0.2', '127.0.0.3', '127.0.0.4']
>>> commands = foreach(hosts, 'echo $SSH_CONNECTION')
>>> pprint([cmd.output for cmd in commands])
['127.0.0.1 57278 127.0.0.1 22',
 '127.0.0.1 52385 127.0.0.2 22',
 '127.0.0.1 49228 127.0.0.3 22',
 '127.0.0.1 40628 127.0.0.4 22']ContactThe latest version ofexecutoris available onPyPIandGitHub. The
documentation is hosted onRead the Docsand includes achangelog. For bug
reports please create an issue onGitHub. If you have questions, suggestions,
etc. feel free to send me an e-mail atpeter@peterodding.com.LicenseThis software is licensed under theMIT license.© 2020 Peter Odding.