Package: concurrent-executor
concurrent-executorExecuting multiple commands concurrently using Python asyncioInstallationpipinstallconcurrent-executor# Or from the latest GitHub versionpipinstallgit+https://github.com/DCsunset/concurrent-executorCLI UsageAll the following tools can handle signals as follows:Upon receiving oneSIGINT(including keyboard interrupt) orSIGTERM, the program will sendSIGTERMto all spawned processes.Upon receiving more than one of them, the program will instead sendSIGKILLto kill all spawned processes.csshcsshis a command line tool provided by this package.
It is used to executing commands concurrently on remote servers vis SSH.Use-Hor--hoststo specify the hosts to run the commands on:cssh-Hhost1host2...host_n--command# pass extra ssh optionscssh-o="-q -4"-Hhost1host2...host_n--command# read hosts from filecssh-fhosts.txt--command# string interpolation (to include host name in command by {0})cssh-Hhost1host2--command--host{0}# pipe to stdinecho"yes"|cssh-Hhost1host2--command# Pipe file content to stdincssh-Hhost1host2--command<input.txtNote that--is necessary to separate the options and the command.
For-o/--optionsto work correctly, use=to prevent it from being parsed as another option.The standard input (stdin) of thecsshprocess is piped to the stdin of every spawned processes.For more details, seecssh -h.cexeccexecis another command line tool provided by this package.
It is used to execute arbitrary shell commands concurrently using template (string interpolation in Python).The command itself can container placeholder in strings: (SeePython string interpolationfor more detail.)# The variables are a, b, c in the template command# This command creates 3 directories and write to a file in each directorycexec-Vabc--"mkdir {0} && echo 1 > {0}/out"# Read variables from a filecexec-fvars.txt--"mkdir {0} && echo 1 > {0}/out"# Run different commands directlycexec-V"cmd1""cmd2""cmd3"--"{}"# pipe to stdinecho"yes"|cexec-Vabc--"cat -"For more details, seecexec -h.LibraryIt can also be used as a library:importasynciofromconcurrent_executor.executorimportSshExecutorasyncdefmain():hosts=["host1","host2"]executor=SshExecutor(hosts)# running concurrentlyawaitexecutor.run("some_command --test")# access stdout for all hosts (or stderr)asyncforindex,outinexecutor.stdout:print(f"{host[index]}:{out}")# wait until all finishedret_codes=awaitexecutor.wait()asyncio.run(main())See more usage inconcurrent_executor/cli.py.DevelopmentTo set up the development environment,
first clone this repo.Then it's recommended to usevenv:# suppose PWD is the root dir of the repopython-mvenvvenv# activate the environment``sourcevenv/bin/activate
pipinstall-rrequirements.txtTo deactivate, rundeactivate.LicenseThis project is licensed under AGPL-3.0. Copyright notice:concurrent-executor
Copyright (C) 2023 DCsunset

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.