Package: loon
loon - A Python toolkit for operating remote hostDescriptionloonis a Python toolkit for operating remote host based on SSH. Idea for developinglooncomes fromsync-deploy, which is limited by its pure bash code. Therefore, I use Python to implement it and more features will be added to it in the future.InstallationInstall from pypi:pipinstallloonInstall from GitHub:pipinstallgit+https://github.com/ShixiangWang/loonUsageConfigurationTo access remote host without typing password, you have to generate ssh key withssh-keygencommand if it is not available.$ssh-keygenFollow the commands, for simplicity, just typeENTERto proceed.Copy your key to remote server, replaceuserwith yourusernameandhostwith your remote host ip address.$ssh-copy-id-i~/.ssh/id_rsauser@hostHost managementAdd a remote host$loonadd-Uwsx-H127.0.0.1-P22=>Addedsuccessfully!# Default port is 22, so don't need to specify it# And we can create a host alias, otherwise# it is same as username of remote host$loonadd-Uwsx-H127.0.0.2-Nhost2=>Addedsuccessfully!List all remote hosts$loonlist
+-----+--------+----------+----+|Alias|Username|IPaddress|Port|+-----+--------+----------+----+|<wsx>|wsx|127.0.0.1|22|+-----+--------+----------+----+|host2|wsx|127.0.0.2|22|+-----+--------+----------+----+
<activehost>Rename alias$loonrenamewsxhost1
$loonlist
+-------+--------+----------+----+|Alias|Username|IPaddress|Port|+-------+--------+----------+----+|<host1>|wsx|127.0.0.1|22|+-------+--------+----------+----+|host2|wsx|127.0.0.2|22|+-------+--------+----------+----+
<activehost>Switch active remote host$loonswitch-Nhost2=>Activated.
$loonlist
+-------+--------+----------+----+|Alias|Username|IPaddress|Port|+-------+--------+----------+----+|host1|wsx|127.0.0.1|22|+-------+--------+----------+----+|<host2>|wsx|127.0.0.2|22|+-------+--------+----------+----+
<activehost>Delete a host$loondelete-Nhost2=>Removinghostfromavailablelist...=>Removingactivehost...=>Changingactivehosttohost1
$loonlist
+-------+--------+----------+----+|Alias|Username|IPaddress|Port|+-------+--------+----------+----+|<host1>|wsx|127.0.0.1|22|+-------+--------+----------+----+
<activehost>Common tasksRun commands$loonrun'ls -l ~'total168drwxr-xr-x2wsxliulab25Apr723:26bin
drwxr-xr-x.2wsxliulab6Apr410:36Desktop
drwxr-xr-x.2wsxliulab6Apr410:36Documents
drwxr-xr-x.3wsxliulab69Jun1016:57Downloads
drwxr-xr-x2wsxliulab6Sep3010:23facet
drwxr-xr-x11wsxliulab4096Sep2220:13metawho
drwxr-xr-x.2wsxliulab6Apr410:36Music
drwxr-xr-x3wsxliulab60Apr3017:50Notebooks
drwxr-xr-x.2wsxliulab6Apr410:36Pictures
drwxr-xr-x6wsxliulab114Sep2717:33projects
drwxr-xr-x6wsxliulab96Jun2716:50projects_bk
drwxr-xr-x.2wsxliulab6Apr410:36Public
drwxr-xr-x.2wsxliulab6Apr410:36Templates
drwxr-xr-x5wsxliulab4096Oct312:24testdrwxr-xr-x3480wsxliulab114688Oct313:44tmp
drwxr-xr-x3wsxliulab32Aug2217:13tools
drwxr-xr-x.2wsxliulab6Apr410:36VideosRun local scriptsThis will upload scripts to remote host firstly, then run them.$loonrun-f../../tests/scripts/t*.py=>Startingupload...

t1.py100%5049.0KB/s00:00t2.py100%5077.6KB/s00:00=>Finisheduploadingin1s=>Gettingresults:
Thisist1script.
Thisist2script.If input contains both files and directories, all files in directory will not be executed. This is a way to include child scripts which does not need to be executed.If input is only a directory, all scripts (not including scripts in subdirectories) under it will be executed. This is the way to maintain an independent project.You can include data directory using--dataflag, specify program likebashorpythonusing--progflag and set remote directory using--dirflag.Upload and download filesUse them likecpcommand. At default, usescpcommand to do the job, set--rsyncto usersynccommand (--rsyncis disabled in Windows). Note there are some differences between scp and rsync, especially processing directory.$loonupload-h
usage:loonupload[-h][-v][--rsync]source[source...]destination

positionalarguments:sourceSourcefilestouploaddestinationRemotedestinationdirectory

optionalarguments:-h,--helpshowthishelpmessageandexit-v,--verbosesetlogleveltoINFO--rsyncUsersyncinsteadofscp

optionalarguments:-h,--helpshowthishelpmessageandexit-v,--verbosesetlogleveltoINFO

$loondownload-h
usage:loondownload[-h][-v][--rsync]source[source...]destination

positionalarguments:sourceSourcefilestodownloaddestinationLocaldestinationdirectory,note'~'shouldbequotedinsomecases

optionalarguments:-h,--helpshowthishelpmessageandexit-v,--verbosesetlogleveltoINFO--rsyncUsersyncinsteadofscpBatch process commandsBy providing a structed stdin/file (CSV, TSV etc) and a sample command with placeholders{index}refer to column index (0 based) or column name of file,batchcommand can be used to execute a batch of commands. Users can set thread number by-Tflag and use--dryflag to dry run the code.$loonbatch-fsrc/loon/data/samplefile.csv'echo hello {0}'helloTCGA-2A-A8VO-01
helloTCGA-2A-A8VT-01
helloTCGA-2A-A8VV-01
helloTCGA-2A-A8VX-01

$loonbatch-fsrc/loon/data/samplefile.csv'echo hello {0}'-T4helloTCGA-2A-A8VO-01
helloTCGA-2A-A8VT-01
helloTCGA-2A-A8VV-01
helloTCGA-2A-A8VX-01

$loonbatch-fsrc/loon/data/samplefile.csv'echo hello {0}'--dry=>RunningechohelloTCGA-2A-A8VO-01=>RunningechohelloTCGA-2A-A8VT-01=>RunningechohelloTCGA-2A-A8VV-01=>RunningechohelloTCGA-2A-A8VX-01You can also read the input from pipe.$echo"yes,no"|loonbatch'echo {0} is not {1}'yesisnotno

$catsrc/loon/data/samplefile.csv|loonbatch'echo sample {0} has a longer name {1}'sampleTCGA-2A-A8VO-01hasalongernameTCGA-2A-A8VO-01-01
sampleTCGA-2A-A8VT-01hasalongernameTCGA-2A-A8VT-01-01
sampleTCGA-2A-A8VV-01hasalongernameTCGA-2A-A8VV-01-01
sampleTCGA-2A-A8VX-01hasalongernameTCGA-2A-A8VX-01-01You can also handle header and refer to column names with index or name!$ cat tests/header.txt | loon batch 'echo hello {0}' 
hello user
hello wsx
hello zd

$ cat tests/header.txt | loon batch 'echo hello {0}' --header
hello wsx
hello zd
$ cat tests/header.txt | loon batch 'echo hello {0}, your score is {1}' --header
hello wsx, your score is 100
hello zd, your score is 100

$ cat tests/header.txt | loon 'echo hello {user}, your score is {score}' --header
hello wsx, your score is 100
hello zd, your score is 100
$ cat tests/header2.txt | loon 'echo hello {user name}, your score is {score}' --header
hello wsx, your score is 100
hello zd, your score is 100Generate a batch of (script) filesgencommand is similar to thepbsgencommand below, but removes the.pbsfile extension,
so users have to add the file extension in the first column of the SAMPLEFILE if necessary.The example files can be generated usingpbsgen_example.usage: loon gen [-h] [-v] [-t TEMPLATE] [-s SAMPLEFILE] [-m MAPFILE]
                       [-o OUTPUT]

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         set loglevel to INFO
  -t TEMPLATE, --template TEMPLATE
                        A template file containing placeholders
  -s SAMPLEFILE, --samplefile SAMPLEFILE
                        A csv file containing unique filenames (the first
                        column) and replacing labels
  -m MAPFILE, --mapfile MAPFILE
                        A csv file containing placeholders and column index
                        (0-based) indicating replacing labels in samplefile
  -o OUTPUT, --output OUTPUT
                        Output directoryPBS management and taskspbstemp- Generate a PBS template filepbsgen- Generate a batch of PBS filespbsgen_example- Generate example files for pbsgen commandpbssub- Submit PBS taskspbsdeploy- Upload a target directory and submit containing PBS files (have.pbsextension)pbscheck- Check status of PBS job on remote hostMore details please see-hoption of the commands above.Current usage infousage:loon[-h][-V][--author]{add,delete,switch,list,rename,run,upload,download,gen,batch,pbstemp,pbsgen,pbsgen_example,pbssub,pbsdeploy,pbscheck}...

Beanefficientloon.

optionalarguments:-h,--helpshowthishelpmessageandexit--versionshowprogram's version number and exit--author              show info of program'sauthor

subcommands:{add,delete,switch,list,rename,run,upload,download,gen,batch,pbstemp,pbsgen,pbsgen_example,pbssub,pbsdeploy,pbscheck}descriptionaddAddaremotehostdeleteDeletearemotehostswitchSwitchactiveremotehostlistListallremotehostsrenameRenamehostaliasrunRuncommandsorscriptsonremoteuploadUploadfilestoactiveremotehostdownloadDownloadfilesfromactiveremotehostgenGenerateabatchof(script)filesbatchBatchprocesscommandswithplaceholderspbstempGenerateaPBStemplatefilepbsgenGenerateabatchofPBSfiles(with.pbsextension)pbsgen_exampleGenerateexamplefilesforpbsgencommandpbssubSubmitPBStaskspbsdeployDeploytargetdestinationtoremotehostpbscheckCheckstatusofPBSjobonremotehostNoteThis project has been set up using PyScaffold 3.2.2. For details and usage
information on PyScaffold seehttps://pyscaffold.org/.