Package: cmdy
cmdy"Shell language" to run command in pythonInstallationpipinstallcmdyUsageSeeDemoBasic usagefromcmdyimportlsprint(ls())forlineinls().iter():print('Got:',line,end='')With non-keyword argumentsfromcmdyimporttarprint(tar("cvf","/tmp/test.tar","./cmdy"))With keyword argumentsfromcmdyimportcurlcurl("http://duckduckgo.com/",o="/tmp/page.html",silent=True)# curl http://duckduckgo.com/ -o /tmp/page.html --silentOrder keyword argumentscurl("http://duckduckgo.com/","-o","/tmp/page.html","--silent")# orfromdiotimportOrderedDiotkwargs=OrderedDiot()kwargs.silent=Truekwargs.o='/tmp/page.html'curl("http://duckduckgo.com/",kwargs)# You can also use collections.OrderedDictPrefix and separator for keyword argumentsfromcmdyimportbedtools,bcftoolsbedtools.intersect(wa=True,wb=True,a='query.bed',b=['d1.bed','d2.bed','d3.bed'],names=['d1','d2','d3'],sorted=True,_prefix='-').h().strcmd# 'bedtools intersect -wa -wb -a query.bed \# -b d1.bed d2.bed d3.bed -names d1 d2 d3 -sorted'# default prefix is autobcftools.query(_=['a.vcf','b.vcf'],H=True,format='%CHROM\t%POS\t%REF\t%ALT\n').h().strcmd# "bcftools query -H --format '%CHROM\t%POS\t%REF\t%ALT\n' a.vcf b.vcf"ls(l=True,block_size='KB',_sep='auto').h().cmd['ls','-l','--block-size=KB']Mixed combinations of prefices and separators in one commandfromcmdyimportjava# Note this is just an example for old verion picard.# Picard is changing it's stylepicard=java(jar='picard.jar',_prefix='',_sep='=',_sub=True)c=picard.SortSam(I='input.bam',O='sorted.bam',SORTED_ORDER='coordinate',_prefix='',_sep='=',_deform=None).hprint(c.cmd)# same as the abovejava({'jar':'picard.jar','_prefix':'-','_sep':' '},'SortSam',I='input.bam',O='sorted.bam',SORTED_ORDER='coordinate',_prefix='',_sep='=',_deform=None).h().cmd# _deform prevents SORTED_ORDER to be deformed to SORTED-ORDER# ['java', 'jar=picard.jar',#  'SortSam', 'I=input.bam', 'O=sorted.bam', 'SORTED_ORDER=coordinate']Subcommandsfromcmdyimportgitgit.branch(v=True).fg# <CmdyResult: ['git', 'branch', '-v']># What if I have separate arguments for main and sub-command?git(git_dir='.',_sub=True).branch(v=True).h# <CmdyHolding: ['git', '--git-dir', '.', 'branch', '-v']>Duplicated keys for list arguments:fromcmdyimportsortprint(sort(k=['1,1','2,2'],t='_',_='./.editorconfig',_dupkey=True))# sort -k 1,1 -k 2,2 ./.editorconfigReturn code and exceptionfromcmdyimportxx()Traceback(mostrecentcalllast):File"<ipython-input-16-092cc5b72e61>",line2,in<module>x()/path/.../to/cmdy/__init__.py", line 146, in __call__ready_cfgargs,ready_popenargs,_will())/path/.../to/cmdy/__init__.py", line 201, in __new__result=holding.run()/path/.../to/cmdy/__init__.py", line 854, in runreturnorig_run(self,wait)/path/.../to/cmdy/__init__.py", line 717, in runreturnorig_run(self,wait)/path/.../to/cmdy/__init__.py", line 327, in runret=CmdyResult(self._run(),self)/path/.../to/cmdy/__init__.py", line 271, in _runraiseCmdyExecNotFoundError(str(fnfe))fromNonecmdy.cmdy_util.CmdyExecNotFoundError:[Errno2]Nosuchfileordirectory:'x':'x'fromcmdyimportlsls('non-existing-file')Traceback(mostrecentcalllast):File"<ipython-input-17-132683fc2227>",line2,in<module>ls('non-existing-file')/path/.../to/cmdy/__init__.py", line 146, in __call__ready_cfgargs,ready_popenargs,_will())/path/.../to/cmdy/__init__.py", line 204, in __new__returnresult.wait()/path/.../to/cmdy/__init__.py", line 407, in waitraiseCmdyReturnCodeError(self)cmdy.cmdy_util.CmdyReturnCodeError:UnexpectedRETURNCODE2,expecting:[0][PID]167164[CMD]['ls non-existing-file'][STDOUT][STDERR]ls:cannotaccessnon-existing-file:NosuchfileordirectoryDon't raise exception but store the return codefromcmdyimportlsresult=ls('non-existing-file',_raise=False)result.rc# 2Tolerance on return codefromcmdyimportls# or _okcode=[0,2]ls('non-existing-file',_okcode='0,2').rc# 2Timeoutsfromcmdyimportsleepsleep(3,_timeout=1)Traceback(mostrecentcalllast):File"<ipython-input-20-47b0ec7af55f>",line2,in<module>sleep(3,_timeout=1)/path/.../to/cmdy/__init__.py", line 146, in __call__ready_cfgargs,ready_popenargs,_will())/path/.../to/cmdy/__init__.py", line 204, in __new__returnresult.wait()/path/.../to/cmdy/__init__.py", line 404, in wait)fromNonecmdy.cmdy_util.CmdyTimeoutError:Timeoutafter1seconds.Redirectionsfromcmdyimportcatcat('./pytest.ini').redirect>'/tmp/pytest.ini'print(cat('/tmp/pytest.ini'))Appending# r short for redirectcat('./pytest.ini').r>>'/tmp/pytest.ini'print(cat('/tmp/pytest.ini'))# content doubledRedirecting to a file handlerwithopen('/tmp/pytest.ini','w')asfcat('./pytest.ini').r>fprint(cat('/tmp/pytest.ini'))STDIN, STDOUT and/or STDERR redirectionsfromcmdyimportSTDIN,STDOUT,STDERR,DEVNULLc=cat().r(STDIN)<'/tmp/pytest.ini'print(c)# Mixedc=cat().r(STDIN,STDOUT)^'/tmp/pytest.ini'>DEVNULL# we can't fetch result from a redirected pipeprint(c.stdout)# Why not '<' for STDIN?# Because the priority of the operator is not in sequential order.# We can use < for STDIN, but we need to ensure it runs firstc=(cat().r(STDIN,STDOUT)<'/tmp/pytest.ini')>DEVNULLprint(c.stdout)# A simple rule for multiple redirections to always use ">" in the last place# Redirect stderr to stdoutfromcmdyimportbashc=bash(c="cat 1>&2").r(STDIN,STDERR)^'/tmp/pytest.ini'>STDOUTprint(c.stdout)# Redirect the worldc=bash(c="cat 1>&2").r(STDIN,STDOUT,STDERR)^'/tmp/pytest.ini'^DEVNULL>STDOUTprint(c.stdout)# Noneprint(c.stderr)# NonePipingsfromcmdyimportgrepc=ls().p|grep('README')print(c)# README.md# README.rst# p short for pipec=ls().p|grep('README').p|grep('md')print(c)# README.mdprint(c.piped_strcmds)# ['ls', 'grep README', 'grep md']fromcmdyimport_CMDY_EVENT# !!! Pipings should be consumed immediately!# !!! DO NOT do thisls().pls()# <- Will not run as expected# All commands will be locked as holding until pipings are consumed_CMDY_EVENT.clear()print(ls())# runs# See Advanced/Holdings if you want to hold a piping command for a whileRunning command in foregroundls().fgfromcmdyimporttailtail('/tmp/pytest.ini',f=True,_timeout=3).fg# This mimics the `tail -f` program# You will see the content comes out one after another# and then program hangs for 3sYou can also write anecho-likeprogram easily. See 'echo.py'Iterating on outputforlineinls().iter():print(line,end='')Iterating on stderrforlineinbash(c="cat /tmp/pytest.ini 1>&2").iter(STDERR):print(line,end='')Getting live output# Like we did for `tail -f` program# This time, we can do something with each output line# Let's use a thread to write content to a file# And we try to get the live contents using cmdyimporttimefromthreadingimportThreaddeflive_write(file,n):withopen(file,'w',buffering=1)asf:# Let's write something every half secondforiinrange(n):f.write(str(i)+'\n')time.sleep(.5)test_file='/tmp/tail-f.txt'Thread(target=live_write,args=(test_file,10)).start()fromcmdyimporttailtail_iter=tail(f=True,_=test_file).iter()forlineintail_iter:# Do whatever you want with the lineprint('We got:',line,end='')ifline.strip()=='8':break# make sure thread endstime.sleep(2)# What about timeout?# Of course you can use a timer to check inside the loop# You can also set a timeout for each fetch# Terminate after 10 queriesThread(target=live_write,args=(test_file,10)).start()fromcmdyimporttailtail_iter=tail(f=True,_=test_file).iter()foriinrange(10):print('We got:',tail_iter.next(timeout=1),end='')AdvancedBaking thecmdyobjectSometimes, you may want to run the same program a couple of times, with the same set of arguments or configurations, and you don't want to type those arguments every time, then you can bake the Cmdy object with that same arguments or configurations.For example, if you want to run ls as ls -l all the time:fromcmdyimportlsll=ls._(l=True)print(ll().h.cmd)# ['ls', '-l']print(ll(a=True).h.cmd)# ['ls', '-l', '-a']# I don't want the l flag for some commands occasionallyprint(ll(l=False).h.cmd)# ['ls']# Bake a baked commandlla=ll._(a=True)print(lla().h.cmd)# ['ls', '-l', '-a']# I know git is always gonna run with subcommandgit=git._(_sub=True)# don't bother to pass _sub=True every timeprint(git(git_dir='.').branch(v=True).h)# <CmdyHolding: ['git', '--git-dir', '.', 'branch', '-v']>print(git().status().h)# <CmdyHolding: ['git', 'status']># What if I have a subcommand call bake?fromcmdyimportgit,CmdyActionErrorprint(git.branch().h.cmd)# ['git', 'branch']print(type(git._()))# <class 'cmdy.Cmdy'># run the git with _subprint(git(_sub=True).bake().h.cmd)# ['git', 'bake']Baking the whole moduleimportcmdy# run version of the whole worldsh=cmdy(version=True)# anything under sh directly will be supposed to have subcommandfromshimportgit,gccprint(git().h)# <CmdyHolding: ['git', '--version']>print(gcc().h)# <CmdyHolding: ['gcc', '--version']>Note that module baking is deep copying, except the exception classes and some utils. This means, you would expect following behavior:importcmdyfromcmdyimportCmdyHolding,CmdyExecNotFoundErrorsh=cmdy()c=sh.echo().hprint(type(c))# <class 'cmdy.CmdyHolding'>print(isinstance(c,CmdyHolding))# Falseprint(isinstance(c,sh.CmdyHolding))# Truetry:sh.notexisting()exceptCmdyExecNotFoundError:# we can catch it, as CmdyExecNotFoundError is sh.CmdyExecNotFoundErrorprint('Catched!')Holding objectsYou may have noticed that we have a couple of examples above with a final call .h or .h(), which is holding the command from running.You can do that, too, if you have multiple operationsprint(ls().h)# <CmdyHolding: ['ls']># however, you cannot hold after some actionsls().r.h# CmdyActionError: Should be called in the first place: .h() or .hold()Once a command is on hold (by .h, .hold, .h() or .hold())You have to explictly call run() to set the command runningfromtimeimporttimetic=time()c=sleep(2).hprint(f'Time elapsed:{time()-tic:.3f}s')# Time elapsed: 0.022 s# not running even with fgc.fgprint(f'Time elapsed:{time()-tic:.3f}s')# Time elapsed: 0.034 sc.run()print(f'Time elapsed:{time()-tic:.3f}s')# Time elapsed: 2.043 sReuse of command# After you set a command running,# you can retrieve the holding object,# and reuse itfromcmdyimportlsc=ls().fg# nothing will be producedc.holding.reset().r>DEVNULLAsync modeimportcuriofromcmdyimportlsa=ls().a# async command is never blocking!asyncdefmain():asyncforlineina:print(line,end='')curio.run(main())ExtendingcmdyAll those actions for holding/result objects were implemented internally as plugins. You can right your own plugins, too.A plugin has to be defined as a class and then instantiated.There are 5 APIs for developing a plugin forcmdycmdy._plugin_factory.register: A decorator for the plugin classcmdy._plugin_factory.hold_then: A decorator to decorate methods in the plugin class, which define actions after a holding object. Arguments:alias: The alias of this action (e.g.r/redirforredirect)final: Whether this is a final action, meaning no other actions should be followedprop: Whether this action can be called as a propertyhold_right: Should I put right following action on hold? This is useful when we have connectors which then can set the command running. (e.g>for redirect and|for pipe)cmdy._plugin_factory.run_then: A decorator to decorate methods in the plugin class, which define actions after a sync result object. Arguments are similar ascmdy._plugin_factory.hold_thenexcept thatpropandhold_rightare not avaialbe.async_run_then.add_method: A decorator to decorate methods in the plugin class, which add methods to theCmdyHolding,CmdyResultorCmdyAsyncResultclass.clsis the only argument that specifies which class we are hacking.async_run_then.add_property: Property version ofcmdy_plugin_add_methodNotes on name conflicts:If we need to add the methods to multiple classes in the plugin with the same name, you can define a different name with extra underscore suffix(es).Notes on module baking:Always use the baked module to get those classesimport cmdy
cmdy2 = cmdy()

@cmdy2._plugin_factory.register
class MyPlugin:
    ...Plugin enable and disable only take effect within the same module. For example:importcmdysh=cmdy()# only affects cmdy not shsh._plugins.fg.disable()# to disable this plugin for sh as well:sh._plugins.fg.disable()# An example to define a pluginimportcmdy@cmdy._plugin_factory.registerclassMyPlugin:@cmdy._plugin_factory.add_method(cmdy.CmdyHolding)defsay_hello(self):return'Hello world!'@cmdy._plugin_factory.hold_then('hello')defhelloworld(self):print(self.say_hello())# keep chainingreturnselfmyplugin=MyPlugin()# command will never run,# because we didn't do self.run() in helloworld(self)ls().helloworld()# prints Hello world!# property calls enabled by defaultls().helloworld# prints Hello world!# we have aliasls().hello# prints Hello world!