Package: pyscriptlib
Pyscriptlib - Python Bash Script Helperspyscriptlibis a collection of helper functions that make it easy to invokebashshell commands from a python script.These helpers simplify the use of thesys,os, andsubprocessmodules for common cases.Fromsyswe get:arg(n:int) -> stra simple parse ofsys.argvthat always returns a stringalso considerargparseorclick-shellif you are building a complex CLInargs(n:int) -> List(str)retrieve the remaining command line args starting witharg(n)shift(n:int=1) -> List[str]shiftssys.argvbynpreservingsys.argv[0]returning removed argsexit(return_code:int=0)exeunt, pursued by a bear (WS)Fromoswe get:env(var:str) -> str|Noneeasy retrieval of environment variableskill(pid:int, signal:int=9)this subprocess is killing me, so kill or be killedFromsubprocesswe get several useful variations ofsh*(cmd:str, **kwargs)to invoke the/bin/bashshell for us (includingsubprocess.Popen(**kwargs)if necessary):sh(cmd:str, **kwargs)  -> (stdout+stderr).strip()output, output, gimme outputshl(cmd:str, **kwargs) -> list(rc, stdout, stderr)ok, ok, maybe we should take a look first:if rc == 0shk(cmd:str, **kwargs) -> list(is_ok, rc, stdout, stderr)let's test with a concise booleanis_okinstead ofrc == 0sho(cmd:str, **kwargs) -> CompletedProcess(is_ok, rc, stdout, stderr)cp = sho(cmd)is the full Montycustomized withcp.is_okandcp.rcshx(cmd:str, **kwargs) -> Nonedisplays live output to terminal just like you want it to in living colorshb(cmd:str, **kwargs) -> int(pid)runscmdin background, returning immediatelyusepidtokill(pid), or not, you daemon youclass Shellcan be used to select your preferred shellShell.executable = '/bin/bash'is the default bash shell used bysh*()functionsShell.shellis the property used to access the currentShell.executablevalueAnd, from nowhere in particular, we gethumanize(seconds:int, style='compact', days='days', zerodays=True) -> str()returns a human readable form of elapsed seconds such as/proc/uptimefull'05 days 03:59:27'or compact'05d 03h 59m 27s'joinlines(lines:list) -> strthis is the inverse ofstr.splitlines()returns list as a string concatenated with LF'sConstants:SP,TAB2,TAB = TAB4,LF,CRLFspaces and line feed constants useful in string formatsInstallationpyscriptlibis available atpypi.orgpipinstallpyscriptlibExample UsageSeepyscriptlib/example_script.pybelow#! /usr/bin/env python3'''Usage: python3 /path/to/site-packages/pyscriptlib/example_script.py ~/some/example/dir arg2 arg3'''frompathlibimportPathimporttimefrompyscriptlibimportarg,nargs,shift,env,sh,shx,shl,shk,sho,shb,kill,humanize,joinlines,TAB2,TAB4,LFdeftitle(descr,code):print(f'\n>>>{descr}\n>>>{code}')title('Retrieve first sys.argv or os.environ variable MY_DIR_PATH or None if not present','dir_path = arg(1) or env("MY_DIR_PATH")')dir_path=arg(1)orenv('MY_DIR_PATH')print(f'{dir_path= }')title('Get remaining command line args beginning at arg(1)','remaining_args = nargs(1)')remaining_args=nargs(2)print(f'{remaining_args= }')title('Shift args','removed = shift()); arg1 = arg(1)')removed=shift()arg1=arg(1)print(f'{removed= }{arg1= }')title('Open the dir_path and verify it is a directory','if Path(dir_path).expanduser().is_dir() else exit(2)')ifdir_path:my_dir=Path(dir_path).expanduser()ifnotmy_dir.is_dir():print(f'Exiting: cannot find{dir_path}')exit(2)else:print('usage: ./myscript dir_path')print('       -- or -- ')print('       MY_DIR_PATH=~/git/pyscriptlib; ./myscript')exit(1)print(f'{my_dir= }')title('Capture the stripped output from stdout+stderr','output = sh(cmd)')output=sh(f'ls -alh{my_dir}')print(output)title('Execute the command directly sending output to the terminal','shx(cmd)')shx(f'tree{my_dir}')title('Get a list of return values from subprocess.CompletedProcess object -- test with rc == 0','rc, stdout, stderr = shl(cmd); if rc == 0:')rc,stdout,stderr=shl(f'ls -alh{my_dir}')ifrc==0:print(f'{rc= }\n{stdout= }')else:print(f'{rc= }\n{stderr= }')title('Get a list of return values from subprocess.CompletedProcess object -- test with boolean is_ok','is_ok, rc, stdout, stderr = shk(cmd); if is_ok:')is_ok,rc,stdout,stderr=shk(f'ls -alh{my_dir}')ifis_ok:print(f'{is_ok= }{rc= }\n{stdout= }')else:print(f'{is_ok= }{rc= }\n{stderr= }')title('Get the customized subprocess.CompletedProcess object -- test with boolean cp.is_ok','cp = sho(cmd); if cp.is_ok:')cmd=f'''cd{my_dir}grep -r\--exclude='*.pyc'\--exclude-dir='.git' --exclude-dir='dist' --exclude-dir='*.egg-info'\pyscriptlib'''print('cmd = ',cmd)cp=sho(cmd)ifcp.is_ok:text=cp.stdout.splitlines()print(text)else:print(f'grep failed:{cp.rc= }{cp.stderr= }')exit(cp.rc)title('Create and kill background process with pid','pid = shb(cmd); kill(pid)')pid=shb(f'echo "hello from background process"; sleep 10')print(f'{pid= }')time.sleep(1)kill(pid)title('Humanize the uptime for this host','uptimes = sh("cat /proc/uptime"); humanize(uptimes[0]))')uptimes=sh('cat /proc/uptime')ifuptimes:uptime=float(uptimes.split(' ')[0])print(f'{humanize(uptime)= }')print(f'{humanize(uptime,style="full")= }')A Personal NoteI've been usingbashfor over 35 years (since the days of Bell Labs Unix where it was invented by Stephen Bourne), but I never really felt very confident with its (to me) arcane syntax.No, I don't mean just the things like this# I'm not always sure which condition form to use# and watch those spaces around the brackets!!!# not to mention tests like -x, -n, -z and $? == 0 is success?if<condition>|[<condition>]|[[<condition>]]then<statements>else<statements>fi# seriously, );; );; esac anyone?case<value>in<match>)<statements>;;<match>)<statements>;;*)<statements>;;esacBut mostly the magic stuff you can do with array notation that is so cryptic it makes my eyes water.# I'm still not sure what all this means :-(locations=("New York"ChicagoAtlantaMiami)forvalin${!locations[@]}doecho"index =${val}, value =${locations[$val]}"doneSigh, and I thought thatperlwas noisy ...So I started looking for a better shell and what I actually found waspython. It's a more powerful interpreter with a much more comfortable and much less cryptic syntax thanbash.Along the way, I also considered the pythonconchshell, but it wants to be a new hybrid language in a REPL, and I don't really want the overhead of another layer of shell even if it isipythonat its heart.In particular, all I really want to do is easily invoke thebash shellto run useful tools likegrep,ls -alh,tree,ip address,et alia, but otherwise I'm happy with python syntax as the medium in a script file.Unfortunately, the down-side to using python for scripting is that it is still a real programming language -- so creatingbash equivalentscripts can get a bit hairy, to say the least.This is especially true when you start usingsubprocess.Popen()and its incredibly powerful capabilities to fork any process you want with any options you want. But the price of power can be overwhealming complexity.And so, I finally realized that all I really need is a concise set of wrapper functions that I can use to embedbash commandsin my python code and let them actually do all the heavy lifting usingsubprocess.Popen()-- and why not throw in somesysandossugar as well?And, thus, complexity begatpyscriptliband here we are.A Testimonial to pathlib.Path -- Can I have an Amen?I am also highly impressed with thePathclass from the standardpathlibmodule. ThePathobject model helps my python scripts become much more concise as I'm often navigating the file system such asPath.home()/'subdir'before I launch somesh*(cmd)that accesses the files.Ooh, ooh, Mr. Kotter: this is not a typo, rather, it is the coolest use of a class dunder method to create aPath slash(/) join operatorthat I have seen:# using the Path slash(/) join operatorfile_path=Path.home()/'subdir'/'file'# is the same as using the Path.joinpath() methodfile_path=Path.home().joinpath('subdir','file')# and the Path instance has direct access methods to the file systemiffile_path.is_file():text=file_path.read_text()# bang, zoom, that's all folksThe old way of doing this withos.pathwas only slightly less painful thansubprocess.Popen. I leave that as a heartless exercise for the reader :-).