Package: shalchemy
ShalchemyConveniently call upon binaries from Python as if you were in sh.fromshalchemyimportsh,run,binfromshalchemy.binimportcat,curl,grepifcat('/etc/hosts')|grep('localhost'):sh.run(curl('example.com')>'file.txt')forlineincat('file.txt'):print(line)sh.run(bin.rm('file.txt'))Note that none of these are Python functions. We just call the system binaries usingsubprocessand do an unhealthy amount of magic to tie everything together.Installation$> pip install shalchemyTutorialYou create expressions by chainingshalchemy.shinstances together.importshalchemyps_aux=shalchemy.sh('ps','aux')grep=shalchemy.sh('grep','python')piped_expression=ps_aux|grepThese expressions on their own don’t immediately run the underlying executables. They are evaluated in these three circumstances:They are passed tosqlalchemy.sh.runThey are converted to abool,bytes,str, orintThey are iterated overDuring the evaluation phase, subprocesses are created, files are opened, and things are piped together with Linux magic. The Python process blocks until everything is finished. Once all the processes are done, things are cleaned up, and the correct data type is provided to the user.Methods of ExecutionThe different methods of evaluation do different things:Evaluating throughsh.runwill run the command and dump its output to stdout and stderr. The return value ofsh.runwill be the exit code of the command.Converting tobytes,str, orintwill run the command, capture its output, and parse it into the corresponding data type.Converting to aboolwill run the command, dump its output to stdout/stderr, then return True if the exit code was 0 and False otherwise.Iterating over the expression will run it, capture the standard output, parse it as a string, then split the string by newlines.If you want both the exit code and the stdout, you should pipe the stdout into anio.StringIOand usesh.run.Pipes and Redirectsshalchemy expressions support pipes|and redirects (<,>,>>) for stdout.Sadly, Python doesn’t support overloading the2>operation for stderr. But because we are crazy, we used>=instead!fromshalchemyimportshfromsqlalchemy.binimportrmsh.run(((rm('nonexistent_file')>'log.txt')>='&1')sh.run(((rm('nonexistent_file2')>>'log.txt')>='errors.txt')There are also issues with Python’s operator precedence and chaining. That is,1 < x < 3expands to1 < x and x < 3which is not very sh-friendly.If you’re going to do any sort of complex redirect chaining, it might be best to use thein_,out_anderr_methods.fromshalchemyimportshfromsqlalchemy.binimportrmsh.run(rm('nonexistent_file').in_('input.txt').out_('log.txt',append=True).err_('&1'))Argumentsshalchemy.shis used to create expressions. Calling it creates an internalCommandExpression. TheseCommandExpressionshold arguments and curry them. You can also access their attributes to naturally generate curried expressions for subcommands. As a result, these four different python lines will create the sameCommandExpression:fromshalchemyimportshfromshalchemy.binimportgitexpr1=sh('git','show','.')expr2=sh(['git','show','.'])expr3=git('show','.')expr4=git.show('.')expr5=sh('git show .')# SpecialThere is something special aboutexpr5that should be noted. Ifsh(or anyCommandExpression) receives a single string as the only argument, it will assume that you wanted to type a sh-compatible string and it’ll automatically tokenize it for you usingshlex.In other words,sh('gitshow .')will create the Commandsh(['git','show','.']). If you don’t like the automatic tokenization, you can explicitly provide a list with a single string inside likesh(['gitshow.']). Note that this second version will attempt to search your$PATHfor a binary named"git\ show\ ."which is almost always not what anybody wants. Just a small warning for this special automatic tokenization thing that might become a gotcha one day.shalchemy.binTheshalchemy.binmodule is a magic module that wraps whatever you want to import inshalchemy.shin a straightforward way. Importinggrepfromsqlalchemy.binwill just give you the result ofsh('grep')Multiple commandsshalchemy does not currently (and probably never will) support multiple commands chained with&&like sh does.Python IO Redirectsshalchemy supports redirects directly from standard Python io objects. That means this is fully supported:fromioimportStringIOfromshalchemyimportshfromshalchemy.binimportcatsh.run(cat<StringIO('my string'))Process SubstitutionsProcess substitution is a technique to make the output of a command
look like a file to the receiving process. One very common use of
this is when using the diff command. Suppose you wanted to diff the
file you have on disk with something on the internet. Normally, you
would do:curlexample.com/file.txt>tempfile.txtdifffile.txttempfile.txtrmtempfile.txtBut actually you can do:difffile.txt<(curlexample.com/file.txt)The<(command)syntax makes sh create a temporary file in /dev/fd/xxxx. This
is called Process Substitution.The way you do the same with shalchemy is:diff('file.txt',curl('example.com/file.txt').read_sub())Once an expression’sread_submethod is called, the result is a
ProcessSubstituteExpression which can no longer be composed with
other expressions. It can only be used as an argument directly to
other commands.fromioimportStringIOfromshalchemyimportshfromshalchemy.binimportcatsh.run(cat<StringIO('my string'))There is also awrite_subequivalent to sh’s>(expr).sh.run(cat('/usr/share/dict/words')|bin.tee((cat>'./words1.txt').write_sub(),(cat>'./words2.txt').write_sub(),)>'/dev/null')