Package: temporython
temporythonGenerate temporary Python scripts to quickly process lines of text or whole text files.Synpopsistemporythonis both a command line tool and a Python library. It creates boilerplate Python scripts to help you quickly solve text processing problems.If you want to build quick, one-off scripts in Python to process lines of text from files or standard input, this tool may speed up your productivity by setting you up with a boilerplate script that already has command line and input processing handled.
All you need to do is edit the code to process the lines of input, which may be as little as one line of code.Quick exampleLet's say your manager emails you two text files in a random format and you need to analyze/convert/transform/correct/import that data in some manner.
The files are namedbounced_email_logs_yesterday.logandbounced_email_logs_today.log.
She asks if you can get your processing work done quickly because an important customer is waiting.
If you find yourself writing custom Python scripts or writing in an interactive shell, thentemporythoncould help accelerate getting you set up. Let's see how that is done.You type the command...temporython lines process_bounced_email_logs.py...and the fileprocess_bounced_email_logs.pyis created. Here is what is inside of that file.#! /usr/bin/env python3# a quick script initially generated by `temporython` that:#   * reads in a list of filenames provided on the command line, or defaults to stdin.#   * processes each line# nice reference for Python 3 text processing: https://docs.python.org/3/library/text.htmlimportstringimporttextwrapimportrefromtemporythonimportmain### custom processing #########################################################classLineProcessor:def__init__(self):"""called before processing begins"""passdefprocess_line(self,filename,line_number,line):# NOTE: filename can be '-' if processing stdin.line=line.strip()print(line)defpost_process(self):"""called once after all lines in all files have been processed."""passif__name__=='__main__':main(LineProcessor)Nice! We have a pretty clear boilerplate script. Now let's customize it.You open up your new script in your favorite text editor and edit theprocess_line()function to your liking. You write code to parse the line and print out the digested results if certain conditions are met (as per your boss's email).Now let's process the files with the new script.$ ./process_bounced_email_logs.py bounced_email_logs_yesterday.log bounced_email_logs_today.logOr we can pipe the files in like this instead...$ cat bounced_email_logs_yesterday.log bounced_email_logs_today.log | ./process_bounced_email_logs.pyAnd now the data is processed and you manager is happy because you completed the task and you completed it quickly.
Yay!temporythonset up a great boilerplate to get you processing the log data quickly and allowed you to focus on writing your custom logic.Featurestemplate typestemporythongenerates three main kinds of text processing scripts:lines- boilerplate is set up so that you can process lines of text, and know which file and line number each line comes from. The lines can be piped in tostdinor filenames can be provided to the generated script via command line arguments.pipe- boilerplate is set up so that you can process lines of text that only come fromstdin.files- boilerplate is set up so that you can process contents of whole files. Filenames are provided as command line arguments.importing (default) vs. inliningBy default, the generated scripts will rely on thetemporythonlibrary to provide functionality.
This helps keep the generated scripts short.
But if you do not want your scripts to depend on thetemporythonlibrary, you can use the--inlineoption.Installationpip3installtemporythonRequirementsThis software requires Python 3.5 or above.UsageDisplay helpYou can display help withtemporython --helportemporython -h.$temporython--helpusage: temporython [-h] [-i] {files,lines,pipe} [FILENAME]positional arguments:{files,lines,pipe}FILENAME            Name of file to generateoptional arguments:-h, --help          show this help message and exit-i, --inline        Inline the temporython library in the generated code insteadof including it.Command line switches-i,--inline- Inline the required pieces of thetemporythonlibrary into the generated script instead of relying ontemporythonbeing available for import when the script is run. This option is useful if you want to create a script that has no external dependencies.Generate a script to process lines of textThis command generates a script called 'my_cool_script.py'$temporythonlinesmy_cool_script.pyNote: If you do not provide a name, then a default name ofprocess_lines.tmpwill be chosen.Now edit the code to your liking.LineProcessor.__init__()- add code in here that you wish to run only once when your script begins.LineProcessor.process_line(filename, line_number, line)- edit the code here to process each line as you wish.LineProcessor.post_process()- add code here that you wish to run once after all files & lines have been processed.You can process lines by running your new script like this...$./my_cool_script.pyfile1.txtfile2.textfile3.textOr you can process lines by piping them in like this...$catfile1.txtfile2.textfile3.text|./my_cool_script.pyGenerate a script to process text piped in viastdinThis command generates a script called 'my_cool_script.py'$temporythonpipemy_cool_script.pyNote: If you do not provide a name, then a default name ofprocess_pipe.tmpwill be chosen.Now edit the code to your liking.LineProcessor.__init__()- add code in here that you wish to run only once when your script begins.LineProcessor.process_line(filename, line_number, line)- edit the code here to process each line as you wish.LineProcessor.post_process()- add code here that you wish to run once after all files & lines have been processed.You can run your script by piping in content like this...$catfile1.txtfile2.textfile3.text|./my_cool_script.pyGenerate a script to process whole filesperformance warning when processing large files:-temporythongenerates a script that will load entire files into memory at one time. If you are processing really large files, your script may run really slowly or run out of memory.This command generates a script called 'my_cool_script.py'$temporythonfilesmy_cool_script.pyNote: If you do not provide a name, then a default name ofprocess_files.tmpwill be chosen.Now edit the code to your liking.FileProcessor.__init__()- add code in here that you wish to run only once when your script begins.FileProcessor.process_file(filename, contents)- edit the code here to process the contents of each file as you wish.FileProcessor.post_process()- add code here that you wish to run once after all files & lines have been processed.You can process lines by running your new script like this...$./my_cool_script.pyfile1.txtfile2.textfile3.textYou can also pipe data into your file processing script and it will process all input as on large file named-.$catfile1.txtfile2.textfile3.text|./my_cool_script.pyWhat if my temporary script isn't temporary any more?Perhaps after creating your script you find yourself reusing or maintaining what was supposed to be a one-off script.
This is not a problem.You could ensure your script is named well and then check it in to the code repo of the project it supports.
You could installtemporythonin your environment if your script imports the libary, or you do not need to install it if you used the--inlineoption.It is also possible that your script may outgrowtemporython.
You can refactor your script to remove the dependency ontemporython.
This may include writing your own command line argument parser using Python's built-inargparselibrary.AlternativesText processing problems can be solved in a variety of ways other than usingtemporython.Here is only a short list of possibilities...shell scripts - usinggrep,sed,awk, etc.spreadsheetsJupyter notebooksinteractive Python shellother text processing Python librariesother code generatorstext editor automationmanual editingEach of these have have advantages and disadvantages, but in the end it the choice of tool(s) comes down to your personal preference, your comfort level, and the constraints/requirements of the environment that you work in.If you are comfortable writing ad hoc code to slice and dice strings in Python,temporythonmay be a great tool to add to your toolbelt.
You can usetemporythoninstead of or or along with the alternatives listed above depending on the text processing problem you face.LicenseMIT