Package: lc-task
task-py: Python Task Execution FrameworkOverviewOne day you need to write a script to process some data, so you sit down and write it. The filepath and other parameters
are hard-coded, but it works. A few days later, you need to do the same thing, but need to accept a filepath on the
command line and maybe one more parameter, so you refactor some functionality into functions and create a simple CLI
usingsys.argv.  Then a coworker asks if he/she/they could use the script, so you finally break down and useargparseto write a better CLI.  If this cycle sounds familiar,task-pycan help by allowing you to skip steps one and two, while
writing interoperable, composable tools.InstallationInstall from PyPI (preferred method)pipinstalllc_taskInstall Directly with Pip and Gitpipinstallgit+https://github.com/libcommon/task-py.git@vx.x.x#egg=lc_taskwherex.x.xis the version you want to download.Install from Cloned Repogitclonessh://git@github.com/libcommon/task-py.git&&\cdtask-py&&\pipinstall.DependenciesThetool.poetry.*dependenciessections inpyproject.tomlcontain the app and dev dependencies.
SeeDEVELOPMENT.mdfor local development and build dependencies.Getting StartedCreating a TaskCreating aTaskis simple: specify some (optional) attributes and implementTask._perform_task, which performs the primary unit of work.
For example, you could implement a task that takes a file path and prints the data of that file to stdout:importtypingfrompathlibimportPathfromlc_taskimportTask,taskclass@taskclassclassCatFileTask(Task):"""Print contents of a file to stdout."""input_path:typing.Optional[Path]=Nonedef_perform_task(self)->None:ifself.input_pathisNone:raiseValueError("No input path provided")ifnotself.input_path.is_file():raiseFileNotFoundError(self.input_path)print(self.input_path.read_text())if__name__=="__main__":importsys# Example of how to use CatFileTask.run() to run the taskCatFileTask(input_path=Path(sys.argv[0])).run()In the example above, we define an attributeinput_pathonCatFileTaskto store the input file path.
For tasks that require some setup and teardown steps, use theTask._preambleandTask._postamblefunctions,
which get called before and after the primary unit of work is completed.
To run this task explicitly, use theCatFileTask.runmethod.Accepting Command Line InputTo create a task that takes command line input (usingargparse), create a child class ofCliTaskthat defines
the name of the command, a brief description, and the command line arguments (plus optional command aliases). Taking the example above:importtypingfrompathlibimportPathfromlc_taskimportCliTask,Task,taskclass@taskclassclassCatFileTask(Task):"""Print contents of a file to stdout."""input_path:typing.Optional[Path]=Nonedef_perform_task(self)->None:ifself.input_pathisNone:raiseValueError("No input path provided")ifnotself.input_path.is_file():raiseFileNotFoundError(self.input_path)print(self.input_path.read_text())@taskclassclassCatFileCliTask(CliTask):_task_cls=CatFileTaskcommand="cat"aliases=["c"]description="Print contents of a file to stdout."@classmethoddefgen_command_parser(cls,parser:typing.Optional[ArgumentParser]=None)->ArgumentParser:parser=super(CatFileCliTask,cls).gen_command_parser()parser.add_argument("input_path",type=Path,help="Path to input file")returnparserif__name__=="__main__":CatFileTask.run_command()This example shows a few key features of the library:The command line parser for a CLI task is defined using theCliTask.gen_command_parserfunction.Command aliases can be added using theCliTask.aiasesattribute.Tasks andCLiTasks are easily composable using the_task_clsattribute ofCliTask.
By default,CliTaskmerges the command line arguments with an instance of_task_clsand runs the task.
This behavior can be customized by overridingClitask._gen_taskandClitask._perform_task.For defining more complex, hierarchical command line interfaces with subcommands, look at the documentation forcli.gen_cli_parser.
It allows you to define your command line app with a mapping of commands to actions, then generates the CLI for you.Pipelines and Message PassingAnyone who has used Bash, PowerShell, or other scripting languages is familiar with the idea of composability and pipes:
writing simple commands that return structured data recognized by other commands can be very powerful.
Python does not support this style of programming out of the box, but it does support overloading operators for custom types,such as the bitwise or operator (|).
In Python, the__ror__builtin methodis called when using the bitwise or operator on two objects where the first (left side) doesn't implement__or__.task-pytakes advantage of this flexibility to allow pipingTasks together to create pipelines. For example, suppose
you were writing a CSV handing toolkit and wanted to create a command line app that reads data from a CSV and removes some columns.
There are two clear steps in this pipeline:Read data from the CSV into some data structureRemove specified columns (and write to stdout)importcsvimporttypingfrompathlibimportPathfromlc_taskimportCliTask,Task,TaskResult,taskclass@taskclassclassCsvColumnRemovalTask(Task):"""Remove specified columns from frows in a CSV."""__slots__=("columns","input_file","reader")columns_to_remove:typing.Optional[typing.List[str]]=Nonerecords:typing.Optional[typing.List[typing.Dict[str,typing.Any]]]=Nonedef_perform_task(self)->None:ifself.columnsandself.records:# Print the header rowprint(",".join(records[0].keys()))forrecordinrecords:forcolumninself.columns:delrecord[column]# Print the recordprint(", ".join(record.values()))@taskclassclassCsvReaderTaskResult(TaskResult):records:typing.Optional[typing.List[typing.Dict[str,typing.Any]]]=None@taskclassclassCsvReaderTask(Task):"""Read data from a CSV into a `csv.DictReader` object."""_result_cls=CsvReaderTaskResultinput_path:typing.Optional[Path]=Nonedef_perform_task(self)->None:ifself.input_pathisNone:raiseValueError("Must provide path to or file handle of CSV file")ifnotself.input_path.is_file():raiseFileNotFoundError(self.input_file)withself.input_path.open(newline="")asinput_file:self.result.records=[recordforrecordincsv.DictReader(input_file,csv.unix_dialect)]@taskclassclassRemoveColumnsCliTask(CliTask):command="remove-columns"description="Remove columns from a CSV and write to disk."@classmethoddefgen_command_parser(cls,parser:Optional[ArgumentParser]=None)->ArgumentParser:parser=super(RemoveColumnsCliTask,cls).gen_command_parser()parser.add_argument("input_path",type=Path,help="Path to input file")parser.add_argument("columns",nargs=+,help="Column names to remove")returnparserdef_perform_task(self)->None:columns_to_remove:typing.List[str]=self.args.columnsinput_path:Path=self.args.path=self.args.input_path# Pipe the output of CSVReaderTask to CSVColumnRemovalTask(CSVReaderTask(input_path=input_path).run()|CSVColumnRemovalTask(columns_to_remove=columns_to_remove))if__name__=="__main__":RemoveColumnsCLITask.run_command()DevelopmentSeeDEVELOPMENT.mdfor development instructions.Contributing/SuggestionsContributions and suggestions are welcome! To make a feature request, report a bug, or otherwise comment on existing functionality, please file an issue.
For contributions please submit a PR, but make sure to lint, type-check, and test your code before doing so. Thanks in advance!