Module Expr Str Import alias Import alias Import alias Import alias Import alias Import alias Import alias Assign NameStore Call AttributeLoad NameLoad attr Str ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam defaults body Expr Str Raise NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad NameLoad attr Num Return Call AttributeLoad Str attr Call NameLoad Lambda arguments args NameParam defaults Call AttributeLoad NameLoad attr Str Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad Call NameLoad NameLoad Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str Call AttributeLoad Str attr NameLoad Assign TupleStore NameStore NameStore TupleLoad Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword NameLoad keyword NameLoad keyword NameLoad TryFinally body With Call NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Assign NameStore CompareEq AttributeLoad NameLoad attr Num Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad If NameLoad body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr NameLoad If NameLoad body If BoolOpOr AttributeLoad NameLoad attr UnaryOpNot NameLoad body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr NameLoad If UnaryOpNot NameLoad body Raise Call NameLoad Call AttributeLoad Str attr AttributeLoad NameLoad attr NameLoad keyword NameLoad keyword NameLoad keyword NameLoad finalbody Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body If CompareIs NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr If CompareIsNot NameLoad NameLoad body Expr Call AttributeLoad NameLoad attr BinOpAdd Num NameLoad decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad NameLoad NameLoad NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore AttributeLoad NameLoad attr AugAssignAdd NameStore Call AttributeLoad Str attr Call AttributeLoad Str attr AttributeLoad NameLoad attr AugAssignAdd NameStore Call AttributeLoad Str attr BoolOpOr AttributeLoad NameLoad attr Str AugAssignAdd NameStore Call AttributeLoad Str attr BoolOpOr AttributeLoad NameLoad attr Str Assign NameStore NameLoad If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Str attr ListComp Call AttributeLoad Str attr ListLoad NameLoad Call AttributeLoad Str attr NameLoad comprehension TupleStore NameStore NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AugAssignAdd NameStore Call AttributeLoad Str attr BoolOpOr NameLoad Str Return NameLoad decorator_list decorator_list ClassDef bases NameLoad body Expr Str Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword Str Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad keyword Str FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad Call NameLoad NameLoad NameLoad attr If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Str attr ListLoad AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Dict Str NameLoad If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Str attr ListLoad Call AttributeLoad Str attr AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Dict Str Str NameLoad AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Str NameLoad Return NameLoad decorator_list decorator_list
N/A N/A 
Template tasks for running external programs as luigi tasks.

This module is primarily intended for when you need to call a single external
program or shell script, and it's enough to specify program arguments and
environment variables.

If you need to run multiple commands, chain them together or pipe output
from one command to the next, you're probably better off using something like
`plumbum`_, and wrapping plumbum commands in normal luigi
:py:class:`~luigi.task.Task` s.

.. _plumbum: https://plumbum.readthedocs.org/
 N/A logging N/A os N/A signal N/A subprocess N/A sys N/A tempfile N/A luigi N/A logger N/A N/A logging getLogger luigi-interface ExternalProgramTask N/A N/A luigi Task N/A N/A 
    Template task for running an external program in a subprocess

    The program is run using :py:class:`subprocess.Popen`, with ``args`` passed
    as a list, generated by :py:meth:`program_args` (where the first element should
    be the executable). See :py:class:`subprocess.Popen` for details.

    Your must override :py:meth:`program_args` to specify the arguments you want,
    and you can optionally override :py:meth:`program_environment` if you want to
    control the environment variables (see :py:class:`ExternalPythonProgramTask`
    for an example).
     program_args N/A N/A self N/A N/A N/A 
        Override this method to map your task parameters to the program arguments

        :return: list to pass as ``args`` to :py:class:`subprocess.Popen`
         N/A NotImplementedError N/A program_environment N/A N/A self N/A N/A N/A 
        Override this method to control environment variables for the program

        :return: dict mapping environment variable names to values
         N/A env N/A N/A N/A os environ copy N/A env N/A always_log_stderr N/A N/A self N/A N/A N/A 
        When True, stderr will be logged even if program execution succeeded

        Override to False to log stderr only when program execution fails.
         N/A True N/A property _clean_output_file N/A N/A self file_object N/A N/A N/A N/A N/A file_object seek 0 N/A N/A N/A join N/A map N/A N/A N/A s N/A N/A N/A s decode utf-8 N/A N/A file_object readlines N/A run N/A N/A self N/A N/A N/A args N/A list N/A map str N/A N/A self program_args N/A N/A N/A logger info Running command: %s N/A N/A join args N/A N/A tmp_stdout tmp_stderr N/A N/A N/A tempfile TemporaryFile N/A N/A tempfile TemporaryFile N/A env N/A N/A self program_environment N/A proc N/A N/A subprocess Popen args env env stdout tmp_stdout stderr tmp_stderr N/A N/A N/A N/A ExternalProgramRunContext proc N/A N/A N/A N/A proc wait N/A success N/A N/A proc returncode 0 N/A stdout N/A N/A self _clean_output_file tmp_stdout N/A stderr N/A N/A self _clean_output_file tmp_stderr N/A stdout N/A N/A N/A N/A logger info N/A N/A Program stdout:
{} format stdout N/A stderr N/A N/A N/A N/A self always_log_stderr N/A success N/A N/A N/A N/A logger info N/A N/A Program stderr:
{} format stderr N/A N/A success N/A N/A N/A ExternalProgramRunError N/A N/A Program failed with return code={}: format N/A proc returncode args env env stdout stdout stderr stderr N/A N/A N/A N/A tmp_stderr close N/A N/A N/A tmp_stdout close N/A N/A ExternalProgramRunContext N/A object N/A __init__ N/A N/A self proc N/A N/A N/A N/A self proc proc N/A __enter__ N/A N/A self N/A N/A N/A N/A self __old_signal N/A N/A signal getsignal N/A signal SIGTERM N/A N/A N/A signal signal N/A signal SIGTERM N/A self kill_job N/A self N/A __exit__ N/A N/A self exc_type exc_val exc_tb N/A N/A N/A N/A exc_type KeyboardInterrupt N/A N/A N/A N/A self kill_job N/A N/A N/A signal signal N/A signal SIGTERM N/A self __old_signal N/A kill_job N/A N/A self captured_signal stack_frame N/A None None N/A N/A N/A N/A N/A self proc kill N/A N/A captured_signal None N/A N/A N/A N/A sys exit N/A 128 captured_signal N/A N/A ExternalProgramRunError N/A RuntimeError N/A __init__ N/A N/A self message args env stdout stderr N/A None None None N/A N/A N/A N/A N/A super ExternalProgramRunError self __init__ message args env stdout stderr N/A N/A self message message N/A N/A self args args N/A N/A self env env N/A N/A self out stdout N/A N/A self err stderr N/A __str__ N/A N/A self N/A N/A N/A info N/A self message N/A info N/A N/A 
COMMAND: {} format N/A N/A join N/A self args N/A info N/A N/A 
STDOUT: {} format N/A N/A self out [empty] N/A info N/A N/A 
STDERR: {} format N/A N/A self err [empty] N/A env_string None N/A N/A self env N/A N/A env_string N/A N/A join N/A N/A N/A = join N/A k N/A N/A '{}' format v N/A N/A k v N/A N/A N/A self env items N/A info N/A N/A 
ENVIRONMENT: {} format N/A env_string [empty] N/A info N/A N/A ExternalPythonProgramTask N/A ExternalProgramTask N/A N/A 
    Template task for running an external Python program in a subprocess

    Simple extension of :py:class:`ExternalProgramTask`, adding two
    :py:class:`luigi.parameter.Parameter` s for setting a virtualenv and for
    extending the ``PYTHONPATH``.
     N/A virtualenv N/A N/A luigi Parameter default None positional False description path to the virtualenv directory to use. It should point to the directory containing the ``bin/activate`` file used for enabling the virtualenv. N/A extra_pythonpath N/A N/A luigi Parameter default None positional False description extend the search path for modules by prepending this value to the ``PYTHONPATH`` environment variable. program_environment N/A N/A self N/A N/A N/A env N/A N/A N/A super ExternalPythonProgramTask self program_environment N/A N/A self extra_pythonpath N/A N/A pythonpath N/A N/A : join N/A N/A self extra_pythonpath N/A N/A env get PYTHONPATH N/A N/A N/A env update N/A PYTHONPATH pythonpath N/A N/A self virtualenv N/A N/A path N/A N/A : join N/A N/A N/A {}/bin format N/A self virtualenv N/A N/A env get PATH N/A N/A N/A env update N/A PATH VIRTUAL_ENV path N/A self virtualenv N/A N/A N/A env pop PYTHONHOME None N/A env N/A N/A