Module Import alias Import alias Import alias Import alias Import alias Import alias Import alias Import alias Import alias Import alias Import alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias Assign AttributeStore AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr NameLoad ClassDef bases AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad FunctionDef arguments args NameParam NameParam defaults body If AttributeLoad NameLoad attr body Return Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr BinOpAdd Str NameLoad keyword BinOpRShift Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr orelse Return Call NameLoad NameLoad decorator_list decorator_list ClassDef bases AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr NameLoad body FunctionDef arguments args NameParam defaults body If AttributeLoad NameLoad attr body Return Call AttributeLoad NameLoad attr orelse Return Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr Str Expr Call AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam defaults body For NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr body Expr Call AttributeLoad NameLoad attr Str NameLoad Num Expr Yield TupleLoad NameLoad Num decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Yield TupleLoad NameLoad Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr Str decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr Num For NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Str body Assign TupleStore NameStore NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr AugAssignAdd AttributeStore NameLoad attr Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body For NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr body Expr Yield TupleLoad NameLoad BinOpDiv Num AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Yield TupleLoad NameLoad Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Yield TupleLoad NameLoad Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr Str decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Str Index Num Return ListLoad TupleLoad NameLoad Str decorator_list FunctionDef arguments args NameParam defaults body Expr Call NameLoad Str decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam defaults body For NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr body Expr Yield TupleLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr Str decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam defaults body Expr Yield TupleLoad Str Num Expr Yield TupleLoad Str Num decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Yield TupleLoad NameLoad Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr Str decorator_list decorator_list ClassDef bases NameLoad body Assign NameStore Str FunctionDef arguments args NameParam NameParam defaults body Expr Yield TupleLoad Str Dict Str Str decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Yield TupleLoad Str Call AttributeLoad NameLoad attr SubscriptLoad Call NameLoad NameLoad Index Num decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr Str decorator_list decorator_list ClassDef bases NameLoad body Pass decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam defaults body Raise Call NameLoad Str decorator_list FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr Str decorator_list decorator_list ClassDef bases AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr body Assign NameStore Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Dict For NameStore Call AttributeLoad NameLoad attr Str body Assign TupleStore NameStore NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Assign SubscriptStore NameLoad Index NameLoad NameLoad Return NameLoad decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr ListLoad NameLoad keyword NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call NameLoad SubscriptLoad NameLoad Index Str Num decorator_list NameLoad FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr ListLoad NameLoad keyword NameLoad Assign NameStore Call NameLoad Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Call NameLoad SubscriptLoad NameLoad Index Str BinOpDiv Num Num decorator_list NameLoad FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr ListLoad NameLoad keyword NameLoad Assign NameStore ListLoad For NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Str body Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num Str Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num Str decorator_list NameLoad FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr ListLoad NameLoad keyword NameLoad Assign NameStore ListLoad For NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Str body Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr Call NameLoad NameLoad Num Expr Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Num Str decorator_list NameLoad FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr ListLoad NameLoad keyword NameLoad Assign NameStore ListLoad For NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Str body Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad ListLoad Str decorator_list NameLoad FunctionDef arguments args NameParam defaults body Assign NameStore Call NameLoad keyword AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr ListLoad NameLoad keyword NameLoad Expr Call AttributeLoad NameLoad attr NameLoad decorator_list NameLoad decorator_list ClassDef bases AttributeLoad NameLoad attr body Assign NameStore NameLoad FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd ListLoad Str Str NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call NameLoad Str decorator_list FunctionDef arguments args NameParam defaults body ClassDef bases AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr body Assign NameStore Call AttributeLoad NameLoad attr FunctionDef arguments args NameParam defaults body If CompareNotIn Str Call AttributeLoad Str attr Call AttributeLoad NameLoad attr body Raise Call NameLoad Str decorator_list decorator_list Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr ListLoad Str Str Str Str Str Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr ListLoad Str Str Str Str Str decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str Assign NameStore NameLoad FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list decorator_list Call NameLoad Str ClassDef bases AttributeLoad NameLoad attr body FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr Str Str decorator_list FunctionDef arguments args NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr Str Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad NameLoad attr Str Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Str Str Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Str Str Expr Call AttributeLoad NameLoad attr Str Str Assert CompareEq AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad NameLoad Str Expr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad NameLoad Str Expr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad NameLoad Str Expr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad NameLoad Str Expr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad NameLoad Str Expr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad NameLoad Str Expr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list Call AttributeLoad NameLoad attr Str FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad Str NameLoad NameLoad Str Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad NameLoad Str Expr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list Call AttributeLoad NameLoad attr Str decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call AttributeLoad Str attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body If CompareEq Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Return AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad Num decorator_list FunctionDef arguments args NameParam defaults body If CompareIsNot Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad body Raise NameLoad decorator_list decorator_list ClassDef bases AttributeLoad NameLoad attr body FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr ListLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body With Call AttributeLoad NameLoad attr Str NameStore body Assign AttributeStore AttributeLoad NameLoad attr attr Call NameLoad NameLoad NameLoad Assign TupleStore NameStore NameStore Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad Str attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Str Assign NameStore ListLoad Str Str BinOpMod Str NameLoad Str Str Str Expr Call AttributeLoad NameLoad attr NameLoad Num Expr Call AttributeLoad NameLoad attr ListLoad NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Str Assign NameStore ListLoad BinOpMod Str NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Num Expr Call AttributeLoad NameLoad attr ListLoad NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore ListLoad Str Str Str Assign NameStore ListLoad Str BinOpMod Str SubscriptLoad NameLoad Index Num Str Str BinOpMod Str SubscriptLoad NameLoad Index Num Str Str BinOpMod Str SubscriptLoad NameLoad Index Num Str Expr Call AttributeLoad NameLoad attr NameLoad Num Expr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Str Assign NameStore ListLoad BinOpMod Str NameLoad With Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr body Expr Call AttributeLoad NameLoad attr NameLoad Num Expr Call AttributeLoad NameLoad attr ListLoad NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call NameLoad NameLoad With Call AttributeLoad NameLoad attr Str NameStore body Assign AttributeStore AttributeLoad NameLoad attr attr NameLoad With Call AttributeLoad NameLoad attr NameLoad body Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr ListLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Str Assign NameStore ListLoad BinOpMod Str NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Str Assign NameStore ListLoad Str BinOpMod Str NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Str Assign NameStore ListLoad BinOpMod Str NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str NameLoad decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Str Assign NameStore ListLoad Str Str BinOpMod Str NameLoad Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad Str Str Str NameLoad decorator_list decorator_list
N/A N/A os N/A sys N/A json N/A unittest N/A luigi N/A luigi.format N/A luigi.contrib.hadoop N/A luigi.contrib.hdfs N/A luigi.mrrunner N/A luigi.notifications N/A minicluster N/A mock luigi.mock MockTarget luigi.six StringIO nose.plugins.attrib attr N/A N/A N/A luigi notifications DEBUG True N/A N/A N/A N/A N/A luigi contrib hadoop attach minicluster OutputMixin N/A N/A luigi Task N/A N/A use_hdfs N/A N/A luigi BoolParameter default False get_output N/A N/A self fn N/A N/A N/A N/A self use_hdfs N/A N/A N/A N/A N/A N/A luigi contrib hdfs HdfsTarget N/A /tmp/ fn format N/A N/A N/A N/A luigi format get_default_format N/A N/A N/A luigi contrib hdfs PlainDir N/A N/A N/A MockTarget fn N/A N/A HadoopJobTask N/A N/A N/A N/A luigi contrib hadoop JobTask OutputMixin N/A job_runner N/A N/A self N/A N/A N/A N/A self use_hdfs N/A N/A N/A N/A minicluster MiniClusterHadoopJobRunner N/A N/A N/A N/A N/A N/A luigi contrib hadoop LocalJobRunner N/A N/A Words N/A OutputMixin N/A output N/A N/A self N/A N/A N/A N/A N/A self get_output words N/A run N/A N/A self N/A N/A N/A f N/A N/A N/A N/A self output open w N/A N/A N/A f write kj kj lkj lkj ljoi j iljlk jlk jlk jk jkl jlk jlkj j ioj ioj kuh kjh
 N/A N/A N/A f write kjsfsdfkj sdjkf kljslkj flskjdfj jkkd jjfk jk jk jk jk jk jklkjf kj lkj lkj
 N/A N/A N/A f close N/A N/A WordCountJob N/A HadoopJobTask N/A mapper N/A N/A self line N/A N/A N/A word N/A N/A N/A N/A line strip split N/A N/A N/A N/A self incr_counter word word 1 N/A N/A N/A word 1 N/A reducer N/A N/A self word occurences N/A N/A N/A N/A N/A word N/A sum occurences N/A requires N/A N/A self N/A N/A N/A N/A Words N/A self use_hdfs N/A output N/A N/A self N/A N/A N/A N/A N/A self get_output wordcount N/A N/A WordFreqJob N/A HadoopJobTask N/A init_local N/A N/A self N/A N/A N/A N/A self n 0 N/A line N/A N/A N/A N/A self input_local open r N/A N/A N/A word count N/A N/A N/A N/A line strip split N/A N/A self n N/A int count N/A mapper N/A N/A self line N/A N/A N/A word N/A N/A N/A N/A line strip split N/A N/A N/A N/A word N/A 1.0 N/A self n N/A combiner N/A N/A self word occurrences N/A N/A N/A N/A N/A word N/A sum occurrences N/A reducer N/A N/A self word occurences N/A N/A N/A N/A N/A word N/A sum occurences N/A requires_local N/A N/A self N/A N/A N/A N/A WordCountJob N/A self use_hdfs N/A requires_hadoop N/A N/A self N/A N/A N/A N/A Words N/A self use_hdfs N/A output N/A N/A self N/A N/A N/A N/A N/A self get_output luigitest-2 N/A extra_files N/A N/A self N/A N/A N/A fn N/A N/A N/A os listdir . N/A 0 N/A N/A N/A fn my_dir/my_file N/A init_remote N/A N/A self N/A N/A N/A N/A open my_dir/my_file N/A N/A MapOnlyJob N/A HadoopJobTask N/A mapper N/A N/A self line N/A N/A N/A word N/A N/A N/A N/A line strip split N/A N/A N/A N/A word N/A requires_hadoop N/A N/A self N/A N/A N/A N/A Words N/A self use_hdfs N/A output N/A N/A self N/A N/A N/A N/A N/A self get_output luigitest-3 N/A N/A UnicodeJob N/A HadoopJobTask N/A mapper N/A N/A self line N/A N/A N/A N/A N/A test 1 N/A N/A N/A test 1 N/A reducer N/A N/A self word occurences N/A N/A N/A N/A N/A word N/A sum occurences N/A requires N/A N/A self N/A N/A N/A N/A Words N/A self use_hdfs N/A output N/A N/A self N/A N/A N/A N/A N/A self get_output luigitest-4 N/A N/A UseJsonAsDataInteterchangeFormatJob N/A HadoopJobTask N/A N/A data_interchange_format json mapper N/A N/A self line N/A N/A N/A N/A N/A json N/A data type json N/A reducer N/A N/A self _ vals N/A N/A N/A N/A N/A N/A N/A json dumps N/A N/A list vals N/A 0 N/A requires N/A N/A self N/A N/A N/A  Two lines from Word.task will cause two `mapper` call.  N/A N/A Words N/A self use_hdfs N/A output N/A N/A self N/A N/A N/A N/A N/A self get_output luigitest-5 N/A N/A FailingJobException N/A Exception N/A N/A N/A FailingJob N/A HadoopJobTask N/A init_hadoop N/A N/A self N/A N/A N/A N/A FailingJobException failure N/A output N/A N/A self N/A N/A N/A N/A N/A self get_output failing N/A N/A MyStreamingJob N/A N/A N/A N/A luigi contrib hadoop JobTask N/A N/A param N/A N/A luigi Parameter N/A read_wordcount_output N/A N/A p N/A N/A N/A count N/A N/A line N/A N/A p open r N/A N/A N/A k v N/A N/A N/A N/A line strip split N/A N/A count N/A k v N/A count N/A CommonTests N/A object N/A test_run N/A N/A test_case N/A N/A N/A job N/A WordCountJob use_hdfs N/A test_case use_hdfs N/A N/A N/A luigi build N/A job local_scheduler True N/A c N/A read_wordcount_output N/A N/A job output N/A N/A N/A test_case assertEqual N/A int N/A c N/A jk 6 N/A staticmethod test_run_2 N/A N/A test_case N/A N/A N/A job N/A WordFreqJob use_hdfs N/A test_case use_hdfs N/A N/A N/A luigi build N/A job local_scheduler True N/A c N/A read_wordcount_output N/A N/A job output N/A N/A N/A test_case assertAlmostEquals N/A float N/A c N/A jk N/A 6.0 33.0 N/A staticmethod test_map_only N/A N/A test_case N/A N/A N/A job N/A MapOnlyJob use_hdfs N/A test_case use_hdfs N/A N/A N/A luigi build N/A job local_scheduler True N/A c N/A N/A line N/A N/A N/A N/A job output open r N/A N/A N/A N/A c append N/A N/A line strip N/A N/A N/A test_case assertEqual N/A c N/A 0 kj N/A N/A N/A test_case assertEqual N/A c N/A 4 ljoi N/A staticmethod test_unicode_job N/A N/A test_case N/A N/A N/A job N/A UnicodeJob use_hdfs N/A test_case use_hdfs N/A N/A N/A luigi build N/A job local_scheduler True N/A c N/A N/A line N/A N/A N/A N/A job output open r N/A N/A N/A N/A c append line N/A N/A N/A test_case assertEqual N/A len c 2 N/A N/A N/A test_case assertEqual N/A c N/A 0 test	2
 N/A staticmethod test_use_json_as_data_interchange_format_job N/A N/A test_case N/A N/A N/A job N/A UseJsonAsDataInteterchangeFormatJob use_hdfs N/A test_case use_hdfs N/A N/A N/A luigi build N/A job local_scheduler True N/A c N/A N/A line N/A N/A N/A N/A job output open r N/A N/A N/A N/A c append line N/A N/A N/A test_case assertEqual c N/A {"data type": "json"}
 N/A staticmethod test_failing_job N/A N/A test_case N/A N/A N/A job N/A FailingJob use_hdfs N/A test_case use_hdfs N/A success N/A N/A luigi build N/A job local_scheduler True N/A N/A N/A test_case assertFalse success N/A staticmethod N/A MapreduceLocalTest N/A N/A unittest TestCase N/A N/A use_hdfs False run_and_check N/A N/A self args N/A N/A N/A run_exit_status N/A N/A luigi run N/A N/A --local-scheduler --no-lock args N/A run_exit_status N/A test_run N/A N/A self N/A N/A N/A N/A N/A CommonTests test_run self N/A test_run_2 N/A N/A self N/A N/A N/A N/A N/A CommonTests test_run_2 self N/A test_map_only N/A N/A self N/A N/A N/A N/A N/A CommonTests test_map_only self N/A test_unicode_job N/A N/A self N/A N/A N/A N/A N/A CommonTests test_unicode_job self N/A test_use_json_as_data_interchange_format_job N/A N/A self N/A N/A N/A N/A N/A CommonTests test_use_json_as_data_interchange_format_job self N/A test_failing_job N/A N/A self N/A N/A N/A N/A N/A CommonTests test_failing_job self N/A test_instantiate_job N/A N/A self N/A N/A N/A N/A MyStreamingJob param_value N/A test_cmd_line N/A N/A self N/A N/A DummyHadoopTask N/A N/A N/A N/A luigi contrib hadoop JobTask N/A N/A param N/A N/A luigi Parameter run N/A N/A self N/A N/A N/A N/A mypool N/A N/A join N/A N/A self jobconfs N/A N/A N/A ValueError noooooo N/A N/A N/A N/A N/A self assertTrue N/A N/A self run_and_check N/A DummyHadoopTask --param myparam --pool mypool N/A N/A N/A self assertTrue N/A N/A self run_and_check N/A DummyHadoopTask --param myparam --hadoop-pool mypool N/A setUp N/A N/A self N/A N/A N/A N/A N/A N/A MockTarget fs clear N/A N/A MapreduceIntegrationTest N/A N/A minicluster MiniClusterTestCase N/A N/A  Uses the Minicluster functionality to test this against Hadoop  N/A use_hdfs True test_run N/A N/A self N/A N/A N/A N/A N/A CommonTests test_run self N/A test_run_2 N/A N/A self N/A N/A N/A N/A N/A CommonTests test_run_2 self N/A test_map_only N/A N/A self N/A N/A N/A N/A N/A CommonTests test_map_only self N/A test_failing_job N/A N/A self N/A N/A N/A N/A N/A CommonTests test_failing_job self N/A N/A N/A attr minicluster CreatePackagesArchive N/A N/A unittest TestCase N/A setUp N/A N/A self N/A N/A N/A N/A N/A N/A sys path append N/A N/A N/A os path join test create_packages_archive_root N/A tearDown N/A N/A self N/A N/A N/A N/A N/A N/A sys path remove N/A N/A N/A os path join test create_packages_archive_root N/A _assert_module N/A N/A self add N/A N/A N/A N/A N/A add assert_called_once_with test/create_packages_archive_root/module.py module.py N/A _assert_package N/A N/A self add N/A N/A N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/__init__.py package/__init__.py N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/submodule.py package/submodule.py N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/submodule_with_absolute_import.py package/submodule_with_absolute_import.py N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/submodule_without_imports.py package/submodule_without_imports.py N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/subpackage/__init__.py package/subpackage/__init__.py N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/subpackage/submodule.py package/subpackage/submodule.py N/A N/A N/A add assert_any_call test/create_packages_archive_root/package.egg-info/top_level.txt package.egg-info/top_level.txt N/A N/A N/A add call_count 7 N/A _assert_package_subpackage N/A N/A self add N/A N/A N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/__init__.py package/__init__.py N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/subpackage/__init__.py package/subpackage/__init__.py N/A N/A N/A add assert_any_call test/create_packages_archive_root/package/subpackage/submodule.py package/subpackage/submodule.py N/A N/A N/A add call_count 3 N/A test_create_packages_archive_module N/A N/A self tar N/A N/A N/A module N/A __import__ module None None dummy N/A N/A N/A N/A N/A luigi contrib hadoop create_packages_archive N/A module /dev/null N/A N/A N/A self _assert_module N/A N/A tar return_value add N/A N/A N/A mock patch tarfile.open test_create_packages_archive_package N/A N/A self tar N/A N/A N/A package N/A __import__ package None None dummy N/A N/A N/A N/A N/A luigi contrib hadoop create_packages_archive N/A package /dev/null N/A N/A N/A self _assert_package N/A N/A tar return_value add N/A N/A N/A mock patch tarfile.open test_create_packages_archive_package_submodule N/A N/A self tar N/A N/A N/A package_submodule N/A __import__ package.submodule None None dummy N/A N/A N/A N/A N/A luigi contrib hadoop create_packages_archive N/A package_submodule /dev/null N/A N/A N/A self _assert_package N/A N/A tar return_value add N/A N/A N/A mock patch tarfile.open test_create_packages_archive_package_submodule_with_absolute_import N/A N/A self tar N/A N/A N/A package_submodule_with_absolute_import N/A __import__ package.submodule_with_absolute_import None None dummy N/A N/A N/A N/A N/A luigi contrib hadoop create_packages_archive N/A package_submodule_with_absolute_import /dev/null N/A N/A N/A self _assert_package N/A N/A tar return_value add N/A N/A N/A mock patch tarfile.open test_create_packages_archive_package_submodule_without_imports N/A N/A self tar N/A N/A N/A package_submodule_without_imports N/A __import__ package.submodule_without_imports None None dummy N/A N/A N/A N/A N/A luigi contrib hadoop create_packages_archive N/A package_submodule_without_imports /dev/null N/A N/A N/A self _assert_package N/A N/A tar return_value add N/A N/A N/A mock patch tarfile.open test_create_packages_archive_package_subpackage N/A N/A self tar N/A N/A N/A package_subpackage N/A __import__ package.subpackage None None dummy N/A N/A N/A N/A N/A luigi contrib hadoop create_packages_archive N/A package_subpackage /dev/null N/A N/A N/A self _assert_package_subpackage N/A N/A tar return_value add N/A N/A N/A mock patch tarfile.open test_create_packages_archive_package_subpackage_submodule N/A N/A self tar N/A N/A N/A package_subpackage_submodule N/A __import__ package.subpackage.submodule None None dummy N/A N/A N/A N/A N/A luigi contrib hadoop create_packages_archive N/A package_subpackage_submodule /dev/null N/A N/A N/A self _assert_package_subpackage N/A N/A tar return_value add N/A N/A N/A mock patch tarfile.open N/A MockProcess N/A object N/A __init__ N/A N/A self err_lines returncode N/A N/A N/A err N/A N/A join err_lines N/A N/A self __err_len N/A len err N/A N/A self stderr N/A StringIO err N/A N/A self __rc returncode N/A N/A self returncode None N/A poll N/A N/A self N/A N/A N/A N/A N/A N/A N/A self stderr tell N/A self __err_len N/A N/A N/A self returncode N/A self __rc N/A N/A self returncode N/A N/A KeyboardInterruptedMockProcess N/A MockProcess N/A __init__ N/A N/A self err_lines N/A N/A N/A N/A N/A N/A super KeyboardInterruptedMockProcess self __init__ err_lines 0 N/A poll N/A N/A self N/A N/A N/A N/A N/A N/A N/A super KeyboardInterruptedMockProcess self poll None N/A N/A KeyboardInterrupt N/A N/A JobRunnerTest N/A N/A unittest TestCase N/A setUp N/A N/A self N/A N/A N/A N/A self tracking_urls N/A N/A track N/A N/A self url N/A N/A N/A N/A N/A N/A self tracking_urls append url N/A _run_and_track N/A N/A self err_lines returncode N/A N/A N/A N/A N/A mock patch luigi.contrib.hadoop.subprocess subprocess N/A N/A N/A N/A subprocess Popen return_value N/A MockProcess err_lines returncode N/A N/A _ err N/A N/A N/A N/A luigi contrib hadoop run_and_track_hadoop_job N/A N/A self track N/A N/A N/A self assertEqual err N/A N/A join err_lines N/A test_tracking_url_yarn N/A N/A self N/A N/A N/A url http://example.jobtracker.com:8080/proxy/application_1234_5678/ N/A yarn_lines N/A INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1234_5678
 INFO impl.YarnClientImpl: Submitted application application_1234_5678
 N/A INFO mapreduce.Job: The url to track the job: %s
 url INFO mapreduce.Job: Running job: job_1234_5678
 INFO mapreduce.Job: Job job_1234_5678 running in uber mode : false
 INFO mapreduce.Job: Job job_1234_5678 completed successfully
 N/A N/A N/A self _run_and_track yarn_lines 0 N/A N/A N/A self assertEqual N/A url N/A self tracking_urls N/A test_tracking_url_old_version N/A N/A self N/A N/A N/A url http://tracker.com/1234_5678 N/A err_lines N/A N/A INFO tracking url: %s
 url N/A N/A N/A self _run_and_track err_lines 0 N/A N/A N/A self assertEqual N/A url N/A self tracking_urls N/A test_multiple_tracking_urls N/A N/A self N/A N/A N/A urls N/A http://tracking/1 http://tracking/2 http://tracking/3 N/A err_lines N/A running...
 N/A The url to track the job: %s
 N/A urls N/A 0 done
 running another stage...
 N/A The url to track the job: %s
 N/A urls N/A 1 done
 running another stage...
 N/A The url to track the job: %s
 N/A urls N/A 2 done
 N/A N/A N/A self _run_and_track err_lines 0 N/A N/A N/A self assertEqual urls N/A self tracking_urls N/A test_tracking_url_captured_on_fail N/A N/A self N/A N/A N/A url http://tracking/ N/A err_lines N/A N/A The url to track the job: %s
 url N/A N/A N/A self assertRaises N/A N/A N/A luigi contrib hadoop HadoopJobError N/A N/A N/A N/A self _run_and_track err_lines 1 N/A N/A N/A self assertEqual N/A url N/A self tracking_urls N/A _run_and_track_with_interrupt N/A N/A self err_lines N/A N/A N/A proc N/A KeyboardInterruptedMockProcess err_lines N/A N/A N/A mock patch luigi.contrib.hadoop.subprocess subprocess N/A N/A N/A N/A subprocess Popen return_value proc N/A N/A N/A self assertRaises KeyboardInterrupt N/A N/A N/A N/A N/A N/A luigi contrib hadoop run_and_track_hadoop_job N/A proc N/A subprocess N/A test_kill_job_on_interrupt N/A N/A self N/A N/A N/A job_id job_1234_5678 N/A err_lines N/A N/A FlowStep: [SomeJob()] submitted hadoop job: %s
 job_id some other line
 N/A subprocess N/A N/A self _run_and_track_with_interrupt err_lines N/A N/A N/A N/A subprocess call assert_called_once_with N/A mapred job -kill job_id N/A test_kill_last_mapreduce_on_interrupt N/A N/A self N/A N/A N/A job_id job_1234_5678 N/A err_lines N/A FlowStep: [SomeJob()] submitted hadoop job: job_0000_0000
 N/A FlowStep: [SomeJob()] submitted hadoop job: %s
 job_id some other line
 N/A subprocess N/A N/A self _run_and_track_with_interrupt err_lines N/A N/A N/A N/A subprocess call assert_called_once_with N/A mapred job -kill job_id N/A test_kill_application_on_interrupt N/A N/A self N/A N/A N/A application_id application_1234_5678 N/A err_lines N/A N/A YarnClientImpl: Submitted application %s
 application_id FlowStep: [SomeJob()] submitted hadoop job: job_1234_5678
 N/A subprocess N/A N/A self _run_and_track_with_interrupt err_lines N/A N/A N/A N/A subprocess call assert_called_once_with N/A yarn application -kill application_id N/A test_kill_last_application_on_interrupt N/A N/A self N/A N/A N/A application_id application_1234_5678 N/A err_lines N/A YarnClientImpl: Submitted application application_0000_0000
 FlowStep: [SomeJob()] submitted hadoop job: job_0000_0000
 N/A YarnClientImpl: Submitted application %s
 application_id FlowStep: [SomeJob()] submitted hadoop job: job_1234_5678
 N/A subprocess N/A N/A self _run_and_track_with_interrupt err_lines N/A N/A N/A N/A subprocess call assert_called_once_with N/A yarn application -kill application_id N/A N/A