Module Import alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias alias ImportFrom alias alias alias Import alias identifier Import alias identifier Import alias identifier Import alias identifier Import alias identifier ImportFrom alias ImportFrom alias alias alias alias ImportFrom alias alias identifier alias identifier alias alias Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num FunctionDef arguments args NameParam NameParam defaults body If Call AttributeLoad NameLoad attr body Return Call NameLoad NameLoad orelse Return Call NameLoad NameLoad decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call NameLoad NameLoad NameLoad Return Call AttributeLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Return Call NameLoad AttributeLoad NameLoad attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults kwarg body Expr Str Assign NameStore Dict Str Str Str Str Str Str NameLoad TupleLoad AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr TupleLoad AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr TupleLoad AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr TupleLoad AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr TupleLoad AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr TupleLoad AttributeLoad AttributeLoad NameLoad attr attr NameLoad TupleLoad AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore Call AttributeLoad NameLoad attr Str Assign NameStore AttributeLoad Call SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index Num attr If NameLoad body AugAssignAdd NameStore AttributeLoad Call AttributeLoad AttributeLoad NameLoad attr attr SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index Num attr Return Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults kwarg body Assign NameStore BoolOpAnd UnaryOpNot Call AttributeLoad NameLoad attr UnaryOpNot AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Str Num Return SubscriptLoad ListComp Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad comprehension NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad BoolOpOr UnaryOpNot NameLoad AttributeLoad NameLoad attr CompareEq AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Slice NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam NameParam defaults NameLoad NameLoad body FunctionDef arguments args NameParam defaults body Expr Str If BoolOpAnd NameLoad CompareNotIn NameLoad AttributeLoad AttributeLoad NameLoad attr attr body Return NameLoad If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore NameLoad For NameStore ListLoad AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr body If CompareIn NameLoad Call AttributeLoad NameLoad attr body Assign NameStore NameLoad Break If UnaryOpNot NameLoad body Return NameLoad Return NameLoad decorator_list Return Call NameLoad NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults kwarg body Return Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str NameLoad BinOpMult NameLoad BinOpSub SubscriptLoad NameLoad Index Str Num decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad decorator_list NameLoad decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults kwarg body Assign NameStore Dict Str Str Str Str Str Str Str Str Assign NameStore Dict If SubscriptLoad NameLoad Index Str body Assign SubscriptStore NameLoad Index Str SubscriptLoad NameLoad Index Str If BoolOpAnd SubscriptLoad NameLoad Index Str CompareNotEq SubscriptLoad NameLoad Index Str Str body Assign SubscriptStore NameLoad Index Str SubscriptLoad NameLoad Index SubscriptLoad NameLoad Index Str If Call AttributeLoad NameLoad attr Str body Assign SubscriptStore NameLoad Index Str SubscriptLoad NameLoad Index Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If BoolOpAnd CompareEq Call NameLoad NameLoad NameLoad CompareIn Str NameLoad body Raise Call NameLoad NameLoad If SubscriptLoad NameLoad Index Str body Assign NameStore ListComp Call NameLoad NameLoad comprehension NameStore SubscriptLoad SubscriptLoad NameLoad Index Str Index Str orelse Return ListLoad If SubscriptLoad NameLoad Index Str body Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index Str attr Assign NameStore Call NameLoad Lambda arguments args NameParam defaults BoolOpOr CompareIn NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr CompareIn NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr CompareIn NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr CompareIn NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Return Call AttributeLoad NameLoad attr NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults kwarg body Assign NameStore BoolOpAnd UnaryOpNot Call AttributeLoad NameLoad attr UnaryOpNot AttributeLoad NameLoad attr Return Call NameLoad Lambda arguments args NameParam defaults BoolOpOr UnaryOpNot NameLoad AttributeLoad NameLoad attr CompareEq AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr Str Str Assign NameStore Call AttributeLoad NameLoad attr Str Str TryExcept body Assign NameStore SubscriptLoad Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Index Str If CompareIn SubscriptLoad NameLoad Index Str TupleLoad Str Str Str body If CompareEq SubscriptLoad NameLoad Index Str Str body Assign NameStore Call NameLoad NameLoad keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr orelse If CompareEq SubscriptLoad NameLoad Index Str Str body Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad orelse Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str orelse If CompareEq SubscriptLoad NameLoad Index Str Str body Raise Call NameLoad NameLoad NameLoad orelse If BoolOpAnd CompareIn SubscriptLoad NameLoad Index Str TupleLoad Str Str Str CompareEq SubscriptLoad NameLoad Index Str Str body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str orelse Assign NameStore Call NameLoad NameLoad AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad name NameStore body If CompareEq AttributeLoad NameLoad attr Num body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr SubscriptLoad NameLoad Index Str orelse Raise Call NameLoad NameLoad ExceptHandler type NameLoad name NameStore body Raise NameLoad ExceptHandler type NameLoad name NameStore body Raise Call NameLoad BinOpMod Str TupleLoad NameLoad NameLoad keyword NameLoad Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults kwarg body Expr Call AttributeLoad NameLoad attr Str Return Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Return Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr BinOpAdd Str NameLoad Return Call NameLoad Call AttributeLoad NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list decorator_list ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr BinOpMod Str NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list decorator_list
N/A N/A logging desktop.lib.exceptions_renderable PopupException desktop.lib.paginator Paginator desktop.lib.rest.http_client RestException hadoop cluster hadoop.cluster jt_ha rm_ha hadoop.api.jobtracker.ttypes ThriftJobPriority TaskTrackerNotFoundException ThriftJobState N/A hadoop.yarn.history_server_api history_server_api N/A hadoop.yarn.mapreduce_api mapreduce_api N/A hadoop.yarn.node_manager_api node_manager_api N/A hadoop.yarn.resource_manager_api resource_manager_api N/A hadoop.yarn.spark_history_server_api spark_history_server_api jobbrowser.conf SHARE_JOBS jobbrowser.models Job JobLinkage TaskList Tracker jobbrowser.yarn_models Application Job YarnJob KilledJob KilledYarnJob Container SparkJob N/A LOG N/A N/A logging getLogger __name__ N/A _DEFAULT_OBJ_PER_PAGINATION 10 get_api N/A N/A user jt N/A N/A N/A N/A N/A cluster is_yarn N/A N/A N/A YarnApi user N/A N/A N/A JtApi jt N/A JobBrowserApi N/A object N/A paginate_task N/A N/A self task_list pagenum N/A N/A N/A paginator N/A Paginator task_list _DEFAULT_OBJ_PER_PAGINATION N/A N/A N/A paginator page pagenum N/A N/A JtApi N/A JobBrowserApi N/A __init__ N/A N/A self jt N/A N/A N/A N/A self jt jt N/A get_job_link N/A N/A self jobid N/A N/A N/A N/A JobLinkage N/A self jt jobid N/A jt_ha get_job N/A N/A self jobid N/A N/A N/A N/A N/A Job from_id jt N/A self jt jobid jobid N/A jt_ha get_jobs N/A N/A self user N/A kwargs N/A N/A 
    Returns an array of jobs where the returned
    jobs are matched by the provided filter arguments.

    If a filter argument is in kwargs it will supersede the same argument
    in the request object.

    Filter arguments may be jobid, pools, user, tasks, text and state.

    Filter by user ownership if check_permission is set to true.
     N/A jobfunc N/A completed succeeded running failed killed all None N/A N/A N/A self jt completed_jobs N/A ThriftJobState SUCCEEDED N/A N/A N/A self jt completed_jobs N/A ThriftJobState SUCCEEDED N/A N/A N/A self jt running_jobs N/A ThriftJobState RUNNING N/A N/A N/A self jt failed_jobs N/A ThriftJobState FAILED N/A N/A N/A self jt killed_jobs N/A ThriftJobState KILLED N/A N/A N/A self jt all_jobs None N/A N/A N/A self jt all_jobs None N/A selection N/A N/A kwargs pop state N/A retired N/A N/A kwargs pop retired N/A jobs N/A N/A N/A N/A jobfunc N/A selection N/A 0 jobs N/A retired N/A N/A jobs N/A N/A N/A N/A self jt retired_jobs N/A N/A jobfunc N/A selection N/A 1 jobs N/A N/A N/A self filter_jobs user jobs kwargs N/A jt_ha filter_jobs N/A N/A self user jobs N/A kwargs N/A N/A check_permission N/A N/A N/A N/A SHARE_JOBS get N/A N/A user is_superuser N/A limit N/A N/A kwargs pop limit 10000 N/A N/A N/A N/A N/A Job from_thriftjob N/A self jt j N/A j N/A N/A self _filter_jobs jobs kwargs N/A N/A check_permission N/A user is_superuser N/A N/A N/A j profile user N/A user username N/A limit N/A jt_ha _filter_jobs N/A N/A self jobs username text N/A None None N/A predicate N/A N/A job N/A N/A N/A 
      Return True if a ThriftJobInProgress structure matches the supplied filters.

      If a filter argument is None, everything matches it.
       N/A N/A username N/A username N/A N/A job profile user N/A N/A False N/A text N/A N/A search N/A N/A text lower N/A saw_text False N/A t N/A N/A N/A job profile user N/A N/A job profile name N/A N/A job jobID asString N/A N/A job profile queueName N/A job priorityAsString N/A N/A N/A search N/A N/A t lower N/A N/A saw_text True N/A N/A N/A saw_text N/A N/A False N/A True N/A N/A N/A filter predicate jobs N/A get_tasks N/A N/A self jobid N/A filters N/A N/A N/A N/A TaskList select N/A self jt jobid N/A filters N/A task_types N/A filters N/A task_states N/A filters N/A task_text _DEFAULT_OBJ_PER_PAGINATION N/A _DEFAULT_OBJ_PER_PAGINATION N/A N/A filters N/A pagenum 1 N/A jt_ha get_tracker N/A N/A self trackerid N/A N/A N/A N/A N/A Tracker from_name N/A self jt trackerid N/A jt_ha N/A YarnApi N/A JobBrowserApi N/A N/A 
  List all the jobs with Resource Manager API.
  Get running single job information with MapReduce API.
  Get finished single job information with History Server API.

  The trick is that we use appid when the job is running and jobid when it is finished.
  We also suppose that each app id has only one MR job id.
  e.g. job_1355791146953_0105, application_1355791146953_0105

  A better alternative might be to call the Resource Manager instead of relying on the type of job id.
  The perfect solution would be to have all this logic embedded
   __init__ N/A N/A self user N/A N/A N/A N/A self user user N/A N/A self resource_manager_api N/A N/A resource_manager_api get_resource_manager N/A user username N/A N/A self mapreduce_api N/A N/A mapreduce_api get_mapreduce_api N/A user username N/A N/A self history_server_api N/A N/A history_server_api get_history_server_api N/A user username N/A N/A self spark_history_server_api N/A N/A spark_history_server_api get_history_server_api N/A get_job_link N/A N/A self job_id N/A N/A N/A N/A N/A self get_job job_id N/A get_jobs N/A N/A self user N/A kwargs N/A N/A state_filters N/A running completed failed killed UNDEFINED SUCCEEDED FAILED KILLED N/A filters N/A N/A N/A kwargs N/A username N/A N/A N/A filters N/A user N/A kwargs N/A username N/A N/A N/A kwargs N/A state N/A N/A kwargs N/A state all N/A N/A N/A filters N/A finalStatus N/A state_filters N/A N/A kwargs N/A state N/A N/A N/A kwargs get limit N/A N/A N/A filters N/A limit N/A kwargs N/A limit N/A json N/A N/A N/A self resource_manager_api apps filters N/A N/A N/A N/A type json str N/A This is standby RM json N/A N/A N/A Exception json N/A N/A json N/A apps N/A N/A jobs N/A N/A Application app N/A app N/A N/A json N/A apps N/A app N/A N/A N/A N/A N/A kwargs N/A text N/A N/A text N/A N/A N/A kwargs N/A text lower N/A jobs N/A filter N/A N/A N/A job N/A N/A N/A text N/A N/A N/A job name lower N/A text N/A N/A N/A job id lower N/A text N/A N/A N/A job user lower N/A text N/A N/A N/A job queue lower jobs N/A N/A N/A self filter_jobs user jobs N/A rm_ha filter_jobs N/A N/A self user jobs N/A kwargs N/A N/A check_permission N/A N/A N/A N/A SHARE_JOBS get N/A N/A user is_superuser N/A N/A filter N/A N/A N/A job N/A N/A N/A check_permission N/A user is_superuser N/A N/A job user N/A user username jobs N/A get_job N/A N/A self jobid N/A N/A N/A job_id N/A N/A jobid replace application job N/A app_id N/A N/A jobid replace job application N/A N/A N/A app N/A N/A N/A N/A self resource_manager_api app app_id N/A app N/A N/A N/A app N/A finalStatus N/A SUCCEEDED FAILED KILLED N/A N/A N/A N/A app N/A applicationType SPARK N/A N/A job N/A SparkJob app rm_api N/A self resource_manager_api hs_api N/A self spark_history_server_api N/A N/A N/A N/A app N/A state KILLED N/A N/A job N/A KilledYarnJob N/A self resource_manager_api app N/A N/A resp N/A N/A N/A self history_server_api job N/A self user job_id N/A job N/A YarnJob N/A self history_server_api N/A resp N/A job N/A N/A N/A N/A app N/A state ACCEPTED N/A N/A N/A ApplicationNotRunning app_id app N/A N/A N/A N/A N/A app N/A state N/A NEW SUBMITTED RUNNING N/A N/A app N/A applicationType MAPREDUCE N/A N/A resp N/A N/A N/A self mapreduce_api job N/A self user job_id N/A job N/A YarnJob N/A self mapreduce_api N/A resp N/A job N/A N/A job N/A Application app N/A self resource_manager_api N/A N/A N/A RestException N/A e N/A N/A N/A N/A e code 404 N/A N/A resp N/A N/A N/A self history_server_api job N/A self user job_id N/A job N/A YarnJob N/A self history_server_api N/A resp N/A job N/A N/A N/A JobExpired app_id N/A N/A ApplicationNotRunning N/A e N/A N/A e N/A N/A Exception N/A e N/A N/A N/A PopupException N/A Job %s could not be found: %s N/A jobid e detail e N/A job N/A rm_ha get_tasks N/A N/A self jobid N/A filters N/A N/A N/A N/A filters pop pagenum N/A N/A N/A N/A N/A self get_job jobid filter_tasks filters N/A get_task N/A N/A self jobid task_id N/A N/A N/A N/A N/A N/A N/A self get_job jobid task task_id N/A get_tracker N/A N/A self node_manager_http_address container_id N/A N/A N/A api N/A N/A node_manager_api get_node_manager_api N/A http:// node_manager_http_address N/A N/A Container N/A N/A api container container_id N/A N/A ApplicationNotRunning N/A Exception N/A __init__ N/A N/A self application_id job N/A N/A N/A N/A self application_id application_id N/A N/A self job job N/A N/A JobExpired N/A Exception N/A __init__ N/A N/A self job N/A N/A N/A N/A N/A N/A super JobExpired self __init__ N/A JobExpired: %s job N/A N/A self job job N/A N/A