Module Expr Str ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ClassDef bases NameLoad body Expr Str Assign NameStore Str Assign NameStore AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore ListLoad NameLoad NameLoad NameLoad NameLoad FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr keyword Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Str decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Return Call NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr If AttributeLoad AttributeLoad NameLoad attr attr body Return AttributeLoad NameLoad attr orelse Return AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Expr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Expr Call AttributeLoad NameLoad attr Return Call AttributeLoad AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Num body Expr Str Expr Call AttributeLoad NameLoad attr Return Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assert CompareEq AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr For NameStore NameLoad body Expr Yield Call NameLoad Call AttributeLoad AttributeLoad NameLoad attr attr Str ListLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr Str Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr If NameLoad body Assign NameStore AttributeLoad NameLoad attr orelse Assign NameStore AttributeLoad NameLoad attr Assign NameStore NameLoad FunctionDef arguments args defaults body Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Return NameLoad decorator_list Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call NameLoad Return Call NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Str If AttributeLoad AttributeLoad NameLoad attr attr body Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If CompareIs NameLoad NameLoad body Raise Call NameLoad BinOpMod Str NameLoad Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If AttributeLoad NameLoad attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr keyword NameLoad Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign NameStore Dict Str AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr keyword BinOpAdd BinOpAdd AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr keyword Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr keyword NameLoad If AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad NameLoad attr NameLoad TryExcept body Expr Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr Return handlers ExceptHandler type NameLoad body Pass Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr keyword NameLoad decorator_list NameLoad decorator_list
N/A N/A User API for controlling Map job execution. google.appengine.api taskqueue google.appengine.datastore datastore_rpc google.appengine.ext db mapreduce model mapreduce util mapreduce.api.map_job map_job_config Job N/A object N/A N/A The job submitter's view of the job.

  The class allows user to submit a job, control a submitted job,
  query its state and result.
   N/A RUNNING running N/A FAILED N/A N/A model MapreduceState RESULT_FAILED N/A ABORTED N/A N/A model MapreduceState RESULT_ABORTED N/A SUCCESS N/A N/A model MapreduceState RESULT_SUCCESS N/A STATUS_ENUM N/A RUNNING FAILED ABORTED SUCCESS __init__ N/A N/A self state N/A None N/A N/A Init the job instance representing the job with id job_id.

    Do not directly call this method. Use class methods to construct
    new instances.

    Args:
      state: model.MapreduceState.
     N/A N/A self _state state N/A N/A self job_config N/A N/A N/A map_job_config JobConfig _to_map_job_config N/A state mapreduce_spec queue_name N/A N/A N/A N/A state mapreduce_spec params get queue_name N/A get_job_by_id N/A N/A cls job_id N/A None N/A N/A Gets the job instance representing the job with id job_id.

    Args:
      job_id: a job id, job_config.job_id, of a submitted job.

    Returns:
      A Job instance for job_id.
     N/A state N/A N/A cls __get_state_by_id job_id N/A N/A cls state N/A classmethod get_status N/A N/A self N/A N/A N/A Get status enum.

    Returns:
      One of the status enum.
     N/A N/A N/A self __update_state N/A N/A N/A self _state active N/A N/A N/A self RUNNING N/A N/A N/A N/A self _state result_status N/A abort N/A N/A self N/A N/A N/A Aborts the job. N/A N/A N/A N/A model MapreduceControl abort N/A N/A self job_config job_id N/A get_counters N/A N/A self N/A N/A N/A Get counters from this job.

    When a job is running, counter values won't be very accurate.

    Returns:
      An iterator that returns (counter_name, value) pairs of type
      (basestring, int)
     N/A N/A N/A self __update_state N/A N/A N/A N/A N/A N/A self _state counters_map counters iteritems N/A get_counter N/A N/A self counter_name default N/A 0 N/A N/A Get the value of the named counter from this job.

    When a job is running, counter values won't be very accurate.

    Args:
      counter_name: name of the counter in string.
      default: default value if the counter doesn't exist.

    Returns:
      Value in int of the named counter.
     N/A N/A N/A self __update_state N/A N/A N/A N/A N/A self _state counters_map get counter_name default N/A get_outputs N/A N/A self N/A N/A N/A Get outputs of this job.

    Should only call if status is SUCCESS.

    Yields:
      Iterators, one for each shard. Each iterator is
      from the argument of map_job.output_writer.commit_output.
     N/A N/A N/A self SUCCESS N/A N/A self get_status N/A ss N/A N/A N/A model ShardState find_all_by_mapreduce_state N/A self _state N/A s ss N/A N/A N/A N/A iter N/A N/A N/A s writer_state get outs N/A N/A submit N/A N/A cls job_config in_xg_transaction N/A False N/A N/A Submit the job to run.

    Args:
      job_config: an instance of map_job.MapJobConfig.
      in_xg_transaction: controls what transaction scope to use to start this MR
        job. If True, there has to be an already opened cross-group transaction
        scope. MR will use one entity group from it.
        If False, MR will create an independent transaction to start the job
        regardless of any existing transaction scopes.

    Returns:
      a Job instance representing the submitted job.
     N/A N/A N/A cls __validate_job_config job_config N/A mapper_spec N/A N/A job_config _get_mapper_spec N/A mapreduce_params N/A N/A job_config _get_mr_params N/A mapreduce_spec N/A N/A model MapreduceSpec N/A job_config job_name N/A job_config job_id N/A N/A mapper_spec to_json mapreduce_params N/A N/A util _obj_to_path N/A job_config _hooks_cls N/A in_xg_transaction N/A N/A propagation N/A db MANDATORY N/A N/A propagation N/A db INDEPENDENT N/A state None _txn N/A N/A N/A N/A N/A state N/A N/A cls __create_and_save_state job_config mapreduce_spec N/A N/A N/A cls __add_kickoff_task job_config mapreduce_spec N/A state N/A N/A N/A db transactional propagation propagation N/A state N/A _txn N/A N/A cls state N/A classmethod __update_state N/A N/A self N/A N/A N/A Fetches most up to date state from db. N/A N/A N/A self _state active N/A N/A N/A self _state N/A N/A self __get_state_by_id N/A N/A self job_config job_id N/A __get_state_by_id N/A N/A cls job_id N/A N/A N/A Get job state by id.

    Args:
      job_id: job id.

    Returns:
      model.MapreduceState for the job.

    Raises:
      ValueError: if the job state is missing.
     N/A state N/A N/A N/A model MapreduceState get_by_job_id job_id N/A N/A state None N/A N/A N/A ValueError N/A Job state for job %s is missing. job_id N/A state N/A classmethod __validate_job_config N/A N/A cls job_config N/A N/A N/A N/A N/A N/A job_config input_reader_cls validate job_config N/A N/A job_config output_writer_cls N/A N/A N/A N/A N/A job_config output_writer_cls validate N/A N/A job_config _get_mapper_spec N/A classmethod __create_and_save_state N/A N/A cls job_config mapreduce_spec N/A N/A N/A Save map job state to datastore.

    Save state to datastore so that UI can see it immediately.

    Args:
      job_config: map_job.JobConfig.
      mapreduce_spec: model.MapreduceSpec.

    Returns:
      model.MapreduceState for this job.
     N/A state N/A N/A N/A model MapreduceState create_new N/A job_config job_id N/A N/A state mapreduce_spec mapreduce_spec N/A N/A state active True N/A N/A state active_shards 0 N/A N/A state app_id N/A job_config _app N/A config N/A N/A datastore_rpc Configuration force_writes N/A job_config _force_writes N/A N/A N/A state put config config N/A state N/A classmethod __add_kickoff_task N/A N/A cls job_config mapreduce_spec N/A N/A N/A Add kickoff task to taskqueue.

    Args:
      job_config: map_job.JobConfig.
      mapreduce_spec: model.MapreduceSpec,
     N/A params N/A mapreduce_id N/A job_config job_id N/A kickoff_task N/A N/A taskqueue Task url N/A N/A N/A job_config _base_path /kickoffjob_callback/ N/A job_config job_id headers N/A N/A util _get_task_headers N/A job_config job_id params params N/A N/A job_config _hooks_cls N/A N/A hooks N/A N/A job_config _hooks_cls mapreduce_spec N/A N/A N/A N/A N/A hooks enqueue_kickoff_task kickoff_task N/A job_config queue_name N/A N/A N/A N/A NotImplementedError N/A N/A N/A N/A N/A kickoff_task add N/A job_config queue_name transactional True N/A classmethod N/A