Module Expr Str Assign NameStore ListLoad Str Str Str Str Str Str Str Str Str Import alias Import alias Import alias Import alias Import alias ImportFrom alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Assign NameStore TupleLoad Str Str Str Str ClassDef bases AttributeLoad NameLoad attr body Expr Str Assign NameStore Call AttributeLoad NameLoad attr FunctionDef arguments args NameParam defaults body Expr Str Return Str decorator_list NameLoad decorator_list ClassDef bases NameLoad body Expr Str Assign NameStore Str Assign NameStore Str Assign NameStore BinOpSub AttributeLoad NameLoad attr Num Assign NameStore AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Str Assign NameStore Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Dict Str AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr If NameLoad body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Str If CompareGt Call NameLoad NameLoad AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad NameLoad attr NameLoad If UnaryOpNot NameLoad body Assign AttributeStore NameLoad attr NameLoad orelse If CompareLt Call NameLoad NameLoad AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr BinOpAdd AttributeLoad NameLoad attr NameLoad orelse If CompareGt Call NameLoad NameLoad AttributeLoad NameLoad attr body Raise Call NameLoad BinOpMod Str TupleLoad AttributeLoad NameLoad attr Call NameLoad NameLoad orelse If UnaryOpNot NameLoad body Raise Call NameLoad Str Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr BinOpAdd AttributeLoad NameLoad attr Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults NameLoad body Expr Str Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareNotEq Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Raise Call NameLoad Str Return Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Assign NameStore NameLoad If Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign NameStore SubscriptLoad NameLoad Slice Call NameLoad AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr orelse If Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign NameStore SubscriptLoad NameLoad Slice Call NameLoad AttributeLoad NameLoad attr If NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad orelse Assign NameStore NameLoad Assign NameStore Dict For TupleStore NameStore NameStore Call AttributeLoad Call AttributeLoad NameLoad attr NameLoad attr body If CompareEq Call NameLoad NameLoad Num body Assign SubscriptStore NameLoad Index NameLoad SubscriptLoad NameLoad Index Num orelse Assign SubscriptStore NameLoad Index NameLoad NameLoad Return NameLoad decorator_list NameLoad decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Str If NameLoad body Assign AttributeStore NameLoad attr NameLoad orelse Assign AttributeStore NameLoad attr Dict decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return BinOpMod Str AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults Num body Expr Str Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Num Assign NameStore BinOpAdd NameLoad NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str For NameStore AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str For NameStore AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad UnaryOpUSub SubscriptLoad AttributeLoad NameLoad attr Index NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign AttributeStore NameLoad attr Dict decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Dict Str AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Str Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Expr Str Return AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list Assign NameStore Call NameLoad NameLoad FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return BoolOpAnd AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Dict Str Str Str Str AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index Str AttributeLoad NameLoad attr Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return BinOpMod Str TupleLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Return Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str Call AttributeLoad NameLoad attr Str decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body If UnaryOpNot Call NameLoad NameLoad AttributeLoad NameLoad attr body Return NameLoad Return CompareEq Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str Assign NameStore Str Assign NameStore Str Assign NameStore Str Assign NameStore Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam defaults Dict NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str If BoolOpAnd CompareIs AttributeLoad NameLoad attr NameLoad CompareIsNot AttributeLoad NameLoad attr NameLoad body Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr If UnaryOpNot Call NameLoad NameLoad NameLoad body Raise Call NameLoad BinOpMod Str AttributeLoad Call NameLoad NameLoad attr If UnaryOpNot Call NameLoad NameLoad AttributeLoad NameLoad attr body Raise Call NameLoad Str Assign AttributeStore NameLoad attr Call NameLoad NameLoad Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Return Dict Str Str Str Str Str AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str Call AttributeLoad NameLoad attr Str Call AttributeLoad NameLoad attr Str Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam defaults body Return Call NameLoad Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body If UnaryOpNot Call NameLoad NameLoad AttributeLoad NameLoad attr body Return NameLoad Return CompareEq Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore BinOpMod Str NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad If NameLoad body Return Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad Return NameLoad decorator_list NameLoad decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str Assign NameStore Str Assign NameStore Str Assign NameStore Str Assign NameStore Call NameLoad ListLoad NameLoad NameLoad NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword Call NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Str Assign NameStore Call AttributeLoad NameLoad attr keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Str Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad FunctionDef arguments args NameParam defaults body Expr Str Return Str decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Return Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr Call NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Return Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad NameLoad If NameLoad body Assign NameStore Call NameLoad Num BinOpDiv NameLoad Num Assign AttributeStore AttributeLoad NameLoad attr attr ListLoad For NameStore Call NameLoad NameLoad body If BoolOpOr CompareEq BinOpMod NameLoad NameLoad Num CompareEq NameLoad BinOpSub NameLoad Num body Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr NameLoad orelse Expr Call AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr Str Assign AttributeStore AttributeLoad NameLoad attr attr ListLoad Str Call NameLoad Call NameLoad NameLoad Assign AttributeStore AttributeLoad NameLoad attr attr Num Assign AttributeStore NameLoad attr Call NameLoad Num Call NameLoad Num BinOpMult NameLoad Num Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Num decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr decorator_list Assign NameStore Call NameLoad NameLoad FunctionDef arguments args NameParam NameParam defaults NameLoad AttributeLoad AttributeLoad NameLoad attr attr body Expr Str If UnaryOpNot NameLoad body Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call NameLoad keyword NameLoad keyword Call NameLoad Expr Call AttributeLoad NameLoad attr ListLoad Return NameLoad decorator_list NameLoad FunctionDef arguments args defaults body Expr Str Return Call AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body If UnaryOpNot Call NameLoad NameLoad AttributeLoad NameLoad attr body Return NameLoad Return CompareEq Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad Num NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Num AugAssignAdd AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Str If NameLoad body AugAssignAdd AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr orelse AugAssignAdd AttributeStore NameLoad attr Num decorator_list FunctionDef arguments args NameParam defaults body Expr Str Assign NameStore Dict Str Str Str Str Str Str Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr Call NameLoad AttributeLoad NameLoad attr Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad AttributeLoad NameLoad attr attr Call NameLoad AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index Str Call AttributeLoad AttributeLoad NameLoad attr attr Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr If NameLoad body Assign SubscriptStore NameLoad Index Str NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str keyword AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str keyword AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr NameLoad Assign NameStore NameLoad If Call AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str Str keyword AttributeLoad NameLoad attr Assert Call NameLoad NameLoad Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str If UnaryOpNot NameLoad body Assign NameStore AttributeLoad AttributeLoad NameLoad attr attr Return Call NameLoad SubscriptLoad AttributeLoad NameLoad attr Index Str NameLoad Call NameLoad Call AttributeLoad NameLoad attr Str Call NameLoad Call AttributeLoad NameLoad attr Str NameLoad NameLoad keyword NameLoad keyword Call NameLoad Call AttributeLoad NameLoad attr Str keyword NameLoad decorator_list NameLoad decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str Assign NameStore Str Assign NameStore Str Assign NameStore Str Assign NameStore Call NameLoad ListLoad NameLoad NameLoad NameLoad Assign NameStore Num Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword Call NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Num keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword Str Assign NameStore Call AttributeLoad NameLoad attr keyword Str FunctionDef arguments args NameParam defaults body Assign NameStore Dict Str Str Str Str AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index Str AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index Str AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index Str AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index Str AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index Str AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Assign SubscriptStore NameLoad Index Str AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr Assign NameStore Str For NameStore NameLoad body AugAssignAdd NameStore BinOpAdd BinOpAdd BinOpAdd NameLoad Str Call NameLoad SubscriptLoad NameLoad Index NameLoad Str AugAssignAdd NameStore Str Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str AugAssignAdd AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Call NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults NameLoad body Expr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad If NameLoad body AugAssignAdd AttributeStore NameLoad attr Num orelse AugAssignAdd AttributeStore NameLoad attr Num decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Num Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str For NameStore Call AttributeLoad Call AttributeLoad NameLoad attr attr body Expr Call NameLoad NameLoad AttributeLoad NameLoad attr Call NameLoad NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body If UnaryOpNot Call NameLoad NameLoad AttributeLoad NameLoad attr body Return NameLoad Return CompareEq Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Expr Str Return Call NameLoad SubscriptLoad Call AttributeLoad Call AttributeLoad Call AttributeLoad NameLoad attr attr attr Str Index Num decorator_list Assign NameStore Call NameLoad NameLoad FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad Call AttributeLoad NameLoad attr attr decorator_list Assign NameStore Call NameLoad NameLoad FunctionDef arguments args NameParam defaults body Expr Str Return Str decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Return BinOpMod Str TupleLoad NameLoad NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Return Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Return Call AttributeLoad NameLoad attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Return Call NameLoad Call AttributeLoad NameLoad attr NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Num While CompareLt NameLoad Call NameLoad NameLoad body FunctionDef arguments args NameParam defaults body Return Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Slice NameLoad BinOpAdd NameLoad AttributeLoad NameLoad attr decorator_list AttributeLoad NameLoad attr Assign NameStore Call NameLoad NameLoad For NameStore NameLoad body AugAssignAdd NameStore Num If CompareIsNot NameLoad NameLoad body Expr Yield NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str If CompareIs NameLoad NameLoad body Return ListLoad Assign NameStore ListLoad For NameStore Call NameLoad AttributeLoad AttributeLoad AttributeLoad NameLoad attr attr attr body Assign NameStore Call AttributeLoad NameLoad attr Call AttributeLoad Call AttributeLoad NameLoad attr attr NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list NameLoad FunctionDef arguments args NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign NameStore Call NameLoad keyword NameLoad keyword NameLoad Return NameLoad decorator_list NameLoad decorator_list ClassDef bases AttributeLoad NameLoad attr body Expr Str Assign NameStore Str Assign NameStore Call NameLoad ListLoad NameLoad Assign NameStore Str Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword NameLoad FunctionDef arguments args NameParam defaults body Expr Str Return Str decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults body Expr Str Return Call AttributeLoad AttributeLoad NameLoad attr attr Call AttributeLoad NameLoad attr BinOpMod Str TupleLoad NameLoad AttributeLoad NameLoad attr decorator_list NameLoad FunctionDef arguments args NameParam NameParam defaults kwarg body Expr Str Expr Call AttributeLoad Call NameLoad keyword BinOpMod Str TupleLoad NameLoad AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr attr NameLoad decorator_list NameLoad decorator_list ClassDef bases NameLoad body Expr Str Assign NameStore Num FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr BoolOpOr NameLoad NameLoad Assign AttributeStore NameLoad attr BoolOpOr NameLoad NameLoad Assign AttributeStore NameLoad attr BoolOpOr NameLoad AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Return Dict Str Str Str Str Str Str Str AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam defaults body Return Call NameLoad SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str SubscriptLoad NameLoad Index Str decorator_list NameLoad decorator_list
N/A N/A Model classes which are used to communicate between parts of implementation.

These model classes are describing mapreduce, its current state and
communication messages. They are either stored in the datastore or
serialized to/from json and passed around with other means.
 N/A __all__ N/A MapreduceState MapperSpec MapreduceControl MapreduceSpec ShardState CountersMap TransientShardState QuerySpec HugeTask N/A cgi N/A datetime N/A urllib N/A zlib N/A google google.appengine._internal.graphy.backends google_chart_api N/A simplejson google.appengine.api memcache google.appengine.api taskqueue google.appengine.datastore datastore_rpc google.appengine.ext db google.appengine.ext.mapreduce context google.appengine.ext.mapreduce hooks google.appengine.ext.mapreduce json_util google.appengine.ext.mapreduce util N/A _MAP_REDUCE_KINDS N/A _GAE_MR_MapreduceControl _GAE_MR_MapreduceState _GAE_MR_ShardState _GAE_MR_TaskPayload _HugeTaskPayload N/A N/A db Model N/A N/A Model object to store task payload. N/A payload N/A N/A db BlobProperty kind N/A N/A cls N/A N/A N/A Returns entity kind. N/A _GAE_MR_TaskPayload N/A classmethod N/A HugeTask N/A object N/A N/A HugeTask is a taskqueue.Task-like class that can store big payloads.

  Payloads are stored either in the task payload itself or in the datastore.
  Task handlers should inherit from base_handler.HugeTaskHandler class.
   N/A PAYLOAD_PARAM __payload N/A PAYLOAD_KEY_PARAM __payload_key N/A MAX_TASK_PAYLOAD N/A N/A taskqueue MAX_PUSH_TASK_SIZE_BYTES 1024 N/A MAX_DB_PAYLOAD N/A N/A datastore_rpc BaseConnection MAX_RPC_BYTES N/A PAYLOAD_VERSION_HEADER AE-MR-Payload-Version N/A PAYLOAD_VERSION 1 __init__ N/A N/A self url params name eta countdown parent headers N/A None None None None None N/A N/A Init.

    Args:
      url: task url in str.
      params: a dict from str to str.
      name: task name.
      eta: task eta.
      countdown: task countdown.
      parent: parent entity of huge task's payload.
      headers: a dict of headers for the task.

    Raises:
      ValueError: when payload is too big even for datastore, or parent is
    not specified when payload is stored in datastore.
     N/A N/A self url url N/A N/A self name name N/A N/A self eta eta N/A N/A self countdown countdown N/A N/A self _headers N/A Content-Type N/A self PAYLOAD_VERSION_HEADER application/octet-stream N/A self PAYLOAD_VERSION N/A headers N/A N/A N/A N/A N/A self _headers update headers N/A payload_str N/A N/A urllib urlencode params N/A compressed_payload N/A N/A N/A len payload_str N/A self MAX_TASK_PAYLOAD N/A N/A compressed_payload N/A N/A zlib compress payload_str N/A N/A compressed_payload N/A N/A N/A self _payload payload_str N/A N/A N/A N/A len compressed_payload N/A self MAX_TASK_PAYLOAD N/A N/A N/A self _payload N/A N/A self PAYLOAD_PARAM compressed_payload N/A N/A N/A N/A len compressed_payload N/A self MAX_DB_PAYLOAD N/A N/A N/A ValueError N/A Payload from %s to big to be stored in database: %s N/A N/A self name N/A len compressed_payload N/A N/A N/A parent N/A N/A N/A ValueError Huge tasks should specify parent entity. N/A payload_entity N/A _HugeTaskPayload payload compressed_payload parent parent N/A payload_key N/A N/A payload_entity put N/A N/A self _payload N/A N/A self PAYLOAD_KEY_PARAM N/A str payload_key N/A add N/A N/A self queue_name transactional N/A False N/A N/A Add task to the queue. N/A task N/A N/A self to_task N/A N/A N/A task add queue_name transactional N/A to_task N/A N/A self N/A N/A N/A Convert to a taskqueue task. N/A N/A N/A taskqueue Task url N/A self url payload N/A self _payload name N/A self name eta N/A self eta countdown N/A self countdown headers N/A self _headers N/A decode_payload N/A N/A cls request N/A N/A N/A Decode task payload.

    HugeTask controls its own payload entirely including urlencoding.
    It doesn't depend on any particular web framework.

    Args:
      request: a webapp Request instance.

    Returns:
      A dict of str to str. The same as the params argument to __init__.

    Raises:
      DeprecationWarning: When task payload constructed from an older
        incompatible version of mapreduce.
     N/A N/A N/A N/A N/A request headers get N/A cls PAYLOAD_VERSION_HEADER N/A cls PAYLOAD_VERSION N/A N/A N/A DeprecationWarning Task is generated by an older incompatible version of mapreduce. Please kill this job manually N/A N/A N/A cls _decode_payload N/A request body N/A classmethod _decode_payload N/A N/A cls body N/A N/A N/A compressed_payload_str None N/A N/A N/A body startswith N/A cls PAYLOAD_KEY_PARAM N/A N/A payload_key N/A body N/A N/A len N/A cls PAYLOAD_KEY_PARAM N/A payload_entity N/A N/A _HugeTaskPayload get payload_key N/A compressed_payload_str N/A payload_entity payload N/A N/A N/A N/A body startswith N/A cls PAYLOAD_PARAM N/A N/A compressed_payload_str N/A body N/A N/A len N/A cls PAYLOAD_PARAM N/A compressed_payload_str N/A N/A payload_str N/A N/A zlib decompress compressed_payload_str N/A N/A payload_str body N/A result N/A N/A N/A name value N/A N/A N/A N/A cgi parse_qs payload_str items N/A N/A N/A N/A len value 1 N/A N/A N/A result N/A name N/A value N/A 0 N/A N/A N/A result N/A name value N/A result N/A classmethod N/A CountersMap N/A N/A json_util JsonMixin N/A N/A Maintains map from counter name to counter value.

  The class is used to provide basic arithmetics of counter values (buil
  add/remove), increment individual values and store/load data from json.
   __init__ N/A N/A self initial_map N/A None N/A N/A Constructor.

    Args:
      initial_map: initial counter values map from counter name (string) to
        counter value (int).
     N/A initial_map N/A N/A N/A self counters initial_map N/A N/A N/A self counters N/A N/A __repr__ N/A N/A self N/A N/A N/A Compute string representation. N/A N/A mapreduce.model.CountersMap(%r) N/A self counters N/A get N/A N/A self counter_name default N/A 0 N/A N/A Get current counter value.

    Args:
      counter_name: counter name as string.
      default: default value if one doesn't exist.

    Returns:
      current counter value as int. 0 if counter was not set.
     N/A N/A N/A N/A self counters get counter_name default N/A increment N/A N/A self counter_name delta N/A N/A N/A Increment counter value.

    Args:
      counter_name: counter name as String.
      delta: increment delta as Integer.

    Returns:
      new counter value.
     N/A current_value N/A N/A N/A self counters get counter_name 0 N/A new_value N/A current_value delta N/A N/A N/A self counters N/A counter_name new_value N/A new_value N/A add_map N/A N/A self counters_map N/A N/A N/A Add all counters from the map.

    For each counter in the passed map, adds its value to the counter in this
    map.

    Args:
      counters_map: CounterMap instance to add.
     N/A counter_name N/A counters_map counters N/A N/A N/A N/A self increment counter_name N/A N/A counters_map counters N/A counter_name N/A sub_map N/A N/A self counters_map N/A N/A N/A Subtracts all counters from the map.

    For each counter in the passed map, subtracts its value to the counter in
    this map.

    Args:
      counters_map: CounterMap instance to subtract.
     N/A counter_name N/A counters_map counters N/A N/A N/A N/A self increment counter_name N/A N/A N/A counters_map counters N/A counter_name N/A clear N/A N/A self N/A N/A N/A Clear all values. N/A N/A self counters N/A N/A to_json N/A N/A self N/A N/A N/A Serializes all the data in this map into json form.

    Returns:
      json-compatible data representation.
     N/A N/A counters N/A self counters N/A from_json N/A N/A cls json N/A N/A N/A Create new CountersMap from the json data structure, encoded by to_json.

    Args:
      json: json representation of CountersMap .

    Returns:
      an instance of CountersMap with all data deserialized from json.
     N/A counters_map N/A cls N/A N/A counters_map counters N/A json N/A counters N/A counters_map N/A classmethod to_dict N/A N/A self N/A N/A N/A Convert to dictionary.

    Returns:
      a dictionary with counter name as key and counter values as value.
     N/A N/A self counters N/A N/A MapperSpec N/A N/A json_util JsonMixin N/A N/A Contains a specification for the mapper phase of the mapreduce.

  MapperSpec instance can be changed only during mapreduce starting process,
  and it remains immutable for the rest of mapreduce execution. MapperSpec is
  passed as a payload to all mapreduce tasks in JSON encoding as part of
  MapreduceSpec.

  Specifying mapper handlers:
    * '<module_name>.<class_name>' - __call__ method of class instance will be
      called
    * '<module_name>.<function_name>' - function will be called.
    * '<module_name>.<class_name>.<method_name>' - class will be instantiated
      and method called.
   __init__ N/A N/A self handler_spec input_reader_spec params shard_count output_writer_spec N/A None N/A N/A Creates a new MapperSpec.

    Args:
      handler_spec: handler specification as string (see class doc for
        details).
      input_reader_spec: The class name of the input reader to use.
      params: Dictionary of additional parameters for the mapper.
      shard_count: number of shards to process in parallel.

    Properties:
      handler_spec: name of handler class/function to use.
      input_reader_spec: The class name of the input reader to use.
      params: Dictionary of additional parameters for the mapper.
      shard_count: number of shards to process in parallel.
      output_writer_spec: The class name of the output writer to use.
     N/A N/A self handler_spec handler_spec N/A N/A self input_reader_spec input_reader_spec N/A N/A self output_writer_spec output_writer_spec N/A N/A self shard_count N/A int shard_count N/A N/A self params params N/A get_handler N/A N/A self N/A N/A N/A Get mapper handler instance.

    This always creates a new instance of the handler. If the handler is a
    callable instance, MR only wants to create a new instance at the
    beginning of a shard or shard retry. The pickled callable instance
    should be accessed from TransientShardState.

    Returns:
      handler instance as callable.
     N/A N/A N/A util handler_for_name N/A self handler_spec N/A N/A handler N/A property get_handler input_reader_class N/A N/A self N/A N/A N/A Get input reader class.

    Returns:
      input reader class object.
     N/A N/A N/A util for_name N/A self input_reader_spec N/A output_writer_class N/A N/A self N/A N/A N/A Get output writer class.

    Returns:
      output writer class object.
     N/A N/A N/A self output_writer_spec N/A N/A util for_name N/A self output_writer_spec N/A to_json N/A N/A self N/A N/A N/A Serializes this MapperSpec into a json-izable object. N/A result N/A mapper_handler_spec mapper_input_reader mapper_params mapper_shard_count N/A self handler_spec N/A self input_reader_spec N/A self params N/A self shard_count N/A N/A self output_writer_spec N/A N/A N/A result N/A mapper_output_writer N/A self output_writer_spec N/A result N/A __str__ N/A N/A self N/A N/A N/A N/A MapperSpec(%s, %s, %s, %s) N/A N/A self handler_spec N/A self input_reader_spec N/A self params N/A self shard_count N/A from_json N/A N/A cls json N/A N/A N/A Creates MapperSpec from a dict-like object. N/A N/A cls N/A json N/A mapper_handler_spec N/A json N/A mapper_input_reader N/A json N/A mapper_params N/A json N/A mapper_shard_count N/A N/A json get mapper_output_writer N/A classmethod __eq__ N/A N/A self other N/A N/A N/A N/A N/A isinstance other N/A self __class__ N/A N/A False N/A N/A N/A N/A self to_json N/A N/A other to_json N/A N/A MapreduceSpec N/A N/A json_util JsonMixin N/A N/A Contains a specification for the whole mapreduce.

  MapreduceSpec instance can be changed only during mapreduce starting process,
  and it remains immutable for the rest of mapreduce execution. MapreduceSpec is
  passed as a payload to all mapreduce tasks in json encoding.
   N/A PARAM_DONE_CALLBACK done_callback N/A PARAM_DONE_CALLBACK_QUEUE done_callback_queue N/A PARAM_DONE_CALLBACK_TARGET done_callback_target N/A PARAM_MAPREDUCE_TARGET mapreduce_target __init__ N/A N/A self name mapreduce_id mapper_spec params hooks_class_name N/A N/A None N/A N/A Create new MapreduceSpec.

    Args:
      name: The name of this mapreduce job type.
      mapreduce_id: ID of the mapreduce.
      mapper_spec: JSON-encoded string containing a MapperSpec.
      params: dictionary of additional mapreduce parameters.
      hooks_class_name: The fully qualified name of the hooks class to use.

    Properties:
      name: The name of this mapreduce job type.
      mapreduce_id: unique id of this mapreduce as string.
      mapper: This MapreduceSpec's instance of MapperSpec.
      params: dictionary of additional mapreduce parameters.
      hooks_class_name: The fully qualified name of the hooks class to use.
     N/A N/A self name name N/A N/A self mapreduce_id mapreduce_id N/A N/A self mapper N/A N/A MapperSpec from_json mapper_spec N/A N/A self params params N/A N/A self hooks_class_name hooks_class_name N/A N/A self __hooks None N/A N/A N/A self get_hooks N/A get_hooks N/A N/A self N/A N/A N/A Returns a hooks.Hooks class or None if no hooks class has been set. N/A N/A N/A N/A self __hooks None N/A N/A self hooks_class_name None N/A N/A hooks_class N/A N/A util for_name N/A self hooks_class_name N/A N/A N/A isinstance hooks_class type N/A N/A N/A ValueError N/A hooks_class_name must refer to a class, got %s N/A N/A type hooks_class __name__ N/A N/A N/A issubclass hooks_class N/A hooks Hooks N/A N/A N/A ValueError hooks_class_name must refer to a hooks.Hooks subclass N/A N/A self __hooks N/A hooks_class self N/A N/A self __hooks N/A to_json N/A N/A self N/A N/A N/A Serializes all data in this mapreduce spec into json form.

    Returns:
      data in json format.
     N/A mapper_spec N/A N/A N/A self mapper to_json N/A N/A name mapreduce_id mapper_spec params hooks_class_name N/A self name N/A self mapreduce_id mapper_spec N/A self params N/A self hooks_class_name N/A from_json N/A N/A cls json N/A N/A N/A Create new MapreduceSpec from the json, encoded by to_json.

    Args:
      json: json representation of MapreduceSpec.

    Returns:
      an instance of MapreduceSpec with all data deserialized from json.
     N/A mapreduce_spec N/A cls N/A json N/A name N/A json N/A mapreduce_id N/A json N/A mapper_spec N/A N/A json get params N/A N/A json get hooks_class_name N/A mapreduce_spec N/A classmethod __str__ N/A N/A self N/A N/A N/A N/A str N/A N/A self to_json N/A __eq__ N/A N/A self other N/A N/A N/A N/A N/A isinstance other N/A self __class__ N/A N/A False N/A N/A N/A N/A self to_json N/A N/A other to_json N/A _get_mapreduce_spec N/A N/A cls mr_id N/A N/A N/A Get Mapreduce spec from mr id. N/A key N/A GAE-MR-spec: %s mr_id N/A spec_json N/A N/A memcache get key N/A spec_json N/A N/A N/A N/A cls from_json spec_json N/A state N/A N/A MapreduceState get_by_job_id mr_id N/A spec N/A state mapreduce_spec N/A spec_json N/A N/A spec to_json N/A N/A N/A memcache set key spec_json N/A spec N/A classmethod N/A MapreduceState N/A N/A db Model N/A N/A Holds accumulated state of mapreduce execution.

  MapreduceState is stored in datastore with a key name equal to the
  mapreduce ID. Only controller tasks can write to MapreduceState.

  Properties:
    mapreduce_spec: cached deserialized MapreduceSpec instance. read-only
    active: if this MR is still running.
    last_poll_time: last time controller job has polled this mapreduce.
    counters_map: shard's counters map as CountersMap. Mirrors
      counters_map_json.
    chart_url: last computed mapreduce status chart url. This chart displays the
      progress of all the shards the best way it can.
    sparkline_url: last computed mapreduce status chart url in small format.
    result_status: If not None, the final status of the job.
    active_shards: How many shards are still processing. This starts as 0,
      then set by KickOffJob handler to be the actual number of input
      readers after input splitting, and is updated by Controller task
      as shards finish.
    start_time: When the job started.
    writer_state: Json property to be used by writer to store its state.
      This is filled when single output per job. Will be deprecated.
      Use OutputWriter.get_filenames instead.
   N/A RESULT_SUCCESS success N/A RESULT_FAILED failed N/A RESULT_ABORTED aborted N/A _RESULTS N/A frozenset N/A RESULT_SUCCESS RESULT_FAILED RESULT_ABORTED N/A mapreduce_spec N/A N/A json_util JsonProperty MapreduceSpec indexed False N/A active N/A N/A db BooleanProperty default True indexed False N/A last_poll_time N/A N/A db DateTimeProperty required True N/A counters_map N/A N/A json_util JsonProperty CountersMap default N/A CountersMap indexed False N/A app_id N/A N/A db StringProperty required False indexed True N/A writer_state N/A N/A json_util JsonProperty dict indexed False N/A active_shards N/A N/A db IntegerProperty default 0 indexed False N/A failed_shards N/A N/A db IntegerProperty default 0 indexed False N/A aborted_shards N/A N/A db IntegerProperty default 0 indexed False N/A result_status N/A N/A db StringProperty required False choices _RESULTS N/A chart_url N/A N/A db TextProperty default N/A chart_width N/A N/A db IntegerProperty default 300 indexed False N/A sparkline_url N/A N/A db TextProperty default N/A start_time N/A N/A db DateTimeProperty auto_now_add True kind N/A N/A cls N/A N/A N/A Returns entity kind. N/A _GAE_MR_MapreduceState N/A classmethod get_key_by_job_id N/A N/A cls mapreduce_id N/A N/A N/A Retrieves the Key for a Job.

    Args:
      mapreduce_id: The job to retrieve.

    Returns:
      Datastore Key that can be used to fetch the MapreduceState.
     N/A N/A N/A N/A db Key from_path N/A N/A cls kind N/A str mapreduce_id N/A classmethod get_by_job_id N/A N/A cls mapreduce_id N/A N/A N/A Retrieves the instance of state for a Job.

    Args:
      mapreduce_id: The mapreduce job to retrieve.

    Returns:
      instance of MapreduceState for passed id.
     N/A N/A N/A db get N/A N/A cls get_key_by_job_id mapreduce_id N/A classmethod set_processed_counts N/A N/A self shards_processed N/A N/A N/A Updates a chart url to display processed count for each shard.

    Args:
      shards_processed: list of integers with number of processed entities in
        each shard
     N/A chart N/A N/A google_chart_api BarChart shards_processed N/A shard_count N/A len shards_processed N/A shards_processed N/A N/A stride_length N/A max 1 N/A shard_count 16 N/A N/A N/A chart bottom labels N/A N/A x N/A xrange shard_count N/A N/A N/A N/A N/A x stride_length 0 N/A x N/A shard_count 1 N/A N/A N/A N/A N/A N/A chart bottom labels append x N/A N/A N/A N/A N/A N/A chart bottom labels append N/A N/A N/A chart left labels N/A 0 N/A str N/A max shards_processed N/A N/A N/A chart left min 0 N/A N/A self chart_width N/A min 700 N/A max 300 N/A shard_count 20 N/A N/A self chart_url N/A N/A N/A chart display Url N/A self chart_width 200 N/A get_processed N/A N/A self N/A N/A N/A Number of processed entities.

    Returns:
      The total number of processed entities as int.
     N/A N/A N/A N/A self counters_map get N/A context COUNTER_MAPPER_CALLS N/A N/A processed N/A property get_processed create_new N/A N/A mapreduce_id gettime N/A None N/A N/A datetime datetime now N/A N/A Create a new MapreduceState.

    Args:
      mapreduce_id: Mapreduce id as string.
      gettime: Used for testing.
     N/A N/A mapreduce_id N/A N/A mapreduce_id N/A N/A MapreduceState new_mapreduce_id N/A state N/A MapreduceState key_name mapreduce_id last_poll_time N/A gettime N/A N/A N/A state set_processed_counts N/A N/A state N/A staticmethod new_mapreduce_id N/A N/A N/A N/A N/A Generate new mapreduce id. N/A N/A N/A util _get_descending_key N/A staticmethod __eq__ N/A N/A self other N/A N/A N/A N/A N/A isinstance other N/A self __class__ N/A N/A False N/A N/A N/A N/A self properties N/A N/A other properties N/A N/A TransientShardState N/A object N/A N/A A shard's states that are kept in task payload.

  TransientShardState holds two types of states:
  1. Some states just don't need to be saved to datastore. e.g.
     serialized input reader and output writer instances.
  2. Some states are duplicated from datastore, e.g. slice_id, shard_id.
     These are used to validate the task.
   __init__ N/A N/A self base_path mapreduce_spec shard_id slice_id input_reader initial_input_reader output_writer retries handler N/A None 0 None N/A N/A Init.

    Args:
      base_path: base path of this mapreduce job. Deprecated.
      mapreduce_spec: an instance of MapReduceSpec.
      shard_id: shard id.
      slice_id: slice id. When enqueuing task for the next slice, this number
        is incremented by 1.
      input_reader: input reader instance for this shard.
      initial_input_reader: the input reader instance before any iteration.
        Used by shard retry.
      output_writer: output writer instance for this shard, if exists.
      retries: the number of retries of the current shard. Used to drop
        tasks from old retries.
      handler: map/reduce handler.
     N/A N/A self base_path base_path N/A N/A self mapreduce_spec mapreduce_spec N/A N/A self shard_id shard_id N/A N/A self slice_id slice_id N/A N/A self input_reader input_reader N/A N/A self initial_input_reader initial_input_reader N/A N/A self output_writer output_writer N/A N/A self retries retries N/A N/A self handler handler N/A N/A self _input_reader_json N/A N/A N/A self input_reader to_json N/A reset_for_retry N/A N/A self output_writer N/A N/A N/A Reset self for shard retry.

    Args:
      output_writer: new output writer that contains new output files.
     N/A N/A self input_reader N/A self initial_input_reader N/A N/A self slice_id 0 N/A N/A self retries 1 N/A N/A self output_writer output_writer N/A N/A self handler N/A N/A N/A self mapreduce_spec mapper handler N/A advance_for_next_slice N/A N/A self recovery_slice N/A False N/A N/A Advance relavent states for next slice.

    Args:
      recovery_slice: True if this slice is running recovery logic.
        See handlers.MapperWorkerCallbackHandler._attempt_slice_recovery
        for more info.
     N/A recovery_slice N/A N/A N/A self slice_id 2 N/A N/A self input_reader N/A N/A N/A self input_reader from_json N/A self _input_reader_json N/A N/A N/A self slice_id 1 N/A to_dict N/A N/A self N/A N/A N/A Convert state to dictionary to save in task payload. N/A result N/A mapreduce_spec shard_id slice_id input_reader_state initial_input_reader_state retries N/A N/A N/A self mapreduce_spec to_json_str N/A self shard_id N/A str N/A self slice_id N/A N/A N/A self input_reader to_json_str N/A N/A N/A self initial_input_reader to_json_str N/A str N/A self retries N/A N/A self output_writer N/A N/A N/A result N/A output_writer_state N/A N/A N/A self output_writer to_json_str N/A serialized_handler N/A N/A util try_serialize_handler N/A self handler N/A serialized_handler N/A N/A N/A result N/A serialized_handler serialized_handler N/A result N/A from_request N/A N/A cls request N/A N/A N/A Create new TransientShardState from webapp request. N/A mapreduce_spec N/A N/A MapreduceSpec from_json_str N/A N/A request get mapreduce_spec N/A mapper_spec N/A mapreduce_spec mapper N/A input_reader_spec_dict N/A N/A simplejson loads N/A N/A request get input_reader_state cls N/A json_util JsonDecoder N/A input_reader N/A N/A N/A N/A mapper_spec input_reader_class from_json input_reader_spec_dict N/A initial_input_reader_spec_dict N/A N/A simplejson loads N/A N/A request get initial_input_reader_state cls N/A json_util JsonDecoder N/A initial_input_reader N/A N/A N/A N/A mapper_spec input_reader_class from_json initial_input_reader_spec_dict N/A output_writer None N/A N/A N/A mapper_spec output_writer_class N/A N/A output_writer N/A N/A N/A N/A mapper_spec output_writer_class from_json N/A N/A simplejson loads N/A N/A request get output_writer_state {} cls N/A json_util JsonDecoder N/A N/A isinstance output_writer N/A N/A mapper_spec output_writer_class N/A %s.from_json returned an instance of wrong class: %s N/A N/A N/A mapper_spec output_writer_class N/A output_writer __class__ N/A handler N/A N/A util try_deserialize_handler N/A N/A request get serialized_handler N/A N/A handler N/A N/A handler N/A N/A mapreduce_spec mapper handler N/A N/A cls N/A N/A mapreduce_spec params N/A base_path mapreduce_spec N/A str N/A N/A request get shard_id N/A int N/A N/A request get slice_id input_reader initial_input_reader output_writer output_writer retries N/A int N/A N/A request get retries handler handler N/A classmethod N/A ShardState N/A N/A db Model N/A N/A Single shard execution state.

  The shard state is stored in the datastore and is later aggregated by
  controller task. ShardState key_name is equal to shard_id.

  Shard state contains critical state to ensure the correctness of
  shard execution. It is the single source of truth about a shard's
  progress. For example:
  1. A slice is allowed to run only if its payload matches shard state's
     expectation.
  2. A slice is considered running only if it has acquired the shard's lock.
  3. A slice is considered done only if it has successfully committed shard
     state to db.

  Properties about the shard:
    active: if we have this shard still running as boolean.
    counters_map: shard's counters map as CountersMap. All counters yielded
      within mapreduce are stored here.
    mapreduce_id: unique id of the mapreduce.
    shard_id: unique id of this shard as string.
    shard_number: ordered number for this shard.
    retries: the number of times this shard has been retried.
    result_status: If not None, the final status of this shard.
    update_time: The last time this shard state was updated.
    shard_description: A string description of the work this shard will do.
    last_work_item: A string description of the last work item processed.
    writer_state: writer state for this shard. The shard's output writer
      instance can save in-memory output references to this field in its
      "finalize" method.

   Properties about slice management:
    slice_id: slice id of current executing slice. A slice's task
      will not run unless its slice_id matches this. Initial
      value is 0. By the end of slice execution, this number is
      incremented by 1.
    slice_start_time: a slice updates this to now at the beginning of
      execution. If the transaction succeeds, the current task holds
      a lease of slice duration + some grace period. During this time, no
      other task with the same slice_id will execute. Upon slice failure,
      the task should try to unset this value to allow retries to carry on
      ASAP.
    slice_request_id: the request id that holds/held the lease. When lease has
      expired, new request needs to verify that said request has indeed
      ended according to logs API. Do this only when lease has expired
      because logs API is expensive. This field should always be set/unset
      with slice_start_time. It is possible Logs API doesn't log a request
      at all or doesn't log the end of a request. So a new request can
      proceed after a long conservative timeout.
    slice_retries: the number of times a slice has been retried due to
      processing data when lock is held. Taskqueue/datastore errors
      related to slice/shard management are not counted. This count is
      only a lower bound and is used to determined when to fail a slice
      completely.
    acquired_once: whether the lock for this slice has been acquired at
      least once. When this is True, duplicates in outputs are possible.
   N/A RESULT_SUCCESS success N/A RESULT_FAILED failed N/A RESULT_ABORTED aborted N/A _RESULTS N/A frozenset N/A RESULT_SUCCESS RESULT_FAILED RESULT_ABORTED N/A _MAX_STATES_IN_MEMORY 10 N/A mapreduce_id N/A N/A db StringProperty required True N/A active N/A N/A db BooleanProperty default True indexed False N/A input_finished N/A N/A db BooleanProperty default False indexed False N/A counters_map N/A N/A json_util JsonProperty CountersMap default N/A CountersMap indexed False N/A result_status N/A N/A db StringProperty choices _RESULTS indexed False N/A retries N/A N/A db IntegerProperty default 0 indexed False N/A writer_state N/A N/A json_util JsonProperty dict indexed False N/A slice_id N/A N/A db IntegerProperty default 0 indexed False N/A slice_start_time N/A N/A db DateTimeProperty indexed False N/A slice_request_id N/A N/A db ByteStringProperty indexed False N/A slice_retries N/A N/A db IntegerProperty default 0 indexed False N/A acquired_once N/A N/A db BooleanProperty default False indexed False N/A update_time N/A N/A db DateTimeProperty auto_now True indexed False N/A shard_description N/A N/A db TextProperty default N/A last_work_item N/A N/A db TextProperty default __str__ N/A N/A self N/A N/A N/A kv N/A active slice_id last_work_item update_time N/A self active N/A self slice_id N/A self last_work_item N/A self update_time N/A N/A self result_status N/A N/A N/A kv N/A result_status N/A self result_status N/A N/A self retries N/A N/A N/A kv N/A retries N/A self retries N/A N/A self slice_start_time N/A N/A N/A kv N/A slice_start_time N/A self slice_start_time N/A N/A self slice_retries N/A N/A N/A kv N/A slice_retries N/A self slice_retries N/A N/A self slice_request_id N/A N/A N/A kv N/A slice_request_id N/A self slice_request_id N/A N/A self acquired_once N/A N/A N/A kv N/A acquired_once N/A self acquired_once N/A keys N/A N/A kv keys N/A N/A N/A keys sort N/A result ShardState is { N/A k keys N/A N/A result N/A N/A N/A k : N/A str N/A kv N/A k , N/A result } N/A result N/A reset_for_retry N/A N/A self N/A N/A N/A Reset self for shard retry. N/A N/A self retries 1 N/A N/A self last_work_item N/A N/A self active True N/A N/A self result_status None N/A N/A self input_finished False N/A N/A self counters_map N/A CountersMap N/A N/A self slice_id 0 N/A N/A self slice_start_time None N/A N/A self slice_request_id None N/A N/A self slice_retries 0 N/A N/A self acquired_once False N/A advance_for_next_slice N/A N/A self recovery_slice N/A False N/A N/A Advance self for next slice.

    Args:
      recovery_slice: True if this slice is running recovery logic.
        See handlers.MapperWorkerCallbackHandler._attempt_slice_recovery
        for more info.
     N/A N/A self slice_start_time None N/A N/A self slice_request_id None N/A N/A self slice_retries 0 N/A N/A self acquired_once False N/A recovery_slice N/A N/A N/A self slice_id 2 N/A N/A N/A self slice_id 1 N/A set_for_failure N/A N/A self N/A N/A N/A N/A self active False N/A N/A self result_status N/A self RESULT_FAILED N/A set_for_abort N/A N/A self N/A N/A N/A N/A self active False N/A N/A self result_status N/A self RESULT_ABORTED N/A set_input_finished N/A N/A self N/A N/A N/A N/A self input_finished True N/A is_input_finished N/A N/A self N/A N/A N/A N/A self input_finished N/A set_for_success N/A N/A self N/A N/A N/A N/A self active False N/A N/A self result_status N/A self RESULT_SUCCESS N/A N/A self slice_start_time None N/A N/A self slice_request_id None N/A N/A self slice_retries 0 N/A N/A self acquired_once False N/A copy_from N/A N/A self other_state N/A N/A N/A Copy data from another shard state entity to self. N/A prop N/A N/A N/A N/A self properties values N/A N/A N/A setattr self N/A prop name N/A getattr other_state N/A prop name N/A __eq__ N/A N/A self other N/A N/A N/A N/A N/A isinstance other N/A self __class__ N/A N/A False N/A N/A N/A N/A self properties N/A N/A other properties N/A get_shard_number N/A N/A self N/A N/A N/A Gets the shard number from the key name. N/A N/A int N/A N/A N/A N/A N/A N/A N/A self key name split - N/A -1 N/A N/A shard_number N/A property get_shard_number get_shard_id N/A N/A self N/A N/A N/A Returns the shard ID. N/A N/A N/A N/A N/A self key name N/A N/A shard_id N/A property get_shard_id kind N/A N/A cls N/A N/A N/A Returns entity kind. N/A _GAE_MR_ShardState N/A classmethod shard_id_from_number N/A N/A cls mapreduce_id shard_number N/A N/A N/A Get shard id by mapreduce id and shard number.

    Args:
      mapreduce_id: mapreduce id as string.
      shard_number: shard number to compute id for as int.

    Returns:
      shard id as string.
     N/A N/A %s-%d N/A mapreduce_id shard_number N/A classmethod get_key_by_shard_id N/A N/A cls shard_id N/A N/A N/A Retrieves the Key for this ShardState.

    Args:
      shard_id: The shard ID to fetch.

    Returns:
      The Datatore key to use to retrieve this ShardState.
     N/A N/A N/A N/A db Key from_path N/A N/A cls kind shard_id N/A classmethod get_by_shard_id N/A N/A cls shard_id N/A N/A N/A Get shard state from datastore by shard_id.

    Args:
      shard_id: shard id as string.

    Returns:
      ShardState for given shard id or None if it's not found.
     N/A N/A N/A cls get_by_key_name shard_id N/A classmethod find_by_mapreduce_state N/A N/A cls mapreduce_state N/A N/A N/A Find all shard states for given mapreduce.

    Deprecated. Use find_all_by_mapreduce_state.
    This will be removed after 1.8.9 release.

    Args:
      mapreduce_state: MapreduceState instance

    Returns:
      A list of ShardStates.
     N/A N/A list N/A N/A cls find_all_by_mapreduce_state mapreduce_state N/A classmethod find_all_by_mapreduce_state N/A N/A cls mapreduce_state N/A N/A N/A Find all shard states for given mapreduce.

    Args:
      mapreduce_state: MapreduceState instance

    Yields:
      shard states sorted by shard id.
     N/A keys N/A N/A cls calculate_keys_by_mapreduce_state mapreduce_state N/A i 0 N/A N/A i N/A len keys N/A no_tx_get N/A N/A i N/A N/A N/A N/A N/A db get N/A keys N/A i N/A i N/A cls _MAX_STATES_IN_MEMORY N/A N/A db non_transactional N/A states N/A no_tx_get i N/A s states N/A N/A i 1 N/A N/A s None N/A N/A N/A s N/A classmethod calculate_keys_by_mapreduce_state N/A N/A cls mapreduce_state N/A N/A N/A Calculate all shard states keys for given mapreduce.

    Args:
      mapreduce_state: MapreduceState instance

    Returns:
      A list of keys for shard states, sorted by shard id.
      The corresponding shard states may not exist.
     N/A N/A mapreduce_state None N/A N/A N/A N/A keys N/A N/A i N/A range N/A N/A N/A mapreduce_state mapreduce_spec mapper shard_count N/A N/A shard_id N/A N/A cls shard_id_from_number N/A N/A N/A N/A mapreduce_state key name i N/A N/A N/A keys append N/A N/A cls get_key_by_shard_id shard_id N/A keys N/A classmethod create_new N/A N/A cls mapreduce_id shard_number N/A N/A N/A Create new shard state.

    Args:
      mapreduce_id: unique mapreduce id as string.
      shard_number: shard number for which to create shard state.

    Returns:
      new instance of ShardState ready to put into datastore.
     N/A shard_id N/A N/A cls shard_id_from_number mapreduce_id shard_number N/A state N/A cls key_name shard_id mapreduce_id mapreduce_id N/A state N/A classmethod N/A MapreduceControl N/A N/A db Model N/A N/A Datastore entity used to control mapreduce job execution.

  Only one command may be sent to jobs at a time.

  Properties:
    command: The command to send to the job.
   N/A ABORT abort N/A _COMMANDS N/A frozenset N/A ABORT N/A _KEY_NAME command N/A command N/A N/A db TextProperty choices _COMMANDS required True kind N/A N/A cls N/A N/A N/A Returns entity kind. N/A _GAE_MR_MapreduceControl N/A classmethod get_key_by_job_id N/A N/A cls mapreduce_id N/A N/A N/A Retrieves the Key for a mapreduce ID.

    Args:
      mapreduce_id: The job to fetch.

    Returns:
      Datastore Key for the command for the given job ID.
     N/A N/A N/A N/A db Key from_path N/A N/A cls kind N/A %s:%s N/A mapreduce_id N/A cls _KEY_NAME N/A classmethod abort N/A N/A cls mapreduce_id N/A kwargs N/A N/A Causes a job to abort.

    Args:
      mapreduce_id: The job to abort. Not verified as a valid job.
     N/A N/A N/A N/A cls key_name N/A %s:%s N/A mapreduce_id N/A cls _KEY_NAME command N/A cls ABORT put kwargs N/A classmethod N/A QuerySpec N/A object N/A N/A Encapsulates everything about a query needed by DatastoreInputReader. N/A DEFAULT_BATCH_SIZE 50 __init__ N/A N/A self entity_kind keys_only filters batch_size model_class_path app ns N/A None None None None None None N/A N/A N/A self entity_kind entity_kind N/A N/A self keys_only N/A keys_only False N/A N/A self filters N/A filters None N/A N/A self batch_size N/A batch_size N/A self DEFAULT_BATCH_SIZE N/A N/A self model_class_path model_class_path N/A N/A self app app N/A N/A self ns ns N/A to_json N/A N/A self N/A N/A N/A N/A entity_kind keys_only filters batch_size model_class_path app ns N/A self entity_kind N/A self keys_only N/A self filters N/A self batch_size N/A self model_class_path N/A self app N/A self ns N/A from_json N/A N/A cls json N/A N/A N/A N/A cls N/A json N/A entity_kind N/A json N/A keys_only N/A json N/A filters N/A json N/A batch_size N/A json N/A model_class_path N/A json N/A app N/A json N/A ns N/A classmethod N/A