Module Expr Str ImportFrom alias Import alias Import alias Import alias Import alias Import alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias ImportFrom alias Import alias identifier Import alias identifier ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam NameParam NameParam defaults Num body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AttributeLoad NameLoad attr decorator_list Call AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AttributeLoad NameLoad attr decorator_list Call AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AttributeLoad NameLoad attr decorator_list Call AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AttributeLoad NameLoad attr decorator_list Call AttributeLoad NameLoad attr FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad AttributeLoad NameLoad attr decorator_list Call AttributeLoad NameLoad attr decorator_list ClassDef bases NameLoad body Expr Str Assign NameStore BinOpLShift Num Num Assign NameStore Num Assign NameStore Num FunctionDef arguments args NameParam NameParam defaults body Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Str Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Str Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Str Assign AttributeStore NameLoad attr Call AttributeLoad NameLoad attr Num Assign NameStore SubscriptLoad NameLoad Index Str Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Str Expr Call AttributeLoad NameLoad attr Str NameLoad NameLoad Assign AttributeStore NameLoad attr Call NameLoad NameLoad NameLoad Assign AttributeStore NameLoad attr Dict Assign AttributeStore NameLoad attr Dict decorator_list FunctionDef arguments args NameParam defaults body Assign AttributeStore NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam defaults body Assign NameStore Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr Return Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr If CompareNotIn NameLoad AttributeLoad NameLoad attr body Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Return Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr keyword ListLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad For NameStore AttributeLoad NameLoad attr body Assign SubscriptStore AttributeLoad NameLoad attr Index AttributeLoad NameLoad attr AttributeLoad NameLoad attr For NameStore AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Str NameLoad NameLoad Expr Call AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr NameLoad Return NameLoad decorator_list FunctionDef arguments args NameParam defaults body While AttributeLoad NameLoad attr body Assign NameStore Num While AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad NameLoad attr If AttributeLoad NameLoad attr body Break Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad Num BinOpMult NameLoad Num For NameStore AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Assign NameStore AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr If CompareNotIn NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad For NameStore AttributeLoad NameLoad attr body Assign NameStore NameLoad While NameLoad body TryExcept body Expr Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad NameLoad Break handlers ExceptHandler body Expr Call AttributeLoad NameLoad attr Str NameLoad NameLoad Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad Call AttributeLoad NameLoad attr If NameLoad body Expr Call AttributeLoad NameLoad attr Str NameLoad AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr orelse Expr Call AttributeLoad NameLoad attr Str NameLoad AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str Assign TupleStore NameStore NameStore NameStore NameLoad Assign NameStore Call NameLoad GeneratorExp Call AttributeLoad NameLoad attr comprehension NameStore BinOpAdd Call AttributeLoad NameLoad attr NameLoad NameLoad Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword ListLoad NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad Return UnaryOpNot AttributeLoad NameLoad attr decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr keyword AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad AttributeLoad NameLoad attr AttributeLoad NameLoad attr AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr Expr Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad NameLoad NameLoad AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Expr Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr keyword NameLoad keyword ListLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr ListLoad NameLoad Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad decorator_list decorator_list
N/A N/A Python Dataflow streaming worker. __future__ absolute_import N/A logging N/A random N/A sys N/A time N/A traceback grpc.beta implementations google.cloud.dataflow.internal windmill_pb2 google.cloud.dataflow.internal windmill_service_pb2 google.cloud.dataflow.utils retry google.cloud.dataflow.worker executor google.cloud.dataflow.worker maptask google.cloud.dataflow.worker windmillio google.cloud.dataflow.worker windmillstate N/A apitools.base.py apitools_base N/A google.cloud.dataflow.internal.clients.dataflow dataflow WindmillClient N/A object N/A N/A Client for communication with Windmill. __init__ N/A N/A self host port request_timeout N/A 10 N/A N/A N/A self host host N/A N/A self port port N/A N/A self request_timeout request_timeout N/A channel N/A N/A implementations insecure_channel host port N/A N/A self stub N/A N/A windmill_service_pb2 beta_create_CloudWindmillServiceV1Alpha1_stub channel N/A GetWork N/A N/A self request N/A N/A N/A N/A N/A N/A self stub GetWork request N/A self request_timeout N/A N/A N/A retry with_exponential_backoff GetData N/A N/A self request N/A N/A N/A N/A N/A N/A self stub GetData request N/A self request_timeout N/A N/A N/A retry with_exponential_backoff CommitWork N/A N/A self request N/A N/A N/A N/A N/A N/A self stub CommitWork request N/A self request_timeout N/A N/A N/A retry with_exponential_backoff GetConfig N/A N/A self request N/A N/A N/A N/A N/A N/A self stub GetConfig request N/A self request_timeout N/A N/A N/A retry with_exponential_backoff ReportStats N/A N/A self request N/A N/A N/A N/A N/A N/A self stub ReportStats request N/A self request_timeout N/A N/A N/A retry with_exponential_backoff N/A StreamingWorker N/A object N/A N/A A streaming worker that communicates with Windmill. N/A MAX_GET_WORK_FETCH_BYTES N/A 64 20 N/A MAX_GET_WORK_ITEMS 100 N/A RETRY_LOCALLY_DELAY 10.0 __init__ N/A N/A self properties N/A N/A N/A N/A self project_id N/A properties N/A project_id N/A N/A self job_id N/A properties N/A job_id N/A N/A self worker_id N/A properties N/A worker_id N/A N/A self client_id N/A N/A random getrandbits 63 N/A windmill_host N/A properties N/A windmill.host N/A windmill_port N/A int N/A properties N/A windmill.grpc_port N/A N/A N/A logging info Using gRPC to connect to Windmill at %s:%d. windmill_host windmill_port N/A N/A self windmill N/A WindmillClient windmill_host windmill_port N/A N/A self instruction_map N/A N/A N/A self system_name_to_computation_id_map N/A N/A run N/A N/A self N/A N/A N/A N/A self running True N/A N/A N/A self dispatch_loop N/A get_work N/A N/A self N/A N/A N/A request N/A N/A windmill_pb2 GetWorkRequest client_id N/A self client_id max_items N/A StreamingWorker MAX_GET_WORK_ITEMS max_bytes N/A StreamingWorker MAX_GET_WORK_FETCH_BYTES N/A N/A N/A N/A self windmill GetWork request N/A add_computation N/A N/A self map_task N/A N/A N/A computation_id N/A N/A N/A self system_name_to_computation_id_map get N/A map_task systemName N/A map_task systemName N/A N/A computation_id N/A self instruction_map N/A N/A N/A N/A self instruction_map N/A computation_id map_task N/A parse_map_task N/A N/A self serialized_map_task N/A N/A N/A N/A N/A apitools_base JsonToMessage N/A dataflow MapTask serialized_map_task N/A get_config N/A N/A self computation_id N/A N/A N/A Load the config for a given computation from Windmill. N/A request N/A N/A windmill_pb2 GetConfigRequest computations N/A computation_id N/A response N/A N/A N/A self windmill GetConfig request N/A map_entry N/A response system_name_to_computation_id_map N/A N/A N/A N/A self system_name_to_computation_id_map N/A N/A map_entry system_name N/A map_entry computation_id N/A serialized_map_task N/A response cloud_works N/A N/A N/A N/A logging info Adding config for computation %s: %r computation_id serialized_map_task N/A N/A N/A self add_computation N/A N/A self parse_map_task serialized_map_task N/A response N/A dispatch_loop N/A N/A self N/A N/A N/A N/A self running N/A N/A backoff_seconds 0.001 N/A N/A self running N/A N/A work_response N/A N/A self get_work N/A N/A work_response work N/A N/A N/A N/A N/A time sleep backoff_seconds N/A backoff_seconds N/A min 1.0 N/A backoff_seconds 2 N/A computation_work N/A work_response work N/A N/A N/A N/A self process_computation computation_work N/A process_computation N/A N/A self computation_work N/A N/A N/A computation_id N/A computation_work computation_id N/A input_data_watermark N/A N/A windmillio windmill_to_harness_timestamp N/A computation_work input_data_watermark N/A N/A computation_id N/A self instruction_map N/A N/A N/A N/A self get_config computation_id N/A map_task_proto N/A N/A self instruction_map N/A computation_id N/A work_item N/A computation_work work N/A N/A retry_locally True N/A retry_locally N/A N/A N/A N/A N/A N/A self process_work_item computation_id map_task_proto input_data_watermark work_item N/A N/A N/A N/A N/A N/A N/A logging error Exception while processing work item for computation %r: %s, %s computation_id work_item N/A N/A traceback format_exc N/A retry_locally N/A N/A self report_failure computation_id work_item N/A N/A sys exc_info N/A retry_locally N/A N/A N/A N/A logging error Execution of work in computation %s for key %r failed; will retry locally. computation_id N/A work_item key N/A N/A N/A time sleep N/A StreamingWorker RETRY_LOCALLY_DELAY N/A N/A N/A N/A logging error Execution of work in computation %s for key %r failed; Windmill indicated to not retry locally. computation_id N/A work_item key N/A report_failure N/A N/A self computation_id work_item exc_info N/A N/A N/A Send exception details to Windmill; returns whether to retry locally. N/A N/A exc_type exc_value exc_traceback exc_info N/A messages N/A list N/A N/A N/A line strip N/A line N/A N/A N/A traceback format_exception_only exc_type exc_value N/A N/A traceback format_tb exc_traceback N/A wm_exception N/A N/A windmill_pb2 Exception stack_frames messages N/A report_stats_request N/A N/A windmill_pb2 ReportStatsRequest computation_id computation_id key N/A work_item key sharding_key N/A work_item sharding_key work_token N/A work_item work_token exceptions N/A wm_exception N/A response N/A N/A N/A self windmill ReportStats report_stats_request N/A N/A N/A response failed N/A process_work_item N/A N/A self computation_id map_task_proto input_data_watermark work_item N/A N/A N/A Process a work item. N/A workitem_commit_request N/A N/A windmill_pb2 WorkItemCommitRequest key N/A work_item key work_token N/A work_item work_token N/A env N/A N/A maptask WorkerEnvironment N/A context N/A N/A maptask StreamingExecutionContext N/A reader N/A N/A windmillstate WindmillStateReader computation_id N/A work_item key N/A work_item work_token N/A self windmill N/A state_internals N/A N/A windmillstate WindmillStateInternals reader N/A state N/A N/A windmillstate WindmillUnmergedState state_internals N/A output_data_watermark N/A N/A windmillio windmill_to_harness_timestamp N/A work_item output_data_watermark N/A N/A N/A context start computation_id work_item input_data_watermark output_data_watermark workitem_commit_request N/A self windmill state N/A map_task_executor N/A N/A executor MapTaskExecutor N/A map_task N/A N/A maptask decode_map_task map_task_proto env context N/A N/A N/A map_task_executor execute map_task N/A N/A N/A state_internals persist_to workitem_commit_request N/A commit_request N/A N/A windmill_pb2 CommitWorkRequest N/A computation_commit_request N/A N/A windmill_pb2 ComputationCommitWorkRequest computation_id computation_id requests N/A workitem_commit_request N/A N/A N/A N/A commit_request requests extend N/A computation_commit_request N/A N/A N/A N/A self windmill CommitWork commit_request N/A N/A