Project: Armada (Orchestration & Management / Scheduling & Orchestration)
File: python_airflow_operator.md

docname: pythonairflowoperator images: {} path: /python-airflow-operator title: Armada Airflow Operator This class provides integration with Airflow and Armada Bases: `BaseOperator` Implementation of an ArmadaOperator for airflow. Airflow operators inherit from BaseOperator. Parameters* name (str*) The name of the airflow task armada_channel_args (GrpcChannelArgsDict*) GRPC channel arguments to be used when creating a grpc channel to connect to the armada server instance. job_service_channel_args (GrpcChannelArgsDict*) GRPC channel arguments to be used when creating a grpc channel to connect to the job service instance. armada_queue (str*) The queue name for Armada. job_request_items (List[JobSubmitRequestItem]*) A PodSpec that is used by Armada for submitting a job lookout_url_template (str | None*) A URL template to be used to provide users a valid link to the related lookout job in this operators log. The format should be: ?jobid=<jobid> where <job_id> will be replaced with the actual job ID. poll_interval (int*) How often to poll jobservice to get status. Returns* an armada operator instance Executes the Armada Operator. Runs an Armada job and calls the jobserviceclient for polling. Parameters* context The airflow context. Returns* None Return type* None Template all attributes listed in self.template_fields. This mutates the attributes in-place and is irreversible. Parameters* context (Context*) Context dict with values to apply on content. jinja_env (Environment | None*) Jinjas environment to use for rendering. Return type* None Bases: `BaseOperator` Implementation of a deferrable armada operator for airflow. Distinguished from ArmadaOperator by its ability to defer itself after submitting its jobrequestitems. See for more information about deferrable airflow operators. Airflow operators inherit from BaseOperator. Parameters* name (str*) The name of the airflow task. armada_channel_args (GrpcChannelArgsDict*) GRPC channel arguments to be used when creating a grpc channel to connect to the armada server instance. job_service_channel_args (GrpcChannelArgsDict*) GRPC channel arguments to be used when creating a grpc channel to connect to the job service instance. armada_queue (str*) The queue name for Armada. job_request_items (List[JobSubmitRequestItem]*) A PodSpec that is used by Armada for submitting a job. lookout_url_template (str | None*) A URL template to be used to provide users a valid link to the related lookout job in this operators log. The format should be: ?jobid=<jobid> where <job_id> will be replaced with the actual job ID. poll_interval (int*) How often to poll jobservice to get status. Returns* A deferrable armada operator instance. Executes the Armada Operator. Only meant to be called by airflow. Submits an Armada job and defers itself to ArmadaJobCompleteTrigger to wait until the job completes. Parameters* context The airflow context. Returns* None Return type* None Template all attributes listed in

This mutates the attributes in-place and is irreversible. Parameters* context (Context*) Context dict with values to apply on content. jinja_env (Environment | None*) Jinjas environment to use for rendering. Return type* None Resumes this operator after deferring itself to ArmadaJobCompleteTrigger. Only meant to be called from within Airflow. Reports the result of the job and returns. Parameters* context* The airflow context. event (dict*) The payload from the TriggerEvent raised by ArmadaJobCompleteTrigger. job_id (str*) The job ID. Returns* None Return type* None Get a serialized version of this object. Returns* A dict of keyword arguments used when instantiating Return type* dict this object. Bases: `BaseTrigger` An airflow trigger that monitors the job state of an armada job. Triggers when the job is complete. Parameters* job_id (str*) The job ID to monitor. job_service_channel_args (GrpcChannelArgsDict*) GRPC channel arguments to be used when creating a grpc channel to connect to the job service instance. armada_queue (str*) The name of the armada queue. job_set_id (str*) The ID of the job set. airflow_task_name (str*) Name of the airflow task to which this trigger belongs. poll_interval (int*) How often to poll jobservice to get status. Returns* An armada job complete trigger instance. Runs the trigger. Meant to be called by an airflow triggerer process. Return the information needed to reconstruct this Trigger. Returns* Tuple of (class path, keyword arguments needed to re-instantiate). Return type* tuple Bases: `object` The JobService Client Implementation of gRPC stubs from JobService Parameters* channel gRPC channel used for authentication. See for more information. Returns* a job service client instance Get job status of a given job in a queue and jobsetid. Uses the GetJobStatus rpc to get a status of your job Parameters* queue (str*) The name of the queue job_set_id (str*) The name of the job set (a grouping of jobs) job_id (str*) The id of the job Returns* A Job Service Request (State, Error) Return type* JobServiceResponse Health Check for GRPC Request Return type* HealthCheckResponse Get a JobServiceClient that has retry configured Parameters* target (str*) grpc channel target credentials (ChannelCredentials | None*) grpc channel credentials (if needed) compresion* grpc channel compression compression (Compression | None*) Returns* A job service client instance Return type* JobServiceClient Bases: `object` The JobService AsyncIO Client AsyncIO implementation of gRPC stubs from JobService Parameters* channel (Channel) AsyncIO gRPC channel used for authentication. See for more information. Returns* A job service client instance Get job status of a given job in a queue and

Uses the GetJobStatus rpc to get a status of your job Parameters* queue (str*) The name of the queue job_set_id (str*) The name of the job set (a grouping of jobs) job_id (str*) The id of the job Returns* A Job Service Request (State, Error) Return type* JobServiceResponse Health Check for GRPC Request Return type* HealthCheckResponse Get a JobServiceAsyncIOClient that has retry configured Parameters* target (str*) grpc channel target credentials (ChannelCredentials | None*) grpc channel credentials (if needed) compresion* grpc channel compression compression (Compression | None*) Returns* A job service asyncio client instance Return type* JobServiceAsyncIOClient Bases: `Enum` An enumeration. Throw an error on a terminal event if job errored out Parameters* job_state (JobState*) A JobState enum class name (str*) The name of your armada job job_id (str*) The job id that armada assigns to it Returns* No Return or an AirflowFailException. AirflowFailException tells Airflow Schedule to not reschedule the task Annotates the inbound job request items with Airflow context elements Parameters* context* The airflow context. job_request_items (List[JobSubmitRequestItem]*) The job request items to be sent to armada Returns* annotated job request items for armada Return type* List[JobSubmitRequestItem] Parameters* armada_queue (str*) job_set_id (str*) job_id (str*) job_service_client (JobServiceClient*) Return type* JobServiceResponse Provides the annotation key prefix, which can be specified in env var ANNOTATIONKEYPREFIX. A default is provided if the env var is not defined Returns* string annotation key prefix Return type* str Return type* JobState Poll JobService cache until you get a terminated event. A terminated event is SUCCEEDED, FAILED or CANCELLED Parameters* armada_queue (str*) The queue for armada job_set_id (str*) Your jobsetid airflow_task_name (str*) The name of your armada job poll_interval (int*) Polling interval for jobservice to get status. job_id (str*) The name of the job id that armada assigns to it job_service_client (JobServiceClient | None*) A JobServiceClient that is used for polling. It is optional only for testing job_status_callable* A callable object for test injection. time_out_for_failure (int*) The amount of time a job can be in jobidnot_found before we decide it was a invalid job Returns* A tuple of JobStateEnum, message Return type* Tuple[JobState, str] Poll JobService cache asyncronously until you get a terminated event. A terminated event is SUCCEEDED, FAILED or CANCELLED Parameters* armada_queue (str*) The queue for armada job_set_id (str*) Your jobsetid airflow_task_name (str*) The name of your armada job job_id (str*) The name of the job id that armada assigns to it job_service_client (JobServiceAsyncIOClient*) A JobServiceClient that is used for polling. It is optional only for testing poll_interval (int*) How often to poll jobservice to get status. time_out_for_failure (int*) The amount of time a job can be in jobidnot_found before we decide it was a invalid job Returns* A tuple of JobStateEnum, message Return type* Tuple[JobState, str]