Repository: apache/airflow

Issue:
Cluster policies on mapped tasks cause broken DAGs
### Apache Airflow version

2.8.1

### What happened?

Defining a task policy that affects an operator in a mapped task causes a DAG Import Error.

### How to reproduce

Consider the following Airflow Task Policy which changes the `execution_timeout` for `BashOperators` to 30 seconds.

```
from datetime import timedelta
from airflow.models.baseoperator import BaseOperator
from abc import ABC


class TimedOperator(BaseOperator, ABC):
    timeout: timedelta


def task_policy(task: TimedOperator):
    if task.operator_name == 'BashOperator':
        task.execution_timeout = timedelta(seconds=30)

```

If we then create the following DAG and run it, we can see that the task does indeed get killed after 30 seconds according to the policy.

```
from datetime import datetime
from airflow import models

from airflow.operators.bash import BashOperator

with models.DAG(
    dag_id='policy_test',
    start_date=datetime(2024, 1, 1),
    catchup=False,
    schedule='0 0 * * *',
) as dag:

    task1 = BashOperator(
        task_id='single_task',
        bash_command='echo "Hello World!"; sleep 60; echo "Goodbye World!"',
    )
```

```
[2024-01-25, 15:35:21 UTC] {task_command.py:423} INFO - Running <TaskInstance: policy_test.single_task scheduled__2024-01-24T00:00:00+00:00 [running]> on host 67fd7cade381
[2024-01-25, 15:35:21 UTC] {taskinstance.py:2480} INFO - Exporting env vars: AIRFLOW_CTX_DAG_OWNER='airflow' AIRFLOW_CTX_DAG_ID='policy_test' AIRFLOW_CTX_TASK_ID='single_task' AIRFLOW_CTX_EXECUTION_DATE='2024-01-24T00:00:00+00:00' AIRFLOW_CTX_TRY_NUMBER='1' AIRFLOW_CTX_DAG_RUN_ID='scheduled__2024-01-24T00:00:00+00:00'
[2024-01-25, 15:35:21 UTC] {subprocess.py:63} INFO - Tmp dir root location: /tmp
[2024-01-25, 15:35:21 UTC] {subprocess.py:75} INFO - Running command: ['/bin/bash', '-c', 'echo "Hello World!"; sleep 60; echo "Goodbye World!"']
[2024-01-25, 15:35:21 UTC] {subprocess.py:86} INFO - Output:
[2024-01-25, 15:35:21 UTC] {subprocess.py:93} INFO - Hello World!
[2024-01-25, 15:35:51 UTC] {timeout.py:68} ERROR - Process timed out, PID: 4315
[2024-01-25, 15:35:51 UTC] {subprocess.py:104} INFO - Sending SIGTERM signal to process group
[2024-01-25, 15:35:51 UTC] {taskinstance.py:2698} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 428, in _execute_task
    result = execute_callable(context=context, **execute_callable_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/airflow/operators/bash.py", line 203, in execute
    result = self.subprocess_hook.run_command(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/airflow/hooks/subprocess.py", line 91, in run_command
    for raw_line in iter(self.sub_process.stdout.readline, b""):
  File "/usr/local/lib/python3.11/site-packages/airflow/utils/timeout.py", line 69, in handle_timeout
    raise AirflowTaskTimeout(self.error_message)
airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 4315
```

However, if we introduce a dynamically mapped BashOperator task, then we are presented with the following error.

```
mapped_task = BashOperator.partial(
        task_id='mapped_tasks',
    ).expand(
        bash_command=[
            'echo "Hello World!"; sleep 60; echo "Goodbye World!"',
            'echo "Hello World!"; sleep 60; echo "Goodbye World!"'
        ]
    )
```

![Screenshot 2024-01-25 at 15 37 48](https://github.com/apache/airflow/assets/967119/03cd4b56-a412-44fb-a98b-411388eb04d7)


### Operating System

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

### Deployment

Astronomer

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Possibly related: #28313 
@cccs-seb will you have time to check if this is a regression due to #28313 ?
I'm going to fix this as it's quite simple I think, i.e. adding more setter methods.  So, I was initially planning to add methods for `execution_timeout` and `queue` but should I just add them for everything?

Basically, I think that any task attribute that is modified by a policy in a dynamically mapped task will be affected unless it has a setter method.

Resolving patch:
diff --git a/airflow/models/mappedoperator.py b/airflow/models/mappedoperator.py
--- a/airflow/models/mappedoperator.py
+++ b/airflow/models/mappedoperator.py
@@ -420,11 +420,19 @@ def is_teardown(self, value: bool) -> None:
     def depends_on_past(self) -> bool:
         return bool(self.partial_kwargs.get("depends_on_past"))
 
+    @depends_on_past.setter
+    def depends_on_past(self, value: bool) -> None:
+        self.partial_kwargs["depends_on_past"] = value
+
     @property
     def ignore_first_depends_on_past(self) -> bool:
         value = self.partial_kwargs.get("ignore_first_depends_on_past", DEFAULT_IGNORE_FIRST_DEPENDS_ON_PAST)
         return bool(value)
 
+    @ignore_first_depends_on_past.setter
+    def ignore_first_depends_on_past(self, value: bool) -> None:
+        self.partial_kwargs["ignore_first_depends_on_past"] = value
+
     @property
     def wait_for_past_depends_before_skipping(self) -> bool:
         value = self.partial_kwargs.get(
@@ -432,62 +440,122 @@ def wait_for_past_depends_before_skipping(self) -> bool:
         )
         return bool(value)
 
+    @wait_for_past_depends_before_skipping.setter
+    def wait_for_past_depends_before_skipping(self, value: bool) -> None:
+        self.partial_kwargs["wait_for_past_depends_before_skipping"] = value
+
     @property
     def wait_for_downstream(self) -> bool:
         return bool(self.partial_kwargs.get("wait_for_downstream"))
 
+    @wait_for_downstream.setter
+    def wait_for_downstream(self, value: bool) -> None:
+        self.partial_kwargs["wait_for_downstream"] = value
+
     @property
     def retries(self) -> int | None:
         return self.partial_kwargs.get("retries", DEFAULT_RETRIES)
 
+    @retries.setter
+    def retries(self, value: int | None) -> None:
+        self.partial_kwargs["retries"] = value
+
     @property
     def queue(self) -> str:
         return self.partial_kwargs.get("queue", DEFAULT_QUEUE)
 
+    @queue.setter
+    def queue(self, value: str | None) -> None:
+        self.partial_kwargs["queue"] = value
+
     @property
     def pool(self) -> str:
         return self.partial_kwargs.get("pool", Pool.DEFAULT_POOL_NAME)
 
+    @pool.setter
+    def pool(self, value: str | None) -> None:
+        self.partial_kwargs["pool"] = value
+
     @property
     def pool_slots(self) -> str | None:
         return self.partial_kwargs.get("pool_slots", DEFAULT_POOL_SLOTS)
 
+    @pool_slots.setter
+    def pool_slots(self, value: str | None) -> None:
+        self.partial_kwargs["pool_slots"] = value
+
     @property
     def execution_timeout(self) -> datetime.timedelta | None:
         return self.partial_kwargs.get("execution_timeout")
 
+    @execution_timeout.setter
+    def execution_timeout(self, value: datetime.timedelta | None) -> None:
+        self.partial_kwargs["execution_timeout"] = value
+
     @property
     def max_retry_delay(self) -> datetime.timedelta | None:
         return self.partial_kwargs.get("max_retry_delay")
 
+    @max_retry_delay.setter
+    def max_retry_delay(self, value: datetime.timedelta | None) -> None:
+        self.partial_kwargs["max_retry_delay"] = value
+
     @property
     def retry_delay(self) -> datetime.timedelta:
         return self.partial_kwargs.get("retry_delay", DEFAULT_RETRY_DELAY)
 
+    @retry_delay.setter
+    def retry_delay(self, value: datetime.timedelta | None) -> None:
+        self.partial_kwargs["retry_delay"] = value
+
     @property
     def retry_exponential_backoff(self) -> bool:
         return bool(self.partial_kwargs.get("retry_exponential_backoff"))
 
+    @retry_exponential_backoff.setter
+    def retry_exponential_backoff(self, value: bool | None) -> None:
+        self.partial_kwargs["retry_exponential_backoff"] = value
+
     @property
     def priority_weight(self) -> int:  # type: ignore[override]
         return self.partial_kwargs.get("priority_weight", DEFAULT_PRIORITY_WEIGHT)
 
+    @priority_weight.setter
+    def priority_weight(self, value: int | None) -> None:
+        self.partial_kwargs["priority_weight"] = value
+
     @property
     def weight_rule(self) -> str:  # type: ignore[override]
         return self.partial_kwargs.get("weight_rule", DEFAULT_WEIGHT_RULE)
 
+    @weight_rule.setter
+    def weight_rule(self, value: str | None) -> None:
+        self.partial_kwargs["weight_rule"] = value
+
     @property
     def sla(self) -> datetime.timedelta | None:
         return self.partial_kwargs.get("sla")
 
+    @sla.setter
+    def sla(self, value: datetime.timedelta | None) -> None:
+        self.partial_kwargs["sla"] = value
+
     @property
     def max_active_tis_per_dag(self) -> int | None:
         return self.partial_kwargs.get("max_active_tis_per_dag")
 
+    @max_active_tis_per_dag.setter
+    def max_active_tis_per_dag(self, value: int | None) -> None:
+        self.partial_kwargs["max_active_tis_per_dag"] = value
+
     @property
     def max_active_tis_per_dagrun(self) -> int | None:
         return self.partial_kwargs.get("max_active_tis_per_dagrun")
 
+    @max_active_tis_per_dagrun.setter
+    def max_active_tis_per_dagrun(self, value: int | None) -> None:
+        self.partial_kwargs["max_active_tis_per_dagrun"] = value
+
     @property
     def resources(self) -> Resources | None:
         return self.partial_kwargs.get("resources")


Test patch: