Repository: apache/airflow (Python)

Issue:
DbtCloudRunJobOperator `reuse_existing_run` flag does not work properly
### Apache Airflow version

2.9.1

### If "Other Airflow 2 version" selected, which one?

_No response_

### What happened?

When `reuse_existing_run` is set to `True` in `DbtCloudRunJobOperator`, it gives an error:

```
[2024-06-04 20:15:06,729] {http.py:201} ERROR - 
{"status":{"code":400,"is_success":false,"user_message":"The request was invalid. 
Please double check the provided data and try again.","developer_message":""},
"data":{"status__in":"'DbtCloudJobRunStatus.NON_TERMINAL_STATUSES' is not valid under any of the given schemas"}}
```


### What you think should happen instead?

When `reuse_existing_run` is True, `DbtCloudJobRunStatus.NON_TERMINAL_STATUSES` is passed as `status__in` parameter.
https://github.com/apache/airflow/blob/68bd42a7ffc3d2828b1ac4102e65944429d34012/airflow/providers/dbt/cloud/operators/dbt.py#L133-L140
 However, this results to a URL like

```
https://cloud.getdbt.com/api/v2/accounts/<account_id>/runs/?job_definition_id=<job_id>&status__in=DbtCloudJobRunStatus.NON_TERMINAL_STATUSES&order_by=-created_at
```

`DbtCloudJobRunStatus.NON_TERMINAL_STATUSES.value` should be used instead of `DbtCloudJobRunStatus.NON_TERMINAL_STATUSES` to produce the correct URL like below:
```
https://cloud.getdbt.com/api/v2/accounts/<account_id>/runs/?job_definition_id=<job_id>&status__in=1&status__in=2&status__in=3&order_by=-created_at
```

In addition to that, I tested [list runs](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) for `status__in`  after fixing the above issue but it returns 400:
```
{"status":{"code":400,"is_success":false,"user_message":"The request was invalid. Please double check the provided data and try again.","developer_message":""},"data":{"status__in":"3 is not valid under any of the given schemas"}}
```

I am doubting if `status__in` parameter works properly or not. I have created a ticket to dbt-cloud, and will update here once I get a response.

**Update:** They said it should be passed like status__in=[1,2,3].

### How to reproduce

Create a sample `DbtCloudRunJobOperator`  task and  pass `reuse_existing_run` as True.  

```
from __future__ import annotations

import logging
import pendulum
from airflow.providers.dbt.cloud.operators.dbt import DbtCloudRunJobOperator

from airflow.decorators import dag


@dag(
    schedule=None,
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    catchup=False
)
def dbt_test():
    dbt_run = DbtCloudRunJobOperator(
        task_id="dbt_run",
        job_id=<JOB_ID>,
        account_id=<ACCOUNT_ID>,
        reuse_existing_run=True,
        wait_for_termination=True,
    )

    dbt_run


dag = dbt_test()

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    dag.test(conn_file_path="PATH_TO_CONNECTIONS_YAML")
```

You would get the following:

```
[2024-06-04 20:15:06,729] {http.py:201} ERROR - 
{"status":{"code":400,"is_success":false,"user_message":"The request was invalid. 
Please double check the provided data and try again.","developer_message":""},
"data":{"status__in":"'DbtCloudJobRunStatus.NON_TERMINAL_STATUSES' is not valid under any of the given schemas"}}
```

### Operating System

macOS 14.4.1

### Versions of Apache Airflow Providers

_No response_

### Deployment

Official Apache Airflow Helm Chart

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion: Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.




Fix (patch):
diff --git a/airflow/providers/dbt/cloud/operators/dbt.py b/airflow/providers/dbt/cloud/operators/dbt.py
index 8e9f310ce8e5b..18df34ce3ecd4 100644
--- a/airflow/providers/dbt/cloud/operators/dbt.py
+++ b/airflow/providers/dbt/cloud/operators/dbt.py
@@ -135,7 +135,7 @@ def execute(self, context: Context):
                 account_id=self.account_id,
                 payload={
                     "job_definition_id": self.job_id,
-                    "status__in": DbtCloudJobRunStatus.NON_TERMINAL_STATUSES,
+                    "status__in": str(list(DbtCloudJobRunStatus.NON_TERMINAL_STATUSES.value)),
                     "order_by": "-created_at",
                 },
             ).json()["data"]