internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: apache/airflow

Issue:
Add overrides in the template field for the Google Cloud Run Jobs Execute  operator
### Description

The overrides parameter is not in the list of template field and it's impossible to pass runtime values to Cloud Run (date start/end, custom dag parameters,...)

### Use case/motivation

I would like to use Cloud Run Jobs with DBT and pass Airflow parameters (date start/end) to the Cloud Run jobs. For that, I need to use the the context (**kwargs) in a template field

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.


Resolving patch:
diff --git a/airflow/providers/google/cloud/operators/cloud_run.py b/airflow/providers/google/cloud/operators/cloud_run.py
--- a/airflow/providers/google/cloud/operators/cloud_run.py
+++ b/airflow/providers/google/cloud/operators/cloud_run.py
@@ -264,7 +264,7 @@ class CloudRunExecuteJobOperator(GoogleCloudBaseOperator):
     :param deferrable: Run operator in the deferrable mode
     """
 
-    template_fields = ("project_id", "region", "gcp_conn_id", "impersonation_chain", "job_name")
+    template_fields = ("project_id", "region", "gcp_conn_id", "impersonation_chain", "job_name", "overrides")
 
     def __init__(
         self,


Test patch:
diff --git a/tests/providers/google/cloud/operators/test_cloud_run.py b/tests/providers/google/cloud/operators/test_cloud_run.py
--- a/tests/providers/google/cloud/operators/test_cloud_run.py
+++ b/tests/providers/google/cloud/operators/test_cloud_run.py
@@ -40,6 +40,12 @@
 PROJECT_ID = "testproject"
 REGION = "us-central1"
 JOB_NAME = "jobname"
+OVERRIDES = {
+    "container_overrides": [{"args": ["python", "main.py"]}],
+    "task_count": 1,
+    "timeout": "60s",
+}
+
 JOB = Job()
 JOB.name = JOB_NAME
 
@@ -78,11 +84,12 @@ def test_create(self, hook_mock):
 class TestCloudRunExecuteJobOperator:
     def test_template_fields(self):
         operator = CloudRunExecuteJobOperator(
-            task_id=TASK_ID, project_id=PROJECT_ID, region=REGION, job_name=JOB_NAME
+            task_id=TASK_ID, project_id=PROJECT_ID, region=REGION, job_name=JOB_NAME, overrides=OVERRIDES
         )
 
         _assert_common_template_fields(operator.template_fields)
         assert "job_name" in operator.template_fields
+        assert "overrides" in operator.template_fields
 
     @mock.patch(CLOUD_RUN_HOOK_PATH)
     def test_execute_success(self, hook_mock):
Repository: apache/airflow

Issue:
default_webserver_config.py isn't compatible with the latest changes
### Apache Airflow version

2.8.0

### If "Other Airflow 2 version" selected, which one?

_No response_

### What happened?

[default_webserver_config.py
](https://github.com/apache/airflow/blob/2.8.0/airflow/config_templates/default_webserver_config.py) is not compatible with 2.8.X versions due to https://github.com/apache/airflow/pull/35572

specifically, [these imports](https://github.com/apache/airflow/blob/db2b75c233e3e3c59ec9d0563b93ddbe733ad0bf/airflow/config_templates/default_webserver_config.py#L25-L28) are not valid anymore

```python
# from airflow.www.fab_security.manager import AUTH_LDAP
# from airflow.www.fab_security.manager import AUTH_OAUTH
# from airflow.www.fab_security.manager import AUTH_OID
# from airflow.www.fab_security.manager import AUTH_REMOTE_USER
```

they should be

```python
# from airflow.auth.managers.fab.security_manager.override import AUTH_LDAP
# from airflow.auth.managers.fab.security_manager.override import AUTH_OAUTH
# from airflow.auth.managers.fab.security_manager.override import AUTH_OID
# from airflow.auth.managers.fab.security_manager.override import AUTH_REMOTE_USER
```




### What you think should happen instead?

_No response_

### How to reproduce

for oauth users just keep using the old `webserver_config.py`
```
from airflow.www.fab_security.manager import AUTH_OAUTH
```
will cause
```
ImportError: cannot import name 'AUTH_OAUTH' from 'airflow.www.fab_security.manager' (/home/zach/KeplerGroup/KIP-Airflow/.venv/lib/python3.11/site-packages/airflow/www/fab_security/manager.py)
```


### Operating System

Debian GNU/Linux 10 (buster)

### Versions of Apache Airflow Providers

_No response_

### Deployment

Other Docker-based deployment

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
cc: @vincbeck 
It should have been fixed in #35302
Oh, some are missing, let me add them

Resolving patch:
diff --git a/airflow/www/fab_security/manager.py b/airflow/www/fab_security/manager.py
--- a/airflow/www/fab_security/manager.py
+++ b/airflow/www/fab_security/manager.py
@@ -26,8 +26,11 @@
 log = logging.getLogger(__name__)
 
 __lazy_imports = {
+    "AUTH_OID": "flask_appbuilder.const",
     "AUTH_DB": "flask_appbuilder.const",
     "AUTH_LDAP": "flask_appbuilder.const",
+    "AUTH_REMOTE_USER": "flask_appbuilder.const",
+    "AUTH_OAUTH": "flask_appbuilder.const",
     "LOGMSG_WAR_SEC_LOGIN_FAILED": "flask_appbuilder.const",
 }
 


Test patch:
Repository: apache/airflow

Issue:
S3 Hook provides unnecessary log message when using s3hook class instance method: download_file
### Apache Airflow Provider(s)

amazon

### Versions of Apache Airflow Providers

apache-airflow-providers-amazon = "8.7.1"

### Apache Airflow version

2.7.2

### Operating System

Ubuntu 22.04.3 LTS

### Deployment

Amazon (AWS) MWAA

### Deployment details

MWAA Local runner 2.7.2

### What happened

When using s3hook.download_file(), Airflow logs return the message:

"This function shadows the 'download_file' method of S3 API, but it is not the same. If you "
"want to use the original method from S3 API, please call "
"'S3Hook.get_conn().download_file()'"

### What you think should happen instead

It would be better to make it a docstring, to not spam this message inside the task instance logs when calling it from within the custom operator.

The change was introduced in this PR: https://github.com/apache/airflow/pull/26886

### How to reproduce

Call the s3hook.download_file() method inside the custom Airflow operator.

### Anything else

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.


Resolving patch:
diff --git a/airflow/providers/amazon/aws/hooks/s3.py b/airflow/providers/amazon/aws/hooks/s3.py
--- a/airflow/providers/amazon/aws/hooks/s3.py
+++ b/airflow/providers/amazon/aws/hooks/s3.py
@@ -1369,6 +1369,10 @@ def download_file(
         """
         Download a file from the S3 location to the local file system.
 
+        Note:
+            This function shadows the 'download_file' method of S3 API, but it is not the same.
+            If you want to use the original method from S3 API, please use 'S3Hook.get_conn().download_file()'
+
         .. seealso::
             - :external+boto3:py:meth:`S3.Object.download_fileobj`
 
@@ -1386,12 +1390,6 @@ def download_file(
             Default: True.
         :return: the file name.
         """
-        self.log.info(
-            "This function shadows the 'download_file' method of S3 API, but it is not the same. If you "
-            "want to use the original method from S3 API, please call "
-            "'S3Hook.get_conn().download_file()'"
-        )
-
         self.log.info("Downloading source S3 file from Bucket %s with path %s", bucket_name, key)
 
         try:


Test patch:
Repository: apache/airflow

Issue:
Airflow Helm chart fails to deploy with ArgoCD
### Official Helm Chart version

1.12.0 (latest released)

### Apache Airflow version

latest

### Kubernetes Version

1.29

### Helm Chart configuration

No parameters with the simple ArgoCD deployment (5.51.x).

### Docker Image customizations

no

### What happened

ArgoCD fails to deploy with `Unable to create application: application spec for airflow-test is invalid: InvalidSpecError: Unable to generate manifests in : rpc error: code = Unknown desc = illegal filepath in archive: /tmp/d629b287-bcfd-4ca1-b827-33aa871dae80`

### What you think should happen instead

ArgoCD should be able to create application

### How to reproduce

1. Deploy ArgoCD
2. Create a simple ArgoCD application from UI
```
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: 'airflow-test'
spec:
  project: default
  source:
    repoURL: 'https://airflow.apache.org'
    targetRevision: 1.12.0
    helm:
      releaseName: airflow-test
      version: v3
    chart: airflow
  destination:
    server: 'https://kubernetes.default.svc'
    namespace: airflow-test
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
```

### Anything else

Issue in ArgoCD https://github.com/argoproj/argo-cd/issues/17182

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

Thanks! Working on a fix.

Resolving patch:
diff --git a/dev/breeze/src/airflow_breeze/utils/reproducible.py b/dev/breeze/src/airflow_breeze/utils/reproducible.py
--- a/dev/breeze/src/airflow_breeze/utils/reproducible.py
+++ b/dev/breeze/src/airflow_breeze/utils/reproducible.py
@@ -137,6 +137,8 @@ def reset(tarinfo):
                         arcname = entry
                         if prepend_path is not None:
                             arcname = os.path.normpath(os.path.join(prepend_path, arcname))
+                        if arcname == ".":
+                            continue
                         if arcname.startswith("./"):
                             arcname = arcname[2:]
                         tar_file.add(entry, filter=reset, recursive=False, arcname=arcname)


Test patch:
Repository: apache/airflow

Issue:
Automatically generate a list of available dataset URI formats
### Body

Follow-up to https://github.com/apache/airflow/pull/37005#discussion_r1498894629

We’ll need to implement a custom Sphinx derivative for this, similar to `airflow-auth-backends` etc.

### Committer

- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.


Discussion:

Resolving patch:
diff --git a/docs/exts/operators_and_hooks_ref.py b/docs/exts/operators_and_hooks_ref.py
--- a/docs/exts/operators_and_hooks_ref.py
+++ b/docs/exts/operators_and_hooks_ref.py
@@ -513,6 +513,17 @@ def render_content(self, *, tags: set[str] | None, header_separator: str = DEFAU
         )
 
 
+class DatasetSchemeDirective(BaseJinjaReferenceDirective):
+    """Generate list of Dataset URI schemes"""
+
+    def render_content(self, *, tags: set[str] | None, header_separator: str = DEFAULT_HEADER_SEPARATOR):
+        return _common_render_list_content(
+            header_separator=header_separator,
+            resource_type="dataset-uris",
+            template="dataset-uri-schemes.rst.jinja2",
+        )
+
+
 def setup(app):
     """Setup plugin"""
     app.add_directive("operators-hooks-ref", OperatorsHooksReferenceDirective)
@@ -527,6 +538,7 @@ def setup(app):
     app.add_directive("airflow-executors", ExecutorsDirective)
     app.add_directive("airflow-deferrable-operators", DeferrableOperatorDirective)
     app.add_directive("airflow-deprecations", DeprecationsDirective)
+    app.add_directive("airflow-dataset-schemes", DatasetSchemeDirective)
 
     return {"parallel_read_safe": True, "parallel_write_safe": True}
 


Test patch:
Repository: apache/airflow

Issue:
Change the UI color for ExternalSensor tasks
### Description

With the recent update of the graph display in DAGs, there's a visibility issue regarding external task sensors. Indeed those tasks have a `#19647e` tile color while the status & operator name are displayed in `--chakra-colors-gray-500` (aka `#718096`) on top of it, which makes the text unreadable. You can see a screenshot below:

<img width="239" alt="Current situation" src="https://github.com/apache/airflow/assets/69579602/8497faa7-9a61-4e59-a0ff-576844cfe67f">

I can submit a PR for it if needed but I'm unsure if there's a chart regarding which color to give to what type of operator etc. so please tell me. My idea would be to use same tint but lighter color like `#4db7db` which is more readable (see below) but not perfect for accessibility:

<img width="239" alt="Suggested new color" src="https://github.com/apache/airflow/assets/69579602/e2fa562b-668f-4756-9ef0-421d5da3280e">


### Use case/motivation

The current color makes monitoring and debugging very annoying as it's unreadable. It is also a big issue regarding accessibility as color-blind people may not be able to guess the status name by the status color. As an argument, it was red flagged by Chrome Lighthouse when I ran a test on the webpage.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

Hi, I would like to take this issue

Resolving patch:
diff --git a/airflow/sensors/external_task.py b/airflow/sensors/external_task.py
--- a/airflow/sensors/external_task.py
+++ b/airflow/sensors/external_task.py
@@ -148,7 +148,7 @@ class ExternalTaskSensor(BaseSensorOperator):
     """
 
     template_fields = ["external_dag_id", "external_task_id", "external_task_ids", "external_task_group_id"]
-    ui_color = "#19647e"
+    ui_color = "#4db7db"
     operator_extra_links = [ExternalDagLink()]
 
     def __init__(
@@ -473,7 +473,7 @@ class ExternalTaskMarker(EmptyOperator):
     """
 
     template_fields = ["external_dag_id", "external_task_id", "execution_date"]
-    ui_color = "#19647e"
+    ui_color = "#4db7db"
     operator_extra_links = [ExternalDagLink()]
 
     # The _serialized_fields are lazily loaded when get_serialized_fields() method is called


Test patch:
Repository: scikit-hep/uproot5

Issue:
Uproot4 and Double32_t
Hi,
I'm facing a weird issue when dealing with Double_32t with uproot4.
I have uploaded a small tree to reproduce the problem, in particular:

- `uproot4.open("test_tree.root")["Hyp3O2"]["RHyperTriton"]["m"].array()` converts the floating variable "m" as expected
-` uproot4.open("test_tree.root")["Hyp3O2"]["RHyperTriton"]["mppi_vert"].array() ` fails while converting the double32 variable "mppi_vert" with the following error: `AttributeError: 'NoneType' object has no attribute 'title'`

The link to the tree is:
- https://drive.google.com/file/d/14kF2gEKV7-eIbhSByd-F0cnUuVwrBopp/view?usp=sharing

Please note that the conversion with old uproot works nicely.
Any idea of what is happening here?



Discussion:

Resolving patch:
diff --git a/uproot4/behaviors/TBranch.py b/uproot4/behaviors/TBranch.py
--- a/uproot4/behaviors/TBranch.py
+++ b/uproot4/behaviors/TBranch.py
@@ -1367,19 +1367,21 @@ def streamer(self):
                 if matches is not None:
                     streamerinfo = matches[max(matches)]
 
-                    for element in streamerinfo.elements:
+                    for element in streamerinfo.walk_members(self._file.streamers):
                         if element.name == clean_name:
                             self._streamer = element
                             break
 
                     if self._streamer is None and clean_parentname is not None:
                         clean_parentname = clean_parentname.group(2)
-                        for element in streamerinfo.elements:
+                        for element in streamerinfo.walk_members(self._file.streamers):
                             if element.name == clean_parentname:
                                 substreamerinfo = self._file.streamer_named(
                                     element.typename
                                 )
-                                for subelement in substreamerinfo.elements:
+                                for subelement in substreamerinfo.walk_members(
+                                    self._file.streamers
+                                ):
                                     if subelement.name == clean_name:
                                         self._streamer = subelement
                                         break
diff --git a/uproot4/streamers.py b/uproot4/streamers.py
--- a/uproot4/streamers.py
+++ b/uproot4/streamers.py
@@ -174,6 +174,15 @@ def class_version(self):
     def elements(self):
         return self._members["fElements"]
 
+    def walk_members(self, streamers):
+        for element in self._members["fElements"]:
+            if isinstance(element, Model_TStreamerBase):
+                base = streamers[element.name][element.base_version]
+                for x in base.walk_members(streamers):
+                    yield x
+            else:
+                yield element
+
     @property
     def file_uuid(self):
         return self._file.uuid


Test patch:
Repository: scikit-hep/uproot5

Issue:
Superfluous print statement in iterate
#101 introduced a `print` statement in `iterate`.  Affects `uproot4` versions >0.0.22. I believe this was added for debugging purposes, but is not meant to be in the package.

Reproduce: 
- download https://github.com/scikit-hep/scikit-hep-testdata/raw/master/src/skhep_testdata/data/uproot-HZZ.root
- run the script:

```python
import uproot4

for _ in uproot4.iterate("uproot-HZZ.root"): pass
```

output:
```
uproot-HZZ.root 0
```

due to https://github.com/scikit-hep/uproot4/blob/1accbe3a70eda72cad954187291ddaec5f36e22d/uproot4/behaviors/TBranch.py#L213


Discussion:

Resolving patch:
diff --git a/uproot4/behaviors/TBranch.py b/uproot4/behaviors/TBranch.py
--- a/uproot4/behaviors/TBranch.py
+++ b/uproot4/behaviors/TBranch.py
@@ -210,8 +210,6 @@ def iterate(
 
     global_offset = 0
     for file_path, object_path in files:
-        print(file_path, global_offset)
-
         hasbranches = _regularize_object_path(
             file_path, object_path, custom_classes, allow_missing, options
         )


Test patch:
Repository: scikit-hep/uproot5

Issue:
Calling lazy() on a filename that doesn't exist produces confusing error
This is what happens if you call `lazy()` on a path that doesn't exist:

```python
>>> import uproot4
>>> uproot4.lazy('nonexistent_file.root')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mproffit/miniconda3/lib/python3.8/site-packages/uproot4/behaviors/TBranch.py", line 585, in lazy
    raise ValueError(
ValueError: allow_missing=True and no TTrees found in

    
```

Obviously this doesn't really indicate the actual problem--that the file doesn't exist. It also strangely claims that `allow_missing=True`, which is not correct since the default is `False`. It says the same thing even if you explicitly set `allow_missing=False`:

```python
>>> uproot4.lazy('nonexistent_file.root', allow_missing=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mproffit/miniconda3/lib/python3.8/site-packages/uproot4/behaviors/TBranch.py", line 585, in lazy
    raise ValueError(
ValueError: allow_missing=True and no TTrees found in

    
```


Discussion:
Just leaving a note for fixing this in the future; it's somewhere in here:

https://github.com/scikit-hep/uproot4/blob/7e42e2f71d9fb20dbfb12d177403b3b0839d0cc2/uproot4/behaviors/TBranch.py#L2707-L2786

Resolving patch:
diff --git a/uproot4/behaviors/TBranch.py b/uproot4/behaviors/TBranch.py
--- a/uproot4/behaviors/TBranch.py
+++ b/uproot4/behaviors/TBranch.py
@@ -2830,7 +2830,7 @@ def _regularize_files(files):
             out.append((file_path, object_path))
 
     if len(out) == 0:
-        uproot4._util._file_not_found(files)
+        raise uproot4._util._file_not_found(files)
 
     return out
 


Test patch:
diff --git a/tests/test_0045-lazy-arrays-1.py b/tests/test_0045-lazy-arrays-1.py
--- a/tests/test_0045-lazy-arrays-1.py
+++ b/tests/test_0045-lazy-arrays-1.py
@@ -148,3 +148,10 @@ def test_awkward_pluralization():
     )
     array = uproot4.lazy({files: "sample"})
     assert awkward1.to_list(array[:5, "i4"]) == [-15, -14, -13, -12, -11]
+
+
+def test_lazy_called_on_nonexistent_file():
+    filename = "nonexistent_file.root"
+    with pytest.raises(FileNotFoundError) as excinfo:
+        uproot4.lazy(filename)
+    assert filename in str(excinfo.value)
Repository: scikit-hep/uproot5

Issue:
TH1 raises `kind` instead of returning
On line 108 of `TH1.py`, `kind` raises `COUNT` instead of returning it, leading to the error:
```
----> 1 hasattr(a, 'kind')

/usr/lib/python3.9/site-packages/uproot/behaviors/TH1.py in kind(self)
    106         ``"MEAN"`` for profile plots (TProfile*).
    107         """
--> 108         raise "COUNT"
    109
    110     def values(self, flow=False):

TypeError: exceptions must derive from BaseException
```


Discussion:

Resolving patch:
diff --git a/uproot/behaviors/TH1.py b/uproot/behaviors/TH1.py
--- a/uproot/behaviors/TH1.py
+++ b/uproot/behaviors/TH1.py
@@ -105,7 +105,7 @@ def kind(self):
         The meaning of this object: ``"COUNT"`` for true histograms (TH*) and
         ``"MEAN"`` for profile plots (TProfile*).
         """
-        raise "COUNT"
+        raise NotImplementedError(repr(self))
 
     def values(self, flow=False):
         """
@@ -216,6 +216,10 @@ def weighted(self):
         sumw2 = self.member("fSumw2", none_if_missing=True)
         return sumw2 is not None and len(sumw2) == self.member("fNcells")
 
+    @property
+    def kind(self):
+        return "COUNT"
+
     def values(self, flow=False):
         if hasattr(self, "_values"):
             values = self._values
diff --git a/uproot/behaviors/TH2.py b/uproot/behaviors/TH2.py
--- a/uproot/behaviors/TH2.py
+++ b/uproot/behaviors/TH2.py
@@ -38,6 +38,10 @@ def weighted(self):
         sumw2 = self.member("fSumw2", none_if_missing=True)
         return sumw2 is not None and len(sumw2) == self.member("fNcells")
 
+    @property
+    def kind(self):
+        return "COUNT"
+
     def values(self, flow=False):
         if hasattr(self, "_values"):
             values = self._values
diff --git a/uproot/behaviors/TH3.py b/uproot/behaviors/TH3.py
--- a/uproot/behaviors/TH3.py
+++ b/uproot/behaviors/TH3.py
@@ -42,6 +42,10 @@ def weighted(self):
         sumw2 = self.member("fSumw2", none_if_missing=True)
         return sumw2 is not None and len(sumw2) == self.member("fNcells")
 
+    @property
+    def kind(self):
+        return "COUNT"
+
     def values(self, flow=False):
         if hasattr(self, "_values"):
             values = self._values


Test patch: