Repository: apache/airflow

Issue:
Task hostname does not reflect pod name when it's too long
### Apache Airflow version

2.8.0

### If "Other Airflow 2 version" selected, which one?

_No response_

### What happened?

When a task has a long name (or it's inside one or many task group(s) with long names), the name of the pod that is created for the task is different than the task hostname (from Airflow Metadata DB)

### What you think should happen instead?

The task hostname should be the same as the pod name

### How to reproduce

1. Run the following DAG
  ```python
  import time
from airflow.decorators import task, task_group
from airflow.models import DAG
from datetime import datetime, timedelta

with DAG(
    dag_id="test_long_task_name",
    schedule=None,
    default_args={
        "depends_on_past": False,
        "wait_for_downstream": False,
        "start_date": datetime(2023, 12, 20),
        "retry_delay": timedelta(seconds=10),
        "retries": 1,
    },
    catchup=False,
    max_active_runs=1,
    render_template_as_native_obj=True,
) as dag:

    @task_group
    def task_group_with_long_name():
        @task_group
        def inner_task_group_with_long_name():
            @task
            def task1():
                pass

            @task
            def task2():
                pass

            task1()
            task2()

        inner_task_group_with_long_name()

    task_group_with_long_name()
  ```
![image](https://github.com/apache/airflow/assets/28935464/c5888712-2d4f-4c79-9d13-3f24ea412664)

2. Check the hostname property for both tasks:
![image](https://github.com/apache/airflow/assets/28935464/60950e2f-e3db-4243-8921-6b4826d7d795)
![image](https://github.com/apache/airflow/assets/28935464/95e49871-d91f-4df3-a9e3-aab700cb90bc)

3. Check the pod name for both tasks (scheduler logs):
```
[2023-12-20T09:45:05.497+0000] {kubernetes_executor_utils.py:396} INFO - Creating kubernetes pod for job is TaskInstanceKey(dag_id='test_long_task_name', task_id='task_group_with_long_name.inner_task_group_with_long_name.task2', run_id='manual__2023-12-20T09:45:00+00:00', try_number=1, map_index=-1), with pod name test-long-task-name-task-group-with-long-name-inner-task-group-with-lon-efj7zr6z, annotations: <omitted>
```

```
[2023-12-20T09:45:05.776+0000] {kubernetes_executor_utils.py:396} INFO - Creating kubernetes pod for job is TaskInstanceKey(dag_id='test_long_task_name', task_id='task_group_with_long_name.inner_task_group_with_long_name.task1', run_id='manual__2023-12-20T09:45:00+00:00', try_number=1, map_index=-1), with pod name test-long-task-name-task-group-with-long-name-inner-task-group-with-lon-diwm37yp, annotations: <omitted>
```


## Task1
- hostname: `test-long-task-name-task-group-with-long-name-inner-task-group`
- pod name: `test-long-task-name-task-group-with-long-name-inner-task-group-with-lon-diwm37yp`
## Task2
- hostname: `test-long-task-name-task-group-with-long-name-inner-task-group`
- pod name: `test-long-task-name-task-group-with-long-name-inner-task-group-with-lon-efj7zr6z`

### Operating System

Airflow in Kubernetes (`apache/airflow:2.8.0-python3.11`)

### Versions of Apache Airflow Providers

apache-airflow-providers-cncf-kubernetes==7.11.0

### Deployment

Official Apache Airflow Helm Chart

### Deployment details

_No response_

### Anything else?

This issue also happens in previous Airflow versions.

After some research, I've found out the following:
- The function that is used to determine the hostname is defined with the [hostname_callable](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#hostname-callable) parameter.
- There's a maximum size for this hostname [[source](https://stackoverflow.com/questions/8724954/what-is-the-maximum-number-of-characters-for-a-host-name-in-unix)]. In the Airflow image, this limit is set to **64**.
- In the k8s provider, the pod name maximum size is set to 80 [[source](https://github.com/apache/airflow/blob/2.8.0/airflow/providers/cncf/kubernetes/kubernetes_helper_functions.py#L62)]


So, for me, there are two possible approaches to solve this issue
1. Modify the k8s provider pod name maximum size to 63, matching `HOST_NAME_MAX` minus 1.
2. Increase the `HOST_NAME_MAX` limit in the Airflow image.

I believe the first one is the safest


### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:

Resolving patch:
diff --git a/airflow/providers/cncf/kubernetes/kubernetes_helper_functions.py b/airflow/providers/cncf/kubernetes/kubernetes_helper_functions.py
--- a/airflow/providers/cncf/kubernetes/kubernetes_helper_functions.py
+++ b/airflow/providers/cncf/kubernetes/kubernetes_helper_functions.py
@@ -34,6 +34,8 @@
 
 alphanum_lower = string.ascii_lowercase + string.digits
 
+POD_NAME_MAX_LENGTH = 63  # Matches Linux kernel's HOST_NAME_MAX default value minus 1.
+
 
 def rand_str(num):
     """Generate random lowercase alphanumeric string of length num.
@@ -43,7 +45,7 @@ def rand_str(num):
     return "".join(secrets.choice(alphanum_lower) for _ in range(num))
 
 
-def add_pod_suffix(*, pod_name: str, rand_len: int = 8, max_len: int = 80) -> str:
+def add_pod_suffix(*, pod_name: str, rand_len: int = 8, max_len: int = POD_NAME_MAX_LENGTH) -> str:
     """Add random string to pod name while staying under max length.
 
     :param pod_name: name of the pod
@@ -59,16 +61,12 @@ def create_pod_id(
     dag_id: str | None = None,
     task_id: str | None = None,
     *,
-    max_length: int = 80,
+    max_length: int = POD_NAME_MAX_LENGTH,
     unique: bool = True,
 ) -> str:
     """
     Generate unique pod ID given a dag_id and / or task_id.
 
-    The default of 80 for max length is somewhat arbitrary, mainly a balance between
-    content and not overwhelming terminal windows of reasonable width. The true
-    upper limit is 253, and this is enforced in construct_pod.
-
     :param dag_id: DAG ID
     :param task_id: Task ID
     :param max_length: max number of characters
diff --git a/airflow/providers/cncf/kubernetes/operators/pod.py b/airflow/providers/cncf/kubernetes/operators/pod.py
--- a/airflow/providers/cncf/kubernetes/operators/pod.py
+++ b/airflow/providers/cncf/kubernetes/operators/pod.py
@@ -51,6 +51,7 @@
     convert_volume_mount,
 )
 from airflow.providers.cncf.kubernetes.hooks.kubernetes import KubernetesHook
+from airflow.providers.cncf.kubernetes.kubernetes_helper_functions import POD_NAME_MAX_LENGTH
 from airflow.providers.cncf.kubernetes.pod_generator import PodGenerator
 from airflow.providers.cncf.kubernetes.triggers.pod import KubernetesPodTrigger
 from airflow.providers.cncf.kubernetes.utils import xcom_sidecar  # type: ignore[attr-defined]
@@ -92,7 +93,7 @@ def _rand_str(num):
     return "".join(secrets.choice(alphanum_lower) for _ in range(num))
 
 
-def _add_pod_suffix(*, pod_name, rand_len=8, max_len=253):
+def _add_pod_suffix(*, pod_name, rand_len=8, max_len=POD_NAME_MAX_LENGTH):
     """Add random string to pod name while staying under max len.
 
     TODO: when min airflow version >= 2.5, delete this function and import from kubernetes_helper_functions.
@@ -107,7 +108,7 @@ def _create_pod_id(
     dag_id: str | None = None,
     task_id: str | None = None,
     *,
-    max_length: int = 80,
+    max_length: int = POD_NAME_MAX_LENGTH,
     unique: bool = True,
 ) -> str:
     """
@@ -962,7 +963,7 @@ def build_pod_request_obj(self, context: Context | None = None) -> k8s.V1Pod:
 
         if not pod.metadata.name:
             pod.metadata.name = _create_pod_id(
-                task_id=self.task_id, unique=self.random_name_suffix, max_length=80
+                task_id=self.task_id, unique=self.random_name_suffix, max_length=POD_NAME_MAX_LENGTH
             )
         elif self.random_name_suffix:
             # user has supplied pod name, we're just adding suffix
diff --git a/airflow/providers/cncf/kubernetes/pod_generator.py b/airflow/providers/cncf/kubernetes/pod_generator.py
--- a/airflow/providers/cncf/kubernetes/pod_generator.py
+++ b/airflow/providers/cncf/kubernetes/pod_generator.py
@@ -41,7 +41,11 @@
     AirflowException,
     RemovedInAirflow3Warning,
 )
-from airflow.providers.cncf.kubernetes.kubernetes_helper_functions import add_pod_suffix, rand_str
+from airflow.providers.cncf.kubernetes.kubernetes_helper_functions import (
+    POD_NAME_MAX_LENGTH,
+    add_pod_suffix,
+    rand_str,
+)
 from airflow.providers.cncf.kubernetes.pod_generator_deprecated import (
     PodDefaults,
     PodGenerator as PodGeneratorDeprecated,
@@ -380,11 +384,11 @@ def construct_pod(
             - executor_config
             - dynamic arguments
         """
-        if len(pod_id) > 253:
+        if len(pod_id) > POD_NAME_MAX_LENGTH:
             warnings.warn(
-                "pod_id supplied is longer than 253 characters; truncating and adding unique suffix."
+                f"pod_id supplied is longer than {POD_NAME_MAX_LENGTH} characters; truncating and adding unique suffix."
             )
-            pod_id = add_pod_suffix(pod_name=pod_id, max_len=253)
+            pod_id = add_pod_suffix(pod_name=pod_id, max_len=POD_NAME_MAX_LENGTH)
         try:
             image = pod_override_object.spec.containers[0].image  # type: ignore
             if not image:


Test patch:
diff --git a/tests/providers/cncf/kubernetes/operators/test_pod.py b/tests/providers/cncf/kubernetes/operators/test_pod.py
--- a/tests/providers/cncf/kubernetes/operators/test_pod.py
+++ b/tests/providers/cncf/kubernetes/operators/test_pod.py
@@ -1347,7 +1347,7 @@ def test_task_id_as_name_with_suffix_very_long(self):
         pod = k.build_pod_request_obj({})
         assert (
             re.match(
-                r"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-[a-z0-9]{8}",
+                r"a{54}-[a-z0-9]{8}",
                 pod.metadata.name,
             )
             is not None
diff --git a/tests/providers/cncf/kubernetes/test_kubernetes_helper_functions.py b/tests/providers/cncf/kubernetes/test_kubernetes_helper_functions.py
--- a/tests/providers/cncf/kubernetes/test_kubernetes_helper_functions.py
+++ b/tests/providers/cncf/kubernetes/test_kubernetes_helper_functions.py
@@ -87,14 +87,14 @@ def test_create_pod_id_dag_and_task(self, dag_id, task_id, expected, create_pod_
 
     def test_create_pod_id_dag_too_long_with_suffix(self, create_pod_id):
         actual = create_pod_id("0" * 254)
-        assert len(actual) == 80
-        assert re.match(r"0{71}-[a-z0-9]{8}", actual)
+        assert len(actual) == 63
+        assert re.match(r"0{54}-[a-z0-9]{8}", actual)
         assert re.match(pod_name_regex, actual)
 
     def test_create_pod_id_dag_too_long_non_unique(self, create_pod_id):
         actual = create_pod_id("0" * 254, unique=False)
-        assert len(actual) == 80
-        assert re.match(r"0{80}", actual)
+        assert len(actual) == 63
+        assert re.match(r"0{63}", actual)
         assert re.match(pod_name_regex, actual)
 
     @pytest.mark.parametrize("unique", [True, False])
diff --git a/tests/providers/cncf/kubernetes/test_pod_generator.py b/tests/providers/cncf/kubernetes/test_pod_generator.py
--- a/tests/providers/cncf/kubernetes/test_pod_generator.py
+++ b/tests/providers/cncf/kubernetes/test_pod_generator.py
@@ -572,7 +572,7 @@ def test_ensure_max_identifier_length(self, mock_rand_str):
             base_worker_pod=worker_config,
         )
 
-        assert result.metadata.name == "a" * 244 + "-" + self.rand_str
+        assert result.metadata.name == "a" * 54 + "-" + self.rand_str
         for v in result.metadata.labels.values():
             assert len(v) <= 63