Repository: kubernetes/kubernetes (language: go)

Issue:
HostAliases for static pods do not take effect when the node network is not yet connected
### What happened?

When the node network is not yet connected, start kubelet and create static pod like this 
```
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  name: kube-vip
  namespace: kube-system
spec:
  containers:
  - args:
    - manager
    env:
    - name: cp_enable
      value: "true"
    - name: vip_interface
      value: ens4
    - name: address
      value: 10.255.9.11
    - name: port
      value: "6443"
    - name: vip_arp
      value: "true"
    - name: vip_leaderelection
      value: "true"
    - name: vip_leaseduration
      value: "5"
    - name: vip_renewdeadline
      value: "3"
    - name: vip_retryperiod
      value: "1"
    image: 10.255.128.144/sks/kube-vip/kube-vip:v0.5.12
    imagePullPolicy: IfNotPresent
    name: kube-vip
    resources: {}
    securityContext:
      capabilities:
        add:
        - NET_ADMIN
        - NET_RAW
    volumeMounts:
    - mountPath: /etc/kubernetes/admin.conf
      name: kubeconfig
    - mountPath: /etc/localtime
      name: localtime
      readOnly: true
  hostAliases:
  - hostnames:
    - kubernetes
    ip: 127.0.0.1
  hostNetwork: true
  volumes:
  - hostPath:
      path: /etc/kubernetes/admin.conf
      type: FileOrCreate
    name: kubeconfig
  - hostPath:
      path: /etc/localtime
      type: ""
    name: localtime
status: {}
```
The created container ` /etc/hosts` file does not contain the configuration of hostAliases.
```
::1         ipv6-localhost ipv6-loopback
127.0.0.1   localhost
127.0.0.1   hw-sks-test--1-24-16-controlplane-f4zt8
```

### What did you expect to happen?

The created container ` /etc/hosts` file should contain the configuration of hostAliases.

### How can we reproduce it (as minimally and precisely as possible)?

1. Stop kubelet service.
2. Create static pod with configuration of hostAliases.
3. disconnet network, like `nmcli device disconnect ens4`.
4. Start kubelet service.
5. The created container ` /etc/hosts` file does not contain the configuration of hostAliases.

### Anything else we need to know?

_No response_

### Kubernetes version

<details>

```console
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.17", GitCommit:"953be8927218ec8067e1af2641e540238ffd7576", GitTreeState:"clean", BuildDate:"2023-02-22T13:34:27Z", GoVersion:"go1.19.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.17", GitCommit:"953be8927218ec8067e1af2641e540238ffd7576", GitTreeState:"clean", BuildDate:"2023-02-22T13:27:46Z", GoVersion:"go1.19.6", Compiler:"gc", Platform:"linux/amd64"}
```

</details>


### Cloud provider

<details>

</details>


### OS version

<details>

```console
# On Linux:
$ cat /etc/os-release
NAME="Rocky Linux"
VERSION="8.7 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.7"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.7 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.7"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.7"
$ uname -a
4.18.0-425.3.1.el8.x86_64

```

</details>


### Install tools

<details>
kubeadm
</details>


### Container runtime (CRI) and version (if applicable)

<details>
containerd containerd.io 1.6.4 212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>

</details>



Discussion / hints:
/sig node
> GitVersion:"v1.23.17"

1.23 is EOL please check if this is happening in a supported release, latest supported is 1.25 https://kubernetes.io/releases/
You might want to set up a dependency on network from kubelet (assuming you're using systemd), so that it doesn't start until network is ready.

Also, could you reproduce this on a later version (eg. 1.27)?

/triage needs-information
The kubelet systemd configuration file already declares a network-online target dependency, but it doesn't seem to work.

```
[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=https://kubernetes.io/docs/
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=multi-user.target
```
https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/
Looks like additional dependencies are required such as NetworkManager-wait-online.service.  I think the best way is that kubelet is compatible with this scenario.

Also, I also tested on 1.27 and the problem is still reproducible.

```
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.2", GitCommit:"7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647", GitTreeState:"clean", BuildDate:"2023-05-17T14:20:07Z", GoVersion:"go1.20.4", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.2", GitCommit:"7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647", GitTreeState:"clean", BuildDate:"2023-05-17T14:13:28Z", GoVersion:"go1.20.4", Compiler:"gc", Platform:"linux/amd64"}
```
Is there any progress in dealing with this issue? @mmiranda96 
@Levi080513  I haven't been working on this, not sure if anyone is actively looking into it.
> Looks like additional dependencies are required such as NetworkManager-wait-online.service. I think the best way is that kubelet is compatible with this scenario.

@Levi080513 did you try adding this dependency? Did it work?
@mmiranda96 NetworkManager-wait-online.service only waits for 60s. If the network is still abnormal after 60s, the problem will still occur.
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity, `lifecycle/stale` is applied
- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied
- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed

You can:
- Mark this issue as fresh with `/remove-lifecycle stale`
- Close this issue with `/close`
- Offer to help out with [Issue Triage][1]

Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community).

/lifecycle stale

[1]: https://www.kubernetes.dev/docs/guide/issue-triage/
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity, `lifecycle/stale` is applied
- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied
- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed

You can:
- Mark this issue as fresh with `/remove-lifecycle rotten`
- Close this issue with `/close`
- Offer to help out with [Issue Triage][1]

Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community).

/lifecycle rotten

[1]: https://www.kubernetes.dev/docs/guide/issue-triage/
/remove-lifecycle rotten
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity, `lifecycle/stale` is applied
- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied
- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed

You can:
- Mark this issue as fresh with `/remove-lifecycle stale`
- Close this issue with `/close`
- Offer to help out with [Issue Triage][1]

Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community).

/lifecycle stale

[1]: https://www.kubernetes.dev/docs/guide/issue-triage/
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity, `lifecycle/stale` is applied
- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied
- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed

You can:
- Mark this issue as fresh with `/remove-lifecycle rotten`
- Close this issue with `/close`
- Offer to help out with [Issue Triage][1]

Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community).

/lifecycle rotten

[1]: https://www.kubernetes.dev/docs/guide/issue-triage/
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:
- After 90d of inactivity, `lifecycle/stale` is applied
- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied
- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed

You can:
- Reopen this issue with `/reopen`
- Mark this issue as fresh with `/remove-lifecycle rotten`
- Offer to help out with [Issue Triage][1]

Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community).

/close not-planned

[1]: https://www.kubernetes.dev/docs/guide/issue-triage/
@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

<details>

In response to [this](https://github.com/kubernetes/kubernetes/issues/119852#issuecomment-2255524333):

>The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
>
>This bot triages issues according to the following rules:
>- After 90d of inactivity, `lifecycle/stale` is applied
>- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied
>- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed
>
>You can:
>- Reopen this issue with `/reopen`
>- Mark this issue as fresh with `/remove-lifecycle rotten`
>- Offer to help out with [Issue Triage][1]
>
>Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community).
>
>/close not-planned
>
>[1]: https://www.kubernetes.dev/docs/guide/issue-triage/


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>
/reopen
/remove-lifecycle rotten
@Levi080513: Reopened this issue.

<details>

In response to [this](https://github.com/kubernetes/kubernetes/issues/119852#issuecomment-2257408985):

>/reopen
>/remove-lifecycle rotten


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.
</details>
/sig network

The kubelet wait for the PodIPs to be present to mount the etc hosts files

https://github.com/kubernetes/kubernetes/blob/17d7d286201ae4b8e60014d8d0c41792d741d969/pkg/kubelet/kubelet_pods.go#L243-L256

however, it does not check if the pod is host network.

If the Pod is using hostNetwork, then it seems the PodIPs are ignored

https://github.com/kubernetes/kubernetes/blob/17d7d286201ae4b8e60014d8d0c41792d741d969/pkg/kubelet/kubelet_pods.go#L449-L473

so it seems legit to mount the aliases independently of PodIPs if the pod is using host network

/assign
/sig network
/triage accepted



Fix patch:
diff --git a/pkg/kubelet/kubelet_pods.go b/pkg/kubelet/kubelet_pods.go
index 7d24a90b0e702..96a51c08d2c54 100644
--- a/pkg/kubelet/kubelet_pods.go
+++ b/pkg/kubelet/kubelet_pods.go
@@ -245,10 +245,10 @@ func (kl *Kubelet) makeBlockVolumes(pod *v1.Pod, container *v1.Container, podVol
 // - container is not an infrastructure (pause) container
 // - container is not already mounting on /etc/hosts
 // Kubernetes will not mount /etc/hosts if:
-// - when the Pod sandbox is being created, its IP is still unknown. Hence, PodIP will not have been set.
-// - Windows pod contains a hostProcess container
+// - the Pod is on the pod network and PodIP has not yet been set (e.g., Pod sandbox is being created).
+// - the Pod is on Windows, and contains a hostProcess container.
 func shouldMountHostsFile(pod *v1.Pod, podIPs []string) bool {
-	shouldMount := len(podIPs) > 0
+	shouldMount := len(podIPs) > 0 || pod.Spec.HostNetwork
 	if runtime.GOOS == "windows" {
 		return shouldMount && !kubecontainer.HasWindowsHostProcessContainer(pod)
 	}


Test patch:
diff --git a/pkg/kubelet/kubelet_pods_linux_test.go b/pkg/kubelet/kubelet_pods_linux_test.go
index affa736aa1c90..798d8e167b204 100644
--- a/pkg/kubelet/kubelet_pods_linux_test.go
+++ b/pkg/kubelet/kubelet_pods_linux_test.go
@@ -24,6 +24,8 @@ import (
 
 	"github.com/stretchr/testify/assert"
 	v1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/utils/ptr"
 
 	runtimeapi "k8s.io/cri-api/pkg/apis/runtime/v1"
 	_ "k8s.io/kubernetes/pkg/apis/core/install"
@@ -271,6 +273,104 @@ func TestMakeMounts(t *testing.T) {
 	}
 }
 
+func TestMakeMountsEtcHostsFile(t *testing.T) {
+	testPod := &v1.Pod{
+		ObjectMeta: metav1.ObjectMeta{
+			Namespace:   "test-ns",
+			Name:        "test-pod-name",
+			Annotations: map[string]string{},
+		},
+		Spec: v1.PodSpec{
+			EnableServiceLinks: ptr.To(false), // to avoid errors with kl.makeEnvironmentVariables
+		},
+	}
+	testContainer := v1.Container{
+		Name:  "test-container",
+		Image: "img",
+	}
+	tests := []struct {
+		name        string
+		podFn       func(pod *v1.Pod)
+		containerFn func(container *v1.Container)
+		podVolumes  kubecontainer.VolumeMap
+		podIPs      []string
+		want        bool // check if mounts the /etc/host file
+		wantErr     bool
+	}{
+		{
+			name:   "pod with network",
+			podIPs: []string{"192.168.0.1"},
+			want:   true,
+		},
+		{
+			name: "pod with network but no IPs",
+			want: false,
+		},
+		{
+			name: "pod with host network",
+			podFn: func(pod *v1.Pod) {
+				pod.Spec.HostNetwork = true
+			},
+			podIPs: []string{"192.168.0.1"},
+			want:   true,
+		},
+		{
+			name: "pod with host network but no IPs",
+			podFn: func(pod *v1.Pod) {
+				pod.Spec.HostNetwork = true
+			},
+			want: true,
+		},
+		{
+			name: "pod with network already mounting the hosts file",
+			containerFn: func(container *v1.Container) {
+				container.VolumeMounts = []v1.VolumeMount{
+					{
+						Name:      "etchosts",
+						MountPath: etcHostsPath,
+					},
+				}
+			},
+			podVolumes: kubecontainer.VolumeMap{
+				"etchosts": kubecontainer.VolumeInfo{Mounter: &stubVolume{path: "/mnt/disk"}},
+			},
+			podIPs: []string{"192.168.0.1"},
+			want:   false,
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			fhu := hostutil.NewFakeHostUtil(nil)
+			fsp := &subpath.FakeSubpath{}
+			pod := testPod.DeepCopy()
+			if tt.podFn != nil {
+				tt.podFn(pod)
+			}
+			container := testContainer.DeepCopy()
+			if tt.containerFn != nil {
+				tt.containerFn(container)
+			}
+
+			mounts, _, err := makeMounts(pod, t.TempDir(), container, "fakepodname", "fakedomain", tt.podIPs, tt.podVolumes, fhu, fsp, nil, false, nil)
+
+			if err != nil {
+				t.Fatalf("unexpected error: %v", err)
+			}
+			got := false
+			for _, mount := range mounts {
+				if mount.Name == "k8s-managed-etc-hosts" {
+					got = true
+					break
+				}
+			}
+
+			if got != tt.want {
+				t.Errorf("/etc/hosts file mounted: got = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}
+
 func TestMakeBlockVolumes(t *testing.T) {
 	testKubelet := newTestKubelet(t, false /* controllerAttachDetachEnabled */)
 	defer testKubelet.Cleanup()