Repository: kubernetes
Issue: After the kube-scheduler is scheduled, the kubelet only starts ADD after a half-hour interval
Labels: sig/scheduling,kind/support,needs-triage

### What happened?

When creating a pod，after the kube-scheduler is scheduled, the kubelet only starts ADD after a half-hour interval
kube-scheduler.log:
`I1115 04:12:00.190703      10 schedule_one.go:286] "Successfully bound pod to node" pod="kube-system/registry-6cc84d4599-d7v2z" node="master1" evaluatedNodes=1 feasibleNodes=1`
kubelet.log:
`I1115 04:47:49.244673  504031 kubelet.go:2430] "SyncLoop ADD" source="api" pods=["kube-system/registry-6cc84d4599-d7v2z"]
`

There are a large number of suspicious logs in kubelet：
```
I1115 04:47:45.496346  504031 trace.go:236] Trace[836595825]: "container_status" containerId:c6c4174db0f85f98c4fae710d3f3a81807db30503dd7a75cc480506faba33879 (15-Nov-2024 04:47:44.439) (total time: 1056ms):
Trace[836595825]: [1.056580909s] [1.056580909s] END
I1115 04:47:45.498660  504031 trace.go:236] Trace[1152628921]: "container_status" containerId:f0020fb44ea49684f6b341b6fe58eae00cb6add89eaa217d46f1c0068bf53719 (15-Nov-2024 04:47:44.444) (total time: 1054ms):
Trace[1152628921]: [1.054541648s] [1.054541648s] END
I1115 04:47:45.500464  504031 trace.go:236] Trace[1799469360]: "container_status" containerId:efe80b266c59d35549d0e52033f80d7eae57733907adaf5b7b045bbd7df2bdb0 (15-Nov-2024 04:47:44.437) (total time: 1062ms):
Trace[1799469360]: [1.062659482s] [1.062659482s] END
I1115 04:47:45.500778  504031 trace.go:236] Trace[2101472811]: "container_status" containerId:f00a1f977b47b0948077d16d3e45e923d96dfa3789728af042e18e9976953377 (15-Nov-2024 04:47:44.447) (total time: 1053ms):
Trace[2101472811]: [1.053375625s] [1.053375625s] END
I1115 04:47:45.648080  504031 manager.go:229] "Device plugin connected" resourceName="resource.sop.huawei.com/fuse"
I1115 04:47:45.728218  504031 trace.go:236] Trace[1018875239]: "Calculate volume metrics of sidecar for pod sop/invmgrservice-77bc956598-s2hw7" (15-Nov-2024 04:47:43.386) (total time: 2341ms):
Trace[1018875239]: [2.341411012s] [2.341411012s] END
I1115 04:47:45.728671  504031 trace.go:236] Trace[2068757664]: "Calculate volume metrics of weave-version for pod manager/weave-0" (15-Nov-2024 04:47:42.023) (total time: 3704ms):
Trace[2068757664]: [3.704755612s] [3.704755612s] END
I1115 04:47:45.729295  504031 trace.go:236] Trace[1909652043]: "Calculate volume metrics of pkg for pod sop/odaepipelinemgrservice-cf46cd978-cfbh6" (15-Nov-2024 04:47:43.725) (total time: 2003ms):
Trace[1909652043]: [2.003296282s] [2.003296282s] END
I1115 04:47:45.808842  504031 trace.go:236] Trace[1375719882]: "Calculate volume metrics of log for pod sop/litecaservice-5676c5776f-shg86" (15-Nov-2024 04:47:44.090) (total time: 1718ms):
Trace[1375719882]: [1.718048823s] [1.718048823s] END
I1115 04:47:45.917400  504031 trace.go:236] Trace[700771020]: "Calculate volume metrics of mypkg for pod sop/cloudsopfmdds-0" (15-Nov-2024 04:47:44.447) (total time: 1469ms):
Trace[700771020]: [1.469530583s] [1.469530583s] END
I1115 04:47:45.917850  504031 trace.go:236] Trace[1173662521]: "Calculate volume metrics of pkg for pod manager/commonfeaturetestservice-c66fdfb5-2s7vt" (15-Nov-2024 04:47:44.448) (total time: 1469ms):
Trace[1173662521]: [1.469245066s] [1.469245066s] END
I1115 04:47:45.939300  504031 trace.go:236] Trace[1820919694]: "image_status" image:registry.caas.local/default/sop_base_image-aarch64:25.550.1660 (15-Nov-2024 04:47:44.736) (total time: 1202ms):
Trace[1820919694]: [1.202447499s] [1.202447499s] END
I1115 04:47:46.009009  504031 trace.go:236] Trace[1413003384]: "podsandbox_status" podSandboxId:3e610ebb799daf9f056931b5138e8fd9956efad36b808b9d0a64dde4af2f35cd (15-Nov-2024 04:47:44.441) (total time: 1567ms):
Trace[1413003384]: [1.567586292s] [1.567586292s] END
I1115 04:47:46.022472  504031 trace.go:236] Trace[853699448]: "container_status" containerId:44d27b1d406fb41a25f89d7e78915e5a1b8e1fdba687263907edb772a7dec413 (15-Nov-2024 04:47:44.446) (total time: 1576ms):
Trace[853699448]: [1.576327467s] [1.576327467s] END
I1115 04:47:46.024482  504031 trace.go:236] Trace[1051222000]: "container_status" containerId:ec6907e308677afe48fdf462890686bbb3aeea7ac545b401d7b55a34ebb1c0ed (15-Nov-2024 04:47:44.447) (total time: 1576ms):
Trace[1051222000]: [1.57649245s] [1.57649245s] END
I1115 04:47:46.024723  504031 trace.go:236] Trace[729411205]: "container_status" containerId:a375d2e8092d921fee7974faf450b2aa87fecda0a04b84a975ba209dc8e35842 (15-Nov-2024 04:47:44.447) (total time: 1577ms):
Trace[729411205]: [1.577505491s] [1.577505491s] END
I1115 04:47:46.024943  504031 trace.go:236] Trace[2047128525]: "container_status" containerId:1a4c4aa97ace171ce07842f077bf2215bb5d5275a2f3d5a327d0179b7dbec140 (15-Nov-2024 04:47:44.448) (total time: 1576ms):
Trace[2047128525]: [1.576844697s] [1.576844697s] END
I1115 04:47:46.045702  504031 trace.go:236] Trace[1650587259]: "podsandbox_status" podSandboxId:d5112ea26fbcc0c290f47378e8c5a9821926c48013bafa279afbe890bc921307 (15-Nov-2024 04:47:44.446) (total time: 1598ms):
Trace[1650587259]: [1.598921141s] [1.598921141s] END
I1115 04:47:46.046134  504031 trace.go:236] Trace[512891868]: "container_status" containerId:fc2660fd77348f40b2ac2d1ce733e49350851a2d9cd71d11a0af26286b079dd9 (15-Nov-2024 04:47:44.446) (total time: 1599ms):
Trace[512891868]: [1.599562164s] [1.599562164s] END
I1115 04:47:46.046385  504031 trace.go:236] Trace[125258435]: "list_containers" filter.id:,filter.podSandboxId: (15-Nov-2024 04:47:44.441) (total time: 1605ms):
Trace[125258435]: [1.60535456s] [1.60535456s] END
I1115 04:47:46.046553  504031 trace.go:236] Trace[465435416]: "list_containers" filter.id:,filter.podSandboxId: (15-Nov-2024 04:47:44.447) (total time: 1599ms):
Trace[465435416]: [1.599515223s] [1.599515223s] END
I1115 04:47:46.046716  504031 trace.go:236] Trace[1559500248]: "list_containers" filter.id:,filter.podSandboxId: (15-Nov-2024 04:47:44.445) (total time: 1600ms):
Trace[1559500248]: [1.600790269s] [1.600790269s] END
I1115 04:47:46.046845  504031 trace.go:236] Trace[436961003]: "list_containers" filter.id:,filter.podSandboxId: (15-Nov-2024 04:47:44.447) (total time: 1599ms):
Trace[436961003]: [1.599288828s] [1.599288828s] END
I1115 04:47:46.046969  504031 trace.go:236] Trace[1952747957]: "exec_sync" containerId:26f9bd6bd9764d585e0f492827ae16f28f3c3ed668fb514305501eacfffad827,cmd:[curl --unix-socket /tmp/health.sock http://unix/healthz],timeou
t:10s (15-Nov-2024 04:47:43.311) (total time: 2735ms):
Trace[1952747957]: [2.735205761s] [2.735205761s] END
I1115 04:47:46.092320  504031 prober.go:107] "Probe failed" probeType="Readiness" pod="manager/test3th1kafkaservice-kafka-0" podUID="7d9f9635-db0a-4e03-b59d-4e9e4ee3cee2" containerName="test3th1kafkaservice-kafka" probeR
esult="failure" output="dial tcp 172.18.138.16:8002: connect: connection refused"
I1115 04:47:46.229808  504031 prober.go:107] "Probe failed" probeType="Liveness" pod="manager/cronserviceinfra-849db64866-7cmgm" podUID="30bbc033-fd40-455b-a1f4-2c49790110c9" containerName="cronserviceinfra" probeResult=
"failure" output="dial tcp 172.18.138.0:8001: i/o timeout"
```

```

```

```

```

### What did you expect to happen?

After scheduling, kubelet will be added normally

### How can we reproduce it (as minimally and precisely as possible)?

What factors cause the above reasons and how to solve them

### Anything else we need to know?

_No response_

### Kubernetes version

<details>

```console
$ kubectl version
# paste output here
```
v1.28.1
</details>


### Cloud provider

<details>
na
</details>


### OS version

<details>

```console
# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here
```

</details>


### Install tools

<details>

</details>


### Container runtime (CRI) and version (if applicable)

<details>

</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>

</details>