Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: With CentOS 7.6, the kubelet systemd service may failed, because the numbers of NUMA nodes it reports is different from the actual number of nodes. the kubelet log is: Sep 25 10:26:37 node50 systemd: Started Kubernetes systemd probe. Sep 25 10:26:37 node50 kubelet: I0925 10:26:37. ] Version: v1.16.0 Sep 25 10:26:37 node50 kubelet: I0925 10:26:37. ] No cloud provider specified. Sep 25 10:26:37 node50 kubelet: I0925 10:26:37. ] Client rotation is on, will bootstrap in background Sep 25 10:26:37 node50 kubelet: I0925 10:26:37. ] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-". Sep 25 10:26:37 node50 kubelet: I0925 10:26:37. ] --cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to / Sep 25 10:26:37 node50 kubelet: F0925 10:26:37. ] failed to run Kubelet: open /sys/devices/system/node/node2/cpulist: no such file or directory Sep 25 10:26:37 node50 systemd: kubelet.service: main process exited, code=exited, status=255/n/a Sep 25 10:26:37 node50 systemd: Unit kubelet.service entered failed state. What you expected to happen: But this node has only 2 NUMA nodes, node0 and node1 [root ~]# lscpu Architecture: x8664 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 72 On-line CPU(s) list: 0-71 Thread(s) per core: 2 Core(s) per socket: 18 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) Gold 6240Y CPU @ 2.60GHz Stepping: 7 CPU MHz: 1000.073 CPU max MHz: 3900.0000 CPU min MHz: 1000.0000 BogoMIPS: 5200.00 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 1024K L3 cache: 25344K NUMA node0 CPU(s): 0-17,36-53 NUMA node1 CPU(s): 18-35,54-71 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 ...... How to reproduce it (as minimally and precisely as possible): With Intel Xeon CLX-SP 2 sockets CPUs, in CentOS 7.6, sometimes the sysfs /sys/devices/system/node/possible reports more than actual CPU counts. And the topology in cpumanager read this file, if it reads 4 CPUs, but there are only 2 CPUs, then kubelet service will failed. [root ~]# cat /sys/devices/system/node/possible 0-3 [root ~]# cat /sys/devices/system/node/online 0-1 Anything else we need to know?: Suggest func GetNUMANodeInfo() read /sys/devices/system/node/online instead of /sys/devices/system/node/possible Environment: Kubernetes version (use ): [root ~]# kubectl version Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: 2x socket Intel Xeon Gold 6240Y OS (e.g: ): [root ~]# cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" IDLIKE="rhel fedora" VERSIONID="7" PRETTYNAME="CentOS Linux 7 (Core)" ANSICOLOR="0;31" CPENAME="cpe:/o:centos:centos:7" HOMEURL="" BUGREPORTURL="" CENTOSMANTISBTPROJECT="CentOS-7" CENTOSMANTISBTPROJECTVERSION="7" REDHATSUPPORTPRODUCT="centos" REDHATSUPPORTPRODUCTVERSION="7" Kernel (e.g. ): [root ~]# uname -a Linux node50 3.10.0-957.el7.x8664 SMP Thu Nov 8 23:39:32 UTC 2018 x8664 x8664 x8664 GNU/Linux Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig node

Discussion:
// nil NUMANodeInfo, indicating that no NUMA information is available // on this machine. This should implicitly be interpreted as having a // single NUMA node with id 0 for all CPUs. <del> nodelist, err := ioutil.ReadFile("/sys/devices/system/node/possible") </del> <ins> nodelist, err := ioutil.ReadFile("/sys/devices/system/node/online") </ins> if err != nil { return nil, nil }