Repository: kubernetes
Issue: wrong container_memory_working_set_bytes for DinD container on cgroupv2 only nodes
Labels: kind/bug,sig/node,triage/accepted

### What happened?

Let's assume you have: 
- cgroupv2 only OS (e.g. Ubuntu 22.04 (Azure AKS) or [gardenlinux 934.7](https://github.com/gardenlinux/gardenlinux/releases/tag/934.7), see below)
- A running Kubernetes cluster on that OS (e.g. K8s  v1.26.5 or 1.27.3)
- Create a DinD pod with memory limits (e.g. [docker:24.0.5-dind-alpine3.18](https://hub.docker.com/layers/library/docker/24.0.5-dind-alpine3.18/images/sha256-fe74d395f3a15eb2233848d2ac47fd076aa6fe7c17dd9b4ca13145116893f2f5?context=explore))

Then start some container inside the dind pod and consume e.g. 400 MB memory:
```
kubectl exec -it dind -- sh
docker run -it ubuntu bash
cat /dev/zero | head -c 400000000 | tail
```

Wait 5 minutes and then check k9s. Then the 400 MB memory consumption is not visible. 
Same is true for `kubectl get --raw "/api/v1/nodes/NODE_NAME/proxy/metrics/resource"`

### What did you expect to happen?

container_memory_working_set_bytes{container="dind",namespace="default",pod="dind"} should show the 400 MB
Metrics server gets the right values, which is shown in `top` or in `docker stats`:
![image](https://github.com/kubernetes-sigs/metrics-server/assets/70013998/aa6a7a07-5986-4f69-8fc2-8cde4abcd9d0)

on cgroupv1 enabled OS the CPU/memory are shown correctly (e.g. Ubuntu 20.xx or gardenlinux 5xx.yy)

### How can we reproduce it (as minimally and precisely as possible)?

cgroupv2 only node OS, can be checked via 
``` 
mount | grep cgroup2
```
should return on the node: `cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)`


example  DinD-pod.yaml:
```
apiVersion: v1
kind: Pod
metadata:
    name: dind
spec:
    containers:
      - name: dind
        image: docker:dind
        resources:
            requests:
                cpu: 10m
                memory: 700Mi
            limits:
                cpu: 2
                memory: 700Mi
        securityContext:
            privileged: true
            allowPrivilegeEscalation: true
        volumeMounts:
          - name: docker-graph-storage
            mountPath: /var/lib/docker
    volumes:
      - name: docker-graph-storage
        emptyDir: {}     
```

### Anything else we need to know?

https://github.com/kubernetes-sigs/metrics-server/issues/1301 brings me here.

This DinD setup is used in Github Runners: https://github.com/actions/actions-runner-controller/issues/2284

See also https://github.com/moby/moby/issues/45378

### Kubernetes version

<details>

Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.4", GitCommit:"fa3d7990104d7c1f16943a67f11b154b71f6a132", GitTreeState:"clean", BuildDate:"2023-07-19T12:20:54Z", GoVersion:"go1.20.6", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.3", GitCommit:"25b4e43193bcda6c7328a6d147b1fb73a33f1598", GitTreeState:"clean", BuildDate:"2023-06-19T16:10:31Z", GoVersion:"go1.20.5", Compiler:"gc", Platform:"linux/amd64"}

</details>


### Cloud provider

<details>

</details>


### OS version

<details>
https://learn.microsoft.com/en-us/azure/aks/supported-kubernetes-versions?tabs=azure-cli#aks-components-breaking-changes-by-version


Azure policy 1.0.1Metrics-Server 0.6.3KEDA 2.10.0Open Service Mesh 1.2.3Core DNS V1.9.40.12.0Overlay VPA 0.11.0Azure-Keyvault-SecretsProvider 1.4.1Ingress AppGateway 1.2.1Eraser v1.1.1Azure Workload Identity V1.1.1ASC Defender 1.0.56Azure Active Directory Pod Identity 1.8.13.6GitOps 1.7.0KMS 0.5.0 | Cilium 1.12.8CNI 1.4.44Cluster Autoscaler 1.8.5.3 | 


OS Image Ubuntu 22.04 Cgroups V2ContainerD 1.7 for Linux and 1.6 for Windows

</details>


### Install tools

<details>

</details>


### Container runtime (CRI) and version (if applicable)

<details>

</details>


### Related plugins (CNI, CSI, ...) and versions (if applicable)

<details>

</details>