Code change:
This is a Feature Request Port this: to the migration guide documentation: as a known issue. Note, CRI-O behavior may be different.
/sig node
/language en
Is the ask here to port the Google doc into the Kubernetes docs or just link to the Google documentation (which isn't necessarily allowed)?
We can't really do either, unless Google licenses to the CNCF. We could write new docs to highlight the same known issue (what's the issue? - I deliberated haven't looked).
Here is the GH issue: These metrics are missing in endpoint with Containerd (they were present with Dockershim): Also the format of container name dimension of many metrics was with dockershim, but the format on the containerd node is . This is coming from this file: that even has an internesting note that this is implementaiton details, but changing it will likely break somethings (btw, maybe this may be noted as well in the list of known issues). So workaround for both issues (missing metrics and the name of metric dimension) is to use a standalone cAdvisor, and use one of the releases with the suffix "-containerd-cri"
That sounds quite involved - should we open a related issue to track explaining how folks would do that?
Do we have a document that can be followed for using a standalone cadvisor? I checked the tags in cadvisor and turns out 2 versions have containerd as suffix but none have the "-containerd-cri" suffix. Ex: v0.41.0-containerd-fix-rc.2 Does the latest version v0.44.0 contain the fix?
can you please help here?
The fix was in I think but not sure which releases contain it
So what do we need to do next, Align versions?
/assign
Yeah I'm not sure how to tell which version has the changes. It's for sure in 0.43.x but the current "Latest" tag is 0.39.x and doesn't seem to have the fix. I think we should wait for or to verify.
It looks like updates the page's docker-specific information related to cAdvisor to be more generic to container runtimes. Would that be enough to address this issue or does more runtime-specific information need to be ?
what's your opinion in respect of ?
It's due to it being reverted in which is a real bummer for those of us using GKE and the default for new node pools is containerd.
/triage accepted
What should the next steps on this issue look like?
my thoughts: This doesn't belong on the suggested page because it's specific to containerd. I'd say let's update as follows: In Before you begin, add a line to "Read the known issues section" Add a section at the end, before What's Next, called "Known issues after migrating" Add "Versions of containerd prior to 0.45.0 didn't collect some metrics in the endpoint. For a list of missing metrics, refer to . To collect these metrics, install a containerd package that is version 0.45.0 or later that has the suffix." In , mention "To avoid the missing metrics in [Known issues after migrating](#known-issues-after-migrating), install a containerd package that is version 0.45.0 or later that has the suffix."
According to the comment at the only current work-around for containerd users is to deploy and scrape a standalone cadvisor daemonset, built from the branch. The stats exposed by the kubelet will be broken until 1.25 or 1.26 when is complete. Since it is the kubelet's cadvisor that is broken (and I would argue that it is broken/regressed by this issue), it should be covered in the kubernetes docs.
Is there a more involved process then than just installing the package?
it's not a package, it's a branch of the cadvisor project: I'm not sure if they're publishing binaries or images from that branch. A sample DaemonSet is available, which could be modified to use :
Could we say: "To collect these metrics, . Set the $VERSION variable to any cAdvisor version that has the suffix, such as " ?
I think that needs more work, for example the default DaemonSet only mounts the docker socket path; it does not mount the containerd socket, and even when I modify it to do so the metrics appear to be missing most of the important labels. Perhaps has some suggestions.
This feels like it would need to be its own task page, possibly linked from our existing Metrics doc as well
I've put a ready to run temporary solution here
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale

Related discussion:
text: If you use Docker Engine via dockershim as your container runtime and wish to upgrade to v1.24, it is recommended that you either migrate to another runtime or find an alternative means to obtain Docker Engine support. Check out the [container runtimes](/docs/setup/production-environment/container-runtimes/) <del> section to know your options. Make sure to </del> <ins> section to know your options. The version of Kubernetes with dockershim (1.23) is out of support and the v1.24 will run out of support [soon](/releases/#release-v1-24). Make sure to </ins> [report issues](https://github.com/kubernetes/kubernetes/issues) you encountered with the migration so the issues can be fixed in a timely manner and your cluster would be <del> ready for dockershim removal. </del> <ins> ready for dockershim removal. After v1.24 running out of support, you will need to contact your Kubernetes provider for support or upgrade multiple versions at a time if there are critical issues affecting your cluster. </ins> Your cluster might have more than one kind of node, although this is not a common configuration.