Code change:
This is a Feature Request Port this: to the migration guide documentation: as a known issue. Note, CRI-O behavior may be different.
/sig node
/language en
Is the ask here to port the Google doc into the Kubernetes docs or just link to the Google documentation (which isn't necessarily allowed)?
We can't really do either, unless Google licenses to the CNCF. We could write new docs to highlight the same known issue (what's the issue? - I deliberated haven't looked).
Here is the GH issue: These metrics are missing in endpoint with Containerd (they were present with Dockershim): Also the format of container name dimension of many metrics was with dockershim, but the format on the containerd node is . This is coming from this file: that even has an internesting note that this is implementaiton details, but changing it will likely break somethings (btw, maybe this may be noted as well in the list of known issues). So workaround for both issues (missing metrics and the name of metric dimension) is to use a standalone cAdvisor, and use one of the releases with the suffix "-containerd-cri"
That sounds quite involved - should we open a related issue to track explaining how folks would do that?
Do we have a document that can be followed for using a standalone cadvisor? I checked the tags in cadvisor and turns out 2 versions have containerd as suffix but none have the "-containerd-cri" suffix. Ex: v0.41.0-containerd-fix-rc.2 Does the latest version v0.44.0 contain the fix?
can you please help here?
The fix was in I think but not sure which releases contain it
So what do we need to do next, Align versions?
/assign
Yeah I'm not sure how to tell which version has the changes. It's for sure in 0.43.x but the current "Latest" tag is 0.39.x and doesn't seem to have the fix. I think we should wait for or to verify.
It looks like updates the page's docker-specific information related to cAdvisor to be more generic to container runtimes. Would that be enough to address this issue or does more runtime-specific information need to be ?
what's your opinion in respect of ?
It's due to it being reverted in which is a real bummer for those of us using GKE and the default for new node pools is containerd.
/triage accepted
What should the next steps on this issue look like?
my thoughts: This doesn't belong on the suggested page because it's specific to containerd. I'd say let's update as follows: In Before you begin, add a line to "Read the known issues section" Add a section at the end, before What's Next, called "Known issues after migrating" Add "Versions of containerd prior to 0.45.0 didn't collect some metrics in the endpoint. For a list of missing metrics, refer to . To collect these metrics, install a containerd package that is version 0.45.0 or later that has the suffix." In , mention "To avoid the missing metrics in [Known issues after migrating](#known-issues-after-migrating), install a containerd package that is version 0.45.0 or later that has the suffix."
According to the comment at the only current work-around for containerd users is to deploy and scrape a standalone cadvisor daemonset, built from the branch. The stats exposed by the kubelet will be broken until 1.25 or 1.26 when is complete. Since it is the kubelet's cadvisor that is broken (and I would argue that it is broken/regressed by this issue), it should be covered in the kubernetes docs.
Is there a more involved process then than just installing the package?
it's not a package, it's a branch of the cadvisor project: I'm not sure if they're publishing binaries or images from that branch. A sample DaemonSet is available, which could be modified to use :
Could we say: "To collect these metrics, . Set the $VERSION variable to any cAdvisor version that has the suffix, such as " ?
I think that needs more work, for example the default DaemonSet only mounts the docker socket path; it does not mount the containerd socket, and even when I modify it to do so the metrics appear to be missing most of the important labels. Perhaps has some suggestions.
This feels like it would need to be its own task page, possibly linked from our existing Metrics doc as well
I've put a ready to run temporary solution here
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale

Related discussion:
text: Your container runtime must support at least v1alpha2 of the container runtime interface. <del> Kubernetes {{< skew currentVersion >}}  defaults to using v1 of the CRI API. If a container runtime does not support the v1 API, the kubelet falls back to </del> <ins> Kubernetes [starting v1.26](/blog/2022/11/18/upcoming-changes-in-kubernetes-1-26/#cri-api-removal) _only works_ with v1 of the CRI API. Earlier versions default to v1 version, however if a container runtime does not support the v1 API, the kubelet falls back to </ins> using the (deprecated) v1alpha2 API instead. ## Container runtimes