Repository: kubernetes/kube-state-metrics

Issue #2453: kube-state-metrics doesn't produce metric for Custom Resources
Created: 2024-07-24T12:56:55Z
What happened:
I defined  configuration for Custom Resource State Metrics for my three CRD's (ReplicationGroup, ClusterInstance, RDSInstance)
After deploying it to cluster where I have kubernetes objects of all above types kube-state-metric is producing me expected crossplane_engine_version metric
But when I deploy same configuration to the cluster where I have only objects of RDSInstance type, crossplane_engine_version metric is missing
What you expected to happen:
Metric should be produced by kube-state-metric even if there are no real Objects of one type/kind listed in configuration below
How to reproduce it (as minimally and precisely as possible):
apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-state-metrics-crd-config
  namespace: crossplane-system
data:
  crd-config.yaml: |
    kind: CustomResourceStateMetrics
    spec:
      resources:
        - groupVersionKind:
            group: cache.aws.crossplane.io
            version: "*"
            kind: ReplicationGroup
          metricNamePrefix: crossplane
          labelsFromPath:
            managedResourceKind: [kind]
            claimKind: [metadata, ownerReferences, "[controller=true]", kind]
            cllaimName: [metadata, labels, crossplane.io/claim-name]
            claimNamespace: [metadata, labels, crossplane.io/claim-namespace]
          metrics:
            - name: "engine_version"
              help: "Engine Version"
              each:
                type: Info
                info:
                  labelsFromPath:
                    engineVersion: [spec, forProvider, engineVersion]
              commonLabels:
                custom_metric: "yes"
        - groupVersionKind:
            group: rds.aws.upbound.io
            version: "*"
            kind: ClusterInstance
          metricNamePrefix: crossplane
          labelsFromPath:
            managedResourceKind: [kind]
            claimKind: [metadata, ownerReferences, "[controller=true]", kind]
            cllaimName: [metadata, labels, crossplane.io/claim-name]
            claimNamespace: [metadata, labels, crossplane.io/claim-namespace]
          metrics:
            - name: "engine_version"
              help: "Engine Version"
              each:
                type: Info
                info:
                  labelsFromPath:
                    engineVersion: [status, atProvider, engineVersion]
              commonLabels:
                custom_metric: "yes"
        - groupVersionKind:
            group: database.aws.crossplane.io
            version: "*"
            kind: RDSInstance
          metricNamePrefix: crossplane
          labelsFromPath:
            managedResourceKind: [kind]
            claimKind: [metadata, ownerReferences, "[controller=true]", kind]
            cllaimName: [metadata, labels, crossplane.io/claim-name]
            claimNamespace: [metadata, labels, crossplane.io/claim-namespace]
          metrics:
            - name: "engine_version"
              help: "Engine Version"
              each:
                type: Info
                info:
                  labelsFromPath:
                    engineVersion: [spec, forProvider, engineVersion]
              commonLabels:
                custom_metric: "yes"
Environment:

kube-state-metrics version: 2.12.0
Kubernetes version: v1.28.9
Cloud provider: AWS
Other info:

Duplicate issue #2384: Custom resource state metrics wildcard not working
Created: 2024-04-30T11:11:00Z
What happened:
Metrics from custom resource is not produced when wildcard is used in kind field.
What you expected to happen:
The wildcard should produce metrics for all kinds.
How to reproduce it (as minimally and precisely as possible):
I have the following helm chart values file
rbac:
  extraRules:
    - apiGroups: ["azure.tnnova.io"]
      resources: ["*"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["apiextensions.k8s.io"]
      resources: ["customresourcedefinitions"]
      verbs: ["get", "list", "watch"]
customResourceState:
  enabled: true
  config:
    kind: CustomResourceStateMetrics
    spec:
      resources:
        - groupVersionKind:
            group: "azure.tnnova.io"
            version: "v1alpha1"
            kind: "UserAssignedIdentity"
          labelsFromPath:
            namespace: [metadata, namespace]
          metrics:
              - name: status_conditions
                help: "The condition of the resource"
                each:
                  type: StateSet
                  stateSet:
                    path: [status, conditions, "0"]
                    labelName: status
                    valueFrom: [status]
                    list: ["True", "False", "Unknown"]
                    labelsFromPath:
                      type: [type]

It produces the following metrics:
# HELP kube_customresource_status_conditions The condition of the resource
# TYPE kube_customresource_status_conditions stateset
kube_customresource_status_conditions{customresource_group="azure.tnnova.io",customresource_kind="UserAssignedIdentity",customresource_version="v1alpha1",namespace="s01234-example-system",status="False",type="Ready"} 0
kube_customresource_status_conditions{customresource_group="azure.tnnova.io",customresource_kind="UserAssignedIdentity",customresource_version="v1alpha1",namespace="s01234-example-system",status="True",type="Ready"} 1
kube_customresource_status_conditions{customresource_group="azure.tnnova.io",customresource_kind="UserAssignedIdentity",customresource_version="v1alpha1",namespace="s01234-example-system",status="Unknown",type="Ready"} 0

However, when I replace kind="UserAssignedIdentity" to kind="*" , like:
...
        - groupVersionKind:
            group: "azure.tnnova.io"
            version: "v1alpha1"
            kind: "*"
...

Then it no longer produces any metrics. I have verified that the kube-state-metrics service account has list,watch,get on customresourcedefintions.
Anything else we need to know?:
Environment:

kube-state-metrics version: 2.12.0
Kubernetes version (use kubectl version): v1.30.0
Cloud provider or hardware configuration: kind
Other info:

Closing comment (2025-01-14T09:09:33Z): It looks like this is a duplicate of #2453 (or other way around).