Repository: provectus/kafka-ui
Issue: Latest Ingress changes breaks compatibility with GKE clusters versions (GKE use -gke.XXXX as suffix for version)
Labels: type/bug
status/accepted
scope/k8s

This changes breaks compatibility with GKE clusters versions (GKE use -gke.XXXX as suffix for version).
It can be solved when use semver comparision to full version ("1.19" -> "1.19.0-0")
```
(trimPrefix "v" .Capabilities.KubeVersion.Version | semverCompare ">= 1.19" ) =>
(trimPrefix "v" .Capabilities.KubeVersion.Version | semverCompare ">= 1.19.0-0" )
```

_Originally posted by @sgolod in https://github.com/provectus/kafka-ui/issues/2318#issuecomment-1203893817_

Issue comments:
Hello there sgolod! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀
Hey, thanks for reporting. We'll take a look.
Hello @sgolod , thx for discovering & reporting this
could you please try to use helm chart from this [branch](https://github.com/provectus/kafka-ui/tree/issues/2382) 
I don't have any GKE available to make final tests.
> 

You forgot to make same changes below (line 35 in this branch).
IMHO, but this patch is overcomplication. This can be done more elegantly by comparing with "1.19.0-0", this solution suggested in same original Helm issue:
https://github.com/helm/helm/issues/3810
Hi @sgolod 
you comment about line 35 correct, I also found and made correction in my last PR
The line `1.19.0` will be automatically treated as `1.19.0-0[]` when converting to the full semver format. I agree that your proposal will work as well. I Prefer to use my version, because it could be corner case when some K8s produce specific version value which could not be comparaible just with `1.19.0-0`. 

Pull request review comments:
Quite significant version increase, is it a typo?
I can't follow why use this construct instead of simply checking the presence of the API version?
```
{{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v1" -}}
```
AFAIK k8s version is irrelevant.

Consider following simple tester template:
```
apiVersion: v1
kind: ConfigMap
metadata:
  name: tester
data:
{{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v1" }}
  check_v1: "Has this API"
{{- else }}
  check_v1: "Does not have this API"
{{- end }}
{{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v2" }}
  check_v2: "Has this API"
{{- else }}
  check_v2: "Does not have this API"
{{- end }}
```
If you try installing it with --dry-run it will be rendered as:
```
apiVersion: v1
kind: ConfigMap
metadata:
  name: tester
data:
  check_v1: "Has this API"
  check_v2: "Does not have this API"
```
Which pretty much proves the point.
No, currently the version of helm chart is not track at all. The release job replace chart version during build and push right version of Helm chart in Helm repo, but don't change anything in this repo.
@Haarolean we need to decide, how we can solve this in future.
Long story :)
Please review 
https://github.com/provectus/kafka-ui/issues/2317
https://github.com/provectus/kafka-ui/pull/2318#issuecomment-1203893817
You could also check Kuberntes API reference
https://github.com/instrumenta/kubernetes-json-schema/blob/master/v1.18.1/networkpolicy.json - networking.k8s.io/v1
https://github.com/instrumenta/kubernetes-json-schema/blob/master/v1.18.1/ingress.json - networking.k8s.io/v1beta1





@azatsafin 
Got it. If you will look closely at the output of `.Capabilities.APIVersions` internal, you'll notice that it returns a list of GVKs.

Test stand:
```YAML
apiVersion: v1
kind: ConfigMap
metadata:
  name: tester
data:
  apiVersions: |
    {{ $.Capabilities.APIVersions }}
```

Result(cut to networking.k8s.io resources only):

```
[... networking.k8s.io/v1/NetworkPolicy networking.k8s.io/v1/Ingress networking.k8s.io/v1 networking.k8s.io/v1/IngressClass ...]
```
This means you can be more specific in your checks, eg:

```
{{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" }}
```

I believe this will fix the issue in a more straightforward manner.
This is a good idea, but please take into account that `helm template` command do not provide resource names for apiVersions, only 'helm install --dry-run' will work for it. 
And please have a look [here](https://github.com/provectus/kafka-ui/blob/98257b2b5fd77a0141f23bebb2b903d90d0a3113/.github/workflows/helm.yaml#L31) 
There are no option to write any logic which will not look to the `apiVersion/resourceName` values and produce right template for ingress/networkPolicy. 

Example:
`helm template abc abc`
``` 
apiVersion: v1
kind: ConfigMap
metadata:
  name: tester
data:
  apiVersions: |
    [v1 admissionregistration.k8s.io/v1 admissionregistration.k8s.io/v1beta1 internal.apiserver.k8s.io/v1alpha1 apps/v1 apps/v1beta1 apps/v1beta2 authentication.k8s.io/v1 authentication.k8s.io/v1beta1 authorization.k8s.io/v1 authorization.k8s.io/v1beta1 autoscaling/v1 autoscaling/v2 autoscaling/v2beta1 autoscaling/v2beta2 batch/v1 batch/v1beta1 certificates.k8s.io/v1 certificates.k8s.io/v1beta1 coordination.k8s.io/v1beta1 coordination.k8s.io/v1 discovery.k8s.io/v1 discovery.k8s.io/v1beta1 events.k8s.io/v1 events.k8s.io/v1beta1 extensions/v1beta1 flowcontrol.apiserver.k8s.io/v1alpha1 flowcontrol.apiserver.k8s.io/v1beta1 flowcontrol.apiserver.k8s.io/v1beta2 networking.k8s.io/v1 networking.k8s.io/v1beta1 node.k8s.io/v1 node.k8s.io/v1alpha1 node.k8s.io/v1beta1 policy/v1 policy/v1beta1 rbac.authorization.k8s.io/v1 rbac.authorization.k8s.io/v1beta1 rbac.authorization.k8s.io/v1alpha1 scheduling.k8s.io/v1alpha1 scheduling.k8s.io/v1beta1 scheduling.k8s.io/v1 storage.k8s.io/v1beta1 storage.k8s.io/v1 storage.k8s.io/v1alpha1 apiextensions.k8s.io/v1beta1 apiextensions.k8s.io/v1]
```

`helm install --dry-run  abc abc`
```
apiVersion: v1
kind: ConfigMap
metadata:
  name: tester
data:
  apiVersions: |
    [apps/v1 storage.k8s.io/v1beta1 v1/Secret apps/v1/ControllerRevision apps/v1/Scale events.k8s.io/v1beta1/Event policy/v1beta1/PodSecurityPolicy batch/v1 v1/Node batch/v1beta1/CronJob argoproj.io/v1alpha1/ArgoCDExtension v1/Eviction networking.k8s.io/v1/IngressClass scheduling.k8s.io/v1/PriorityClass node.k8s.io/v1/RuntimeClass apiextensions.k8s.io/v1 node.k8s.io/v1beta1 apps/v1/StatefulSet rbac.authorization.k8s.io/v1/ClusterRole metrics.k8s.io/v1beta1/NodeMetrics admissionregistration.k8s.io/v1 authorization.k8s.io/v1beta1/SubjectAccessReview authorization.k8s.io/v1beta1 rbac.authorization.k8s.io/v1beta1 autoscaling/v2beta2/HorizontalPodAutoscaler apiregistration.k8s.io/v1 scheduling.k8s.io/v1beta1 crd.k8s.amazonaws.com/v1alpha1 v1/ResourceQuota admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration coordination.k8s.io/v1 authorization.k8s.io/v1beta1/SelfSubjectRulesReview networking.k8s.io/v1beta1/Ingress policy/v1/PodDisruptionBudget certificates.k8s.io/v1/CertificateSigningRequest networking.k8s.io/v1/Ingress rbac.authorization.k8s.io/v1beta1/ClusterRole storage.k8s.io/v1/CSIDriver flowcontrol.apiserver.k8s.io/v1beta1/FlowSchema crd.k8s.amazonaws.com/v1alpha1/ENIConfig events.k8s.io/v1 certificates.k8s.io/v1beta1 apiregistration.k8s.io/v1beta1/APIService authorization.k8s.io/v1beta1/LocalSubjectAccessReview autoscaling/v2beta1/HorizontalPodAutoscaler scheduling.k8s.io/v1beta1/PriorityClass admissionregistration.k8s.io/v1beta1 discovery.k8s.io/v1beta1 v1/PodAttachOptions authorization.k8s.io/v1/SubjectAccessReview admissionregistration.k8s.io/v1beta1/MutatingWebhookConfiguration v1/Scale apiextensions.k8s.io/v1beta1/CustomResourceDefinition v1/Binding v1/NodeProxyOptions v1/PodProxyOptions apps/v1/Deployment networking.k8s.io/v1/NetworkPolicy storage.k8s.io/v1/VolumeAttachment storage.k8s.io/v1beta1/CSIDriver metrics.k8s.io/v1beta1/PodMetrics events.k8s.io/v1/Event apps/v1/DaemonSet apps/v1/ReplicaSet authorization.k8s.io/v1beta1/SelfSubjectAccessReview rbac.authorization.k8s.io/v1beta1/ClusterRoleBinding storage.k8s.io/v1beta1/VolumeAttachment coordination.k8s.io/v1beta1/Lease events.k8s.io/v1beta1 autoscaling/v2beta1 autoscaling/v2beta2 coordination.k8s.io/v1beta1 authorization.k8s.io/v1/SelfSubjectAccessReview rbac.authorization.k8s.io/v1/ClusterRoleBinding vpcresources.k8s.aws/v1beta1/SecurityGroupPolicy batch/v1beta1 v1/ComponentStatus v1/PodExecOptions discovery.k8s.io/v1/EndpointSlice authorization.k8s.io/v1 apiextensions.k8s.io/v1beta1 discovery.k8s.io/v1 policy/v1beta1/PodDisruptionBudget v1 certificates.k8s.io/v1 policy/v1 metrics.k8s.io/v1beta1 apiextensions.k8s.io/v1/CustomResourceDefinition apiregistration.k8s.io/v1beta1 v1/LimitRange v1/PodPortForwardOptions authorization.k8s.io/v1/SelfSubjectRulesReview batch/v1/Job autoscaling/v1 v1/ServiceAccount rbac.authorization.k8s.io/v1beta1/Role admissionregistration.k8s.io/v1/MutatingWebhookConfiguration admissionregistration.k8s.io/v1beta1/ValidatingWebhookConfiguration argoproj.io/v1alpha1/AppProject authorization.k8s.io/v1/LocalSubjectAccessReview autoscaling/v1/HorizontalPodAutoscaler authentication.k8s.io/v1 v1/ConfigMap networking.k8s.io/v1beta1/IngressClass storage.k8s.io/v1beta1/StorageClass coordination.k8s.io/v1/Lease node.k8s.io/v1beta1/RuntimeClass networking.k8s.io/v1 node.k8s.io/v1 flowcontrol.apiserver.k8s.io/v1beta1 v1/Service flowcontrol.apiserver.k8s.io/v1beta1/PriorityLevelConfiguration authentication.k8s.io/v1beta1 storage.k8s.io/v1 scheduling.k8s.io/v1 vpcresources.k8s.aws/v1beta1 v1/Endpoints v1/Pod certificates.k8s.io/v1beta1/CertificateSigningRequest rbac.authorization.k8s.io/v1/RoleBinding storage.k8s.io/v1/CSINode argoproj.io/v1alpha1/Application extensions/v1beta1 v1/Namespace extensions/v1beta1/Ingress networking.k8s.io/v1beta1 rbac.authorization.k8s.io/v1 v1/TokenRequest v1/ServiceProxyOptions storage.k8s.io/v1beta1/CSINode v1/PersistentVolume v1/ReplicationController rbac.authorization.k8s.io/v1/Role rbac.authorization.k8s.io/v1beta1/RoleBinding storage.k8s.io/v1/StorageClass argoproj.io/v1alpha1 batch/v1/CronJob discovery.k8s.io/v1beta1/EndpointSlice policy/v1beta1 v1/Event v1/PersistentVolumeClaim v1/PodTemplate apiregistration.k8s.io/v1/APIService authentication.k8s.io/v1/TokenReview authentication.k8s.io/v1beta1/TokenReview storage.k8s.io/v1beta1/CSIStorageCapacity]
```
Last command could be executed only for specific kubernetes context, which are not available for testing. 



You are right, `template` will not provide the correct apiversions but it won't provide .Capabilities.KubeVersion.Version as well for the same reason. In fact detected version will be determined with your helm version. Just did 2 checks:
- with `helm v3.3.0` .Capabilities.KubeVersion.Version == 1.18.0
- with `helm v3.9.2` .Capabilities.KubeVersion.Version == 1.24.0

Either way it's fair because this is a documented behaviour of `helm template`.

If you you are not convinced - let me know and I'll resolve this conversation.
I agree that we should track version of used HELM, and supported kubernetes versions. @Haarolean 

I don't want to link resource definitions to use HELM versions, because one HELM version support at least 3 Kubernetes version, we can find it [here](https://helm.sh/docs/topics/version_skew/) .

Currently, in our test [pipeline](https://github.com/provectus/kafka-ui/blob/757bf9526efa1d66308ca26bc5cae2ef2f3bf343/.github/workflows/helm.yaml#L31), we are using `--kube-version` argument to provide Kubernetes vesions to Charts, I don't want to change that logic now. 

Patch that resolved the issue:
diff --git a/charts/kafka-ui/Chart.yaml b/charts/kafka-ui/Chart.yaml
index 1177e74887c..7eaff3c59aa 100644
--- a/charts/kafka-ui/Chart.yaml
+++ b/charts/kafka-ui/Chart.yaml
@@ -2,6 +2,6 @@ apiVersion: v2
 name: kafka-ui
 description: A Helm chart for kafka-UI
 type: application
-version: 0.0.4
+version: 0.4.2
 appVersion: latest
 icon: https://github.com/provectus/kafka-ui/raw/master/documentation/images/kafka-ui-logo.png
diff --git a/charts/kafka-ui/templates/ingress.yaml b/charts/kafka-ui/templates/ingress.yaml
index 7659867b31a..e4b33439c42 100644
--- a/charts/kafka-ui/templates/ingress.yaml
+++ b/charts/kafka-ui/templates/ingress.yaml
@@ -1,7 +1,9 @@
 {{- if .Values.ingress.enabled -}}
 {{- $fullName := include "kafka-ui.fullname" . -}}
 {{- $svcPort := .Values.service.port -}}
-{{- if and ($.Capabilities.APIVersions.Has "networking.k8s.io/v1") (trimPrefix "v" .Capabilities.KubeVersion.Version | semverCompare ">= 1.19" ) -}}
+{{- $kubeCapabilityVersion := semver .Capabilities.KubeVersion.Version -}}
+{{- $isHigher1p19 := ge (semver "1.19" | $kubeCapabilityVersion.Compare) 0 -}}
+{{- if and ($.Capabilities.APIVersions.Has "networking.k8s.io/v1") $isHigher1p19 -}}
 apiVersion: networking.k8s.io/v1
 {{- else if $.Capabilities.APIVersions.Has "networking.k8s.io/v1beta1" }}
 apiVersion: networking.k8s.io/v1beta1
@@ -30,7 +32,7 @@ spec:
   rules:
     - http:
         paths:
-{{- if and ($.Capabilities.APIVersions.Has "networking.k8s.io/v1") (trimPrefix "v" .Capabilities.KubeVersion.Version | semverCompare ">= 1.19" ) -}}
+{{- if and ($.Capabilities.APIVersions.Has "networking.k8s.io/v1") $isHigher1p19 -}}
           {{- range .Values.ingress.precedingPaths }}
           - path: {{ .path }}
             pathType: Prefix


Test patch: