Repository: GoogleCloudPlatform/spark-on-k8s-operator
Pull request #1427 (2021-12-09 15:52:32)

Kubernetes v1.22 extensions/v1beta1 API removal
Fixes https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/1426.
This is a breaking change, meaning that on kubernetes versions older than 1.19 Ingress objects would fail to be created (everything else should still work as expected). It also defines the `pathType` statically as "ImplementationSpecific". 

TomHellier: @aneagoe please could you update this PR to support both options. It should be possible to write code which detects the k8s version and applies the appropriate ingress code.

I dont think it would be appropriate to merge this as it would move the minimum k8s version from 1.13~1.16 up quite a few versions
alexellis: Glad this was useful to you. Changes like this kind of set the whole ecosystem back. 
TomHellier: LGTM :+1: thanks @aneagoe 
Reamer: Thank you very much for this contribution.
I think a PR like this should also change the corresponding [rbac.yml](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/2919f521ccde71fc54b8d72ba0474c5872c7d4aa/manifest/spark-operator-install/spark-operator-rbac.yaml#L42-L44)
aneagoe: @Reamer I'm not quite sure how but it seems to be gracefully handled. Here's the clusterrole created by the helm chart on my 1.22 cluster:
```
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    helm.sh/hook: pre-install
    helm.sh/hook-delete-policy: hook-failed, before-hook-creation
  creationTimestamp: "2021-12-03T09:00:55Z"
  labels:
    app.kubernetes.io/instance: spark-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: spark-operator
    app.kubernetes.io/version: v1beta2-1.3.1-3.1.1
    helm.sh/chart: spark-operator-1.1.15
  name: spark-operator
  resourceVersion: "9819502"
  uid: 16cc785d-04b4-43f9-81bc-520a53ad2485
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - services
  - configmaps
  - secrets
  verbs:
  - create
  - get
  - delete
  - update
- apiGroups:
  - extensions
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - create
  - get
  - delete
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - update
  - patch
- apiGroups:
  - ""
  resources:
  - resourcequotas
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - apiextensions.k8s.io
  resources:
  - customresourcedefinitions
  verbs:
  - create
  - get
  - update
  - delete
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  - validatingwebhookconfigurations
  verbs:
  - create
  - get
  - update
  - delete
- apiGroups:
  - sparkoperator.k8s.io
  resources:
  - sparkapplications
  - sparkapplications/status
  - scheduledsparkapplications
  - scheduledsparkapplications/status
  verbs:
  - '*'
- apiGroups:
  - batch
  resources:
  - jobs
  verbs:
  - delete
```

As you can see, it contains the `networking.k8s.io` api group. Changing the rbac.yaml would require further modifications of the helm chart logic to support both, but does not seem necessary. 
Reamer: I do not use the helm charts, but it seems that helm uses this file [charts/spark-operator-chart/templates/rbac.yaml](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/2e8b733f5ad029ceb8a949b153a53c434b8fcfe1/charts/spark-operator-chart/templates/rbac.yaml#L28). It would be good if the api group was also added in simple manifest file [manifest/spark-operator-install/spark-operator-rbac.yaml](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/2919f521ccde71fc54b8d72ba0474c5872c7d4aa/manifest/spark-operator-install/spark-operator-rbac.yaml#L42).
Perhaps we should make this change in a separate PR?
aneagoe: > I do not use the helm charts, but it seems that helm uses this file [charts/spark-operator-chart/templates/rbac.yaml](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/2e8b733f5ad029ceb8a949b153a53c434b8fcfe1/charts/spark-operator-chart/templates/rbac.yaml#L28). It would be good if the api group was also added in simple manifest file [manifest/spark-operator-install/spark-operator-rbac.yaml](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/2919f521ccde71fc54b8d72ba0474c5872c7d4aa/manifest/spark-operator-install/spark-operator-rbac.yaml#L42). Perhaps we should make this change in a separate PR?

It would make more sense, yes. At the same time, it might not be just a simple addition. I'm not sure how this behaves on older k8s that don't support `networking.k8s.io` ingresses. 
Reamer: > I'm not sure how this behaves on older k8s that don't support `networking.k8s.io` ingresses.
Neither do I, thank you for your PR.

aneagoe: I've amended the `appVersion` to avoid the build failure. I'm not sure how to tackle the api change check though. @liyinan926 @TomHellier any advice?
TomHellier: @aneagoe I guess technically the api has  changed, but as you say it should be backwards compatible as you've tested with v16 and v22. Perhaps that github workflow step could be amended to search for a string in the github commit like "(ignoreapi)" or something so we can just bypass it.

Unfortunately I think @liyinan926 is the only person with maintainer privileges on this repository, and I think he only checks this place once a month. I think ideally this repo would have more maintainers but as the repo states, this is not an officially supported project.
aneagoe: The `build-api-docs` job fails at step `The API should not change once published`. This check would always fail if anything were to change:
```
0s
Run if ! git diff --quiet origin/master -- pkg/apis/sparkoperator.k8s.io/v1beta1; then
  if ! git diff --quiet origin/master -- pkg/apis/sparkoperator.k8s.io/v1beta1; then
    echo "sparkoperator.k8s.io/v1beta1 api has changed"
    false
  fi
  if ! git diff --quiet origin/master -- pkg/apis/sparkoperator.k8s.io/v1beta2; then
    echo "sparkoperator.k8s.io/v1beta2 api has changed"
    false
  fi
  shell: /usr/bin/bash -e {0}
sparkoperator.k8s.io/v1beta2 api has changed
```

That file had to be changed. 

Patch:
diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml
index ae13c34..778120e 100644
--- a/.github/workflows/main.yaml
+++ b/.github/workflows/main.yaml
@@ -80,8 +80,7 @@ jobs:
 
       - name: Build Spark-Operator Docker Image
         run: |
-          DOCKER_TAG=$(cat charts/spark-operator-chart/Chart.yaml | grep "appVersion: .*" | cut -c13-)
-          docker build -t gcr.io/spark-operator/spark-operator:${DOCKER_TAG} .
+          docker build -t gcr.io/spark-operator/spark-operator:latest .
 
       - name: Check changes in resources used in docker file
         run: |
@@ -90,14 +89,14 @@ jobs:
             # If the resource is different
             if ! git diff  --quiet origin/master -- $resource; then
               ## And the appVersion hasn't been updated
-              if ! git diff --quiet charts/spark-operator-chart/Chart.yaml | grep +appVersion; then
+              if ! git diff origin/master -- charts/spark-operator-chart/Chart.yaml | grep +appVersion; then
                 echo "resource used in gcr.io/spark-operator/spark-operator has changed in $resource, need to update the appVersion in charts/spark-operator-chart/Chart.yaml"
                 git diff origin/master -- $resource;
                 echo "failing the build... " && false
               fi
             fi
           done
-          
+
   build-helm-chart:
     runs-on: ubuntu-latest
     steps:
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
index c8b882b..5c4da69 100644
--- a/.github/workflows/release.yaml
+++ b/.github/workflows/release.yaml
@@ -24,27 +24,31 @@ jobs:
         with:
           version: v3.7.1
 
-      - uses: actions/setup-python@v2
-        with:
-          python-version: 3.10
-
       # TODO: Maintainer of repository to follow:
       # https://github.com/docker/login-action#google-container-registry-gcr to add credentials so
       # we can push from github actions
-      # - name: log in to container registry
+      # - name: log in to google container registry
       #   uses: docker/login-action@v1
       #   with:
       #     registry: gcr.io
       #     username: ${{ secrets.DOCKER_USERNAME }}
       #     password: ${{ secrets.DOCKER_PASSWORD }}
+      
+      - name: Login to Packages Container registry
+        uses: docker/login-action@v1 
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
 
-      - name: Release Spark-Operator Docker Image
+      - name: Release Spark-Operator Docker Image to github container registry
         run: |
           DOCKER_TAG=$(cat charts/spark-operator-chart/Chart.yaml | grep "appVersion: .*" | cut -c13-)
           docker build -t gcr.io/spark-operator/spark-operator:${DOCKER_TAG} .
           echo "Ideally, we'd release the docker container at this point, but the maintainer of this repo needs to approve..."
-          if ! docker pull docker pull gcr.io/spark-operator/spark-operator:${DOCKER_TAG}; then
-            echo "docker push gcr.io/spark-operator/spark-operator:${DOCKER_TAG}"
+          docker tag gcr.io/spark-operator/spark-operator:${DOCKER_TAG} ghcr.io/googlecloudplatform/spark-operator:${DOCKER_TAG}
+          if ! docker pull ghcr.io/googlecloudplatform/spark-operator:${DOCKER_TAG}; then
+            docker push ghcr.io/googlecloudplatform/spark-operator:${DOCKER_TAG}
             git tag $DOCKER_TAG
             git push origin $DOCKER_TAG
           fi
diff --git a/README.md b/README.md
index 0e3eb0e..7c9d07c 100644
--- a/README.md
+++ b/README.md
@@ -48,6 +48,8 @@ The following table lists the most recent few versions of the operator.
 | Operator Version | API Version | Kubernetes Version | Base Spark Version | Operator Image Tag |
 | ------------- | ------------- | ------------- | ------------- | ------------- |
 | `latest` (master HEAD) | `v1beta2` | 1.13+ | `3.0.0` | `latest` |
+| `v1beta2-1.3.3-3.1.1` | `v1beta2` | 1.16+ | `3.1.1` | `v1beta2-1.3.3-3.1.1` |
+| `v1beta2-1.3.2-3.1.1` | `v1beta2` | 1.16+ | `3.1.1` | `v1beta2-1.3.2-3.1.1` |
 | `v1beta2-1.3.0-3.1.1` | `v1beta2` | 1.16+ | `3.1.1` | `v1beta2-1.3.0-3.1.1` |
 | `v1beta2-1.2.3-3.1.1` | `v1beta2` | 1.13+ | `3.1.1` | `v1beta2-1.2.3-3.1.1` |
 | `v1beta2-1.2.0-3.0.0` | `v1beta2` | 1.13+ | `3.0.0` | `v1beta2-1.2.0-3.0.0` |
diff --git a/charts/spark-operator-chart/Chart.yaml b/charts/spark-operator-chart/Chart.yaml
index c37b971..1550c5f 100644
--- a/charts/spark-operator-chart/Chart.yaml
+++ b/charts/spark-operator-chart/Chart.yaml
@@ -1,8 +1,8 @@
 apiVersion: v2
 name: spark-operator
 description: A Helm chart for Spark on Kubernetes operator
-version: 1.1.16
-appVersion: v1beta2-1.3.2-3.1.1
+version: 1.1.17
+appVersion: v1beta2-1.3.3-3.1.1
 keywords:
   - spark
 home: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
diff --git a/main.go b/main.go
index fac4381..b2ddbba 100644
--- a/main.go
+++ b/main.go
@@ -153,6 +153,10 @@ func main() {
 		glog.Fatal(err)
 	}
 
+	if err = util.InitializeIngressCapabilities(kubeClient); err != nil {
+		glog.Fatalf("Error retrieving Kubernetes cluster capabilities: %s", err.Error())
+	}
+
 	var batchSchedulerMgr *batchscheduler.SchedulerManager
 	if *enableBatchScheduler {
 		if !*enableWebhook {
diff --git a/pkg/apis/sparkoperator.k8s.io/v1beta2/types.go b/pkg/apis/sparkoperator.k8s.io/v1beta2/types.go
index 8583664..bd4aee2 100644
--- a/pkg/apis/sparkoperator.k8s.io/v1beta2/types.go
+++ b/pkg/apis/sparkoperator.k8s.io/v1beta2/types.go
@@ -18,7 +18,7 @@ package v1beta2
 
 import (
 	apiv1 "k8s.io/api/core/v1"
-	extensions "k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 )
 
@@ -322,7 +322,7 @@ type SparkUIConfiguration struct {
 	IngressAnnotations map[string]string `json:"ingressAnnotations,omitempty"`
 	// TlsHosts is useful If we need to declare SSL certificates to the ingress object
 	// +optional
-	IngressTLS []extensions.IngressTLS `json:"ingressTLS,omitempty"`
+	IngressTLS []networkingv1.IngressTLS `json:"ingressTLS,omitempty"`
 }
 
 // ApplicationStateType represents the type of the current state of an application.
diff --git a/pkg/apis/sparkoperator.k8s.io/v1beta2/zz_generated.deepcopy.go b/pkg/apis/sparkoperator.k8s.io/v1beta2/zz_generated.deepcopy.go
index 7c5a4d9..8099514 100644
--- a/pkg/apis/sparkoperator.k8s.io/v1beta2/zz_generated.deepcopy.go
+++ b/pkg/apis/sparkoperator.k8s.io/v1beta2/zz_generated.deepcopy.go
@@ -24,7 +24,7 @@ package v1beta2
 
 import (
 	v1 "k8s.io/api/core/v1"
-	v1beta1 "k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
 	runtime "k8s.io/apimachinery/pkg/runtime"
 )
 
@@ -1009,6 +1009,13 @@ func (in *SparkUIConfiguration) DeepCopyInto(out *SparkUIConfiguration) {
 		*out = new(v1.ServiceType)
 		**out = **in
 	}
+	if in.ServiceAnnotations != nil {
+		in, out := &in.ServiceAnnotations, &out.ServiceAnnotations
+		*out = make(map[string]string, len(*in))
+		for key, val := range *in {
+			(*out)[key] = val
+		}
+	}
 	if in.IngressAnnotations != nil {
 		in, out := &in.IngressAnnotations, &out.IngressAnnotations
 		*out = make(map[string]string, len(*in))
@@ -1018,7 +1025,7 @@ func (in *SparkUIConfiguration) DeepCopyInto(out *SparkUIConfiguration) {
 	}
 	if in.IngressTLS != nil {
 		in, out := &in.IngressTLS, &out.IngressTLS
-		*out = make([]v1beta1.IngressTLS, len(*in))
+		*out = make([]networkingv1.IngressTLS, len(*in))
 		for i := range *in {
 			(*in)[i].DeepCopyInto(&(*out)[i])
 		}
diff --git a/pkg/controller/sparkapplication/controller.go b/pkg/controller/sparkapplication/controller.go
index 567e25c..81c3634 100644
--- a/pkg/controller/sparkapplication/controller.go
+++ b/pkg/controller/sparkapplication/controller.go
@@ -872,10 +872,19 @@ func (c *Controller) deleteSparkResources(app *v1beta2.SparkApplication) error {
 
 	sparkUIIngressName := app.Status.DriverInfo.WebUIIngressName
 	if sparkUIIngressName != "" {
-		glog.V(2).Infof("Deleting Spark UI Ingress %s in namespace %s", sparkUIIngressName, app.Namespace)
-		err := c.kubeClient.ExtensionsV1beta1().Ingresses(app.Namespace).Delete(context.TODO(), sparkUIIngressName, metav1.DeleteOptions{GracePeriodSeconds: int64ptr(0)})
-		if err != nil && !errors.IsNotFound(err) {
-			return err
+		if util.IngressCapabilities.Has("networking.k8s.io/v1") {
+			glog.V(2).Infof("Deleting Spark UI Ingress %s in namespace %s", sparkUIIngressName, app.Namespace)
+			err := c.kubeClient.NetworkingV1().Ingresses(app.Namespace).Delete(context.TODO(), sparkUIIngressName, metav1.DeleteOptions{GracePeriodSeconds: int64ptr(0)})
+			if err != nil && !errors.IsNotFound(err) {
+				return err
+			}
+		}
+		if util.IngressCapabilities.Has("extensions/v1beta1") {
+			glog.V(2).Infof("Deleting extensions/v1beta1 Spark UI Ingress %s in namespace %s", sparkUIIngressName, app.Namespace)
+			err := c.kubeClient.ExtensionsV1beta1().Ingresses(app.Namespace).Delete(context.TODO(), sparkUIIngressName, metav1.DeleteOptions{GracePeriodSeconds: int64ptr(0)})
+			if err != nil && !errors.IsNotFound(err) {
+				return err
+			}
 		}
 	}
 
@@ -916,7 +925,7 @@ func (c *Controller) validateSparkResourceDeletion(app *v1beta2.SparkApplication
 
 	sparkUIIngressName := app.Status.DriverInfo.WebUIIngressName
 	if sparkUIIngressName != "" {
-		_, err := c.kubeClient.ExtensionsV1beta1().Ingresses(app.Namespace).Get(context.TODO(), sparkUIIngressName, metav1.GetOptions{})
+		_, err := c.kubeClient.NetworkingV1().Ingresses(app.Namespace).Get(context.TODO(), sparkUIIngressName, metav1.GetOptions{})
 		if err == nil || !errors.IsNotFound(err) {
 			return false
 		}
diff --git a/pkg/controller/sparkapplication/sparkapp_util.go b/pkg/controller/sparkapplication/sparkapp_util.go
index 477a2ad..ef41ade 100644
--- a/pkg/controller/sparkapplication/sparkapp_util.go
+++ b/pkg/controller/sparkapplication/sparkapp_util.go
@@ -26,7 +26,7 @@ import (
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1beta2"
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/config"
 	apiv1 "k8s.io/api/core/v1"
-	"k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
 )
 
 // Helper method to create a key with namespace and appName
@@ -100,8 +100,8 @@ func getIngressResourceAnnotations(app *v1beta2.SparkApplication) map[string]str
 	return ingressAnnotations
 }
 
-func getIngressTlsHosts(app *v1beta2.SparkApplication) []v1beta1.IngressTLS {
-	var ingressTls []v1beta1.IngressTLS
+func getIngressTlsHosts(app *v1beta2.SparkApplication) []networkingv1.IngressTLS {
+	var ingressTls []networkingv1.IngressTLS
 	if app.Spec.SparkUIOptions != nil && app.Spec.SparkUIOptions.IngressTLS != nil {
 		for _, ingTls := range app.Spec.SparkUIOptions.IngressTLS {
 			ingressTls = append(ingressTls, ingTls)
diff --git a/pkg/controller/sparkapplication/sparkui.go b/pkg/controller/sparkapplication/sparkui.go
index f46a2a5..2a0d09f 100644
--- a/pkg/controller/sparkapplication/sparkui.go
+++ b/pkg/controller/sparkapplication/sparkui.go
@@ -27,12 +27,14 @@ import (
 
 	apiv1 "k8s.io/api/core/v1"
 	extensions "k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/util/intstr"
 	clientset "k8s.io/client-go/kubernetes"
 
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1beta2"
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/config"
+	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/util"
 )
 
 const (
@@ -76,11 +78,90 @@ type SparkIngress struct {
 	ingressName string
 	ingressURL  *url.URL
 	annotations map[string]string
-	ingressTLS  []extensions.IngressTLS
+	ingressTLS  []networkingv1.IngressTLS
 }
 
 func createSparkUIIngress(app *v1beta2.SparkApplication, service SparkService, ingressURL *url.URL, kubeClient clientset.Interface) (*SparkIngress, error) {
+	if util.IngressCapabilities.Has("networking.k8s.io/v1") {
+		return createSparkUIIngress_v1(app, service, ingressURL, kubeClient)
+	} else {
+		return createSparkUIIngress_legacy(app, service, ingressURL, kubeClient)
+	}
+}
+
+func createSparkUIIngress_v1(app *v1beta2.SparkApplication, service SparkService, ingressURL *url.URL, kubeClient clientset.Interface) (*SparkIngress, error) {
+	ingressResourceAnnotations := getIngressResourceAnnotations(app)
+	ingressTlsHosts := getIngressTlsHosts(app)
+
+	ingressURLPath := ingressURL.Path
+	// If we're serving on a subpath, we need to ensure we create capture groups
+	if ingressURLPath != "" && ingressURLPath != "/" {
+		ingressURLPath = ingressURLPath + "(/|$)(.*)"
+	}
+
+	implementationSpecific := networkingv1.PathTypeImplementationSpecific
+
+	ingress := networkingv1.Ingress{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:            getDefaultUIIngressName(app),
+			Namespace:       app.Namespace,
+			Labels:          getResourceLabels(app),
+			OwnerReferences: []metav1.OwnerReference{*getOwnerReference(app)},
+		},
+		Spec: networkingv1.IngressSpec{
+			Rules: []networkingv1.IngressRule{{
+				Host: ingressURL.Host,
+				IngressRuleValue: networkingv1.IngressRuleValue{
+					HTTP: &networkingv1.HTTPIngressRuleValue{
+						Paths: []networkingv1.HTTPIngressPath{{
+							Backend: networkingv1.IngressBackend{
+								Service: &networkingv1.IngressServiceBackend{
+									Name: service.serviceName,
+									Port: networkingv1.ServiceBackendPort{
+										Number: service.servicePort,
+									},
+								},
+							},
+							Path:     ingressURLPath,
+							PathType: &implementationSpecific,
+						}},
+					},
+				},
+			}},
+		},
+	}
+
+	if len(ingressResourceAnnotations) != 0 {
+		ingress.ObjectMeta.Annotations = ingressResourceAnnotations
+	}
+
+	// If we're serving on a subpath, we need to ensure we use the capture groups
+	if ingressURL.Path != "" && ingressURL.Path != "/" {
+		if ingress.ObjectMeta.Annotations == nil {
+			ingress.ObjectMeta.Annotations = make(map[string]string)
+		}
+		ingress.ObjectMeta.Annotations["nginx.ingress.kubernetes.io/rewrite-target"] = "/$2"
+	}
+	if len(ingressTlsHosts) != 0 {
+		ingress.Spec.TLS = ingressTlsHosts
+	}
+	glog.Infof("Creating an Ingress %s for the Spark UI for application %s", ingress.Name, app.Name)
+	_, err := kubeClient.NetworkingV1().Ingresses(ingress.Namespace).Create(context.TODO(), &ingress, metav1.CreateOptions{})
+	if err != nil {
+		return nil, err
+	}
+	return &SparkIngress{
+		ingressName: ingress.Name,
+		ingressURL:  ingressURL,
+		annotations: ingress.Annotations,
+		ingressTLS:  ingressTlsHosts,
+	}, nil
+}
+
+func createSparkUIIngress_legacy(app *v1beta2.SparkApplication, service SparkService, ingressURL *url.URL, kubeClient clientset.Interface) (*SparkIngress, error) {
 	ingressResourceAnnotations := getIngressResourceAnnotations(app)
+	// var ingressTlsHosts networkingv1.IngressTLS[]
+	// That we convert later for extensionsv1beta1, but return as is in SparkIngress
 	ingressTlsHosts := getIngressTlsHosts(app)
 
 	ingressURLPath := ingressURL.Path
@@ -129,11 +210,10 @@ func createSparkUIIngress(app *v1beta2.SparkApplication, service SparkService, i
 		ingress.ObjectMeta.Annotations["nginx.ingress.kubernetes.io/rewrite-target"] = "/$2"
 	}
 	if len(ingressTlsHosts) != 0 {
-		ingress.Spec.TLS = ingressTlsHosts
+		ingress.Spec.TLS = convertIngressTlsHostsToLegacy(ingressTlsHosts)
 	}
-	glog.Infof("Creating an Ingress %s for the Spark UI for application %s", ingress.Name, app.Name)
+	glog.Infof("Creating an extensions/v1beta1 Ingress %s for the Spark UI for application %s", ingress.Name, app.Name)
 	_, err := kubeClient.ExtensionsV1beta1().Ingresses(ingress.Namespace).Create(context.TODO(), &ingress, metav1.CreateOptions{})
-
 	if err != nil {
 		return nil, err
 	}
@@ -141,10 +221,21 @@ func createSparkUIIngress(app *v1beta2.SparkApplication, service SparkService, i
 		ingressName: ingress.Name,
 		ingressURL:  ingressURL,
 		annotations: ingress.Annotations,
-		ingressTLS:  ingress.Spec.TLS,
+		ingressTLS:  ingressTlsHosts,
 	}, nil
 }
 
+func convertIngressTlsHostsToLegacy(ingressTlsHosts []networkingv1.IngressTLS) []extensions.IngressTLS {
+	var ingressTlsHosts_legacy []extensions.IngressTLS
+	for _, ingressTlsHost := range ingressTlsHosts {
+		ingressTlsHosts_legacy = append(ingressTlsHosts_legacy, extensions.IngressTLS{
+			Hosts:      ingressTlsHost.Hosts,
+			SecretName: ingressTlsHost.SecretName,
+		})
+	}
+	return ingressTlsHosts_legacy
+}
+
 func createSparkUIService(
 	app *v1beta2.SparkApplication,
 	kubeClient clientset.Interface) (*SparkService, error) {
diff --git a/pkg/util/capabilities.go b/pkg/util/capabilities.go
new file mode 100644
index 0000000..27b7f71
--- /dev/null
+++ b/pkg/util/capabilities.go
@@ -0,0 +1,78 @@
+/*
+Copyright 2017 Google LLC
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    https://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package util
+
+import (
+	"strings"
+
+	"k8s.io/client-go/kubernetes"
+)
+
+type Capabilities map[string]bool
+
+func (c Capabilities) Has(wanted string) bool {
+	return c[wanted]
+}
+
+func (c Capabilities) String() string {
+	keys := make([]string, 0, len(c))
+	for k := range c {
+		keys = append(keys, k)
+	}
+	return strings.Join(keys, ", ")
+}
+
+// getPreferredAvailableAPIs queries the cluster for the preferred resources information and returns a Capabilities
+// instance containing those api groups that support the specified kind.
+//
+// kind should be the title case singular name of the kind. For example, "Ingress" is the kind for a resource "ingress".
+func getPreferredAvailableAPIs(client kubernetes.Interface, kind string) (Capabilities, error) {
+	discoveryclient := client.Discovery()
+	lists, err := discoveryclient.ServerPreferredResources()
+	if err != nil {
+		return nil, err
+	}
+
+	caps := Capabilities{}
+	for _, list := range lists {
+		if len(list.APIResources) == 0 {
+			continue
+		}
+		for _, resource := range list.APIResources {
+			if len(resource.Verbs) == 0 {
+				continue
+			}
+			if resource.Kind == kind {
+				caps[list.GroupVersion] = true
+			}
+		}
+	}
+
+	return caps, nil
+}
+
+var (
+	IngressCapabilities Capabilities
+)
+
+func InitializeIngressCapabilities(client kubernetes.Interface) (err error) {
+	if IngressCapabilities != nil {
+		return
+	}
+	IngressCapabilities, err = getPreferredAvailableAPIs(client, "Ingress")
+	return
+}


Test patch:
diff --git a/pkg/controller/sparkapplication/controller_test.go b/pkg/controller/sparkapplication/controller_test.go
index e70c7a8..3bcb60c 100644
--- a/pkg/controller/sparkapplication/controller_test.go
+++ b/pkg/controller/sparkapplication/controller_test.go
@@ -48,6 +48,7 @@ func newFakeController(app *v1beta2.SparkApplication, pods ...*apiv1.Pod) (*Cont
 	crdclientfake.AddToScheme(scheme.Scheme)
 	crdClient := crdclientfake.NewSimpleClientset()
 	kubeClient := kubeclientfake.NewSimpleClientset()
+	util.IngressCapabilities = map[string]bool{"networking.k8s.io/v1": true}
 	informerFactory := crdinformers.NewSharedInformerFactory(crdClient, 0*time.Second)
 	recorder := record.NewFakeRecorder(3)
 
@@ -1599,10 +1600,13 @@ func TestIngressWithSubpathAffectsSparkConfiguration(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	ingresses, err := ctrl.kubeClient.ExtensionsV1beta1().Ingresses(app.Namespace).List(context.TODO(), metav1.ListOptions{})
+	ingresses, err := ctrl.kubeClient.NetworkingV1().Ingresses(app.Namespace).List(context.TODO(), metav1.ListOptions{})
 	if err != nil {
 		t.Fatal(err)
 	}
+	if ingresses == nil || ingresses.Items == nil || len(ingresses.Items) != 1 {
+		t.Fatal("The ingress does not exist, has no items, or wrong amount of items")
+	}
 	if ingresses.Items[0].Spec.Rules[0].IngressRuleValue.HTTP.Paths[0].Path != "/"+app.Namespace+"/"+app.Name+"(/|$)(.*)" {
 		t.Fatal("The ingress subpath was not created successfully.")
 	}
diff --git a/pkg/controller/sparkapplication/sparkui_test.go b/pkg/controller/sparkapplication/sparkui_test.go
index 91b5c66..f6b8128 100644
--- a/pkg/controller/sparkapplication/sparkui_test.go
+++ b/pkg/controller/sparkapplication/sparkui_test.go
@@ -24,13 +24,14 @@ import (
 	"testing"
 
 	apiv1 "k8s.io/api/core/v1"
-	extensions "k8s.io/api/extensions/v1beta1"
+	networkingv1 "k8s.io/api/networking/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/util/intstr"
 	"k8s.io/client-go/kubernetes/fake"
 
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/apis/sparkoperator.k8s.io/v1beta2"
 	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/config"
+	"github.com/GoogleCloudPlatform/spark-on-k8s-operator/pkg/util"
 )
 
 func TestCreateSparkUIService(t *testing.T) {
@@ -43,6 +44,7 @@ func TestCreateSparkUIService(t *testing.T) {
 	}
 	testFn := func(test testcase, t *testing.T) {
 		fakeClient := fake.NewSimpleClientset()
+		util.IngressCapabilities = map[string]bool{"networking.k8s.io/v1": true}
 		sparkService, err := createSparkUIService(test.app, fakeClient)
 		if err != nil {
 			if test.expectError {
@@ -351,7 +353,7 @@ func TestCreateSparkUIIngress(t *testing.T) {
 		if sparkIngress.ingressURL.String() != test.expectedIngress.ingressURL.String() {
 			t.Errorf("Ingress URL wanted %s got %s", test.expectedIngress.ingressURL, sparkIngress.ingressURL)
 		}
-		ingress, err := fakeClient.ExtensionsV1beta1().Ingresses(test.app.Namespace).
+		ingress, err := fakeClient.NetworkingV1().Ingresses(test.app.Namespace).
 			Get(context.TODO(), sparkIngress.ingressName, metav1.GetOptions{})
 		if err != nil {
 			t.Fatal(err)
@@ -394,11 +396,14 @@ func TestCreateSparkUIIngress(t *testing.T) {
 			t.Errorf("No Ingress paths found.")
 		}
 		ingressPath := ingressRule.IngressRuleValue.HTTP.Paths[0]
-		if ingressPath.Backend.ServiceName != sparkService.serviceName {
-			t.Errorf("Service name wanted %s got %s", sparkService.serviceName, ingressPath.Backend.ServiceName)
+		if ingressPath.Backend.Service.Name != sparkService.serviceName {
+			t.Errorf("Service name wanted %s got %s", sparkService.serviceName, ingressPath.Backend.Service.Name)
 		}
-		if ingressPath.Backend.ServicePort.IntVal != sparkService.servicePort {
-			t.Errorf("Service port wanted %v got %v", sparkService.servicePort, ingressPath.Backend.ServicePort)
+		if *ingressPath.PathType != networkingv1.PathTypeImplementationSpecific {
+			t.Errorf("PathType wanted %s got %s", networkingv1.PathTypeImplementationSpecific, *ingressPath.PathType)
+		}
+		if ingressPath.Backend.Service.Port.Number != sparkService.servicePort {
+			t.Errorf("Service port wanted %v got %v", sparkService.servicePort, ingressPath.Backend.Service.Port.Number)
 		}
 	}
 
@@ -451,7 +456,7 @@ func TestCreateSparkUIIngress(t *testing.T) {
 					"kubernetes.io/ingress.class":                    "nginx",
 					"nginx.ingress.kubernetes.io/force-ssl-redirect": "true",
 				},
-				IngressTLS: []extensions.IngressTLS{
+				IngressTLS: []networkingv1.IngressTLS{
 					{Hosts: []string{"host1", "host2"}, SecretName: "secret"},
 				},
 			},
@@ -475,7 +480,7 @@ func TestCreateSparkUIIngress(t *testing.T) {
 				IngressAnnotations: map[string]string{
 					"kubernetes.io/ingress.class": "nginx",
 				},
-				IngressTLS: []extensions.IngressTLS{
+				IngressTLS: []networkingv1.IngressTLS{
 					{Hosts: []string{"host1", "host2"}, SecretName: ""},
 				},
 			},
@@ -520,7 +525,7 @@ func TestCreateSparkUIIngress(t *testing.T) {
 					"kubernetes.io/ingress.class":                    "nginx",
 					"nginx.ingress.kubernetes.io/force-ssl-redirect": "true",
 				},
-				ingressTLS: []extensions.IngressTLS{
+				ingressTLS: []networkingv1.IngressTLS{
 					{Hosts: []string{"host1", "host2"}, SecretName: "secret"},
 				},
 			},
@@ -536,7 +541,7 @@ func TestCreateSparkUIIngress(t *testing.T) {
 					"kubernetes.io/ingress.class":                    "nginx",
 					"nginx.ingress.kubernetes.io/force-ssl-redirect": "true",
 				},
-				ingressTLS: []extensions.IngressTLS{
+				ingressTLS: []networkingv1.IngressTLS{
 					{Hosts: []string{"host1", "host2"}, SecretName: ""},
 				},
 			},